Identifying user corrections automatically in spoken dialogue systems
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
We present results of machine learning experiments designed to identify user corrections of speech recognition errors in a corpus collected from a train information spoken dialogue system. We investigate the predictive power of features automatically computable from the prosody of the turn, the speech recognition process, experimental conditions, and the dialogue history. Our best performing features reduce classification error from baselines of 25.70-28.99% to 15.72%.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“with users frequently adopting clarification strategies, such as repetition [21, 66] and modulation strategies, such as changes in voice pitch or volume [24, 28, 46].”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [28]
Citations
Cites
Empirical methods for artificial intelligence
(1996) · Choice Reviews Online · doi:10.5860/choice.33-5144 · Cited by 625
Learning Trees an ules with Set-val ed Features
Cohen, W. W. (1996) · National Conference on Artificial Intelligence · Cited by 274
Towards developing general models of usability with PARADISE
WALKER, M., KAMM, C., & LITMAN, D. (2000) · Natural Language Engineering · doi:10.1017/s1351324900002503 · Cited by 262
Self-citationCorrections in spoken dialogue systems
Swerts, M., Litman, D., & Hirschberg, J. (2000) · 6th International Conference on Spoken Language Processing (ICSLP 2000) · doi:10.21437/icslp.2000-344 · Cited by 81
Self-citationCharacterizing and recognizing spoken corrections in human-computer dialogue
Levow, G.-A. (1998) · Proceedings of the 36th annual meeting on Association for Computational Linguistics - · doi:10.3115/980845.980969 · Cited by 80
Walker, M. A., Langkilde-Geary, I., Wright, J. H., Gorin, A. L., & Litman, D. J. (2000) · Cited by 78
Self-citationPredicting automatic speech recognition performance using prosodic cues
Litman, D. J., Hirschberg, J., & Swerts, M. G. J. (2000) · OLAC - Open Language Archives Community · Cited by 78
Self-citationEmpirically Evaluating an Adaptable Spoken Dialogue System
Litman, D. J., & Pan, S. (1999) · CISM International Centre for Mechanical Sciences · doi:10.1007/978-3-7091-2490-1_6 · Cited by 67
Self-citationAutomatic detection of poor speech recognition at the dialogue level
Litman, D. J., Walker, M. A., & Kearns, M. S. (1999) · Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics - · doi:10.3115/1034678.1034729 · Cited by 65
Self-citationModeling hyperarticulate speech during human-computer error resolution
Oviatt, S., Levow, G.-A., MacEachern, M., & Kuhn, K. (2002) · Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96 · doi:10.1109/icslp.1996.607722 · Cited by 44
Cited by
Designing and Evaluating an Adaptive Spoken Dialogue System
Litman, D. J., & Pan, S. (2002) · User Modeling and User-Adapted Interaction · doi:10.1023/a:1015036910358 · Cited by 160
Self-citationThe Disambiguation of Nominalizations
Lapata, M. (2002) · Computational Linguistics · doi:10.1162/089120102760276018 · Cited by 131
Prosodic and other cues to speech recognition failures
Hirschberg, J., Litman, D. J., & Swerts, M. G. J. (2004) · Speech Communication · doi:10.1016/j.specom.2004.01.006 · Cited by 108
Self-citationAutomatically Training a Problematic Dialogue Predictor for a Spoken Dialogue System
Walker, M. A., Langkilde-Geary, I., Wright Hastie, H., Wright, J., & Gorin, A. (2002) · Journal of Artificial Intelligence Research · doi:10.1613/jair.971 · Cited by 77
Audiovisual representation of prosody in expressive speech communication
Granström, B., & House, D. (2005) · Speech Communication · doi:10.1016/j.specom.2005.02.017 · Cited by 71
Voice Interaction Design: Crafting the New Conversational Speech Systems
Harris, R. A. (2004) · Cited by 66
A “K Hypotheses + Other” Belief Updating Model
Bohus, D., & Rudnicky, A. I. (1994) · KiltHub Repository · doi:10.1184/r1/6587174 · Cited by 54
Rieser, V., & Lemon, O. (2011) · Computational Linguistics · doi:10.1162/coli_a_00038 · Cited by 50
Partially Observable Markov Decision Processes for Spoken Dialogue Management
Williams, J. D. (2006) · Cited by 46
Problem detection in human–machine interactions based on facial expressions of users
Barkhuysen, P. N., Krahmer, E., & Swerts, M. G. J. (2004) · Speech Communication · doi:10.1016/j.specom.2004.10.004 · Cited by 43
Self-citationMultimodal feedback cues in human-machine interactions
Granström, B., House, D., & Swerts, M. G. J. (2002) · Speech prosody · doi:10.21437/speechprosody.2002-72 · Cited by 41
Self-citationConstructing accurate beliefs in spoken dialog systems
Bohus, D., & Rudnicky, A. I. (2005) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2005.1566504 · Cited by 37
Alghamdi, E., Halvey, M., & Nicol, E. (2024) · ACM Conversational User Interfaces 2024 · doi:10.1145/3640794.3665558 · Cited by 36
Error-correction detection and response generation in a spoken dialogue system
Bulyko, I., Kirchhoff, K., Ostendorf, M., & Goldberg, J. (2005) · Speech Communication · doi:10.1016/j.specom.2004.09.009 · Cited by 36
Error awareness and recovery in conversational spoken language interfaces
Rudnicky, A. I., Rosenfeld, R., & Bohus, D. (2007) · Cited by 33
Using word-level pitch features to better predict student emotions during spoken tutoring dialogues
Rotaru, M., & Litman, D. J. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-398 · Cited by 25
Self-citationPredicting user reactions to system error
Litman, D. J., Hirschberg, J., & Swerts, M. G. J. (2001) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1073012.1073060 · Cited by 23
Self-citationMachine learning for shallow interpretation of user utterances in spoken dialogue systems
Lendvai, P., van den Bosch, A. P. J., & Krahmer, E. (2003) · Research portal (Tilburg University) · Cited by 23
The Impact of Response Wording in Error Correction Subdialogs
Goldberg, J. H., Ostendorf, M., & Kirchhoff, K. (2003) · Cited by 21
The effect of response complexity and media on user restatement with multimodal virtual assistants
Freeman, C., & Beaver, I. (2018) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2018.06.002 · Cited by 21
Feedback interpretation based on facial expressions in human-robot interaction
Lang, C., Hanheide, M., Lohse, M., Wersing, H., & Sagerer, G. (2009) · IEEE RO-MAN · doi:10.1109/roman.2009.5326199 · Cited by 18
Detecting Misrecognitions and Corrections in Spoken Dialogue Systems from ‘Aware’ Sites
Hirschberg, J., Litman, D. J., & Swerts, M. G. J. (2003) · Cited by 17
Self-citationModelling and Evaluating Verbal and Non-Verbal Communication in Talking Animated Interface Agents
Granström, B., & House, D. (2007) · Springer eBooks · doi:10.1007/978-1-4020-5817-2_3 · Cited by 13
Predicting Causes of Reformulation in Intelligent Assistants
Sano, S., Kaji, N., & Sassano, M. (2017) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w17-5536 · Cited by 11
Acoustic correlates of user response to error in human-computer dialogues
Kazemzadeh, A., Lee, S., & Narayanan, S. (2004) · 2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No.03EX721) · doi:10.1109/asru.2003.1318443 · Cited by 10
“Also the increased percentage of voiced speech with the decreased percent silence gives evidence of increased speaking rate, which we confirmed by estimating the speech rate by dividing the number of words from the transcription by the duration (other studies have estimated speech rate using purely acoustic features b…”
BackgroundAudiovisual representation of prosody in expressive speech communication
Granström, B., & House, D. (2004) · Speech prosody · doi:10.21437/speechprosody.2004-90 · Cited by 9
Effective Interaction with Talking Animated Agents an Dialogue Systems
Granström, B., & House, D. (2005) · Text, speech and language technology · doi:10.1007/1-4020-3933-6_10 · Cited by 9
Compensating hyperarticulation for automatic speech recognition
Soltau, H. (2005) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.5445/ir/1000003651 · Cited by 9
Labeling corrections and aware sites in spoken dialogue systems
Hirschberg, J., Swerts, M. G. J., & Litman, D. J. (2001) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.3115/1118078.1118088 · Cited by 7
Self-citationTowards conversational QA: automatic identification of problematic situations and user intent
Chai, J. Y., Zhang, C., & Baldwin, T. (2006) · International Conference on Computational Linguistics and Annual Meeting of the Association for Computational Linguistics (COLING/ACL) · doi:10.3115/1273073.1273081 · Cited by 7
Detection and recognition of correction utterances on misrecognition of spoken dialog system
Kitaoka, N., Kakutani, N., & Nakagawa, S. (2005) · Systems and Computers in Japan · doi:10.1002/scj.20341 · Cited by 6
Kontogiorgos, D. (2022) · Thesis Commons (OSF Preprints) · doi:10.31237/osf.io/fpts4 · Cited by 5
Autonomous Self-Assessment of Autocorrections: Exploring Text Message Dialogues
Baldwin, T., & Chai, J. Y. (2012) · North American Chapter of the Association for Computational Linguistics · Cited by 5
MULTIMODAL COMMUNICATION ERROR DETECTION FOR DRIVER-CAR INTERACTION
Wang, S. B., Demirdjian, D., Darrell, T., & Kjellström, H. (2007) · International Conference on Informatics in Control, Automation and Robotics (ICINCO) · doi:10.5220/0001637603650371 · Cited by 4
Using model trees for evaluating dialog error conditions based on acoustic information
Kazemzadeh, A., Lee, S., & Narayanan, S. S. (2006) · doi:10.1145/1178745.1178763 · Cited by 4
Detecting communication errors from visual cues during the system's conversational turn
Wang, S. B., Demirdjian, D., & Darrell, T. (2007) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1322192.1322248 · Cited by 3
Exceptionality and natural language learning
Rotaru, M., & Litman, D. J. (2003) · Conference on Computational Natural Language Learning · doi:10.3115/1119176.1119185 · Cited by 3
Self-citationvan den Bosch, A. P. J. (2005) · Research portal (Tilburg University) · Cited by 3
Speech Driven by Artificial Larynx: Potential Advancement Using Synthetic Pitch Contours
Jian, H. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-20684-4_30 · Cited by 2
(2005) · Elsevier eBooks · doi:10.1016/b978-155860768-2/50018-9
Word Embeddings and Their Role in Synonym Ring Construction
Dermentzi, M. (2021) · OpenEdition (OpenEdition) · doi:10.58079/nkm9
Metadata
- Authors
- Hirschberg, J., Litman, D., & Swerts, M.
- Affiliations
- AT&T (United States) (US)
- Venue
- Second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies 2001 - NAACL '01
- Published
- 2001
- Pages
- 1-8
- OpenAlex
- W2064094392
- Cited by
- 40
- References
- 16
- Citations by year
- 2024: 1 · 2022: 1 · 2018: 2 · 2017: 1 · 2015: 1 · 2012: 1
- FWCI
- 3.56
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- spoken dialogue systems, user corrections, speech recognition, dialogue management
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.