Analysis of user behavior under error conditions in spoken dialogs
Abstract
We focus on developing an account of user behavior under error conditions, working with annotated data from real human-machine mixed initiative dialogs. In particular, we examine categories of error perception, user behavior under error, effect of user strategies on error recovery, and the role of user initiative in error situations. A conditional probability model smoothed by weighted ASR error rate is proposed. Results show that users discovering errors through implicit confirmations are less likely to get back on track (or succeed) and take a longer time in doing so than other forms of error discovery such as system reject and reprompts. Further successful user error-recovery strategies included more rephrasing, less contradicting, and a tendency to terminate error episodes (cancel and startover) than to attempt at repairing a chain of errors.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Studies within the customer service domain utilised telephones [9, 11, 15, 27, 32, 66] or computer speakers [29, 38].
“By contrast, studies within the customer service domain utilised telephones [9, 11, 15, 27, 32, 66] or computer speakers [29, 38].”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [66]
Some studies show that correction often manifests as a contradiction, wherein the user interrupts or 'barges in' to correct and guide the SDS's incorrect process.
“Some studies (e.g. [32, 66]) show that correction often manifests as a contradiction, wherein the user interrupts or 'barges in' to correct and guide the SDS's incorrect process.”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [66]
Users frequently adopt clarification strategies such as repetition and modulation strategies such as changes in voice pitch or volume.
“with users frequently adopting clarification strategies, such as repetition [21, 66] and modulation strategies, such as changes in voice pitch or volume [24, 28, 46].”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [66]
Cites
A stochastic model of human-machine interaction for learning dialog strategies
Levin, E., Pieraccini, R., & Eckert, W. (2000) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.817450 · Cited by 516 · see citing works
Gorin, A. L., Riccardi, G., & Wright, J. H. (1997) · Speech Communication · doi:10.1016/s0167-6393(97)00040-x · Cited by 510 · see citing works
JUPlTER: a telephone-based conversational interface for weather information
Zue, V. W., Seneff, S., Glass, J. R., Polifroni, J. H., Pao, C., Hazen, T. J., & Hetherington, L. (2000) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.817460 · Cited by 437 · see citing works
DARPA communicator dialog travel planning systems: the june 2000 data collection
Walker, M. A., Aberdeen, J. S., Boland, J., Bratt, E. O., Garofolo, J. S., Hirschman, L., Le, A., Lee, S. C., Narayanan, S. S., Papineni, K., et al. (2001) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.2001-355 · Cited by 109 · see citing works
Self-citationCharacterizing and recognizing spoken corrections in human-computer dialogue
Levow, G. (1998) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/980845.980969 · Cited by 80 · see citing works
The AT&t-DARPA communicator mixed-initiative spoken dialog system
Levin, E., Narayanan, S. S., Pieraccini, R., Biatov, K., Bocchieri, E., Fabbrizio, G. D., Eckert, W., Lee, S. C., Pokrovsky, A., Rahim, M. G., et al. (2000) · International Conference on Spoken Language Processing (ICSLP) · doi:10.21437/icslp.2000-224 · Cited by 56 · see citing works
Self-citationFinding errors automatically in semantically tagged dialogues
Aberdeen, J. S., Doran, C., Damianos, L. E., Bayer, S., & Hirschman, L. (2001) · International Conference on Human Language Technology Research · doi:10.3115/1072133.1072180 · Cited by 7 · see citing works
Flagged first, then by citations
1 of the 8 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
SCHATZMANN, J., WEILHAMMER, K., STUTTLE, M., & YOUNG, S. (2006) · The Knowledge Engineering Review · doi:10.1017/s0269888906000944 · Cited by 336 · see citing works
Let's go public! taking a spoken dialog system to the real world
Raux, A., Langner, B., Bohus, D., Black, A. W., & Eskénazi, M. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-399 · Cited by 226 · see citing works
Example-based dialog modeling for practical multi-domain dialog system
Lee, C., Jung, S., Kim, S., & Lee, G. G. (2009) · Speech Communication · doi:10.1016/j.specom.2009.01.008 · Cited by 139 · see citing works
Exploring human error recovery strategies: Implications for spoken dialogue systems
Skantze, G. (2005) · Speech Communication · doi:10.1016/j.specom.2004.11.005 · Cited by 137 · see citing works
Sorry, I Didn’t Catch That! An Investigation of Non-understanding Errors and Recovery Strategies
Bohus, D., & Rudnicky, A. I. (2018) · doi:10.1184/r1/6609683 · Cited by 109 · see citing works
Recent Approaches to Dialog Management for Spoken Dialog Systems
Lee, C., Jung, S., Kim, K., Lee, D., & Lee, G. G. (2010) · Journal of Computing Science and Engineering · doi:10.5626/jcse.2010.4.1.001 · Cited by 82 · see citing works
(1992) · doi:10.1201/b12577 · Cited by 57 · see citing works
Feil-Seifer, D., & Matarić, M. J. (2009) · Springer eBooks · doi:10.1007/978-0-387-30440-3_274 · Cited by 52 · see citing works
Partially Observable Markov Decision Processes for Spoken Dialogue Management
Williams, J. D. (2006) · Cited by 39 · see citing works
Error-correction detection and response generation in a spoken dialogue system
Bulyko, I., Kirchhoff, K., Ostendorf, M., & Goldberg, J. (2005) · Speech Communication · doi:10.1016/j.specom.2004.09.009 · Cited by 36 · see citing works
Constructing accurate beliefs in spoken dialog systems
Bohus, D., & Rudnicky, A. I. (2005) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2005.1566504 · Cited by 32 · see citing works
Error awareness and recovery in conversational spoken language interfaces
Rudnicky, A. I., Rosenfeld, R., & Bohus, D. (2007) · Cited by 31 · see citing works
Transparency strategies to help users handle system errors
Karsenty, L., & Botherel, V. (2005) · Speech Communication · doi:10.1016/j.specom.2004.10.018 · Cited by 25 · see citing works
Hybrid approach to robust dialog management using agenda and dialog examples
Lee, C., Jung, S., Kim, K., & Lee, G. G. (2009) · Computer Speech & Language · doi:10.1016/j.csl.2009.08.003 · Cited by 23 · see citing works
ASR error segment localization for spoken recovery strategy
Béchet, F., & Favre, B. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6638986 · Cited by 20 · see citing works
Handling speech input in the ritel QA dialogue system
van Schooten, B. W., Rosset, S., Galibert, O., Max, A., den Akker, R. O., & Illouz, G. (2007) · Interspeech (USB) · doi:10.21437/interspeech.2007-55 · Cited by 20 · see citing works
The Impact of Response Wording in Error Correction Subdialogs
Goldberg, J. H., Mari Ostendorf, E., & Kirchhoff, K. (2003) · Cited by 18 · see citing works
Spoken Dialog Systems for Automated Survey Interviewing
Johnston, M. O., Ehlen, P., Conrad, F. G., Schober, M. F., Antoun, C., Stefanie, F., Hupp, A. L., Vickers, L., Yan, H. Y., & Zhang, C. (2013) · Cited by 15 · see citing works
Robot perception errors and human resolution strategies in situated human–robot dialogue
Schütte, N., Namee, B. M., & Kelleher, J. D. (2017) · Advanced Robotics · doi:10.1080/01691864.2016.1268973 · Cited by 15 · see citing works
Alghamdi, E., Halvey, M., & Nicol, E. (2024) · ACM Conversational User Interfaces 2024 · doi:10.1145/3640794.3665558 · Cited by 36 · see citing works
Opfermann, C., & Pitsch, K. (2017) · 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) · doi:10.1109/roman.2017.8172319 · Cited by 12 · see citing works
Predicting concept types in user corrections in dialog
Stoyanchev, S., & Stent, A. J. (2009) · doi:10.3115/1626296.1626302 · Cited by 11 · see citing works
Acoustic correlates of user response to error in human-computer dialogues
Kazemzadeh, A., Lee, S., & Narayanan, S. (2004) · 2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No.03EX721) · doi:10.1109/asru.2003.1318443 · Cited by 10 · see citing works
Self-citationUsers’ Perceived Difficulties and Corresponding Reformulation Strategies in Voice Search
Jeng, W., He, D., & Jiang, J. (2013) · D-Scholarship@Pitt (University of Pittsburgh) · Cited by 9 · see citing works
Weitz, K., Vanderlyn, L., Vu, N. T., & André, E. (2021) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2021.conll-1.1 · Cited by 8 · see citing works
Shin, J., Georgiou, P., & Narayanan, S. S. (2012) · Computer Speech & Language · doi:10.1016/j.csl.2012.02.001 · Cited by 8 · see citing works
Self-citationA framework for improving error detection and correction in spoken dialog systems
Griol, D., & Molina, J. M. (2016) · Soft Computing · doi:10.1007/s00500-016-2290-z · Cited by 8 · see citing works
Socially Assistive Robot-Based Intervention for Children with Autism Spectrum Disorder
Feil-Seifer, D. (2008) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · Cited by 6 · see citing works
“Errare Humanum Est”: Simulation of communication error among a virtual team in crisis situation
Huguet, L., Sabouret, N., & Lourdeaux, D. (2016) · IEEE International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC) · doi:10.1109/icci-cc.2016.7862058 · Cited by 6 · see citing works
Shin, J., Georgiou, P. G., & Narayanan, S. S. (2010) · Computer Speech and Language · doi:10.1016/j.csl.2009.04.008 · Cited by 7 · see citing works
Self-citationExample-based error recovery strategy for spoken dialog system
Lee, C., Jung, S., Lee, D., & Lee, G. G. (2007) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2007.4430169 · Cited by 6 · see citing works
Matarić, M. J., & Feil-Seifer, D. (2012) · Cited by 5 · see citing works
Improving user interaction with spoken dialog systems via shaping
Tomko, S. (2005) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1056808.1056846 · Cited by 5 · see citing works
Crowd-supervised training of spoken language systems
Seneff, S., & Mcgraw, I. C. (2012) · Cited by 5 · see citing works
Natural language description of emotion
Kazemzadeh, A. (2013) · ProQuest LLC eBooks · doi:10.25549/usctheses-c3-298137 · Cited by 4 · see citing works
Self-citationReranked aligners for interactive transcript correction
Favre, B., Rouvier, M., & Béchet, F. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6853575 · Cited by 4 · see citing works
Using model trees for evaluating dialog error conditions based on acoustic information
Kazemzadeh, A., Lee, S., & Narayanan, S. S. (2006) · doi:10.1145/1178745.1178763 · Cited by 4 · see citing works
Self-citationProblem-Sensitive Response Generation in Human-Robot Dialogs
Gieselmann, P., & Ostendorf, M. (2007) · Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue · doi:10.18653/v1/2007.sigdial-1.38 · Cited by 4 · see citing works
Towards a comprehensive repair framework for human-chatbot interaction
Asisof, A. (2022) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3514197.3549641 · Cited by 3 · see citing works
The Effect of Sensor Errors in Situated Human-Computer Dialogue
Schütte, N., Kelleher, J. D., & Namee, B. M. (2014) · Workshop on Vision and Language (VL) · doi:10.3115/v1/w14-5401 · Cited by 2 · see citing works
The effect of miscommunication rate on user response preferences
Ai, H., Harris, T., & Rosé, C. P. (2006) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1125451.1125551 · Cited by 2 · see citing works
Concept Type Prediction and Responsive Adaptation in a Dialogue System
Stoyanchev, S., & Stent, A. J. (2012) · Dialogue & Discourse · doi:10.5087/dad.2012.101 · Cited by 2 · see citing works
Software Monitoring in Data Centers
Wu, C., & Guo, J. (2015) · Springer eBooks · doi:10.1007/978-1-4939-2092-1_42 · Cited by 2 · see citing works
User modeling in a speech translation driven mediated interaction setting
Shin, J., Georgiou, P., & Narayanan, S. S. (2006) · doi:10.1145/1178745.1178759 · Cited by 1 · see citing works
Self-citationToward a Computational Approach for Natural Language Description of Emotions
Kazemzadeh, A. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24571-8_23 · Cited by 1 · see citing works
Self-citationProceedings of SRSL 2009, the 2nd Workshop on Semantic Representation of Spoken Language
Plá, M. A., & Declerck, T. (2009) · Cited by 1 · see citing works
Examining the Effects of Rephrasing User Input on Two Mobile Spoken Language Systems
Tsourakis, N., Lisowska, A., Rayner, M., & Bouillon, P. (2010) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/38sk3m6ato23
Schütte, N., Kelleher, J. D., & Namee, B. M. (2021) · Arrow@dit (Dublin Institute of Technology) · doi:10.21427/eg9g-ve04
Clarification Dialogues for Perception-based Errors in Situated Human-Computer Dialogues
Schütte, N., Kelleher, J. D., & Namee, B. M. (2014) · doi:10.1145/2666499.2666506
Impact of Responsive and Directive Adaptation on Local
Stoyanchev, S. (2009)
Speech Recognition Standard Procedures, Error Recognition and Repair Strategies
Chandra, E., & Ramaraj, E. (2006) · IEEE International Conference on Communication Technology (ICCT) · doi:10.1109/icct.2006.341808
김민지 (2018) · Seoul National University Open Repository (Seoul National University)
Feil-Seifer, D., & Matarić, M. J. (2020) · Springer eBooks · doi:10.1007/978-3-642-27737-5_274-5 · Cited by 1 · see citing works
Perception Based Misunderstandings in Human-Computer Dialogues
Schütte, N. (2014) · Arrow@dit (Dublin Institute of Technology) · doi:10.21427/d79c96
- Authors
- Shin, J., Narayanan, S. S., Gerber, L., Kazemzadeh, A., & Byrd, D.
- Venue
- 7th International Conference on Spoken Language Processing (ICSLP 2002)
- Publisher
- ISCA
- Published
- 2002-09-16
- Pages
- 2069-2072
- OpenAlex
- W2151600365
- Cited by
- 54
- References
- 8
- Citations by year
- 2024: 1 · 2022: 1 · 2021: 2 · 2020: 1 · 2018: 2 · 2017: 2 · 2016: 2 · 2015: 1 · 2014: 4 · 2013: 4 · 2012: 4
- FWCI
- 2.17
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Spoken Language Processing (ICSLP) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- spoken dialogue systems, error recovery, user behavior, error perception, user strategies, conditional probability model, paraphrase
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review