Acoustic correlates of user response to error in human-computer dialogues
The DOI matches its Crossref record; 8 of the 11 references it cites were found in a registry or an index.
Abstract
Using tagged data from the DARPA Communicator Project, we investigate acoustic features of user responses to system errors. We measure acoustic parameters such as energy, fundamental frequency, sub-band energy, ratios of voiced, unvoiced and silent regions of speech, fundamental frequency slope, spectral slope, and spectral center of gravity. We investigate different types of user responses to the errors, including frustration and various types of corrections. It is confirmed that the most prominent acoustic parameter for responses to the errors is fundamental frequency maximum and range, while other features are found to be salient for specific reaction types. More interestingly, acoustic characteristics of user responses to the errors are found to be different depending on whether the responses are the initial or continued responses to the errors. Similarly, normal user responses can differ acoustically depending on whether or not they were preceded by responses to error. We also present results on automatic classification of error response types using these features.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“By contrast, studies within the customer service domain utilised telephones [9, 11, 15, 27, 32, 66] or computer speakers [29, 38].”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [32]
In the citing paper
“For example,[32] analysed the acoustic features of user-repair strategies and found that contradictions, as a form of correction user strategies, were associated with higher energy and frequency compared to other responses, such as repetition.”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [32]
In the citing paper
“Some studies (e.g. [32, 66]) show that correction often manifests as a contradiction, wherein the user interrupts or 'barges in' to correct and guide the SDS's incorrect process.”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [32]
Citations
Cites
Murray, I. R., & Arnott, J. L. (1993) · The Journal of the Acoustical Society of America · doi:10.1121/1.405558 · Cited by 1,098 · cited 2×
“This approach of using data tagged by annotators reading transcriptions has been used in detecting corrections in [2], while approaches to detecting emotions have generally favored using human listeners [9] or actors [10] [11].”
BackgroundMethodDARPA communicator dialog travel planning systems: the june 2000 data collection
Walker, M., Aberdeen, J., Boland, J., Bratt, E., Garofolo, J., Hirschman, L., Le, A., Lee, S., Narayanan, S., Papineni, K., et al. (2001) · 7th European Conference on Speech Communication and Technology (Eurospeech 2001) · doi:10.21437/eurospeech.2001-355 · Cited by 109 · cited 2×
“The data and the collection procedure are described in detail in [6].”
MethodSelf-citationCorrections in spoken dialogue systems
Swerts, M., Litman, D., & Hirschberg, J. (2000) · 6th International Conference on Spoken Language Processing (ICSLP 2000) · doi:10.21437/icslp.2000-344 · Cited by 81 · cited 2×
“This approach of using data tagged by annotators reading transcriptions has been used in detecting corrections in [2], while approaches to detecting emotions have generally favored using human listeners [9] or actors [10] [11].”
BackgroundMethodThe AT&t-DARPA communicator mixed-initiative spoken dialog system
Levin, E., Narayanan, S., Pieraccini, R., Biatov, K., Bocchieri, E., Fabbrizio, G. D., Eckert, W., Lee, S., Pokrovsky, A., Rahim, M., et al. (2000) · 6th International Conference on Spoken Language Processing (ICSLP 2000) · doi:10.21437/icslp.2000-224 · Cited by 55
Self-citationAnalysis of user behavior under error conditions in spoken dialogs
Shin, J., Narayanan, S. S., Gerber, L., Kazemzadeh, A., & Byrd, D. (2002) · 7th International Conference on Spoken Language Processing (ICSLP 2002) · doi:10.21437/icslp.2002-567 · Cited by 54 · cited 5×
“Previous studies [1,2] have shown that, in human-computer dialogues, users’ reactions to errors have a varying degree of effectiveness in the correction of the errors.”
BackgroundMethodSelf-citationWard, W., & Pellom, B. L. (1999) · Cited by 49
Identifying user corrections automatically in spoken dialogue systems
Hirschberg, J., Litman, D., & Swerts, M. (2001) · Second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies 2001 - NAACL '01 · doi:10.3115/1073336.1073363 · Cited by 40 · cited 1×
“Also the increased percentage of voiced speech with the decreased percent silence gives evidence of increased speaking rate, which we confirmed by estimating the speech rate by dividing the number of words from the transcription by the duration (other studies have estimated speech rate using purely acoustic features b…”
BackgroundA macroscopic analysis of an emotional speech corpus
Noad, J. E. H., Whiteside, S. P., & Green, P. (1997) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1997-172 · Cited by 9 · cited 1×
“This approach of using data tagged by annotators reading transcriptions has been used in detecting corrections in [2], while approaches to detecting emotions have generally favored using human listeners [9] or actors [10] [11].”
Method
Research works printed but not linked: 3
- swaim, 3, A simple c++ regression class, C/C++ User's Journal
- narayanan, 2002, Toward modeling user behavior in human-machine interactions: Effects of Errors and Emotions, Proc ISLE Workshop Multimodal Dialogue Tagging
- lee, 0, Towards Detecting Emotion in Spoken Dialogs, IEEE Transactions on Speech and Audio Processing
Cited by
Alghamdi, E., Halvey, M., & Nicol, E. (2024) · ACM Conversational User Interfaces 2024 · doi:10.1145/3640794.3665558 · Cited by 36
“By contrast, studies within the customer service domain utilised telephones [9, 11, 15, 27, 32, 66] or computer speakers [29, 38].”
BackgroundMethodResultInfluentialError-correction detection and response generation in a spoken dialogue system
Bulyko, I., Kirchhoff, K., Ostendorf, M., & Goldberg, J. (2005) · Speech Communication · doi:10.1016/j.specom.2004.09.009 · Cited by 36
“Researchers have shown in (Kazemzadeh et al., 2003) that the particular combination of acoustic–prosodic correlates often depends on the type of error response (e.g. repetition/rephrase vs. signaling frustration), and that it changes with the distance to the original error occurrence, suggesting…”
BackgroundÁlvarez, A., Cearreta, I., López, J. M., Arruti, A., Lazkano, E., Sierra, B., & Garay-Vitoria, N. (2006) · Lecture notes in computer science · doi:10.1007/11846406_71 · Cited by 24
“We have computed energy in low band, between 0 and 1300 Hz, medium band, between 1300 and 2600 Hz and high band from 2600 to 4000 Hz [ 15 ].”
MethodCorrection grammars for error handling in a speech dialog system
Sagawa, H., Mitamura, T., & Nyberg, E. (2004) · Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL) · doi:10.3115/1613984.1614000 · Cited by 13
“Most of the forms of correction utterances can be expected in advance because correction utterances include many repetitions of words or phrases from previous turns (Kazemzadeh et al., 2003).”
BackgroundÁlvarez, A., Cearreta, I., López, J. M., Arruti, A., Lazkano, E., Sierra, B., & Garay-Vitoria, N. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-77347-4_24 · Cited by 7
“We have computed energy in Low band, between 0 and 1300 Hz, Medium band, between 1300 and 2600 Hz and High band from 2600 to 4000 Hz [ 17 ].”
Method
Metadata
- Authors
- Kazemzadeh, A., Lee, S., & Narayanan, S.
- Affiliations
- University of Southern California (US)
- Venue
- 2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No.03EX721)
- Publisher
- IEEE
- Published
- 2004-09-07
- Pages
- 215-220
- OpenAlex
- W2171913036
- Cited by
- 10
- References
- 11
- Citations by year
- 2025: 1 · 2024: 1 · 2013: 1
- FWCI
- 1.30
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- human-computer dialogue, acoustic features, fundamental frequency, frustration, error correction, speech energy, spectral properties
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Acoustic correlates of user response to error in human-computer dialogues. https://app.cochat.ai/papers/10.1109/asru.2003.1318443
- The paper
- Kazemzadeh, A., Lee, S., & Narayanan, S. (2004). Acoustic correlates of user response to error in human-computer dialogues. 2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No.03EX721). https://doi.org/10.1109/asru.2003.1318443
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.