The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
User Preference and Categories for Error Responses in Conversational User Interfaces
Open accessAbstract
Error messages are frequent in interactions with Conversational User Interfaces (CUI). Smart speakers respond to about every third user request with an error message. Errors can heavily affect user experience (UX) in interaction with CUI. However, there is limited research on how error responses should be formulated. In this paper, we present a system to study how people classify different categories (acknowledgement of user sentiment, acknowledgement of error and apology) of error messages, and evaluate peoples' preference of error responses with clear categories. The results indicate that if an error response has only one element (i.e. neutral acknowledgement of error, apology or sentiment), responses that acknowledge errors neutrally are preferred by participants. Moreover, we find that when interviewed, participants like error messages to include an apology, an explanation of what went wrong, and a suggestion how to fix the problem in addition to a neutral acknowledgement of an error. Our study has two main contributions: (1) our results inform about the design of error messages and (2) we present a framework for error response categorization and validation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Importantly, the comparison and combination of different repair strategies have gained attention as researchers have discovered that employing multiple repair strategies is more effective than relying on a single repair strategy [37, 77].”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [77]
In the citing paper
“One study [77], which did not involve direct interaction with SDSs, investigated user classification of error messages and preferences for error responses using categorisation and sorting surveys.”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [77]
In the citing paper
“Approximately half of the SDSs (N=16) employed in the reviewed studies were mock-ups in which participants interacted with the SDSs through Wizard of Oz (WoZ) setups(e.g. [17, 77]).”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [77]
In the citing paper
“while some studies (e.g., [72, 75, 77]) suggest that the effectiveness of apologies employed as social repair strategies may be limited, others (e.g., [22, 26, 27]) report significant positive impact on users' perceptions of SDS.”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [77]
In the citing paper
“Research shows the effectiveness of system-repair strategies is stronger when combined with other strategies as a multiple-repair strategy [37, 77].”
From System and User Strategies to Repair Conversational Breakdowns of Spoken Dialogue Systems: A Scoping Review (2024) · cites this paper as [77]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,312
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Miller, G. A. (1956) · Psychological Review · doi:10.1037/h0043158 · Cited by 17,812
Content Analysis: An Introduction to Its Methodology
Krippendorff, K. (2019) · doi:10.4135/9781071878781 · Cited by 15,448
The Content Analysis Guidebook
Neuendorf, K. A. (2017) · doi:10.4135/9781071802878 · Cited by 9,286
Inter-Coder Agreement for Computational Linguistics
Artstein, R., & Poesio, M. (2008) · Computational Linguistics · doi:10.1162/coli.07-034-r2 · Cited by 1,665
Mitzias, P., Riga, M., Waddington, S. N., Kontopoulos, E., Μεδίτσκος, Γ., Laurenson, P., & Kompatsiaris, I. Y. (2015) · Cited by 1,240
Qualitative data analysis: the framework approach
Smith, J., & Firth, J. (2011) · Nurse Researcher · doi:10.7748/nr2011.01.18.2.52.c8284 · Cited by 1,206 · cited 1×
“Firstly, the experimenter separates qualitative material into units, secondly, category-sets are established that represent patterns in the data and can be used to establish a theory afterward [12, 38].”
BackgroundNachar, N. (2008) · Tutorials in Quantitative Methods for Psychology · doi:10.20982/tqmp.04.1.p013 · Cited by 1,166
Schema, promise and mutuality: The building blocks of the psychological contract
Rousseau, D. M. (2001) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317901167505 · Cited by 1,148
Reliability in content analysis: some common misconceptions and recommendations.
Krippendorff, K. (2004) · Human Communication Research · doi:10.1093/hcr/30.3.411 · Cited by 1,027
Modeling Local Coherence: An Entity-Based Approach
Barzilay, R., & Lapata, M. (2008) · Computational Linguistics · doi:10.1162/coli.2008.34.1.1 · Cited by 671
The Content Analysis Guidebook
Neuendorf, K. A. (2017) · doi:10.4135/9781071873045 · Cited by 480
Unitizing and categorizing problems in coding qualitative data
Guetzkow, H. (1950) · Journal of Clinical Psychology · doi:10.1002/1097-4679(195001)6:1<47::aid-jclp2270060111>3.0.co;2-i · Cited by 440
Computers that recognise and respond to user emotion: theoretical and practical implications
Picard, R. W., & Klein, J. (2002) · Interacting with Computers · doi:10.1016/s0953-5438(01)00055-8 · Cited by 392
Personality types in software engineering
Capretz, L. F. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(02)00137-4 · Cited by 249
Research Methods in Second Language Acquisition: A Practical Guide
(2011) · Wiley eBooks · doi:10.1002/9781444347340 · Cited by 243
How to Collect and Analyze Qualitative Data
Friedman, D. A. (2011) · Wiley eBooks · doi:10.1002/9781444347340.ch10 · Cited by 225
An Exploration of the Structure of Effective Apologies
Lewicki, R. J., Polin, B., & Lount, R. B. (2016) · Negotiation and Conflict Management Research · doi:10.1111/ncmr.12073 · Cited by 220
Emotion Recognition Based on Joint Visual and Audio Cues
Sebe, N., Cohen, I., Gevers, T., & Huang, T. S. (2006) · 18th International Conference on Pattern Recognition (ICPR'06) · doi:10.1109/icpr.2006.489 · Cited by 202 · cited 1×
“Automatic emotion recognition is investigated by other HCI researchers [36] and applied in industry [5].”
BackgroundHow do users respond to voice input errors? lexical and phonetic query reformulation in voice search
Jiang, J., Jeng, W., & He, D. (2013) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2484028.2484092 · Cited by 110
Toward a more civilized design: studying the effects of computers that apologize
Tzeng, J. (2004) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2004.01.002 · Cited by 107
Kuligowska, K. (2015) · Professionals Center for Business Research · doi:10.18483/pcbr.22 · Cited by 97
This computer responds to user frustration
Klein, J., Moon, Y., & Picard, R. W. (1999) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/632716.632866 · Cited by 82
Modality in Germanic Languages
(1997) · doi:10.1515/9783110889932 · Cited by 79
(1968) · doi:10.2307/2218560 · Cited by 52
Empathic Communication in Virtual Education for Nursing Students: I’m Sorry to Hear That
Strekalova, Y. A., Krieger, J. L. R., Kleinheksel, A. J., & Kotranza, A. (2016) · Nurse Educator · doi:10.1097/nne.0000000000000308 · Cited by 40
Do you care if a computer says sorry? user experience design through affective messages
Park, S. J., MacDonald, C. M., & Khoo, M. J. (2012) · Designing Interactive Systems Conference · doi:10.1145/2317956.2318067 · Cited by 31
Akgun, M., Çağıltay, K., & Zeyrek, D. (2010) · Journal of Pragmatics · doi:10.1016/j.pragma.2009.12.011 · Cited by 27
Profile of the Mechanical Engineer III. Personality
(1955) · doi:10.1111/j.1744-6570.1955.tb01224.x · Cited by 13
Voice assistant technology is in danger of trying to be too human
Clark, L., & Cowan, B. (2019) · doi:10.64628/ab.49jejdhdh
Research works printed but not linked: 8
- Philip R Doyle , Justin Edwards , Odile Dumbleton , Leigh Clark , and Benjiamin R . Cowan . 2019 . Mapping Perceptions of Humanness in Speech-Based Intelligent Personal Assistant Interaction . arXiv preprint arXiv:1907.11585, 2019. (2019). Philip R Doyle, Justin Edwards, Odile Dumbleton, Leigh Clark, and Benjiamin R. Cowan. 2019. Mapping Perceptions of Humanness in Speech-Based Intelligent Personal Assistant Interaction. arXiv preprint arXiv:1907.11585, 2019. (2019). · arXiv
- Eric Enge . 2018 . RATING THE SMARTS OF THE DIGITAL PERSONAL ASSISTANTS IN 2018. https://www.perficientdigital.com/insights/our-research/digital-personal-assistants-study{#}Smartest Eric Enge. 2018. RATING THE SMARTS OF THE DIGITAL PERSONAL ASSISTANTS IN 2018. https://www.perficientdigital.com/insights/our-research/digital-personal-assistants-study{#}Smartest
- Sture Holm . 1979 . Board of the Foundation of the Scandinavian Journal of Statistics A Simple Sequentially Rejective Multiple Test Procedure A Simple Sequentially Rejective Multiple Test Procedure. Source : Scandinavian Journal of Statistics Scand J Statist 6 , 6 (1979), 65 -- 70 . Sture Holm. 1979. Board of the Foundation of the Scandinavian Journal of Statistics A Simple Sequentially Rejective Multiple Test Procedure A Simple Sequentially Rejective Multiple Test Procedure. Source: Scandinavian Journal of Statistics Scand J Statist 6, 6 (1979), 65--70.
- Market Research Engine . 2020. Chatbot Market By Platform Analysis (Web-based , Mobile, Stand-alone) ; By Enterprise size Analysis (Small Enterprises, Medium Enterprises, Large Enterprises) and By Regional Analysis - Global Forecast by 2018 - 2024 | Marketresearch ., 140 pages. https://www.marketresearchengine.com/chatbot-market-report Market Research Engine. 2020. Chatbot Market By Platform Analysis (Web-based, Mobile, Stand-alone); By Enterprise size Analysis (Small Enterprises, Medium Enterprises, Large Enterprises) and By Regional Analysis - Global Forecast by 2018 - 2024 | Marketresearch., 140 pages. https://www.marketresearchengine.com/chatbot-market-report
- Marketsandmarkets. 2018. Smart Manufacturing Market Size , Growth, Trend and Forecast to 2023 . https://www.marketsandmarkets.com/Market-Reports/smart-speaker-market-44984088.htmlhttps://www.marketsandmarkets.com/Market-Reports/smart-manufacturing-market-105448439.html Marketsandmarkets. 2018. Smart Manufacturing Market Size, Growth, Trend and Forecast to 2023. https://www.marketsandmarkets.com/Market-Reports/smart-speaker-market-44984088.htmlhttps://www.marketsandmarkets.com/Market-Reports/smart-manufacturing-market-105448439.html
- Marshall McLuhan and Quentin Fiore . 1967 . The medium is the message . New York 123 (1967), 126 -- 128 . Marshall McLuhan and Quentin Fiore. 1967. The medium is the message. New York 123 (1967), 126--128.
- C Pearl . 2016. Designing voice user interfaces: principles of conversational experiences . O'Reilly Media, Inc. C Pearl. 2016. Designing voice user interfaces: principles of conversational experiences. O'Reilly Media, Inc.
- 10.5555/2467342.2467523 · doi:10.5555/2467342.2467523
Cited by
Theories of Conversation for Conversational IR
Thomas, P., Czerwinksi, M., McDuff, D., & Craswell, N. (2021) · ACM Transactions on Information Systems · doi:10.1145/3439869 · Cited by 29
Glasser, A., Watkins, M., Hart, K., Lee, S., & Huenerfauth, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501987 · Cited by 22
Virtual Customer Assistants in finance: From state of the art and practices to design guidelines
Iovine, A., Narducci, F., Musto, C., de Gemmis, M., & Semeraro, G. M. (2023) · Computer Science Review · doi:10.1016/j.cosrev.2023.100534 · Cited by 29
Perceptions and reactions to conversational privacy initiated by a conversational user interface
Brüggemeier, B., & Lalone, P. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101269 · Cited by 25
Self-citationUser experience with conversational agent: a systematic review of assessment methods
Tubin, C., Rodriguez, J. P. M., & Marchi, A. C. B. D. (2021) · Behaviour and Information Technology · doi:10.1080/0144929x.2021.2001047 · Cited by 24
“With the interviews, it is possible to generate insights from a qualitative perspective (Yuan et al. 2020).”
Background
Metadata
- Authors
- Yuan, S., Brüggemeier, B., Hillmann, S., & Michael, T.
- Affiliations
- Fraunhofer Institute for Integrated Circuits (DE)
- Technische Universität Berlin (DE)
- Venue
- Proceedings of the 2nd Conference on Conversational User Interfaces
- Publisher
- ACM
- Published
- 2020-07-22
- Pages
- 1-8
- OpenAlex
- W3043404834
- Cited by
- 16
- References
- 32
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 2 · 2023: 1 · 2022: 3 · 2021: 6 · 2020: 1
- FWCI
- 1.05
- Type
- conference paper
- Language
- en
- Open access
- Locations
- ACM Conference on Conversational User Interfaces (CUI) (publishedVersion)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion, other-oa)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- Funders
- German Federal Ministry for Economic Affairs and Energy (01MK20011A)
- Federal Ministry of Education and Research (16SV8151)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- error messages, conversational interfaces, smart speakers, user preferences, apology, user sentiment, user experience, voice assistants
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.