Conversational Agents Trust Calibration: A User-Centred Perspective to Design
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Previous work identified trust as one of the key requirements for adoption and continued use of conversational agents (CAs). Given recent advances in natural language processing and deep learning, it is currently possible to execute simple goal-oriented tasks by using voice. As CAs start to provide a gateway for purchasing products and booking services online, the question of trust and its impact on users’ reliance and agency becomes ever-more pertinent. This paper collates trust-related literature and proposes four design suggestions that are illustrated through example conversations. Our goal is to encourage discussion on ethical design practices to develop CAs that are capable of employing trust-calibration techniques that should, when relevant, reduce the user’s trust in the agent. We hope that our reflections, based on the synthesis of insights from the fields of human-agent interaction, explainable ai, and information retrieval, can serve as a reminder of the dangers of excessive trust in automation and contribute to more user-centred CA design.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 1×
“of Human-Automation Interaction (HAI), especially from a human factors and ergonomics perspective [25, 30].”
BackgroundA model for types and levels of human interaction with automation
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/3468.844354 · Cited by 4,195 · cited 1×
“Complacent behaviour can be harmful to the user in the long term as it increases the risk of missing automation failures and reduces the situational awareness of the user, ultimately resulting in poor decision-making [35].”
BackgroundTrust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743
Luger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176
O'Donovan, J., & Smyth, B. (2005) · Proceedings of the 10th international conference on Intelligent user interfaces · doi:10.1145/1040830.1040870 · Cited by 865
Qiu, L., & Benbasat, I. (2009) · Journal of Management Information Systems · doi:10.2753/mis0742-1222250405 · Cited by 737
What Makes a Good Conversation?: Challenges in Designing Truly Conversational Agents
Clark, L., Pantidi, N., Cooney, O., Doyle, P., Garaialde, D., Edwards, J., Spillane, B., Gilmartin, E., Murad, C., Munteanu, C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300705 · Cited by 461
Systematic Review: Trust-Building Factors and Implications for Conversational Agent Design
Rheu, M., Shin, J. Y., Peng, W., & Huh-Yoo, J. (2021) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1807710 · Cited by 360
"What can i help you with?": infrequent users' experiences of intelligent personal assistants
Cowan, B. R., Pantidi, N., Coyle, D., Morrissey, K., Clarke, P., Al-Shehri, S., Earley, D., & Bandeira, N. (2017) · Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/3098279.3098539 · Cited by 351
Cited by
Cohn, M., Pushkarna, M., Olanubi, G. O., Moran, J. M., Padgett, D., Mengesha, Z., & Heldreth, C. (2024) · Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613905.3650818 · Cited by 71
Generative AI Considered Harmful
Fischer, J. E. (2023) · Proceedings of the 5th International Conference on Conversational User Interfaces · doi:10.1145/3571884.3603756 · Cited by 47
The Art of Talking Machines: A Comprehensive Literature Review of Conversational User Interfaces
Namvarpour, M. (., & Razi, A. (2025) · Proceedings of the 7th ACM Conference on Conversational User Interfaces · doi:10.1145/3719160.3736621 · Cited by 19
Impact of Voice Fidelity on Decision Making: A Potential Dark Pattern?
Dubiel, M., Sergeeva, A., & Leiva, L. A. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645202 · Cited by 16
Self-citationRadensky, M., Séguin, J. A., Lim, J. S., Olson, K., & Geiger, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594043 · Cited by 14
Metadata
- Authors
- Dubiel, M., Daronnat, S., & Leiva, L. A.
- Venue
- Proceedings of the 4th Conference on Conversational User Interfaces
- Publisher
- ACM
- Published
- 2022-07-26
- Pages
- 1-6
- OpenAlex
- W4295919094
- Cited by
- 27
- References
- 19
- FWCI
- 3.00
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Conversational Agents Trust Calibration: A User-Centred Perspective to Design. https://app.cochat.ai/papers/10.1145/3543829.3544518
- The paper
- Dubiel, M., Daronnat, S., & Leiva, L. A. (2022). Conversational Agents Trust Calibration: A User-Centred Perspective to Design. Proceedings of the 4th Conference on Conversational User Interfaces. https://doi.org/10.1145/3543829.3544518
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.