Understanding Uncertainty: How Lay Decision-makers Perceive and Interpret Uncertainty in Human-AI Decision Making
Open accessAbstract
Decision Support Systems (DSS) based on Machine Learning (ML) often aim to assist lay decision-makers, who are not math-savvy, in making high-stakes decisions. However, existing ML-based DSS are not always transparent about the probabilistic nature of ML predictions and how uncertain each prediction is. This lack of transparency could give lay decision-makers a false sense of reliability. Growing calls for AI transparency have led to increasing efforts to quantify and communicate model uncertainty. However, there are still gaps in knowledge regarding how and why the decision-makers utilize ML uncertainty information in their decision process. Here, we conducted a qualitative, think-aloud user study with 17 lay decision-makers who interacted with three different DSS: 1) interactive visualization, 2) DSS based on an ML model that provides predictions without uncertainty information, and 3) the same DSS with uncertainty information. Our qualitative analysis found that communicating uncertainty about ML predictions forced participants to slow down and think analytically about their decisions. This in turn made participants more vigilant, resulting in reduction in over-reliance on ML-based DSS. Our work contributes empirical knowledge on how lay decision-makers perceive, interpret, and make use of uncertainty information when interacting with DSS. Such foundational knowledge informs the design of future ML-based DSS that embrace transparent uncertainty communication.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Communicating (un)certainty in AI-assisted decision-making settings increases vigilance.
“in AI-assisted decision-making settings, communicating (un)certainty has been shown to support trust calibration [118], increase vigilance [85], and improve task performance [7].”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [85]
Prabhudesai et al. found that visualizing a system's uncertainty using quantile dot plots forced participants to slow down and think analytically.
“In the context of house valuation, Prabhudesai et al. [85] found that visualizing a system's uncertainty using quantile dot plots forced participants to slow down and think analytically.”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as Prabhudesai et al. [85]
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810 · cited 3× · see citing works
“However, heuristic-based reasoning can lead to grave errors of judgement [100].”
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 1× · see citing works
“Additionally, suppressing uncertainty grossly oversimplifies the complex, dynamic and ambiguous nature of the world [42], which can lead to over-reliance of decision-makers on inaccurate DSS predictions that may be rooted in harmful historical biases [2, 10, 12, 35, 81, 82].”
BackgroundThe mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 2× · see citing works
“…of not only uncertainty quantification (UQ) algorithms [40, 75, 101], but also uncertainty communication mechanisms [37, 41, 58, 66], transparent model reporting practices [3, 79, 85], and calls for embracing uncertainty in the design of interactive computer technology [11, 15, 50, 88, 94].”
BackgroundBuçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124 · see citing works
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · see citing works
Self-citationDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 1× · see citing works
“Most existing quantitative empirical studies in ML-supported decision-making measure the effects of transparency and communicating uncertainty on efficiency [4, 102], quality of decisions [36, 69, 73], trust and reliance [5, 7, 18, 19, 77, 108], and the decision-maker’s confidence in decisions [51].”
BackgroundThe need for uncertainty quantification in machine-assisted medical decision making
Begoli, E., Bhattacharya, T., & Kusnezov, D. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-018-0004-1 · Cited by 597 · see citing works
Expanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538 · cited 1× · see citing works
“Transparency mechanisms [28, 30], including uncertainty communication [13], enable decision-makers to understand model limitations and have appropriate trust and reliance on AI [74].”
BackgroundSelf-citationThe Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · see citing works
FactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434 · see citing works
The Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392 · cited 1× · see citing works
“More recently, Machine Learning (ML) has been used in DSS, to extract patterns from large volumes of historic data, and make inferences to inform decision-makers [48].”
MethodResearchers Misunderstand Confidence Intervals and Standard Error Bars.
Belia, S., Fidler, F., Williams, J., & Cumming, G. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.389 · Cited by 390 · see citing works
Impact of a deep learning assistant on the histopathologic classification of liver cancer
Kiani, A., Uyumazturk, B., Rajpurkar, P., Wang, A., Gao, R., Jones, E., Yu, Y., Langlotz, C. P., Ball, R. L., Montine, T. J., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0232-8 · Cited by 325 · see citing works
Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Bansal, G., Nushi, B., Kamar, E., Weld, D. S., Lasecki, W. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012429 · Cited by 284 · see citing works
The flaws of policies requiring human oversight of government algorithms
Green, B. (2022) · Computer Law & Security Review · doi:10.1016/j.clsr.2022.105681 · Cited by 263 · cited 1× · see citing works
“However, when ML-based DSS are opaque about the uncertainty in their predictions (including but not limited to the uncertainty arising from complex, dynamic and ambiguous nature of the world [42]), they may give a false sense of reliability to users [47], whilst covertly reinforcing harmful societal biases [2] and mai…”
BackgroundUncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Bhatt, U., Antorán, J., Zhang, Y., Liao, Q. V., Sattigeri, P., Fogliato, R., Melançon, G., Krishnan, R., Stanley, J., Tickoo, O., et al. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462571 · Cited by 225 · cited 5× · see citing works
“Transparency mechanisms [28, 30], including uncertainty communication [13], enable decision-makers to understand model limitations and have appropriate trust and reliance on AI [74].”
BackgroundInfluentialSelf-citationDual-process cognitive interventions to enhance diagnostic reasoning: a systematic review
Lambe, K. A., O'Reilly, G., Kelly, B. D., & Curristan, S. (2016) · BMJ Quality & Safety · doi:10.1136/bmjqs-2015-004417 · Cited by 215 · cited 1× · see citing works
“Our findings also show that future designs could leverage uncertainty representations as cognitive forcing functions [72] to promote vigilance and further reduce over-reliance on ML algorithms [17].”
BackgroundDecision Support Systems: A Historical Overview
Power, D. J. (2008) · International Handbooks on Information Systems · doi:10.1007/978-3-540-48713-5_7 · Cited by 202 · see citing works
Machine learning for environmental monitoring
Hino, M., Benami, E., & Brooks, N. (2018) · Nature Sustainability · doi:10.1038/s41893-018-0142-9 · Cited by 201 · cited 1× · see citing works
“Recently, ML-based DSS [84] have seen deployment in high-stakes decision-making scenarios, such as production [44], policy [54], welfare [32], and healthcare [89].”
BackgroundRapid Trust Calibration through Interpretable and Uncertainty-Aware AI
Tomsett, R., Preece, A., Braines, D., Cerutti, F., Chakraborty, S., Srivastava, M., Pearson, G., & Kaplan, L. (2020) · Patterns · doi:10.1016/j.patter.2020.100049 · Cited by 186 · cited 2× · see citing works
“Our work shows further evidence that communicating uncertainty could provide a form of algorithmic transparency [13] and enable decision-makers to have appropriate confidence in and reliance on ML predictions [98, 99].”
BackgroundIndependent real‐world application of a clinical‐grade automated prostate cancer detection system
da Silva, L. M., Pereira, E. M., Salles, P. G., Godrich, R., Ceballos, R., Kunz, J. D., Casson, A., Viret, J., Chandarlapaty, S., Ferreira, C. G., et al. (2021) · The Journal of Pathology · doi:10.1002/path.5662 · Cited by 175 · see citing works
Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making
Fernandes, M., Walls, L., Munson, S., Hullman, J., & Kay, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173718 · Cited by 175 · see citing works
The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality
Gordon, M. L., Zhou, K., Patel, K., Hashimoto, T., & Bernstein, M. S. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445423 · Cited by 101 · see citing works
Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies
Lai, V., Chen, C., Liao, Q. V., Smith-Renner, A., & Tan, C. (2021) · arXiv · doi:10.48550/arxiv.2112.11471 · Cited by 75 · see citing works
Self-citationBanovic, N., Yang, Z., Ramesh, A., & Liu, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579460 · Cited by 55 · cited 1× · see citing works
“Most existing quantitative empirical studies in ML-supported decision-making measure the effects of transparency and communicating uncertainty on efficiency [4, 102], quality of decisions [36, 69, 73], trust and reliance [5, 7, 18, 19, 77, 108], and the decision-maker’s confidence in decisions [51].”
Background
Flagged first, then by citations
Cited by
Sorted by citations
Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S., & Vaughan, J. W. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658941 · Cited by 147 · see citing works
Self-citationTrusting AI: does uncertainty visualization affect decision-making?
Reyes, J., Batmaz, A. U., & Kersten-Oertel, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1464348 · Cited by 27 · see citing works
Xu, Z., Song, T., & Lee, Y.-C. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103455 · Cited by 18 · see citing works
Radensky, M., Séguin, J. A., Lim, J. S., Olson, K., & Geiger, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594043 · Cited by 14 · see citing works
Ashktorab, Z., Desmond, M., Pan, Q., Johnson, J., Brachman, M., Dugan, C., Danilevsky, M., & Geyer, W. (2025) · Proceedings of the 4th Annual Symposium on Human-Computer Interaction for Work · doi:10.1145/3729176.3729179 · Cited by 10 · see citing works
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5 · see citing works
Performance and Metacognition Disconnect when Reasoning in Human-AI Interaction
Fernandes, D., Villa, S., Nicholls, S., Haavisto, O., Buschek, D., Schmidt, A., Kosch, T., Shen, C., & Welsch, R. (2024) · arXiv · doi:10.48550/arxiv.2409.16708 · Cited by 71
- Authors
- Prabhudesai, S., Yang, L., Asthana, S., Huan, X., Liao, Q. V., & Banovic, N.
- Affiliations
- University of Michigan (US)
- Columbia University (US)
- Microsoft (Canada) (CA)
- Electrical Engineering and Computer Science, University of Michigan, United States
- Venue
- Proceedings of the 28th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2023-03-27
- Pages
- 379-396
- OpenAlex
- W4360991197
- Cited by
- 82
- References
- 26
- Citations by year
- 2026: 22 · 2025: 40 · 2024: 17 · 2023: 3
- FWCI
- 14.54
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
- Keywords
- uncertainty communication, human-AI decision-making, decision support system, machine learning transparency, model uncertainty, overreliance on AI, interactive visualization, high-stakes decision making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
