Trusting AI: does uncertainty visualization affect decision-making?
Open accessAbstract
Introduction Decision-making based on AI can be challenging, especially when considering the uncertainty associated with AI predictions. Visualizing uncertainty in AI refers to techniques that use visual cues to represent the level of confidence or uncertainty in an AI model's outputs, such as predictions or decisions. This study aims to investigate the impact of visualizing uncertainty on decision-making and trust in AI. Methods We conducted a user study with 147 participants, utilizing static classic gaming scenarios as a proxy for human-AI collaboration in decision-making. The study measured changes in decisions, trust in AI, and decision-making confidence when uncertainty was visualized in a continuous format compared to a binary output of the AI model. Results Our findings indicate that visualizing uncertainty significantly enhances trust in AI for 58% of participants with negative attitudes toward AI. Additionally, 31% of these participants found uncertainty visualization to be useful. The size of the uncertainty visualization was identified as the method that had the most impact on participants' trust in AI and their confidence in their decisions. Furthermore, we observed a strong association between participants' gaming experience and changes in decision-making when uncertainty was visualized, as well as a strong link between trust in AI and individual attitudes toward AI. Discussion These results suggest that visualizing uncertainty can improve trust in AI, particularly among individuals with negative attitudes toward AI. The findings also have important implications for the design of human-AI decision-support systems, offering insights into how uncertainty can be visualized to enhance decision-making and user confidence.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · see citing works
Developing and validating trust measures for e-commerce: an integrative typology
McKnight, D. H., Choudhury, V., & Kacmar, C. (2002) · Information Systems Research · doi:10.1287/isre.13.3.334.81 · Cited by 5,550 · see citing works
Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making
Jarrahi, M. H. (2018) · Business Horizons · doi:10.1016/j.bushor.2018.03.007 · Cited by 2,772 · see citing works
Foundations for an Empirically Determined Scale of Trust in Automated Systems
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000) · International Journal of Cognitive Ergonomics · doi:10.1207/s15327566ijce0401_04 · Cited by 1,799 · see citing works
Automating the design of graphical presentations of relational information
Mackinlay, J. D. (1986) · ACM Transactions on Graphics · doi:10.1145/22949.22950 · Cited by 1,631 · see citing works
In AI we trust? Perceptions about automated decision-making by artificial intelligence
Araujo, T., Helberger, N., Kruikemeier, S., & de Vreese, C. H. (2020) · AI & SOCIETY · doi:10.1007/s00146-019-00931-w · Cited by 1,038 · see citing works
Initial validation of the general attitudes towards Artificial Intelligence Scale
Schepman, A., & Rodway, P. (2020) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2020.100014 · Cited by 785 · see citing works
Shneiderman, B. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3419764 · Cited by 768 · see citing works
Trust in AI: progress, challenges, and future directions
Afroogh, S., Akbari, A., Malone, E., Kargar, M., & Alambeigi, H. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04044-8 · Cited by 618 · see citing works
Rainbow Color Map (Still) Considered Harmful
Borland, D., & Taylor, R. M. (2007) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2007.323435 · Cited by 503 · see citing works
Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making
Cai, C. J., Reif, E., Hegde, N., Hipp, J., Kim, B., Smilkov, D., Wattenberg, M., Viegas, F., Corrado, G. S., Stumpe, M. C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300234 · Cited by 405 · see citing works
Encyclopedia of Human-Computer Interaction
Baskerville, R., & Stage, J. (2004) · Cited by 401 · see citing works
Artificial Intelligence for Intraoperative Guidance
Madani, A., Namazi, B., Altieri, M. S., Hashimoto, D. A., Rivera, A. M., Pucher, P. H., Navarrete-Welton, A., Sankaranarayanan, G., Brunt, L. M., Okrainec, A., et al. (2020) · Annals of Surgery · doi:10.1097/sla.0000000000004594 · Cited by 395 · see citing works
Xu, W., Dainoff, M. J., Ge, L., & Gao, Z. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2041900 · Cited by 363 · see citing works
Hybrid-augmented intelligence: collaboration and cognition
Zheng, N., Liu, Z.-Y., Ren, P.-J., Ma, Y.-Q., Chen, S.-T., Yu, S.-Y., Xue, J.-R., Chen, B.-D., & Wang, F.-Y. (2017) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700053 · Cited by 303 · see citing works
Kwak, Y., Ahn, J.-W., & Seo, Y. H. (2022) · BMC Nursing · doi:10.1186/s12912-022-01048-0 · Cited by 291 · see citing works
Sue, V. M., & Ritter, L. A. (2007) · Sage eBooks · doi:10.4135/9781412983754 · Cited by 287 · see citing works
Adaptive trust calibration for human-AI collaboration
Okamura, K., & Yamada, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0229132 · Cited by 239 · see citing works
Humans versus AI: whether and why we prefer human-created compared to AI-created artwork
Bellaiche, L., Shahi, R., Turpin, M. H., Ragnhildstveit, A., Sprockett, S., Barr, N., Christensen, A., & Seli, P. (2023) · Cognitive Research: Principles and Implications · doi:10.1186/s41235-023-00499-6 · Cited by 233 · see citing works
Zhang, L., Pentina, I., & Fan, Y. (2021) · Journal of Services Marketing · doi:10.1108/jsm-05-2020-0162 · Cited by 210 · see citing works
The automation of leadership functions: Would people trust decision algorithms?
Höddinghaus, M., Sondern, D., & Hertel, G. (2020) · Computers in Human Behavior · doi:10.1016/j.chb.2020.106635 · Cited by 192 · see citing works
Rapid Trust Calibration through Interpretable and Uncertainty-Aware AI
Tomsett, R., Preece, A., Braines, D., Cerutti, F., Chakraborty, S., Srivastava, M., Pearson, G., & Kaplan, L. (2020) · Patterns · doi:10.1016/j.patter.2020.100049 · Cited by 186 · see citing works
Papageorgiou, E. I., Markinos, A. T., & Gemtos, T. A. (2011) · Applied Soft Computing · doi:10.1016/j.asoc.2011.01.036 · Cited by 157 · see citing works
Common Concerns with MTurk as a Participant Pool: Evidence and SolutionsPreprint
Hauser, D., Paolacci, G., & Chandler, J. (2018) · doi:10.31234/osf.io/uq45c · Cited by 156 · see citing works
Lee, M. K., & Rich, K. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445570 · Cited by 145 · see citing works
AI TOOLS IN DECISION MAKING SUPPORT SYSTEMS: A REVIEW
Phillips-Wren, G. E. (2012) · International Journal of Artificial Intelligence Tools · doi:10.1142/s0218213012400052 · Cited by 123 · see citing works
Predicting risk in criminal procedure: actuarial tools, algorithms, AI and judicial decision-making
McKay, C. (2019) · Current Issues in Criminal Justice · doi:10.1080/10345329.2019.1658694 · Cited by 113 · see citing works
Evaluating Sketchiness as a Visual Variable for the Depiction of Qualitative Uncertainty
Boukhelifa, N., Bezerianos, A., Tobias Isenberg, A., & Fekete, J. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.220 · Cited by 111 · see citing works
Moss, A. J., Rosenzweig, C., Robinson, J., Jaffe, S. N., & Litman, L. (2023) · Behavior Research Methods · doi:10.3758/s13428-022-02005-0 · Cited by 109 · see citing works
Effect of AI Explanations on Human Perceptions of Patient-Facing AI-Powered Healthcare Systems
Zhang, Z., Genc, Y., Wang, D., Ahsen, M. E., & Fan, X. (2021) · Journal of Medical Systems · doi:10.1007/s10916-021-01743-6 · Cited by 108 · see citing works
Will AI Console Me when I Lose my Pet? Understanding Perceptions of AI-Mediated Email Writing
Liu, Y., Mittal, A., Yang, D., & Bruckman, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517731 · Cited by 92 · see citing works
Prabhudesai, S., Yang, L., Asthana, S., Huan, X., Liao, Q. V., & Banovic, N. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584033 · Cited by 82 · see citing works
Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies
Lai, V., Chen, C., Liao, Q. V., Smith-Renner, A., & Tan, C. (2021) · arXiv · doi:10.48550/arxiv.2112.11471 · Cited by 75 · see citing works
Visualizing Uncertainty and Alternatives in Event Sequence Predictions
Guo, S., Du, F., Malik, S., Koh, E., Kim, S., Liu, Z., Kim, Y., Hongyuan Zha, B., & Cao, N. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300803 · Cited by 56 · see citing works
In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making ProcessesPreprint
Ashoori, M., & Weisz, J. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02675 · Cited by 54 · see citing works
Argumentation–based negotiation planning for autonomous agents
Monteserin, A., & Amandi, A. (2011) · Decision Support Systems · doi:10.1016/j.dss.2011.02.016 · Cited by 37 · see citing works
Malof, J. M., Mazurowski, M. A., & Tourassi, G. D. (2011) · Neural Networks · doi:10.1016/j.neunet.2011.07.002 · Cited by 36 · see citing works
Schueller, L. A., Booth, L. M., Fleming, K., et al. (2020) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2020.101802 · Cited by 26 · see citing works
Cognitive Models of the Influence of Color Scale on Data Visualization Tasks
Breslow, L. A., Ratwani, R. M., & Trafton, J. G. (2009) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720809338286 · Cited by 24 · see citing works
How to build trust in answers given by Generative AI for specific and vague financial questions
Zarifis, A., & Cheng, X. (2024) · Journal of Electronic Business & Digital Economics · doi:10.1108/jebde-11-2023-0028 · Cited by 21 · see citing works
Human Information Interaction and the Cognitive Predicting Theory of Trust
Fell, L., Gibson, A., Bruza, P., & Hoyte, P. (2020) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3343413.3377981 · Cited by 15 · see citing works
Measuring and Predicting Human Trust in Recommendations from an AI Teammate
Gurney, N., Pynadath, D. V., & Wang, N. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05643-7_2 · Cited by 9 · see citing works
Taghezout, N., & Zaraté, P. (2011) · Intelligent Decision Technologies · doi:10.3233/idt-2011-0111 · Cited by 6 · see citing works
Incorporating Uncertainty into Decision-Making: An Information Visualisation Approach
Daradkeh, M. K., & Abul-Huda, B. A. H. (2017) · Lecture notes in business information processing · doi:10.1007/978-3-319-57487-5_6 · Cited by 6 · see citing works
Investigating Students' Attitudes & Trust in AI During COVID-19
Arif, A. (2023) · Journal of Student Research · doi:10.47611/jsrhs.v12i3.5015 · Cited by 5 · see citing works
Uncertainty-based decision support system for gaming applications
Jagtap, V., Kulkarni, P. A., & Joshi, P. (2022) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-221611 · Cited by 4 · see citing works
Visualization of Machine Learning Uncertainty in AR-Based See-Through Applications
Doula, A., Schmidt, L., Mühlhäuser, M., et al. (2022) · IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR) · doi:10.1109/aivr56993.2022.00022 · Cited by 4 · see citing works
Representing Uncertainty Information from AI for Human Understanding
Cassenti, D. N., Kaplan, L., & Roy, A. (2023) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/21695067231193649 · Cited by 4 · see citing works
Using AI Uncertainty Quantification to Improve Human Decision-MakingPreprint
Marusich, L. R., Bakdash, J. Z., Zhou, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.10852 · Cited by 2 · see citing works
Measuring Algorithm Aversion and Betrayal Aversion to Humans and AI using Trust Games
Takagi, H., Li, Y., Komori, M., & Terada, K. (2024) · IEEE RO-MAN · doi:10.1109/ro-man60168.2024.10731215 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- Greis, M., Agroudy, P. E., Schuff, H., Machulla, T., and Schmidt, A. (2016). "Decision-making under uncertainty: how the amount of presented uncertainty influences user behavior," in Proceedings of the 9th Nordic Conference on Human- Computer Interaction (New York, NY: ACM), 1-4. doi: 10.1145/2971485.2971535 · DOI
- Kulesza, T., Stumpf, S., Burnett, M., and Kwan, I. (2012). "Tell me more? The effects of mental model soundness on personalizing an intelligent agent," in Proceedings of the Sigchi conference on human factors in computing systems (New York, NY: ACM), 1-10. doi: 10.1145/2207676.2207678 · DOI · Paper page
- Oviatt, S. (2006). "Human-centered design meets cognitive load theory: designing interfaces that help people think," in Proceedings of the 14th ACM international conference on Multimedia (New York, NY: ACM), 871-880. doi: 10.1145/1180639.1180831 · DOI
Cited by
Sorted by citations
Cosic, K., Kuljanac, A., & Palermo, S. (2026) · Frontiers in psychology · doi:10.3389/fpsyg.2026.1821242 · Cited by 3 · see citing works
“In high-stakes settings, however, the usefulness of such systems depends not only on predictive performance but also on trust calibration, and effective teaming and oversight (Reyes et al., 2025; Gonzalez et al., 2026; Kargarnovin et al., 2026).”
- Authors
- Reyes, J., Batmaz, A. U., & Kersten-Oertel, M.
- Affiliations
- Concordia University (CA)
- Corresponding author
- Reyes, Jonatan
- Venue
- Frontiers in Computer Science
- Publisher
- Frontiers Media SA
- Published
- 2025-02-07
- Volume
- 7
- ISSN
- 2624-9898 (ISSN-L)
- OpenAlex
- W4407261951
- Cited by
- 27
- References
- 50
- Citations by year
- 2026: 17 · 2025: 10
- FWCI
- 29.91
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Computer Science (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
- Keywords
- uncertainty visualization, trust in AI, human-AI decision-making, decision confidence, attitudes toward AI, human-AI collaboration, decision support system, AI transparency, video game experience
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
