When Confidence Meets Accuracy: Exploring the Effects of Multiple Performance Indicators on Trust in Machine Learning Models
Open accessAbstract
Previous research shows that laypeople’s trust in a machine learning model can be affected by both performance measurements of the model on the aggregate level and performance estimates on individual predictions. However, it is unclear how people would trust the model when multiple performance indicators are presented at the same time. We conduct an exploratory human-subject experiment to answer this question. We find that while the level of model confidence significantly affects people’s belief in model accuracy, both the model’s stated and observed accuracy generally have a larger impact on people’s willingness to follow the model’s predictions as well as their self-reported levels of trust in the model, especially after observing the model’s performance in practice. We hope the empirical evidence reported in this work could open doors to further studies to advance understanding of how people perceive, process, and react to performance-related information of machine learning.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Poursabzi-Sangdeh et al. found that providing users with more information through increased interpretability and transparency decreased the user's capabilities to detect and correct mistakes by the system.
“Poursabzi-Sangdeh et al. [108] found that providing users with more information (through increased interpretability and transparency) decreased the user's capabilities to detect and correct mistakes by the system.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [108]
Endo calibrations are achieved by presenting information about system confidence throughout interaction using cues or warning signals, or by increasing transparency through explanations.
“This is achieved by, for example, presenting information about a system's confdence throughout the interaction [90, 104, 145] using cues or warning signals or by increasing a system's transparency through explanations [108].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [108]
Cites
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258 · see citing works
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200 · see citing works
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · see citing works
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061 · see citing works
(2011) · Cited by 2,025 · see citing works
Predicting good probabilities with supervised learning
Niculescu-Mizil, A., & Caruana, R. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102430 · Cited by 1,884 · see citing works
Combining satellite imagery and machine learning to predict poverty
Jean, N., Burke, M., Xie, M., Davis, W. M., Lobell, D. B., & Ermon, S. (2016) · Science · doi:10.1126/science.aaf7894 · Cited by 1,664 · see citing works
Ma, X., Dai, Z., He, Z., Ma, J., Wang, Y., & Wang, Y. (2017) · Sensors · doi:10.3390/s17040818 · Cited by 1,417 · see citing works
Transforming classifier scores into accurate multiclass probability estimates
Zadrozny, B., & Elkan, C. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775151 · Cited by 1,280 · see citing works
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230 · see citing works
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · see citing works
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · cited 1× · see citing works
“Many different factors have been identified as influencing people’s trust in ML, such as people’s understanding of how the model works [19, 37] and people’s perceptions on whether the model is biased [9, 62].”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · see citing works
How Much Information?: Effects of Transparency on Trust in an Algorithmic Interface
Kizilcec, R. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858402 · Cited by 615 · see citing works
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 6× · see citing works
“[58] ran a sequence of human-subject experiments and found that both an ML model’s stated accuracy on held-out data and its observed accuracy in practice significantly affect laypeople’s trust in the model.”
BackgroundMethodResultInfluentialSelf-citationGender Differences in Mate Selection: Evidence From a Speed Dating Experiment
Fisman, R., Iyengar, S. S., Kamenica, E., & Simonson, I. (2006) · The Quarterly Journal of Economics · doi:10.1162/qjec.2006.121.2.673 · Cited by 522 · see citing works
A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk
Hara, K., Adams, A., Milland, K., Savage, S., Callison-Burch, C., & Bigham, J. P. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174023 · Cited by 497 · see citing works
Lai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377 · see citing works
Cho, J.-H., Chan, K., & Adalı, S. (2015) · ACM Computing Surveys · doi:10.1145/2815595 · Cited by 346 · see citing works
Wang, X., & Yin, M. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450650 · Cited by 320 · see citing works
Self-citationThe relationship between trust in AI and trustworthy machine learning technologies
Toreini, E., Aitken, M., Coopamootoo, K., Elliott, K., Zelaya, C. G., & van Moorsel, A. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372834 · Cited by 296 · see citing works
Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders
Cheng, H.-F., Wang, R., Zhang, Z., O'Connell, F., Gray, T., Harper, F. M., & Zhu, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300789 · Cited by 292 · cited 1× · see citing works
“was used in [31, 50], agreement or switch fraction was used in [35, 58, 61], and subject’s self-reported level of trust in the model was used in [10, 58].”
MethodUpdates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Bansal, G., Nushi, B., Kamar, E., Weld, D. S., Lasecki, W. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012429 · Cited by 284 · see citing works
Kocielnik, R., Amershi, S., & Bennett, P. N. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300641 · Cited by 277 · see citing works
Kay, M., Kola, T., Hullman, J. R., & Munson, S. A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858558 · Cited by 239 · see citing works
Neural network model for rapid forecasting of freeway link travel time
Dharia, A., & Adeli, H. (2003) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2003.09.011 · Cited by 233 · see citing works
How do visual explanations foster end users' appropriate trust in machine learning?
Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377480 · Cited by 200 · see citing works
Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making
Fernandes, M., Walls, L., Munson, S., Hullman, J., & Kay, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173718 · Cited by 175 · see citing works
Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems
Kunkel, J., Donkers, T., Michael, L., Barbu, C.-M., & Ziegler, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300717 · Cited by 156 · see citing works
Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
Lu, Z., & Yin, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445562 · Cited by 131 · see citing works
Self-citationI can do better than your AI: expertise and explanations
Schaffer, J., O'Donovan, J., Michaelis, J., Raglin, A., & Höllerer, T. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302308 · Cited by 119 · see citing works
Survey on Computational Trust and Reputation Models
Braga, D. D. S., Niemann, M., Hellingrath, B., & Neto, F. B. D. L. (2018) · ACM Computing Surveys · doi:10.1145/3236008 · Cited by 116 · see citing works
Investigating intelligibility for uncertain context-aware applications
Lim, B. Y., & Dey, A. K. (2011) · Proceedings of the 13th international conference on Ubiquitous computing · doi:10.1145/2030112.2030168 · Cited by 101 · see citing works
Towards improving trust in context-aware systems by displaying system confidence
Antifakos, S., Kern, N., Schiele, B., & Schwaninger, A. (2005) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/1085777.1085780 · Cited by 98 · see citing works
User Trust Dynamics: An Investigation Driven by Differences in System Performance
Yu, K., Berkovsky, S., Taib, R., Conway, D., Zhou, J., & Chen, F. (2017) · Proceedings of the 22nd International Conference on Intelligent User Interfaces · doi:10.1145/3025171.3025219 · Cited by 90 · see citing works
Pinsker, R. E. (2011) · Behavioral Research in Accounting · doi:10.2308/bria.2011.23.1.161 · Cited by 78 · see citing works
Allocating Interventions Based on Predicted Outcomes: A Case Study on Homelessness Services
Kube, A., Das, S., & Fowler, P. J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301622 · Cited by 58 · see citing works
You’d Better Stop! Understanding Human Reliance on Machine Learning Models under Covariate Shift
Chiang, C.-W., & Yin, M. (2021) · 13th ACM Web Science Conference 2021 · doi:10.1145/3447535.3462487 · Cited by 53 · see citing works
Self-citationEvaluating the Effects of Displaying Uncertainty in Context-Aware Applications
Antifakos, S., Schwaninger, A., & Schiele, B. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30119-6_4 · Cited by 50 · see citing works
An Exploration of Location Error Estimation
Dearman, D., Varshavsky, A., de Lara, E., & Truong, K. N. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74853-3_11 · Cited by 28 · see citing works
RUSSELL, W. L. (1940) · Journal of Heredity · doi:10.1093/oxfordjournals.jhered.a104798 · Cited by 20 · see citing works
Verame, J. K. M., Costanza, E., & Ramchurn, S. D. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858369 · Cited by 15 · see citing works
Probability Forecasts and Their Combination: A Research PerspectivePreprint
Winkler, R. L., Grushka‐Cockayne, Y., Lichtendahl, K. C., & Jose, V. R. R. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3258627 · Cited by 7 · see citing works
Flagged first, then by citations
1 of the 45 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 145 · see citing works
Wischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146 · see citing works
Gómez, C., Cho, S. M., Ke, S., Huang, C., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 163 · see citing works
Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems
He, G., Kuiper, L., & Gadiraju, U. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581025 · Cited by 107 · see citing works
“They found that factors like first impression [76], AI literacy [8], risk perception [33, 34], and performance feedback [55, 61] among others, play important roles in shaping human trust and reliance on AI systems.”
BackgroundTowards Risk‐Free Trustworthy Artificial Intelligence: Significance and Requirements
Alzubaidi, L., Al-Sabaawi, A., Bai, J., Dukhan, A., Alkenani, A. H., Al‐Asadi, A., Alwzwazy, H. A., Manoufali, M., Fadhel, M. A., Albahri, A. S., et al. (2023) · International Journal of Intelligent Systems · doi:10.1155/2023/4459198 · Cited by 106 · see citing works
Enhancing AI-Assisted Group Decision Making through LLM-Powered Devil's Advocate
Chiang, C.-W., Lu, Z., Li, Z., & Yin, M. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645199 · Cited by 95 · see citing works
Self-citationHow do we assess the trustworthiness of AI? Introducing the trustworthiness assessment model (TrAM)
Schlicker, N., Baum, K. L., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108671 · Cited by 87 · see citing works
Harnessing Biomedical Literature to Calibrate Clinicians’ Trust in AI Decision Support Systems
Yang, Q., Hao, Y., Quan, K., Yang, S., Zhao, Y., Kuleshov, V., & Wang, F. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581393 · Cited by 75 · see citing works
Budhiraja, R., Joshi, I., Challa, J. S., Akolekar, H. D., & Kumar, D. (2024) · Proceedings of the 26th Australasian Computing Education Conference · doi:10.1145/3636243.3636257 · Cited by 52 · see citing works
Will You Accept the AI Recommendation? Predicting Human Behavior in AI-Assisted Decision Making
Wang, X., Lu, Z., & Yin, M. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512240 · Cited by 49 · see citing works
Self-citationUnderstanding the Evolvement of Trust Over Time within Human-AI Teams
Duan, W., Zhou, S., Scalia, M. J., Yin, X., Weng, N., Zhang, R., Freeman, G., McNeese, N. J., Gorman, J. C., & Tolston, M. T. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687060 · Cited by 53 · see citing works
Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant
He, G., Aishwarya, N., & Gadiraju, U. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712133 · Cited by 53 · see citing works
A Missing Piece in the Puzzle: Considering the Role of Task Complexity in Human-AI Decision Making
Salimzadeh, S., He, G., & Gadiraju, U. (2023) · Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3565472.3592959 · Cited by 42 · see citing works
Chiang, C.-W., & Yin, M. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511121 · Cited by 42 · see citing works
Self-citationvan Berkel, N., Sarsenbayeva, Z., & Gonçalves, J. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102954 · Cited by 41 · see citing works
Rezaeian, O., Bayrak, A. E., & Asan, O. (2026) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2539458 · Cited by 39 · see citing works
Rezaeian, O., Bayrak, A. E., & Asan, O. (2025) · arXiv · doi:10.48550/arxiv.2501.16693 · Cited by 39 · see citing works
Chen, C., & Sundar, S. S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580805 · Cited by 38 · see citing works
Li, J., Yang, Y., Liao, Q. V., Zhang, J., & Lee, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713336 · Cited by 40 · see citing works
Visser, R. W., Peters, T. M., Scharlau, I., & Hammer, B. (2025) · Cognitive Systems Research · doi:10.1016/j.cogsys.2025.101357 · Cited by 47 · see citing works
de Jong, S., Paananen, V., Tag, B., & van Berkel, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710946 · Cited by 32 · see citing works
Supporting High-Uncertainty Decisions through AI and Logic-Style Explanations
Cau, F. M., Hauptmann, H., Spano, L. D., & Tintarev, N. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584080 · Cited by 31 · see citing works
Göndöcs, D., Horváth, S., & Dörfler, V. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103622 · Cited by 32 · see citing works
Dorsch, J., & Deroy, O. D. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00752-w · Cited by 26 · see citing works
He, G., Demartini, G., & Gadiraju, U. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713218 · Cited by 23 · see citing works
How Do We Assess the Trustworthiness of AI? Introducing the Trustworthiness Assessment Model (TrAM)
Schlicker, N., Baum, K., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2022) · doi:10.31234/osf.io/qhwvx · Cited by 79 · see citing works
Modeling Human Trust and Reliance in AI-Assisted Decision Making: A Markovian Approach
Li, Z., Lu, Z., & Yin, M. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i5.25748 · Cited by 20 · see citing works
Self-citationShen, Y., Tang, L., Le, H., Tan, S., Zhao, Y., Shen, K., Li, X., Juelich, T., Wang, Q., Gašević, D., et al. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13562 · Cited by 26 · see citing works
From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysis
Li, Z., Zhu, H., Lu, Z., Xiao, Z., & Yin, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713133 · Cited by 18 · see citing works
Self-citationDo Expressions Change Decisions? Exploring the Impact of AI's Explanation Tone on Decision-Making
Okoso, A., Yang, M., & Baba, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713744 · Cited by 19 · see citing works
Towards Hormone Health: An Autoethnography of Long-Term Holistic Tracking to Manage PCOS
Kang, D., Li, J., Leshed, G., Rzeszotarski, J. M., & Lu, X. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713619 · Cited by 18 · see citing works
Explainability for Transparent Conversational Information-Seeking
Łajewska, W., Spina, D., Trippas, J. R., & Balog, K. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657768 · Cited by 18 · see citing works
To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems
He, G., Bharos, A., & Gadiraju, U. (2024) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3648188.3675130 · Cited by 17 · see citing works
Impact of Tone-Aware Explanations in Recommender Systems
Okoso, A., Otaki, K., Koide, S., & Baba, Y. (2025) · ACM Transactions on Recommender Systems · doi:10.1145/3718101 · Cited by 15 · see citing works
Figueiredo, M. C., Ankrah, E. A., Powell, J. E., Epstein, D. A., & Chen, Y. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3631414 · Cited by 18 · see citing works
Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process
Zhang, Z. T., Feger, S. S., Dullenkopf, L., Liao, R., Süsslin, L. E., Liu, Y., & Butz, A. M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687024 · Cited by 16 · see citing works
A Diachronic Perspective on User Trust in AI under Uncertainty
Dhuliawala, S., Zouhar, V., El‐Assady, M., & Sachan, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.339 · Cited by 14 · see citing works
The Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Features
Goyal, N., Baumler, C., Nguyen, T., & Daumé, H. (2023) · International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645210 · Cited by 19 · see citing works
A Lightweight Method for Modeling Confidence in Recommendations with Learned Beta Distributions
Knyazev, N., & Oosterhuis, H. (2023) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3604915.3608788 · Cited by 15 · see citing works
To Solicit or Not to Solicit? Impact of AI Assistance Delivery Mechanisms on Decision-Making
Nair, P., Gressel, G., Anand, M. N., & Achuthan, K. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2536617 · Cited by 12 · see citing works
Fair Machine Guidance to Enhance Fair Decision Making in Biased People
Yang, M., Arai, H., Yamashita, N., & Baba, Y. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642627 · Cited by 11 · see citing works
Do, H. J., Kong, H., Tetali, P., Karahalios, K. G., & Bailey, B. P. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610192 · Cited by 9 · see citing works
Thoughtful Adoption of NLP for Civic Participation: Understanding Differences Among Policymakers
Guridi, J. A., Cheyre, C., & Yang, Q. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711091 · Cited by 12 · see citing works
Visual-Conversational Interface for Evidence-Based Explanation of Diabetes Risk Prediction
Samimi, R., Bhattacharya, A., Gosak, L., Štiglic, G., & Verbert, K. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3736616 · Cited by 10 · see citing works
Strategic Adversarial Attacks in AI-assisted Decision Making to Reduce Human Trust and Reliance
Lu, Z., Li, Z., Chiang, C.-W., & Yin, M. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/337 · Cited by 9 · see citing works
Self-citationThe Influence of Curiosity Traits and On-Demand Explanations in AI-Assisted Decision-Making
Cau, F. M., & Spano, L. D. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712165 · Cited by 8 · see citing works
Raees, M., Khan, V.-J., & Papangelis, K. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3699682.3728320 · Cited by 8 · see citing works
de Jong, S., Bos, M. W., van Berkel, N., & Lamers, M. H. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2535732 · Cited by 13 · see citing works
Schömbs, S., Zhang, Y., Gonçalves, J., & Johal, W. (2025) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3765766.3765795 · Cited by 6 · see citing works
Matzen, L. E., Gastelum, Z., Howell, B. C., Divis, K., & Stites, M. C. (2024) · Cognitive Research Principles and Implications · doi:10.1186/s41235-024-00586-2 · Cited by 7 · see citing works
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5 · see citing works
Abdelaal, Y., Aupetit, M. J. M., Baggag, A., Bashir, M., & Al‐Thani, D. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2400388 · Cited by 5 · see citing works
Exploring the impact of explainable AI and cognitive capabilities on users’ decisions
Cau, F. M., & Spano, L. D. (2025) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-025-09438-0 · Cited by 9 · see citing works
The Impact of Confidence Ratings on User Trust in Large Language Models
Wang, L., Friedman, N., Zhu, C., Zhu, Z., & Mountford, S. J. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3708319.3734178 · Cited by 5 · see citing works
A novel deep learning approach for one-step conformal prediction approximation
Meister, J. A., Nguyen, K. A., Kapetanakis, S., & Luo, Z. (2023) · Annals of Mathematics and Artificial Intelligence · doi:10.1007/s10472-023-09849-y · Cited by 5 · see citing works
Accounting for Human Engagement Behavior to Enhance AI-Assisted Decision Making
Yin, M. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v3i1.31184 · Cited by 5 · see citing works
Self-citationTrust Junk and Evil Knobs: Calibrating Trust in AI Visualization
Wall, E., Matzen, L. E., El‐Assady, M., Masters, P., Hosseinpour, H., Endert, A., Borgo, R., Chau, P., Perer, A., Schupp, H. T., et al. (2024) · IEEE Pacific Visualization Conference (PacificVis) · doi:10.1109/pacificvis60374.2024.00012 · Cited by 7 · see citing works
The Role of Initial Acceptance Attitudes Toward AI Decisions in Algorithmic Recourse
Tominaga, T., Yamashita, N., & Kurashima, T. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713573 · Cited by 4 · see citing works
Using multiple linear regression for biochemical oxygen demand prediction in water
Mutai, I. K., Laerhoven, K. V., Karuri, N. W., & Tewo, R. K. (2024) · Applied Computing and Intelligence · doi:10.3934/aci.2024008 · Cited by 4 · see citing works
Zhou, Y., Chen, E., Pisipati, M., Xiong, A., & Rajtmajer, S. M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757588 · Cited by 3 · see citing works
Newen, C., Müller, E., & Newen, A. (2025) · Topoi · doi:10.1007/s11245-025-10287-0 · Cited by 3 · see citing works
Erlei, A., Cau, F. M., Georgiev, R., Kumar, S. C., Bizer, K., & Gadiraju, U. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791420 · Cited by 4 · see citing works
Łajewska, W., Balog, K., Spina, D., & Trippas, J. R. (2024) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3673791.3698409 · Cited by 3 · see citing works
A meta-heuristic approach to estimate and explain classifier uncertainty
Houston, A. D., & Cosma, G. (2025) · Applied Intelligence · doi:10.1007/s10489-024-06127-0 · Cited by 3 · see citing works
Tretter, S., Platz, A., & Diefenbach, S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581529 · Cited by 3 · see citing works
Building Human-AI Reliance Through Cognitive Engagement and Exploratory AI Assistance
Raees, M., Khan, V.-J., & Papangelis, K. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3708319.3734171 · Cited by 3 · see citing works
Judging Phishing Under Uncertainty: How Do Users Handle Inaccurate Automated Advice?
Saka, T., Vakali, K., Jenkins, A. D. G., Kökciyan, N., & Vaniea, K. E. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714267 · Cited by 2 · see citing works
Gong, Y., Liu, Y., Shang, L., Wei, N., & Wang, D. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757577 · Cited by 2 · see citing works
The Enterprising and Elusive Prospects of Human-AI Collaboration
Gadiraju, U., & Balayn, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-01940-0_7 · Cited by 2 · see citing works
Kaufman, Z., Endres, M., Xiong, C., & Brun, Y. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3634245 · Cited by 3 · see citing works
Rahman, J. S., Delaforce, A., Bradford, D., Li, J., Magrabi, F., Cook, D., & Brankovic, A. (2026) · JMIR Human Factors · doi:10.2196/81460 · Cited by 2 · see citing works
Wang, X., Guo, L., Xu, L., Guan, L., Ge, P., & Gao, T. (2026) · Ergonomics · doi:10.1080/00140139.2026.2621032 · Cited by 2 · see citing works
Retracted: Exploring Reinforcement Learning in Large-Scale Data Processing
Bhambu, P., Kumar, R., Sharmila, P., Patil, V. D., Khurana, S., & Vivek, V. (2023) · International Conference on Smart Generation Computing, Communication and Networking (SMART GENCON) · doi:10.1109/smartgencon60755.2023.10442194 · Cited by 3 · see citing works
RetractedAutomated Trustworthiness Oracle Generation for Machine Learning Text Classifiers
Tung, L. N., Cho, S. B., Du, X., Neelofar, N., Terragni, V., Ruberto, S., & Aleti, A. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729376 · Cited by 4 · see citing works
Toward Industrial Artificial Intelligence: A Discussion of Management, Technology, and Human
Zhang, W., & Bansal, S. (2026) · IEEE Systems Man and Cybernetics Magazine · doi:10.1109/msmc.2024.3520321 · Cited by 1 · see citing works
If It’s Safe, Don’t Ask: Decreasing Frustration through User Involvement for Risky Robot Behaviors
Leusmann, J., Schömbs, S., Diedrich, M., & Müller, F. (2026) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3757279.3785545 · Cited by 1 · see citing works
Can I Trust You? Rethinking Calibration with Controllable Confidence Ranking
Ge, W., Ma, H., & Zhang, C. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651478 · Cited by 1 · see citing works
Beliefs about accuracy shape confidence attributions to humans and artificial agents
Colombatto, C., & Fleming, S. M. (2026) · Communications Psychology · doi:10.1038/s44271-026-00445-4 · Cited by 1 · see citing works
Externalizing Vulnerability and Verbalizing Thoughts to More Adequately Evaluate Trust in AI
Lim, G., & Perrault, S. T. (2023) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3623809.3623962
Moafa, K., Antico, M., Vukovic, D., Edwards, C. R., Canty, D. J., Serra, X. C., Royse, A. G., Royse, C. F., Haji, K., Dowling, J., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-4487345/v1
Agustina, N., Kusrini, K., Utami, E., & Hidayat, T. (2024) · International Conference of Computer and Informatics Engineering (IC2IE) · doi:10.1109/ic2ie63342.2024.10748079
Rahman, J. S., Delaforce, A., Bradford, D., Li, J., Magrabi, F., Cook, D., & Brankovic, A. (2025) · doi:10.2196/preprints.81460
AI Ethics Advisor’s Guide to Trust: Why Perception Matters
Carnevale, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-05526-2_3
Behzad, T., Gurney, N., Wang, N., & Pynadath, D. V. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-13187-4_8
Waiting for GPT: Probing Interaction Patterns in High-Latency RAG Chatbots
Kothbauer, T., Probst, K., & Stöckl, A. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3799266
Mental Workload Prediction Using Physiological Signals: Balancing Performance and Interpretability
Hochgeschurz, S., Schwarz, J., & Witte, T. E. F. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790359
Xu, Y., Zhu, Y., Wang, H., Wu, Y. G., Ouyang, Y., Li, H., Zhou, W., Liu, X., Jiang, C., & Li, Q. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790835 · Cited by 2 · see citing works
Song, J., Cho, A., Lee, S., Choe, K., & Seo, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791657
Does Sycophancy Change Decisions? Effect of LLM Sycophancy on AI-Assisted Decision-Making
Li, Z., Pan, J., Liu, Q., Xi, Y., Zhou, Y., Jin, Y., Mao, R., & Chen, P. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790934 · Cited by 2 · see citing works
The Future of Self-Knowledge: A Call for Socratic AI
Dorsch, J. (2026) · Studies in brain and mind · doi:10.1007/978-3-032-16161-1_9
Popowski, L., Vasconcelos, H., Heinsen, I. J. F., Mgbahurike, C., Herbrich, R., Hancock, J. T., & Bernstein, M. S. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3788067
Offline Reinforcement Learning for Adaptive Support in AI-Assisted Decision-Making
Buçinca, Z., Swaroop, S., Paluch, A. E., Murphy, S. A., & Gajos, K. Z. (2026) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3815418
Yang, M.-Z., Kagawa, R., & Baba, Y. (2026) · Conference on Fairness, Accountability and Transparency · doi:10.1145/3805689.3812288
Linguistic Uncertainty Markers for Trust Calibration in AI-Assisted Decision-Making
Spillner, L., Rockstroh, J., Wenig, N., Zargham, N., Kiefner, L., Porzel, R., & Malaka, R. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816231
AI Chatbot Anthropomorphism and Consumer Decision-Making: A Dual-Pathway Calibration Model
Zhang, Q., & Abdullah, F. (2026) · Behavioral Sciences · doi:10.3390/bs16081269
Not All Trust Is the Same: Effects of Decision Workflow and Explanations in Human-AI Decision Making
Spillner, L., Ringe, R., Porzel, R., & Malaka, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-26717-7_15 · Cited by 1 · see citing works
Enhancing AI Use: How Complementary System Information Drives Delegation Frequency and Effectiveness
Taudien, A., Fügener, A., Gupta, A., & Ketter, W. (2026) · Information Systems Research · doi:10.1287/isre.2024.1330
Li, Z.-Y., Ma, Z.-Z., He, S.-S., Lu, Z.-R., Xiao, Z., & Yin, M. (2026) · Proceedings of the 2026 ACM Conference on Human-AI Complementarity and Alignment · doi:10.1145/3834580.3838738
Self-citationQing, H. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101381
- Authors
- Rechkemmer, A., & Yin, M.
- Affiliations
- Purdue University West Lafayette (US)
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-14
- OpenAlex
- W4224982959
- Cited by
- 97
- References
- 45
- Citations by year
- 2026: 24 · 2025: 35 · 2024: 18 · 2023: 16 · 2022: 4
- FWCI
- 10.09
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- NSF (National Science Foundation) (1850335)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- trust in machine learning, model confidence, model accuracy, performance indicators, laypeople, human-machine interaction, explainable AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
