Cited by 101Explainable Artificial Intelligence (XAI)Human-Automation Interaction and SafetyEthics and Social Impacts of AI

The DOI matches its Crossref record, with a small difference in title or year; 44 of the 45 references it cites were found in a registry or an index.

When Confidence Meets Accuracy: Exploring the Effects of Multiple Performance Indicators on Trust in Machine Learning Models

Open access

Rechkemmer, Amy, Yin, Ming·2022·CHI Conference on Human Factors in Computing Systems

DOI 10.1145/3491102.3501967

Open this paper in CoChat — free account Open PDF

Abstract

Previous research shows that laypeople’s trust in a machine learning model can be affected by both performance measurements of the model on the aggregate level and performance estimates on individual predictions. However, it is unclear how people would trust the model when multiple performance indicators are presented at the same time. We conduct an exploratory human-subject experiment to answer this question. We find that while the level of model confidence significantly affects people’s belief in model accuracy, both the model’s stated and observed accuracy generally have a larger impact on people’s willingness to follow the model’s predictions as well as their self-reported levels of trust in the model, especially after observing the model’s performance in practice. We hope the empirical evidence reported in this work could open doors to further studies to advance understanding of how people perceive, process, and react to performance-related information of machine learning.

Integrity signals

Registry

Crossref · 10.1145/3491102.3501967

Cites

44 of 44 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery.
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200 · cited 1×
“To indicate the size of an e�ect, we also followed Cumming (2013) in considering a con�dence interval’s range in relation to zero [13].”
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Untitled work
(2011) · Cited by 2,025
Predicting good probabilities with supervised learning
Niculescu-Mizil, A., & Caruana, R. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102430 · Cited by 1,884
Combining satellite imagery and machine learning to predict poverty
Jean, N., Burke, M., Xie, M., Davis, W. M., Lobell, D. B., & Ermon, S. (2016) · Science · doi:10.1126/science.aaf7894 · Cited by 1,664
Learning Traffic as Images: A Deep Convolutional Neural Network for Large-Scale Transportation Network Speed Prediction
Ma, X., Dai, Z., He, Z., Ma, J., Wang, Y., & Wang, Y. (2017) · Sensors · doi:10.3390/s17040818 · Cited by 1,417
Transforming classifier scores into accurate multiclass probability estimates
Zadrozny, B., & Elkan, C. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775151 · Cited by 1,280

Show all 45 references

Research works printed but not linked: 18

Gagan Bansal , Besmira Nushi , Ece Kamar , Walter S. Lasecki , Daniel S. Weld , and Eric Horvitz . 2019 . Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance . In Proceedings of the Seventh AAAI Conference on Human Computation and Crowdsourcing. Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S. Lasecki, Daniel S. Weld, and Eric Horvitz. 2019. Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance. In Proceedings of the Seventh AAAI Conference on Human Computation and Crowdsourcing.
James Bennett , Stan Lanning , 2007 . The netflix prize . In Proceedings of KDD cup and workshop, Vol. 2007 . Citeseer, 35. James Bennett, Stan Lanning, 2007. The netflix prize. In Proceedings of KDD cup and workshop, Vol. 2007. Citeseer, 35.
David V Budescu and Adrian K Rantilla . 2000. Confidence in aggregation of expert opinions. Acta psychologica 104, 3 ( 2000 ), 371–398. David V Budescu and Adrian K Rantilla. 2000. Confidence in aggregation of expert opinions. Acta psychologica 104, 3 (2000), 371–398.
Aylin Caliskan , Joanna J Bryson , and Arvind Narayanan . 2017. Semantics derived automatically from language corpora contain human-like biases. Science 356, 6334 ( 2017 ), 183–186. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science 356, 6334 (2017), 183–186.
Berkeley J Dietvorst , Joseph P Simmons , and Cade Massey . 2015 . Algorithm aversion: People erroneously avoid algorithms after seeing them err.Journal of Experimental Psychology : General 144 , 1 (2015), 114 . Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015. Algorithm aversion: People erroneously avoid algorithms after seeing them err.Journal of Experimental Psychology: General 144, 1 (2015), 114.
Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608(2017). Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608(2017). · arXiv
Yuan Gao , Sanmay Das , and Patrick Fowler . 2017 . Homelessness service provision: a data science perspective . In Workshops at the Thirty-First AAAI Conference on Artificial Intelligence. Yuan Gao, Sanmay Das, and Patrick Fowler. 2017. Homelessness service provision: a data science perspective. In Workshops at the Thirty-First AAAI Conference on Artificial Intelligence.
Inga Großmann , André Hottung , and Artus Krohn-Grimberghe . 2019. Machine learning meets partner matching: Predicting the future relationship quality based on personality traits. PloS one 14, 3 ( 2019 ). Inga Großmann, André Hottung, and Artus Krohn-Grimberghe. 2019. Machine learning meets partner matching: Predicting the future relationship quality based on personality traits. PloS one 14, 3 (2019).
10.5555/3305381.3305518 · DOI
Samantha Joel , Paul W Eastwick , and Eli J Finkel . 2017. Is romantic desire predictable? Machine learning applied to initial romantic attraction. Psychological science 28, 10 ( 2017 ), 1478–1489. Samantha Joel, Paul W Eastwick, and Eli J Finkel. 2017. Is romantic desire predictable? Machine learning applied to initial romantic attraction. Psychological science 28, 10 (2017), 1478–1489.
Volodymyr Kuleshov and Percy S Liang. 2015. Calibrated structured prediction. In Advances in Neural Information Processing Systems. 3474–3482. Volodymyr Kuleshov and Percy S Liang. 2015. Calibrated structured prediction. In Advances in Neural Information Processing Systems. 3474–3482.
10.1145/3233231 · DOI · Paper page
Ellen Peters , Daniel Västfjäll , Paul Slovic , CK Mertz , Ketti Mazzocco , and Stephan Dickert . 2006. Numeracy and decision making. Psychological science 17, 5 ( 2006 ), 407–413. Ellen Peters, Daniel Västfjäll, Paul Slovic, CK Mertz, Ketti Mazzocco, and Stephan Dickert. 2006. Numeracy and decision making. Psychological science 17, 5 (2006), 407–413.
John Platt 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10, 3 ( 1999 ), 61–74. John Platt 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10, 3 (1999), 61–74.
Valerie F Reyna and Charles J Brainerd . 2008. Numeracy , ratio bias, and denominator neglect in judgments of risk and probability. Learning and individual differences 18, 1 ( 2008 ), 89–107. Valerie F Reyna and Charles J Brainerd. 2008. Numeracy, ratio bias, and denominator neglect in judgments of risk and probability. Learning and individual differences 18, 1 (2008), 89–107.
Aaron Springer and Steve Whittaker . 2018 . Progressive Disclosure: Designing for Effective Transparency. arXiv preprint arXiv:1811.02164(2018). Aaron Springer and Steve Whittaker. 2018. Progressive Disclosure: Designing for Effective Transparency. arXiv preprint arXiv:1811.02164(2018). · arXiv
Harini Suresh , Natalie Lao , and Ilaria Liccardi . 2020 . Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making. In 12th ACM Conference on Web Science. 315–324 . Harini Suresh, Natalie Lao, and Ilaria Liccardi. 2020. Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making. In 12th ACM Conference on Web Science. 315–324.
Brandon M Turner , Mark Steyvers , Edgar C Merkle , David V Budescu , and Thomas S Wallsten . 2014. Forecast aggregation via recalibration. Machine learning 95, 3 ( 2014 ), 261–289. Brandon M Turner, Mark Steyvers, Edgar C Merkle, David V Budescu, and Thomas S Wallsten. 2014. Forecast aggregation via recalibration. Machine learning 95, 3 (2014), 261–289.

Sorted by citations

Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 220
Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions
Wischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146
Human-AI collaboration is not very collaborative yet: a taxonomy of interaction patterns in AI-assisted decision making from a systematic review
Gómez, C., Cho, S. M., Ke, S., Huang, C., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 207
Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems
He, G., Kuiper, L., & Gadiraju, U. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581025 · Cited by 107
“They found that factors like first impression [76], AI literacy [8], risk perception [33, 34], and performance feedback [55, 61] among others, play important roles in shaping human trust and reliance on AI systems.”
Background
Towards Risk‐Free Trustworthy Artificial Intelligence: Significance and Requirements
Alzubaidi, L., Al-Sabaawi, A., Bai, J., Dukhan, A., Alkenani, A. H., Al‐Asadi, A., Alwzwazy, H. A., Manoufali, M., Fadhel, M. A., Albahri, A. S., et al. (2023) · International Journal of Intelligent Systems · doi:10.1155/2023/4459198 · Cited by 116
Enhancing AI-Assisted Group Decision Making through LLM-Powered Devil's Advocate
Chiang, C.-W., Lu, Z., Li, Z., & Yin, M. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645199 · Cited by 140
Self-citation
How do we assess the trustworthiness of AI? Introducing the trustworthiness assessment model (TrAM)
Schlicker, N., Baum, K. L., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108671 · Cited by 106
Harnessing Biomedical Literature to Calibrate Clinicians’ Trust in AI Decision Support Systems
Yang, Q., Hao, Y., Quan, K., Yang, S., Zhao, Y., Kuleshov, V., & Wang, F. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581393 · Cited by 92
Trust, distrust, and appropriate reliance in (X)AI: A conceptual clarification of user trust and survey of its empirical evaluation
Visser, R., Peters, T. M., Scharlau, I., & Hammer, B. (2025) · Cognitive Systems Research · doi:10.1016/j.cogsys.2025.101357 · Cited by 58
“It's not like Jarvis, but it's pretty close!” - Examining ChatGPT's Usage among Undergraduate Students in Computer Science
Budhiraja, R., Joshi, I., Challa, J. S., Akolekar, H. D., & Kumar, D. (2024) · Proceedings of the 26th Australasian Computing Education Conference · doi:10.1145/3636243.3636257 · Cited by 52
Will You Accept the AI Recommendation? Predicting Human Behavior in AI-Assisted Decision Making
Wang, X., Lu, Z., & Yin, M. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512240 · Cited by 63
Self-citation
Understanding the Evolvement of Trust Over Time within Human-AI Teams
Duan, W., Zhou, S., Scalia, M. J., Yin, X., Weng, N., Zhang, R., Freeman, G., McNeese, N. J., Gorman, J. C., & Tolston, M. T. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687060 · Cited by 62
Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant
He, G., Aishwarya, N., & Gadiraju, U. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712133 · Cited by 87
A Missing Piece in the Puzzle: Considering the Role of Task Complexity in Human-AI Decision Making
Salimzadeh, S., He, G., & Gadiraju, U. (2023) · Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3565472.3592959 · Cited by 42
Exploring the Effects of Machine Learning Literacy Interventions on Laypeople’s Reliance on Machine Learning Models
Chiang, C.-W., & Yin, M. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511121 · Cited by 42
Self-citation
The methodology of studying fairness perceptions in Artificial Intelligence: Contrasting CHI and FAccT
van Berkel, N., Sarsenbayeva, Z., & Gonçalves, J. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102954 · Cited by 48
Explainability and AI Confidence in Clinical Decision Support Systems: Effects on Trust, Diagnostic Performance, and Cognitive Load in Breast Cancer Care
Rezaeian, O., Bayrak, A. E., & Asan, O. (2026) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2539458 · Cited by 39
Explainability and AI confidence in clinical decision support systems: effects on trust, diagnostic performance, and cognitive load in breast Cancer care
Rezaeian, O., Bayrak, A. E., & Asan, O. (2025) · arXiv · doi:10.48550/arxiv.2501.16693 · Cited by 39
Is this AI trained on Credible Data? The Effects of Labeling Quality and Performance Bias on User Trust
Chen, C., & Sundar, S. S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580805 · Cited by 41
As Confidence Aligns: Understanding the Effect of AI Confidence on Human Self-confidence in Human-AI Decision Making
Li, J., Yang, Y., Liao, Q. V., Zhang, J., & Lee, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713336 · Cited by 63
Cognitive Forcing for Better Decision-Making: Reducing Overreliance on AI Systems Through Partial Explanations
de Jong, S., Paananen, V., Tag, B., & van Berkel, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710946 · Cited by 32
Supporting High-Uncertainty Decisions through AI and Logic-Style Explanations
Cau, F. M., Hauptmann, H., Spano, L. D., & Tintarev, N. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584080 · Cited by 35
Uncovering the dynamics of human-AI hybrid performance: A qualitative meta-analysis of empirical studies
Göndöcs, D., Horváth, S., & Dörfler, V. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103622 · Cited by 39
“Quasi-Metacognitive Machines: Why We Don’t Need Morally Trustworthy AI and Communicating Reliability is Enough”
Dorsch, J., & Deroy, O. D. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00752-w · Cited by 30
Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant
He, G., Demartini, G., & Gadiraju, U. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713218 · Cited by 23
How Do We Assess the Trustworthiness of AI? Introducing the Trustworthiness Assessment Model (TrAM)
Schlicker, N., Baum, K., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2022) · doi:10.31234/osf.io/qhwvx · Cited by 79
Modeling Human Trust and Reliance in AI-Assisted Decision Making: A Markovian Approach
Li, Z., Lu, Z., & Yin, M. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i5.25748 · Cited by 25
Self-citation
Aligning and comparing values of ChatGPT and human as learning facilitators: A value‐sensitive design approach
Shen, Y., Tang, L., Le, H., Tan, S., Zhao, Y., Shen, K., Li, X., Juelich, T., Wang, Q., Gašević, D., et al. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13562 · Cited by 27
From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysis
Li, Z., Zhu, H., Lu, Z., Xiao, Z., & Yin, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713133 · Cited by 25
Self-citation
Do Expressions Change Decisions? Exploring the Impact of AI's Explanation Tone on Decision-Making
Okoso, A., Yang, M., & Baba, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713744 · Cited by 26
Towards Hormone Health: An Autoethnography of Long-Term Holistic Tracking to Manage PCOS
Kang, D., Li, J., Leshed, G., Rzeszotarski, J. M., & Lu, X. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713619 · Cited by 20
Explainability for Transparent Conversational Information-Seeking
Łajewska, W., Spina, D., Trippas, J. R., & Balog, K. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657768 · Cited by 20
To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems
He, G., Bharos, A., & Gadiraju, U. (2024) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3648188.3675130 · Cited by 21
Impact of Tone-Aware Explanations in Recommender Systems
Okoso, A., Otaki, K., Koide, S., & Baba, Y. (2025) · ACM Transactions on Recommender Systems · doi:10.1145/3718101 · Cited by 17
Powered by AI
Figueiredo, M. C., Ankrah, E. A., Powell, J. E., Epstein, D. A., & Chen, Y. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3631414 · Cited by 24
Beyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process
Zhang, Z. T., Feger, S. S., Dullenkopf, L., Liao, R., Süsslin, L. E., Liu, Y., & Butz, A. M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687024 · Cited by 22
A Diachronic Perspective on User Trust in AI under Uncertainty
Dhuliawala, S., Zouhar, V., El‐Assady, M., & Sachan, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.339 · Cited by 29
The Impact of Explanations on Fairness in Human-AI Decision-Making: Protected vs Proxy Features
Goyal, N., Baumler, C., Nguyen, T. T., & Daumé, H. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645210 · Cited by 20
A Lightweight Method for Modeling Confidence in Recommendations with Learned Beta Distributions
Knyazev, N., & Oosterhuis, H. (2023) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3604915.3608788 · Cited by 15
To Solicit or Not to Solicit? Impact of AI Assistance Delivery Mechanisms on Decision-Making
Nair, P., Gressel, G., Anand, M. N., & Achuthan, K. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2536617 · Cited by 15
Fair Machine Guidance to Enhance Fair Decision Making in Biased People
Yang, M., Arai, H., Yamashita, N., & Baba, Y. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642627 · Cited by 16
Inform, Explain, or Control: Techniques to Adjust End-User Performance Expectations for a Conversational Agent Facilitating Group Chat Discussions
Do, H. J., Kong, H., Tetali, P., Karahalios, K. G., & Bailey, B. P. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610192 · Cited by 9
Thoughtful Adoption of NLP for Civic Participation: Understanding Differences Among Policymakers
Guridi, J. A., Cheyre, C., & Yang, Q. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711091 · Cited by 12
Visual-Conversational Interface for Evidence-Based Explanation of Diabetes Risk Prediction
Samimi, R., Bhattacharya, A., Gosak, L., Štiglic, G., & Verbert, K. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3736616 · Cited by 14
Strategic Adversarial Attacks in AI-assisted Decision Making to Reduce Human Trust and Reliance
Lu, Z., Li, Z., Chiang, C.-W., & Yin, M. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/337 · Cited by 9
Self-citation
The Influence of Curiosity Traits and On-Demand Explanations in AI-Assisted Decision-Making
Cau, F. M., & Spano, L. D. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712165 · Cited by 13
Exploring Persuasive Engagement to Reduce Over-Reliance on AI-Assistance in a Customer Classification Case
Raees, M., Khan, V.-J., & Papangelis, K. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3699682.3728320 · Cited by 8
Algorithm appreciation or aversion: the effects of accuracy disclosure on users' reliance on algorithmic suggestions
de Jong, S., Bos, M. W., van Berkel, N., & Lamers, M. H. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2535732 · Cited by 19
From Conversation to Orchestration: HCI Challenges and Opportunities in Interactive Multi-Agentic Systems
Schömbs, S., Zhang, Y., Gonçalves, J., & Johal, W. (2025) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3765766.3765795 · Cited by 6
Effects of machine learning errors on human decision-making: manipulations of model accuracy, error types, and error importance
Matzen, L. E., Gastelum, Z., Howell, B. C., Divis, K., & Stites, M. C. (2024) · Cognitive Research Principles and Implications · doi:10.1186/s41235-024-00586-2 · Cited by 7
Do people appropriately rely on AI-advice? An analytical review of HCI research on human-AI decision-making.
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5
How Much Wearable Data is Enough for the Utility and Trust of Augmented Artificial Intelligence Systems? A Scenario-Based Interview with Medical Professionals
Abdelaal, Y., Aupetit, M. J. M., Baggag, A., Bashir, M., & Al‐Thani, D. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2400388 · Cited by 6
Exploring the impact of explainable AI and cognitive capabilities on users’ decisions
Cau, F. M., & Spano, L. D. (2025) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-025-09438-0 · Cited by 16
The Impact of Confidence Ratings on User Trust in Large Language Models
Wang, L., Friedman, N., Zhu, C., Zhu, Z., & Mountford, S. J. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3708319.3734178 · Cited by 5
A novel deep learning approach for one-step conformal prediction approximation
Meister, J. A., Nguyen, K. A., Kapetanakis, S., & Luo, Z. (2023) · Annals of Mathematics and Artificial Intelligence · doi:10.1007/s10472-023-09849-y · Cited by 5
Accounting for Human Engagement Behavior to Enhance AI-Assisted Decision Making
Yin, M. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v3i1.31184 · Cited by 12
Self-citation
Trust Junk and Evil Knobs: Calibrating Trust in AI Visualization
Wall, E., Matzen, L. E., El‐Assady, M., Masters, P., Hosseinpour, H., Endert, A., Borgo, R., Chau, P., Perer, A., Schupp, H. T., et al. (2024) · IEEE Pacific Visualization Conference (PacificVis) · doi:10.1109/pacificvis60374.2024.00012 · Cited by 9
The Role of Initial Acceptance Attitudes Toward AI Decisions in Algorithmic Recourse
Tominaga, T., Yamashita, N., & Kurashima, T. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713573 · Cited by 4
Using multiple linear regression for biochemical oxygen demand prediction in water
Mutai, I. K., Laerhoven, K. V., Karuri, N. W., & Tewo, R. K. (2024) · Applied Computing and Intelligence · doi:10.3934/aci.2024008 · Cited by 4
Effect of AI Performance, Risk Perception, and Trust on Human Dependence in Deepfake Detection AI System
Zhou, Y., Chen, E., Pisipati, M., Xiong, A., & Rajtmajer, S. M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757588 · Cited by 4
Trust and Uncertainties: Characterizing Trustworthy AI Systems Within a Multidimensional Theory of Trust
Newen, C., Müller, E., & Newen, A. (2025) · Topoi · doi:10.1007/s11245-025-10287-0 · Cited by 6
When Life Gives You AI, Will You Turn It Into A Market for Lemons? Understanding How Information Asymmetries About AI System Capabilities Affect Market Outcomes and Adoption
Erlei, A., Cau, F. M., Georgiev, R., Kumar, S. C., Bizer, K., & Gadiraju, U. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791420 · Cited by 7
Can Users Detect Biases or Factual Errors in Generated Responses in Conversational Information-Seeking?
Łajewska, W., Balog, K., Spina, D., & Trippas, J. R. (2024) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3673791.3698409 · Cited by 5
A meta-heuristic approach to estimate and explain classifier uncertainty
Houston, A. D., & Cosma, G. (2025) · Applied Intelligence · doi:10.1007/s10489-024-06127-0 · Cited by 3
Matching Mind and Method
Tretter, S., Platz, A., & Diefenbach, S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581529 · Cited by 3
Building Human-AI Reliance Through Cognitive Engagement and Exploratory AI Assistance
Raees, M., Khan, V.-J., & Papangelis, K. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3708319.3734171 · Cited by 5
Judging Phishing Under Uncertainty: How Do Users Handle Inaccurate Automated Advice?
Saka, T., Vakali, K., Jenkins, A. D. G., Kökciyan, N., & Vaniea, K. E. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714267 · Cited by 2
Designing Effective AI Explanations for Misinformation Detection: A Comparative Study of Content, Social, and Combined Explanations
Gong, Y., Liu, Y., Shang, L., Wei, N., & Wang, D. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757577 · Cited by 4
The Enterprising and Elusive Prospects of Human-AI Collaboration
Gadiraju, U., & Balayn, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-01940-0_7 · Cited by 5
Your Model Is Unfair, Are You Even Aware? Inverse Relationship Between Comprehension and Trust in Explainability Visualizations of Biased ML Models
Kaufman, Z., Endres, M., Xiong, C., & Brun, Y. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3634245 · Cited by 3
The Role of Explanations in AI-Generated Alerts: Qualitative Study of Clinical Views on Explainable AI in Predictive Tools
Rahman, J. S., Delaforce, A., Bradford, D., Li, J., Magrabi, F., Cook, D., & Brankovic, A. (2026) · JMIR Human Factors · doi:10.2196/81460 · Cited by 2
Human-likeness perceptions in automated driving systems: exploring post-usage trust and continuance intention through TAM and automated social presence perspectives
Wang, X., Guo, L., Xu, L., Guan, L., Ge, P., & Gao, T. (2026) · Ergonomics · doi:10.1080/00140139.2026.2621032 · Cited by 2
Retracted: Exploring Reinforcement Learning in Large-Scale Data Processing
Bhambu, P., Kumar, R., Sharmila, P., Patil, V. D., Khurana, S., & Vivek, V. (2023) · International Conference on Smart Generation Computing, Communication and Networking (SMART GENCON) · doi:10.1109/smartgencon60755.2023.10442194 · Cited by 3
Retracted
Automated Trustworthiness Oracle Generation for Machine Learning Text Classifiers
Tung, L. N., Cho, S. B., Du, X., Neelofar, N., Terragni, V., Ruberto, S., & Aleti, A. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729376 · Cited by 6
Toward Industrial Artificial Intelligence: A Discussion of Management, Technology, and Human
Zhang, W., & Bansal, S. (2026) · IEEE Systems Man and Cybernetics Magazine · doi:10.1109/msmc.2024.3520321 · Cited by 1
If It’s Safe, Don’t Ask: Decreasing Frustration through User Involvement for Risky Robot Behaviors
Leusmann, J., Schömbs, S., Diedrich, M., & Müller, F. (2026) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3757279.3785545 · Cited by 1
Can I Trust You? Rethinking Calibration with Controllable Confidence Ranking
Ge, W., Ma, H., & Zhang, C. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651478 · Cited by 1
Beliefs about accuracy shape confidence attributions to humans and artificial agents
Colombatto, C., & Fleming, S. M. (2026) · Communications Psychology · doi:10.1038/s44271-026-00445-4 · Cited by 1
Externalizing Vulnerability and Verbalizing Thoughts to More Adequately Evaluate Trust in AI
Lim, G., & Perrault, S. T. (2023) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3623809.3623962
Convolutional Automatic Identification of B-lines and Interstitial Syndrome in Lung Ultrasound Images Using Pre-Trained Neural Networks with Feature FusionPreprint
Moafa, K., Antico, M., Vukovic, D., Edwards, C. R., Canty, D. J., Serra, X. C., Royse, A. G., Royse, C. F., Haji, K., Dowling, J., et al. (2024) · Research Square · doi:10.21203/rs.3.rs-4487345/v1
Systematic Literature Review in the Development of Datasets and Fact Verification Models for Indonesian Language
Agustina, N., Kusrini, K., Utami, E., & Hidayat, T. (2024) · International Conference of Computer and Informatics Engineering (IC2IE) · doi:10.1109/ic2ie63342.2024.10748079 · Cited by 2
When Explanations Differ: A Qualitative Study of Clinical Views on Explainable AI (XAI) Methods in Healthcare (Preprint)Preprint
Rahman, J. S., Delaforce, A., Bradford, D., Li, J., Magrabi, F., Cook, D., & Brankovic, A. (2025) · doi:10.2196/preprints.81460
AI Ethics Advisor’s Guide to Trust: Why Perception Matters
Carnevale, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-05526-2_3
Beyond Predictions: A Study of AI Strength and Weakness Transparency Communication on Human-AI Collaboration
Behzad, T., Gurney, N., Wang, N., & Pynadath, D. V. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-13187-4_8
Understanding Human-AI Trust in the Context of Decision Making through the Lenses of Academia and Industry : Definitions, Factors, and Evaluation
Vereschak, O. (2022) · theses.fr (ABES) · doi:10.70675/7ecd53bfza7ccz4edfz8f1azdb8052ba1233
Waiting for GPT: Probing Interaction Patterns in High-Latency RAG Chatbots
Kothbauer, T., Probst, K., & Stöckl, A. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3799266 · Cited by 1
Mental Workload Prediction Using Physiological Signals: Balancing Performance and Interpretability
Hochgeschurz, S., Schwarz, J., & Witte, T. E. F. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790359
“Do I Trust the AI?” Towards Trustworthy AI-Assisted Diagnosis: Understanding User Perception in LLM-Supported Clinical Reasoning
Xu, Y., Zhu, Y., Wang, H., Wu, Y. G., Ouyang, Y., Li, H., Zhou, W., Liu, X., Jiang, C., & Li, Q. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790835 · Cited by 4
Good Fences Make Good Learning: How Self-Directed Language Learners Navigate LLM Delegation Decisions
Song, J., Cho, A., Lee, S., Choe, K., & Seo, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791657 · Cited by 3
Does Sycophancy Change Decisions? Effect of LLM Sycophancy on AI-Assisted Decision-Making
Li, Z., Pan, J., Liu, Q., Xi, Y., Zhou, Y., Jin, Y., Mao, R., & Chen, P. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790934 · Cited by 5
The Future of Self-Knowledge: A Call for Socratic AI
Dorsch, J. (2026) · Studies in brain and mind · doi:10.1007/978-3-032-16161-1_9
People Can Accurately Predict Behavior of Complex Algorithms That Are Available, Compact, and Aligned CSCW031
Popowski, L., Vasconcelos, H., Heinsen, I. J. F., Mgbahurike, C., Herbrich, R., Hancock, J. T., & Bernstein, M. S. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3788067 · Cited by 1
Offline Reinforcement Learning for Adaptive Support in AI-Assisted Decision-Making
Buçinca, Z., Swaroop, S., Paluch, A. E., Murphy, S. A., & Gajos, K. Z. (2026) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3815418 · Cited by 48
Do User-Aligned Explanations Steer Human Decisions? Context-Dependent Influence and Ethical Implications
Yang, M.-Z., Kagawa, R., & Baba, Y. (2026) · Conference on Fairness, Accountability and Transparency · doi:10.1145/3805689.3812288
Linguistic Uncertainty Markers for Trust Calibration in AI-Assisted Decision-Making
Spillner, L., Rockstroh, J., Wenig, N., Zargham, N., Kiefner, L., Porzel, R., & Malaka, R. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816231 · Cited by 1
AI Chatbot Anthropomorphism and Consumer Decision-Making: A Dual-Pathway Calibration Model
Zhang, Q., & Abdullah, F. (2026) · Behavioral Sciences · doi:10.3390/bs16081269
Not All Trust Is the Same: Effects of Decision Workflow and Explanations in Human-AI Decision Making
Spillner, L., Ringe, R., Porzel, R., & Malaka, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-26717-7_15 · Cited by 3
Enhancing AI Use: How Complementary System Information Drives Delegation Frequency and Effectiveness
Taudien, A., Fügener, A., Gupta, A., & Ketter, W. (2026) · Information Systems Research · doi:10.1287/isre.2024.1330
Adaptive Selection of Effective AI Assistance in AI-assisted Decision Making Using Multi-Armed Bandits
Li, Z., Ma, Z., He, S., Lu, Z., Xiao, Z., & Yin, M. (2026) · ACM Conference on Human-AI Complementarity and Alignment · doi:10.1145/3834580.3838738
Self-citation
How trust shapes human reliance on AI decision support in healthcare: A simulation-based behavioral study
Qing, H. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101381

Authors: Rechkemmer, A., & Yin, M.
Affiliations: Purdue University West Lafayette (US)
ORCID: Rechkemmer, Amy
Yin, Ming
Venue: CHI Conference on Human Factors in Computing Systems
Publisher: ACM
Published: 2022-04-29
Pages: 1-14
DOI: 10.1145/3491102.3501967
OpenAlex: W4224982959
Cited by: 97
References: 45
Citations by year: 2026: 24 · 2025: 35 · 2024: 18 · 2023: 16 · 2022: 4
FWCI: 10.09
Type: conference paper
Language: en
Open access: bronze
doi.org
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: CHI Conference on Human Factors in Computing Systems (publishedVersion)
Funders: NSF (National Science Foundation) (1850335)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Keywords: trust in machine learning, model confidence, model accuracy, performance indicators, laypeople, human-machine interaction, explainable AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

When Confidence Meets Accuracy: Exploring the Effects of Multiple Performance Indicators on Trust in Machine Learning Models

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks