The DOI matches its Crossref record, with a small difference in title or year; 44 of the 45 references it cites were found in a registry or an index.
When Confidence Meets Accuracy: Exploring the Effects of Multiple Performance Indicators on Trust in Machine Learning Models
Open accessAbstract
Previous research shows that laypeople’s trust in a machine learning model can be affected by both performance measurements of the model on the aggregate level and performance estimates on individual predictions. However, it is unclear how people would trust the model when multiple performance indicators are presented at the same time. We conduct an exploratory human-subject experiment to answer this question. We find that while the level of model confidence significantly affects people’s belief in model accuracy, both the model’s stated and observed accuracy generally have a larger impact on people’s willingness to follow the model’s predictions as well as their self-reported levels of trust in the model, especially after observing the model’s performance in practice. We hope the empirical evidence reported in this work could open doors to further studies to advance understanding of how people perceive, process, and react to performance-related information of machine learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Poursabzi-Sangdeh et al. [108] found that providing users with more information (through increased interpretability and transparency) decreased the user's capabilities to detect and correct mistakes by the system.”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [108]
In the citing paper
“This is achieved by, for example, presenting information about a system's confdence throughout the interaction [90, 104, 145] using cues or warning signals or by increasing a system's transparency through explanations [108].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [108]
Citations
Cites
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200 · cited 1×
“To indicate the size of an e�ect, we also followed Cumming (2013) in considering a con�dence interval’s range in relation to zero [13].”
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
(2011) · Cited by 2,025
Predicting good probabilities with supervised learning
Niculescu-Mizil, A., & Caruana, R. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102430 · Cited by 1,884
Combining satellite imagery and machine learning to predict poverty
Jean, N., Burke, M., Xie, M., Davis, W. M., Lobell, D. B., & Ermon, S. (2016) · Science · doi:10.1126/science.aaf7894 · Cited by 1,664
Ma, X., Dai, Z., He, Z., Ma, J., Wang, Y., & Wang, Y. (2017) · Sensors · doi:10.3390/s17040818 · Cited by 1,417
Transforming classifier scores into accurate multiclass probability estimates
Zadrozny, B., & Elkan, C. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775151 · Cited by 1,280
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · cited 1×
“Many different factors have been identified as influencing people’s trust in ML, such as people’s understanding of how the model works [19, 37] and people’s perceptions on whether the model is biased [9, 62].”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728
How Much Information?: Effects of Transparency on Trust in an Algorithmic Interface
Kizilcec, R. F. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858402 · Cited by 615
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 6×
“[58] ran a sequence of human-subject experiments and found that both an ML model’s stated accuracy on held-out data and its observed accuracy in practice significantly affect laypeople’s trust in the model.”
BackgroundMethodResultInfluentialSelf-citationGender Differences in Mate Selection: Evidence From a Speed Dating Experiment
Fisman, R., Iyengar, S. S., Kamenica, E., & Simonson, I. (2006) · The Quarterly Journal of Economics · doi:10.1162/qjec.2006.121.2.673 · Cited by 522
A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk
Hara, K., Adams, A., Milland, K., Savage, S., Callison-Burch, C., & Bigham, J. P. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174023 · Cited by 497
Lai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377
Cho, J.-H., Chan, K., & Adalı, S. (2015) · ACM Computing Surveys · doi:10.1145/2815595 · Cited by 365
Wang, X., & Yin, M. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450650 · Cited by 320
Self-citationThe relationship between trust in AI and trustworthy machine learning technologies
Toreini, E., Aitken, M., Coopamootoo, K., Elliott, K., Zelaya, C. G., & van Moorsel, A. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372834 · Cited by 296
Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders
Cheng, H.-F., Wang, R., Zhang, Z., O'Connell, F., Gray, T., Harper, F. M., & Zhu, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300789 · Cited by 292 · cited 1×
“was used in [31, 50], agreement or switch fraction was used in [35, 58, 61], and subject’s self-reported level of trust in the model was used in [10, 58].”
MethodUpdates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
Bansal, G., Nushi, B., Kamar, E., Weld, D. S., Lasecki, W. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33012429 · Cited by 284
Kocielnik, R., Amershi, S., & Bennett, P. N. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300641 · Cited by 277
Kay, M., Kola, T., Hullman, J. R., & Munson, S. A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858558 · Cited by 239
Neural network model for rapid forecasting of freeway link travel time
Dharia, A., & Adeli, H. (2003) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2003.09.011 · Cited by 233
How do visual explanations foster end users' appropriate trust in machine learning?
Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377480 · Cited by 200
Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making
Fernandes, M., Walls, L., Munson, S., Hullman, J., & Kay, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173718 · Cited by 175
Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems
Kunkel, J., Donkers, T., Michael, L., Barbu, C.-M., & Ziegler, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300717 · Cited by 156
Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
Lu, Z., & Yin, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445562 · Cited by 131
Self-citationI can do better than your AI: expertise and explanations
Schaffer, J., O'Donovan, J., Michaelis, J., Raglin, A., & Höllerer, T. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302308 · Cited by 119
Survey on Computational Trust and Reputation Models
Braga, D. D. S., Niemann, M., Hellingrath, B., & Neto, F. B. D. L. (2018) · ACM Computing Surveys · doi:10.1145/3236008 · Cited by 116
Towards improving trust in context-aware systems by displaying system confidence
Antifakos, S., Kern, N., Schiele, B., & Schwaninger, A. (2005) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/1085777.1085780 · Cited by 103
Investigating intelligibility for uncertain context-aware applications
Lim, B. Y., & Dey, A. K. (2011) · Proceedings of the 13th international conference on Ubiquitous computing · doi:10.1145/2030112.2030168 · Cited by 101
User Trust Dynamics: An Investigation Driven by Differences in System Performance
Yu, K., Berkovsky, S., Taib, R., Conway, D., Zhou, J., & Chen, F. (2017) · Proceedings of the 22nd International Conference on Intelligent User Interfaces · doi:10.1145/3025171.3025219 · Cited by 90
Pinsker, R. E. (2011) · Behavioral Research in Accounting · doi:10.2308/bria.2011.23.1.161 · Cited by 78
Allocating Interventions Based on Predicted Outcomes: A Case Study on Homelessness Services
Kube, A., Das, S., & Fowler, P. J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301622 · Cited by 58
You’d Better Stop! Understanding Human Reliance on Machine Learning Models under Covariate Shift
Chiang, C.-W., & Yin, M. (2021) · 13th ACM Web Science Conference 2021 · doi:10.1145/3447535.3462487 · Cited by 53
Self-citationEvaluating the Effects of Displaying Uncertainty in Context-Aware Applications
Antifakos, S., Schwaninger, A., & Schiele, B. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30119-6_4 · Cited by 50
An Exploration of Location Error Estimation
Dearman, D., Varshavsky, A., de Lara, E., & Truong, K. N. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74853-3_11 · Cited by 28
RUSSELL, W. L. (1940) · Journal of Heredity · doi:10.1093/oxfordjournals.jhered.a104798 · Cited by 20
Verame, J. K. M., Costanza, E., & Ramchurn, S. D. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858369 · Cited by 15
Probability Forecasts and Their Combination: A Research PerspectivePreprint
Winkler, R. L., Grushka‐Cockayne, Y., Lichtendahl, K. C., & Jose, V. R. R. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3258627 · Cited by 7
Research works printed but not linked: 18
- Gagan Bansal , Besmira Nushi , Ece Kamar , Walter S. Lasecki , Daniel S. Weld , and Eric Horvitz . 2019 . Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance . In Proceedings of the Seventh AAAI Conference on Human Computation and Crowdsourcing. Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S. Lasecki, Daniel S. Weld, and Eric Horvitz. 2019. Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance. In Proceedings of the Seventh AAAI Conference on Human Computation and Crowdsourcing.
- James Bennett , Stan Lanning , 2007 . The netflix prize . In Proceedings of KDD cup and workshop, Vol. 2007 . Citeseer, 35. James Bennett, Stan Lanning, 2007. The netflix prize. In Proceedings of KDD cup and workshop, Vol. 2007. Citeseer, 35.
- David V Budescu and Adrian K Rantilla . 2000. Confidence in aggregation of expert opinions. Acta psychologica 104, 3 ( 2000 ), 371–398. David V Budescu and Adrian K Rantilla. 2000. Confidence in aggregation of expert opinions. Acta psychologica 104, 3 (2000), 371–398.
- Aylin Caliskan , Joanna J Bryson , and Arvind Narayanan . 2017. Semantics derived automatically from language corpora contain human-like biases. Science 356, 6334 ( 2017 ), 183–186. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science 356, 6334 (2017), 183–186.
- Berkeley J Dietvorst , Joseph P Simmons , and Cade Massey . 2015 . Algorithm aversion: People erroneously avoid algorithms after seeing them err.Journal of Experimental Psychology : General 144 , 1 (2015), 114 . Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015. Algorithm aversion: People erroneously avoid algorithms after seeing them err.Journal of Experimental Psychology: General 144, 1 (2015), 114.
- Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608(2017). Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608(2017). · arXiv
- Yuan Gao , Sanmay Das , and Patrick Fowler . 2017 . Homelessness service provision: a data science perspective . In Workshops at the Thirty-First AAAI Conference on Artificial Intelligence. Yuan Gao, Sanmay Das, and Patrick Fowler. 2017. Homelessness service provision: a data science perspective. In Workshops at the Thirty-First AAAI Conference on Artificial Intelligence.
- Inga Großmann , André Hottung , and Artus Krohn-Grimberghe . 2019. Machine learning meets partner matching: Predicting the future relationship quality based on personality traits. PloS one 14, 3 ( 2019 ). Inga Großmann, André Hottung, and Artus Krohn-Grimberghe. 2019. Machine learning meets partner matching: Predicting the future relationship quality based on personality traits. PloS one 14, 3 (2019).
- 10.5555/3305381.3305518 · DOI
- Samantha Joel , Paul W Eastwick , and Eli J Finkel . 2017. Is romantic desire predictable? Machine learning applied to initial romantic attraction. Psychological science 28, 10 ( 2017 ), 1478–1489. Samantha Joel, Paul W Eastwick, and Eli J Finkel. 2017. Is romantic desire predictable? Machine learning applied to initial romantic attraction. Psychological science 28, 10 (2017), 1478–1489.
- Volodymyr Kuleshov and Percy S Liang. 2015. Calibrated structured prediction. In Advances in Neural Information Processing Systems. 3474–3482. Volodymyr Kuleshov and Percy S Liang. 2015. Calibrated structured prediction. In Advances in Neural Information Processing Systems. 3474–3482.
- 10.1145/3233231 · DOI · Paper page
- Ellen Peters , Daniel Västfjäll , Paul Slovic , CK Mertz , Ketti Mazzocco , and Stephan Dickert . 2006. Numeracy and decision making. Psychological science 17, 5 ( 2006 ), 407–413. Ellen Peters, Daniel Västfjäll, Paul Slovic, CK Mertz, Ketti Mazzocco, and Stephan Dickert. 2006. Numeracy and decision making. Psychological science 17, 5 (2006), 407–413.
- John Platt 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10, 3 ( 1999 ), 61–74. John Platt 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10, 3 (1999), 61–74.
- Valerie F Reyna and Charles J Brainerd . 2008. Numeracy , ratio bias, and denominator neglect in judgments of risk and probability. Learning and individual differences 18, 1 ( 2008 ), 89–107. Valerie F Reyna and Charles J Brainerd. 2008. Numeracy, ratio bias, and denominator neglect in judgments of risk and probability. Learning and individual differences 18, 1 (2008), 89–107.
- Aaron Springer and Steve Whittaker . 2018 . Progressive Disclosure: Designing for Effective Transparency. arXiv preprint arXiv:1811.02164(2018). Aaron Springer and Steve Whittaker. 2018. Progressive Disclosure: Designing for Effective Transparency. arXiv preprint arXiv:1811.02164(2018). · arXiv
- Harini Suresh , Natalie Lao , and Ilaria Liccardi . 2020 . Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making. In 12th ACM Conference on Web Science. 315–324 . Harini Suresh, Natalie Lao, and Ilaria Liccardi. 2020. Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making. In 12th ACM Conference on Web Science. 315–324.
- Brandon M Turner , Mark Steyvers , Edgar C Merkle , David V Budescu , and Thomas S Wallsten . 2014. Forecast aggregation via recalibration. Machine learning 95, 3 ( 2014 ), 261–289. Brandon M Turner, Mark Steyvers, Edgar C Merkle, David V Budescu, and Thomas S Wallsten. 2014. Forecast aggregation via recalibration. Machine learning 95, 3 (2014), 261–289.
Cited by
Sorted by citations
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 220
Wischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146
Gómez, C., Cho, S. M., Ke, S., Huang, C., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 207
Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems
He, G., Kuiper, L., & Gadiraju, U. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581025 · Cited by 107
“They found that factors like first impression [76], AI literacy [8], risk perception [33, 34], and performance feedback [55, 61] among others, play important roles in shaping human trust and reliance on AI systems.”
BackgroundTowards Risk‐Free Trustworthy Artificial Intelligence: Significance and Requirements
Alzubaidi, L., Al-Sabaawi, A., Bai, J., Dukhan, A., Alkenani, A. H., Al‐Asadi, A., Alwzwazy, H. A., Manoufali, M., Fadhel, M. A., Albahri, A. S., et al. (2023) · International Journal of Intelligent Systems · doi:10.1155/2023/4459198 · Cited by 116
Metadata
- Authors
- Rechkemmer, A., & Yin, M.
- Affiliations
- Purdue University West Lafayette (US)
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-14
- OpenAlex
- W4224982959
- Cited by
- 97
- References
- 45
- Citations by year
- 2026: 24 · 2025: 35 · 2024: 18 · 2023: 16 · 2022: 4
- FWCI
- 10.09
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- NSF (National Science Foundation) (1850335)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- trust in machine learning, model confidence, model accuracy, performance indicators, laypeople, human-machine interaction, explainable AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
