Does Explainable Artificial Intelligence Improve Human Decision-Making?
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Explainable AI provides insights to users into the why formodel predictions, offering potential for users to better un-derstand and trust a model, and to recognize and correct AIpredictions that are incorrect. Prior research on human andexplainable AI interactions has typically focused on measuressuch as interpretability, trust, and usability of the explanation.There are mixed findings whether explainable AI can improveactual human decision-making and the ability to identify theproblems with the underlying model. Using real datasets, wecompare objective human decision accuracy without AI (con-trol), with an AI prediction (no explanation), and AI predic-tion with explanation. We find providing any kind of AI pre-diction tends to improve user decision accuracy, but no con-clusive evidence that explainable AI has a meaningful impact.Moreover, we observed the strongest predictor for human de-cision accuracy was AI accuracy and that users were some-what able to detect when the AI was correct vs. incorrect, butthis was not significantly affected by including an explana-tion. Our results indicate that, at least in some situations, thewhy information provided in explainable AI may not enhanceuser decision-making, and further research may be needed tounderstand how to integrate explainable AI into real systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
“Clinical versus Actuarial Judgment.”
Dawes, R. M., Faust, D., & Meehl, P. E. (1989) · Science · doi:10.1126/science.2648573 · Cited by 2,298
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 15×
“Anchor LIME is an example of the current state-of-the-art explainable rule-based model [8].”
BackgroundMethodInfluentialHomo Heuristicus: Why Biased Minds Make Better Inferences
Gigerenzer, G., & Brighton, H. (2009) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2008.01006.x · Cited by 1,801
Models of ecological rationality: The recognition heuristic.
Goldstein, D. G., & Gigerenzer, G. (2002) · Psychological Review · doi:10.1037/0033-295x.109.1.75 · Cited by 1,352
The accuracy, fairness, and limits of predicting recidivism
Dressel, J., & Farid, H. (2018) · Science Advances · doi:10.1126/sciadv.aao5580 · Cited by 1,071
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728
Datta, A., Sen, S., & Zick, Y. (2016) · 2016 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2016.42 · Cited by 728
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Vaughan, J. W., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1802.07810 · Cited by 711
Does automation bias decision-making?
SKITKA, L. J., MOSIER, K. L., & BURDICK, M. (1999) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1999.0252 · Cited by 694 · cited 1×
“Sometimes these aids are beneficial for human decision-making, whereas at other times they have negative effects on decisions [22, 23].”
BackgroundIntelligible models for classification and regression
Lou, Y., Caruana, R., & Gehrke, J. (2012) · Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2339530.2339556 · Cited by 547
The Measure and Mismeasure of Fairness
Corbett-Davies, S., Gaebler, J. D., Nilforoshan, H., Shroff, R., & Goel, S. (2018) · arXiv · doi:10.48550/arxiv.1808.00023 · Cited by 444
The Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392
Lai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377
Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307
When Users Want What's not Best for Them
Andre, A. D., & Wickens, C. D. (1995) · Ergonomics in Design: The Quarterly of Human Factors Applications · doi:10.1177/106480469500300403 · Cited by 194
Man versus Machine or Man + Machine?
Cummings, M. M. (2014) · IEEE Intelligent Systems · doi:10.1109/mis.2014.87 · Cited by 146
An Evaluation of the Human-Interpretability of Explanation
Lage, I., Chen, E., He, J., Narayanan, M., Kim, B., Gershman, S., & Doshi-Velez, F. (2019) · arXiv · doi:10.48550/arxiv.1902.00006 · Cited by 125
Uncertainty Forecasts Improve Decision Making Among Nonexperts
Nadav-Greenberg, L., & Joslyn, S. L. (2009) · Journal of Cognitive Engineering and Decision Making · doi:10.1518/155534309x474460 · Cited by 87
Cited by
Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations
Schemmer, M., Kuehl, N., Benz, C., Bartos, A., & Satzger, G. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584066 · Cited by 208
Wysocki, O., Davies, J. K., Vigo, M., Armstrong, A. C., Landers, D., Lee, R., & Freitas, A. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2022.103839 · Cited by 161
Superhuman artificial intelligence can improve human decision-making by increasing novelty
Shin, M., Kim, J., van Opheusden, B., & Griffiths, T. L. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2214840120 · Cited by 106
Fok, R., & Weld, D. S. (2024) · AI Magazine · doi:10.1002/aaai.12182 · Cited by 81
Wang, P., & Ding, H. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103732 · Cited by 68
Metadata
- Authors
- Alufaisan, Y., Marusich, L. R., Bakdash, J. Z., Zhou, Y., & Kantarcioglu, M.
- Publisher
- Center for Open Science
- Published
- 2020-06-18
- OpenAlex
- W3177386843
- Cited by
- 30
- References
- 19
- Type
- posted content
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.