The DOI matches its Crossref record; 62 of the 63 references it cites were found in a registry or an index.
Explainable Artificial Intelligence: Evaluating the Objective and Subjective Impacts of xAI on Human-Agent Interaction
Open accessAbstract
Intelligent agents must be able to communicate intentions and explain their decision-making processes to build trust, foster confidence, and improve human-agent team dynamics. Recognizing this need, academia and industry are rapidly proposing new ideas, methods, and frameworks to aid in the design of more explainable AI. Yet, there remains no standardized metric or experimental protocol for benchmarking new methods, leaving researchers to rely on their own intuition or ad hoc methods for assessing new concepts. In this work, we present the first comprehensive (n = 286) user study testing a wide range of approaches for explainable machine learning, including feature importance, probability scores, decision trees, counterfactual reasoning, natural language explanations, and case-based reasoning, as well as a baseline condition with no explanations. We provide the first large-scale empirical evidence of the effects of explainability on human-agent teaming. Our results will help to guide the future of explainability research by highlighting the benefits of counterfactual explanations and the shortcomings of confidence scores for explainability. We also propose a novel questionnaire to measure explainability with human participants, inspired by relevant prior work and correlated with human-agent teaming metrics.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Explainability is becoming increasingly important as AI systems are being used to make decisions that have significant impacts on people's lives, such as in healthcare, finance, and criminal justice [49].”
From Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence (2024) · cites this paper as [49]
Citations
Cites
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
Classification and regression trees
Praagman, J. (1985) · European Journal of Operational Research · doi:10.1016/0377-2217(85)90321-2 · Cited by 10,053
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
"General Intelligence," Objectively Determined and Measured
Spearman, C. (1904) · The American Journal of Psychology · doi:10.2307/1412107 · Cited by 6,207
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Usability Evaluation In Industry
(1996) · CRC Press eBooks · doi:10.1201/9781498710411 · Cited by 4,354
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Research works printed but not linked: 18
- Angelino E., 2017, The Journal of Machine Learning Research,, 18, 1
- Chen, C. & Rudin, C. (2017). An optimization approach to learning falling rule lists. arXiv preprint arXiv:1710.02572. · arXiv
- Chen, H., Chen, X., Shi, S. & Zhang, Y. (2021). Generate natural language explanations for recommendation. arXiv preprint arXiv:2101.03392. · arXiv
- Doshi-Velez, F. & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608. · arXiv
- Hoffman, R. R., Mueller, S. T., Klein, G. & Litman, J. (2018). Metrics for explainable AI: Challenges and prospects. arXiv preprint arXiv:1812.04608. · arXiv
- Hooker, S., Erhan, D., Kindermans, P.J. & Kim, B. (2018). A benchmark for interpretability methods in deep neural networks. arXiv preprint arXiv:1806.10758. · arXiv
- Jain S., 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 3543
- Karimi, A.H., Barthe, G., Schölkopf, B. & Valera, I. (2020). A survey of algorithmic recourse: Definitions, formulations, solutions, and prospects. arXiv preprint arXiv:2010.04050. · arXiv
- Koh P. W., 2017, International Conference on Machine Learning, 1885
- Lage, I., Chen, E., He, J., Narayanan, M., Gershman, S. & Kim, B. (2018). An evaluation of the human-interpretability of explanation. arXiv:1902.00006. · arXiv
- Paleja R., 2021, Proceedings of the Conference on Neural Information Processing Systems (NeurIPS)
- Ribera M., 2019, IUI Workshops,, 2327, 38
- Rosenfeld A., 2021, Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, 45
- Silva A., 2020, Proceedings of the twenty third international conference on artificial intelligence and statistics, 108, 1855
- Suau, X., Zappella, L. & Apostoloff, N. (2020). Finding experts in transformer models. arXiv preprint arXiv:2005.07647. · arXiv
- Verma, S., Dickerson, J. & Hines, K. (2020). Counterfactual explanations for machine learning: A review. arXiv preprint arXiv:2010.10596. · arXiv
- Wachter S., 2017, Harvard Journal of Law & Technology, 31, 841
- White, A. & Garcez, A. d. (2021). Counterfactual instances explain little. arXiv preprint arXiv:2109.09809. · arXiv
Cited by
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence
Hassija, V., Chamola, V., Mahapatra, A., Singal, A., Goel, D., Huang, K., Scardapane, S., Spinelli, I., Mahmud, M., & Hussain, A. (2024) · Cognitive Computation · doi:10.1007/s12559-023-10179-8 · Cited by 2,495
“Explainability is becoming increasingly important as AI systems are being used to make decisions that have significant impacts on people’s lives, such as in healthcare, finance, and criminal justice [49].”
BackgroundGomez, C., Cho, S. M., Ke, S., Huang, C.-M., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 209
Defining human-AI teaming the human-centered way: a scoping review and network analysis
Berretta, S., Tausch, A., Ontrup, G., Gilles, B., Peifer, C., & Kluge, A. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1250725 · Cited by 171
Explainable Generative AI (GenXAI): a survey, conceptualization, and research agenda
Schneider, J. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10916-x · Cited by 260
A review of evaluation approaches for explainable AI with applications in cardiology
Salih, A. M., Galazzo, I. B., Gkontra, P., Rauseo, E., Lee, A. M., Lekadir, K., Radeva, P. I., Petersen, S. E., & Menegaz, G. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10852-w · Cited by 104
Metadata
- Authors
- Silva, A., Schrum, M., Hedlund-Botti, E., Gopalan, N., & Gombolay, M.
- Affiliations
- Georgia Institute of Technology (US)
- Corresponding author
- Silva, Andrew
- Venue
- International Journal of Human–Computer Interaction
- Publisher
- Informa UK Limited
- Published
- 2022-08-08
- Volume
- 39
- Issue
- 7
- Pages
- 1390-1404
- ISSN
- 1044-7318 (ISSN-L)
- 1532-7590
- OpenAlex
- W4290617519
- Cited by
- 104
- References
- 63
- Citations by year
- 2026: 30 · 2025: 32 · 2024: 29 · 2023: 13
- FWCI
- 6.09
- Type
- journal article
- Language
- en
- Open access
- green
- figshare.com
- Licence
- cc-by
- Locations
- International Journal of Human-Computer Interaction (publishedVersion)
- Figshare (submittedVersion, cc-by)
- Figshare (cc-by)
- Funders
- MIT (7000437192)
- NASA (80HQTR19NOA01-19ECF-B1)
- the National Science Foundation (20-604)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- explainable AI, human-agent interaction, human-agent teaming, explainability evaluation, counterfactual explanations, confidence scores, feature importance, natural language explanations, case-based reasoning, decision tree
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Explainable Artificial Intelligence: Evaluating the Objective and Subjective Impacts of xAI on Human-Agent Interaction. https://app.cochat.ai/papers/10.1080/10447318.2022.2101698
- The paper
- Silva, A., Schrum, M., Hedlund-Botti, E., Gopalan, N., & Gombolay, M. (2023). Explainable Artificial Intelligence: Evaluating the Objective and Subjective Impacts of xAI on Human-Agent Interaction. International Journal of Human–Computer Interaction. https://doi.org/10.1080/10447318.2022.2101698
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.