Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 62 of the 63 references it cites were found in a registry or an index.

Explainable Artificial Intelligence: Evaluating the Objective and Subjective Impacts of xAI on Human-Agent Interaction

Open access
Silva, Andrew, Schrum, Mariah, Hedlund-Botti, Erin, Gopalan, Nakul, Gombolay, Matthew · 2023 · International Journal of Human–Computer Interaction

Abstract

Intelligent agents must be able to communicate intentions and explain their decision-making processes to build trust, foster confidence, and improve human-agent team dynamics. Recognizing this need, academia and industry are rapidly proposing new ideas, methods, and frameworks to aid in the design of more explainable AI. Yet, there remains no standardized metric or experimental protocol for benchmarking new methods, leaving researchers to rely on their own intuition or ad hoc methods for assessing new concepts. In this work, we present the first comprehensive (n = 286) user study testing a wide range of approaches for explainable machine learning, including feature importance, probability scores, decision trees, counterfactual reasoning, natural language explanations, and case-based reasoning, as well as a baseline condition with no explanations. We provide the first large-scale empirical evidence of the effects of explainability on human-agent teaming. Our results will help to guide the future of explainability research by highlighting the benefits of counterfactual explanations and the shortcomings of confidence scores for explainability. We also propose a novel questionnaire to measure explainability with human participants, inspired by relevant prior work and correlated with human-agent teaming metrics.

Overview

Integrity signals

Registry

Crossref · 10.1080/10447318.2022.2101698

Cites

62 of 62 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 63 references

Research works printed but not linked: 18

  1. Angelino E., 2017, The Journal of Machine Learning Research,, 18, 1
  2. Chen, C. & Rudin, C. (2017). An optimization approach to learning falling rule lists. arXiv preprint arXiv:1710.02572. · arXiv
  3. Chen, H., Chen, X., Shi, S. & Zhang, Y. (2021). Generate natural language explanations for recommendation. arXiv preprint arXiv:2101.03392. · arXiv
  4. Doshi-Velez, F. & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608. · arXiv
  5. Hoffman, R. R., Mueller, S. T., Klein, G. & Litman, J. (2018). Metrics for explainable AI: Challenges and prospects. arXiv preprint arXiv:1812.04608. · arXiv
  6. Hooker, S., Erhan, D., Kindermans, P.J. & Kim, B. (2018). A benchmark for interpretability methods in deep neural networks. arXiv preprint arXiv:1806.10758. · arXiv
  7. Jain S., 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 3543
  8. Karimi, A.H., Barthe, G., Schölkopf, B. & Valera, I. (2020). A survey of algorithmic recourse: Definitions, formulations, solutions, and prospects. arXiv preprint arXiv:2010.04050. · arXiv
  9. Koh P. W., 2017, International Conference on Machine Learning, 1885
  10. Lage, I., Chen, E., He, J., Narayanan, M., Gershman, S. & Kim, B. (2018). An evaluation of the human-interpretability of explanation. arXiv:1902.00006. · arXiv
  11. Paleja R., 2021, Proceedings of the Conference on Neural Information Processing Systems (NeurIPS)
  12. Ribera M., 2019, IUI Workshops,, 2327, 38
  13. Rosenfeld A., 2021, Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, 45
  14. Silva A., 2020, Proceedings of the twenty third international conference on artificial intelligence and statistics, 108, 1855
  15. Suau, X., Zappella, L. & Apostoloff, N. (2020). Finding experts in transformer models. arXiv preprint arXiv:2005.07647. · arXiv
  16. Verma, S., Dickerson, J. & Hines, K. (2020). Counterfactual explanations for machine learning: A review. arXiv preprint arXiv:2010.10596. · arXiv
  17. Wachter S., 2017, Harvard Journal of Law & Technology, 31, 841
  18. White, A. & Garcez, A. d. (2021). Counterfactual instances explain little. arXiv preprint arXiv:2109.09809. · arXiv

Cited by

See all 104

Metadata

Authors
Silva, A., Schrum, M., Hedlund-Botti, E., Gopalan, N., & Gombolay, M.
Affiliations
Georgia Institute of Technology (US)
Corresponding author
Silva, Andrew
Venue
International Journal of Human–Computer Interaction
Published
2022-08-08
Volume
39
Issue
7
Pages
1390-1404
ISSN
  • 1044-7318 (ISSN-L)
  • 1532-7590
OpenAlex
W4290617519
Cited by
104
References
63
Citations by year
2026: 30 · 2025: 32 · 2024: 29 · 2023: 13
FWCI
6.09
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • International Journal of Human-Computer Interaction (publishedVersion)
  • Figshare (submittedVersion, cc-by)
  • Figshare (cc-by)
Funders
  • MIT (7000437192)
  • NASA (80HQTR19NOA01-19ECF-B1)
  • the National Science Foundation (20-604)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
Keywords
explainable AI, human-agent interaction, human-agent teaming, explainability evaluation, counterfactual explanations, confidence scores, feature importance, natural language explanations, case-based reasoning, decision tree

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Explainable Artificial Intelligence: Evaluating the Objective and Subjective Impacts of xAI on Human-Agent Interaction. https://app.cochat.ai/papers/10.1080/10447318.2022.2101698
The paper
Silva, A., Schrum, M., Hedlund-Botti, E., Gopalan, N., & Gombolay, M. (2023). Explainable Artificial Intelligence: Evaluating the Objective and Subjective Impacts of xAI on Human-Agent Interaction. International Journal of Human–Computer Interaction. https://doi.org/10.1080/10447318.2022.2101698

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.