DARPA’s Explainable AI (XAI) program: A retrospective
Open accessAbstract
DARPA formulated the Explainable Artificial Intelligence (XAI) program in 2015 with the goal to enable end users to better understand, trust, and effectively manage artificially intelligent systems. In 2017, the four-year XAI research program began. Now, as XAI comes to an end in 2021, it is time to reflect on what succeeded, what failed, and what was learned. This article summarizes the goals, organization, and research progress of the XAI Program.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
It is difficult to trust that deep learning answers are correct and unbiased.
“This makes it difficult to trust that their answers are correct and unbiased, as explored in DARPA's explainable AI retrospective [6].”
From Autonomous Task Completion Based on Goal-directed Answer Set Programming (2025) · cites this paper as [6]
The Abstract describes AI systems as seemingly inscrutable and says people wanted to know how to trust them. XAI Program Goals links explainability to appropriate trust and to understanding a system’s weaknesses and mistakes. These passages support difficulty assessing whether answers are correct, but our copy does not address whether the answers are unbiased. (LLM, conf=0.85)
Experimental — accuracy not yet measuredMost deep learning systems are black boxes that lack explainability and interpretability.
“However, most deep learning systems are black boxes that lack explainability and interpretability.”
From Autonomous Task Completion Based on Goal-directed Answer Set Programming (2025) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Letham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766 · see citing works
Flagged first, then by citations
Research works printed but not linked: 25
- Akula, Arjun R., et al. "CX-ToM: Counterfactual Explanations with Theory-of-Mind for Enhancing Human Trust in Image Recognition Models."arXiv preprint arXiv: 2109.01401 (2021) (accepted to iScience 2021). · arXiv
- Akula, Arjun R., et al. "CoCoX: Generating Conceptual and Counterfactual explanations via Fault-Lines." AAAI, 2020.
- Bau, David, et al. "Gan dissection: Visualizing and understanding generative adversarial networks." arXiv preprint arXiv:1811.10597 (2018). · arXiv
- Cellan-Jones, R. (2014). Stephen Hawking warns artificial intelligence could end mankind. BBC news , 2 (10), 2014.
- Danesh, Mohamad H., et al. "Re-understanding Finite-State Representations of Recurrent Policy Networks." International Conference on Machine Learning . PMLR, 2021.
- Dikkala, Rupika, et al. "Doing Remote Controlled Studies with Humans: Tales from the COVID Trenches." ACM-IEEE CHASE. 2021.
- Folke, Tomas, et al. "Explainable AI for medical imaging: explaining pneumothorax diagnoses with Bayesian teaching." arXiv preprint arXiv:2106.04684 (2021). · arXiv
- Gunning, David, and David Aha. "DARPA's explainable artificial intelligence (XAI) program." AI Magazine 40.2 (2019): 44-58.
- Johnson, W. Lewis. "Agents that Learn to Explain Themselves."AAAI . 1994.
- Jordan, Michael I., and Tom M. Mitchell. "Machine learning: Trends, perspectives, and prospects." Science 349.6245 (2015): 255-260.
- Khorram, Saeed, Tyler Lawson, and Li Fuxin. "iGOS++ integrated gradient optimized saliency by bilateral perturbations." Proceedings of the Conference on Health, Inference, and Learning . 2021.
- Anurag Koul, Alan Fern, and Sam Greydanus. "Learning Finite State Representations of Recurrent Policy Networks." International Conference on Learning Representations. 2019
- Kulesza, Todd, et al. "Principles of explanatory debugging to personalize interactive machine learning." Proceedings of the 20th international conference on intelligent user interfaces . 2015
- Lacave, Carmen, and Francisco J. Díez. "A review of explanation methods for Bayesian networks." The Knowledge Engineering Review 17.2 (2002): 107-127.
- Zhengxian Lin, Kim-Ho Lam, and Alan Fern. "Contrastive Explanations for Reinforcement Learning via Embedded Self Predictions." International Conference on Learning Representations.
- Liu, Hangxin, et al. "Patching Interpretable And-Or Graph Knowledge Representation using Augmented Reality." Applied AI Letters 2021 (under review)
- Mai, Theresa, et al. "Keeping it" organized and logical" after-action review for AI (AAR/AI)." Proceedings of the 25th International Conference on Intelligent User Interfaces . 2020.
- Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. "" Why should i trust you?" Explaining the predictions of any classifier."Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining . 2016
- Richmond, Sheldon. "Superintelligence: Paths, Dangers, Strategies. By Nick Bostrom. Oxford University Press, Oxford, 2014
- Selvaraju, Ramprasaath R., et al. "Grad-CAM: Why did you say that?."arXiv preprint arXiv:1611.07450 (2016). · arXiv
- Shortliffe, Edward H., and Bruce G. Buchanan. "A model of inexact reasoning in medicine." Mathematical biosciences 23.3-4 (1975): 351-379.
- Sun, Mingjie, Siddhant Agarwal, and J. Zico Kolter. "Poisoned classifiers are not only backdoored, they are fundamentally broken."arXiv preprint arXiv:2010.09080 (2020). · arXiv
- Swartout, William, Cecile Paris, and Johanna Moore. "Explanations in knowledge systems: Design for ex- plainable expert systems." IEEE Expert 6.3 (1991): 58-64.
- Van Lent, Michael, William Fisher, and Michael Mancuso. "An explainable artificial intelligence system for small-unit tactical behavior."Proceedings of the national conference on artificial intelligence . Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2004.
- Zeiler, Matthew D., and Rob Fergus. "Visualizing and understanding convolutional networks." European conference on computer vision . Springer, Cham, 2014.
Cited by
Sorted by citations
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762 · see citing works
Dikmen, M., & Burns, C. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102792 · Cited by 116 · see citing works
Consciousness beyond the human case
LeDoux, J., Birch, J., Andrews, K., Clayton, N. S., Daw, N. D., Frith, C., Lau, H., Peters, M. A. K., Schneider, S., Seth, A., et al. (2023) · Current Biology · doi:10.1016/j.cub.2023.06.067 · Cited by 67 · see citing works
Creative Use of OpenAI in Education: Case Studies from Game Development
French, F., Levi, D., Maczo, C., Simonaityte, A., Triantafyllidis, S., & Varda, G. (2023) · Multimodal Technologies and Interaction · doi:10.3390/mti7080081 · Cited by 64 · see citing works
Explainable Anomaly Detection Based on Operational Sequences in Industrial Control Systems
Kim, K.-K., Kim, J.-S., & Euom, I.-C. (2025) · IEEE Access · doi:10.1109/access.2025.3560260 · Cited by 7 · see citing works
- Authors
- Gunning, D., Vorm, E., Wang, Y., & Turek, M.
- Affiliations
- Defense Advanced Research Projects Agency (US)
- United States Naval Research Laboratory (US)
- Quantitative Scientific Solutions
- Publisher
- Wiley
- Published
- 2021-11-15
- OpenAlex
- W4200545033
- Cited by
- 60
- References
- 1
- Citations by year
- 2026: 8 · 2025: 13 · 2024: 17 · 2023: 17 · 2022: 5
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- explainable AI, DARPA, research program evaluation, trust in AI, human-AI interaction, AI transparency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
