Accountability of AI Under the Law: The Role of Explanation
Open accessThe DOI matches its Crossref record; 24 of the 26 references it cites were found in a registry or an index; none of the 17 sources its claims cite is checked for support yet.
Abstract
The ubiquity of systems using artificial intelligence or "AI" has brought increasing attention to how those systems should be regulated. The choice of how to regulate AI systems will require care. AI systems have the potential to synthesize large amounts of data, allowing for greater levels of personalization and precision than ever before|applications range from clinical decision support to autonomous driving and predictive policing. That said, common sense reasoning [McCarthy, 1960] remains one of the holy grails of AI, and there exist legitimate concerns about the intentional and unintentional negative consequences of AI systems [Bostrom, 2003, Amodei et al., 2016, Sculley et al., 2014]. There are many ways to hold AI systems accountable. In this work, we focus on one: explanation. Questions about a legal right to explanation from AI systems was recently debated in the EU General Data Protection Regulation [Goodman and Flaxman, 2016, Wachter et al., 2017], and thus thinking carefully about when and how explanation from AI systems might improve accountability is timely. Good choices about when to demand explanation can help prevent negative consequences from AI systems, while poor choices may not only fail to hold AI systems accountable but also hamper the development of much-needed beneficial AI systems. Below, we briefly review current societal, moral, and legal norms around explanation, and then focus on the different contexts under which explanation is currently required under the law. We find that there exists great variation around when explanation is demanded, but there also exists important consistencies: when demanding explanation from humans, what we typically want to know is how and whether certain input factors affected the final decision or outcome. These consistencies allow us to list the technical considerations that must be considered if we desired AI systems that could provide kinds of explanations that are currently required of humans under the law. Contrary to popular wisdom of AI systems as indecipherable black boxes, we find that this level of explanation should often be technically feasible but may sometimes be practically onerous|there are certain aspects of explanation that may be simple for humans to provide but challenging for AI systems, and vice versa. As an interdisciplinary team of legal scholars, computer scientists, and cognitive scientists, we recommend that for the present, AI systems can and should be held to a similar standard of explanation as humans currently are; in the future we may wish to hold an AI to a different standard.
Overview
CoChat Summary
TL;DR
The paper argues that legally operative explanations from AI systems—human-interpretable information about factors and weights or counterfactual answers—are technically feasible and should initially be demanded of AI systems to the same extent as humans. It reviews legal contexts in the United States, United Kingdom, France, and Germany and proposes treating explanation systems as distinct from AI predictors.
Key findings
- The paper defines an explanation as human-interpretable information about the logic by which a decision-maker reached a conclusion, and says it should let an observer determine the extent to which a particular input was determinative or influential, via either important factors and their relative weight or answers to counterfactual questions.
- Society tends to demand explanations based on three factors: the impact of the decision, the possibility of contesting/correcting/compensating for error, and reason to believe an error has occurred or will occur; no single factor is dispositive.
- The legal survey finds variation: judges are generally required to give reasoned decisions in the UK, France, and Germany, while juries rarely explain; administrative agencies must explain decisions; the U.S. Fair Credit Reporting Act requires key negative credit factors; France’s Digital Republic Act creates a right to explanation for algorithmic public-sector decisions; GDPR Articles 13–15 and 22 address automated decision-making information and rights.
- Technically, local explanation and local counterfactual faithfulness can map to legally operative explanations without full transparency; an explanation system can be separate from the AI system and need not affect predictive accuracy, though concept-mapping and ex post term determination remain challenges; alternatives include empirical evidence and theoretical guarantees.
Methods
- Conceptual, legal, and technical analysis produced from over a dozen meetings of the Berkman Klein Center Working Group on AI Interpretability, a collaboration among legal scholars, computer scientists, and cognitive scientists.
- Comparative legal survey of explanation from two perspectives—the legal system as a decision-maker and as an enforcer of accountability—across the United States, United Kingdom, France, and Germany; the paper states the survey is far from comprehensive.
- No empirical dataset or quantitative experiment is reported; the analysis maps legal explanation requirements to AI technical concepts and offers recommendations.
Limitations
- The paper states there is little data to determine the actual costs of requiring AI systems to generate explanations, so the role of explanation must be re-evaluated as technology changes.
- The paper states technical challenges: mapping high-dimensional inputs to human-interpretable concepts, determining relevant terms ex post in litigation, and reverse-engineering risk from access to many explanations.
- Evident from the text: there is no empirical validation or cost measurement; the legal survey is explicitly not comprehensive; and the recommendations are normative and may not generalize across all AI contexts.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 15 Not valid · 15 Not checked · 138 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Across both perspectives and countries, we find variations regarding the role of the explanation, who is obligated to provide it, and what type or amount of evidence is needed to trigger that obligation.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Across all of the jurisdictions we surveyed, adjudication of individual claims often requires one or both parties to generate explanations.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Society is more likely explanation where the explanation can be acted on in some way.”
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By isolating the determinative factors of a decision, which can then be contested, counterfactual explanations promote accountability.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and access to an explanation might decrease observers' trust in some decisions.”
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
45 more claims
Create a free account to see all 168 claims and to verify the 17 references still open.
- Probably not a substantial claim. 107
- It may state two separate claims as one. 33
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The use of “high dimensional” or “black box” prediction techniques typically favored in the machine learning community are generally problematic (since these approaches can predict race through other variables, whether or not race is explicitly encoded)—although methods that have been proposed to make these models more transparent have recently been adapted to address fairness46.”
From Predictably unequal: understanding and addressing concerns that algorithmic clinical prediction may increase health disparities (2020) · cites this paper as [46]
Experimental — accuracy not yet measured
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Telling more than we can know: Verbal reports on mental processes.
Nisbett, R. E., & Wilson, T. D. (1977) · Psychological Review · doi:10.1037/0033-295x.84.3.231 · Cited by 11,317
(2012) · Cited by 9,062
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936
SmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467
Concrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314
Wachter, S., Mittelstadt, B., & Floridi, L. (2017) · International Data Privacy Law · doi:10.1093/idpl/ipx005 · Cited by 1,308
Counterfactual explanations without opening the black box: automated decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3063289 · Cited by 1,061
Rationalizing Neural Predictions
Lei, T., Barzilay, R., & Jaakkola, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1011 · Cited by 777
Research works printed but not linked: 1
- , 60 BUS. LAW. 737, 744 (2005).
Cited by
Items 401–443 of 443
اثار قيام المسؤولية المدنية الناشئة على استخدام تقنية الذكاء الاصطناعي
الحيارى, ا. ف. م., & المحاسنة, ا. ي. ع. م. (2023) · Journal of Educational and Human Sciences · doi:10.33193/jeahs.29.2023.412
Ouyang, L., Jahng, J., & Woo, S. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2705163
Zhang, X. (2024) · Applied Mathematics and Nonlinear Sciences · doi:10.2478/amns-2024-2506 · Cited by 1
Modèles prédictifs & raisonnement avec les explications
Boumazouza, R. (2022) · theses.fr (ABES) · doi:10.70675/e74c615fz12cbz4425z8bc7z8f651a87f9d8 · Cited by 1
Crockett, K. A. (2022) · Women in engineering and science · doi:10.1007/978-3-030-79092-9_3 · Cited by 2
Penney, J. W. (2025) · Cambridge University Press eBooks · doi:10.1017/9781108641784.017
Trustworthy AI for Modbus/TCP Intrusion Detection in Industrial Control Systems
Tawfik, M. K., & Badawy, W. M. (2025) · International Conference on Intelligent Computing and Information Systems (ICICIS) · doi:10.1109/icicis66182.2025.11313108
Outlining Traceability: A Principle for Operationalizing Accountability in Computing Systems
Kroll, J. A. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445937 · Cited by 2
AutoXAI: a meta-learning approach for recommendation of explanation techniques
Shawi, R. E., & Jamel, L. (2025) · Scientific Reports · doi:10.1038/s41598-025-25872-x · Cited by 1
Yadav, P., Kumari, J., Kumar, B. P., Nagaraj, D. S., Sharma, K., Mahboob, M. S., Glory, K. B., & Gupta, D. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9164-9.ch006
Razakamaharavo, V. T. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3652998 · Cited by 1
Li, J., & Li, Y. (2026) · International Journal of Advertising · doi:10.1080/02650487.2026.2661459 · Cited by 2
Azhari, A. A., Ahmed, W. M., Alhamadani, A., Alfaraj, A., Zhang, M., & Lu, C.-T. (2026) · Dentistry Journal · doi:10.3390/dj14020072 · Cited by 3
Opportunities, Risks and Ethics in Rising AI-Powered Hiring
Niroula, N. P., & Sharma, G. (2026) · The Voice of Creative Research · doi:10.53032/tvcr/2026.v8n1.13 · Cited by 1
A Theoretical Approximation to Artificial Intelligence as an Autopoietic System
Wiesmüller, S. (2023) · Relational economics and organization governance · doi:10.1007/978-3-031-25023-1_2
Behera, N. R., Gurunathan, K., Kaliappan, S., Steffi, T., Rai, V., & Manikandan, V. (2024) · International Conference Big Data and Machine Learning · doi:10.1109/icbdml60909.2024.10577307 · Cited by 1
Introduction to Continuous Time Series
Habiba, M., Pearlmutter, B. A., & Maleki, M. (2026) · Springer eBooks · doi:10.1007/978-3-032-18022-3_1
Future of Work, Protection of Labor’ Rights in Using Artificial Intelligence Based on SDGs 8.9
Wijayanti, A., Chamdani, Sahid, M. M., & Putri, A. (2023) · Advances in Social Science, Education and Humanities Research/Advances in social science, education and humanities research · doi:10.2991/978-2-38476-022-0_10
Simulação de fenômenos sociais
Almada, M. (2018) · doi:10.47749/t/unicamp.2018.1079854
AI Meets Sustainability: A Special Issue on Real-World Applications
Al‐Jumeily, D., Mustafina, J., & Jayabalan, M. (2025) · Sustainability · doi:10.3390/su17209148
Auto Encoding Explanatory Examples with Stochastic Paths
Ojeda, C., Sánchez, R. J., Cvejoski, K., Schücker, J., Bauckhagez, C., & Georgievz, B. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413267
Artificial Intelligence Through the Lens of Islamic Ethics: A Quranic Perspective
Khan, D. N., & Muhammad, G. (2026) · VFAST Transactions on Islamic Research · doi:10.21015/vfast-tir.v14i1.2454
How Artificial Intelligence Can be Useful for Criminal Courts in Indonesia
Santosa, D. N. A. (2023) · Multidiscipline International Conference (MIC) · doi:10.4108/eai.28-10-2023.2341682
Constituting Algorithmic Accountability
Johnson, D. G. (2025) · Philosophy of engineering and technology · doi:10.1007/978-3-031-87974-6_2 · Cited by 1
Unleashing Human Potential: Human-AI Collaboration and Augmentation
Najar, M. A. A. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-5231-1.ch013
Et-Tousy, J., Malek, C., & Zyane, A. (2026) · International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET) · doi:10.1109/iraset68627.2026.11538732
Ethics, Governance, Risks, Compliance, and Sovereignty of AI for Cybersecurity
Ramachandran, M. (2026) · Texts in computer science · doi:10.1007/978-3-032-17367-6_16
Mast, T. (2023) · Nomos Verlagsgesellschaft mbH & Co. KG eBooks · doi:10.5771/9783748936060-141
A Neutrosophic Framework for Assessing Ethical Considerations in Autonomous Systems
Paraskevas, A. (2025) · WSEAS Transactions on Information Science and Applications archive · doi:10.37394/23209.2025.22.24
Explainable pattern modelling and summarization in sensor equipped smart homes of elderly
Jain, A. (2020) · MOspace Institutional Repository (University of Missouri) · doi:10.32469/10355/79503
The Use of Artificial Intelligence in Corporate Management: Legal Issues
Кирилова, М. В. (2026) · Actual Problems of Russian Law · doi:10.17803/1994-1471.2026.185.4.091-108
Durante, M. (2026) · Governance and public management · doi:10.1007/978-3-032-11744-1_2
ARTIFICIAL INTELLIGENCE, ETHICS, AND LEGAL ACCOUNTABILITY IN HIGHER EDUCATION INSTITUTIONS
Srivastava, M., Sahoo, G. R., Bhatt, G., Mukherji, J., Singh, J., & Mohapatra, A. (2026) · ShodhKosh Journal of Visual and Performing Arts · doi:10.29121/shodhkosh.v7.i14s.2026.8537 · Cited by 1
AI as a Collaborative Tool: Redefining Educator Roles in Content Development
Ghosh, S., & Jana, O. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-1122-7.ch005
Verhoturov, D. S. (2025) · Юридические исследования · doi:10.25136/2409-7136.2025.11.76324
Explainable Deep Learning for Modbus/TCP Traffic Classification in Industrial Control Systems
Tawfik, M. K., & Badawy, W. (2025) · doi:10.1109/ficac65757.2025.11341864
Bhatt, N., & Bhatt, J. N. (2025) · Journal of Digital Technologies and Law · doi:10.21202/jdtl.2025.26 · Cited by 1
Oukhouya, H., Rhiouane, A. E., Lfaze, F. E., Zari, T., Guerbaz, R., Belkhoutout, K., & Moussi, A. (2025) · International Journal of Computers and Applications · doi:10.1080/1206212x.2025.2572477 · Cited by 2
Metadata
- Authors
- Doshi-Velez, F., Kortz, M., Budish, R., Bavitz, C., Gershman, S. J., O'Brien, D., Scott, K., Shieber, S., Waldo, J., Weinberger, D., Weller, A., & Wood, A.
- Affiliations
- Harvard University (US)
- University of Manchester (GB)
- University of Cambridge (GB)
- Venue
- SSRN Electronic Journal
- Publisher
- Elsevier BV
- Published
- 2017
- ISSN
- 1556-5068 (ISSN-L)
- OpenAlex
- W2765811634
- Cited by
- 457
- References
- 26
- Citations by year
- 2026: 36 · 2025: 56 · 2024: 64 · 2023: 49 · 2022: 58 · 2021: 64 · 2020: 55 · 2019: 51 · 2018: 21 · 2017: 1
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- SSRN Electronic Journal (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- AI accountability, legal explanation, right to explanation, AI regulation, General Data Protection Regulation, explainable AI, black box problem, human-AI comparison, AI decision transparency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.