CoChat
Sign Up
Cited by 136Explainable Artificial Intelligence (XAI)Ethics and Social Impacts of AISafety Systems Engineering in Autonomy

Artificial intelligence explainability: the technical and ethical dimensions

Open access
McDermid, John A., Jia, Yan, Porter, Zoe, Habli, Ibrahim·2021·Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
DOI 10.1098/rsta.2020.0363PMC PMC8366909
Open this paper in CoChat — free account

Abstract

In recent years, several new technical methods have been developed to make AI-models more transparent and interpretable. These techniques are often referred to collectively as ‘AI explainability’ or ‘XAI’ methods. This paper presents an overview of XAI methods, and links them to stakeholder purposes for seeking an explanation. Because the underlying stakeholder purposes are broadly ethical in nature, we see this analysis as a contribution towards bringing together the technical and ethical dimensions of XAI. We emphasize that use of XAI methods must be linked to explanations of human decisions made during the development life cycle. Situated within that wider accountability framework, our analysis may offer a helpful starting point for designers, safety engineers, service providers and regulators who need to make practical judgements about which XAI methods to employ or to require. This article is part of the theme issue ‘Towards symbiotic autonomous systems’.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1098/rsta.2020.0363

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Provable AI ethics frameworks propose technical architectures for embedding explainability, constraint validation, and ethical guardrails directly into AI systems rather than relying solely on policy-level oversight.

    “Recent evidence50 on provable AI ethics frameworks proposes technical architectures for embedding explainability, constraint validation, and ethical guardrails directly into AI systems rather than relying solely on policy-level oversight.”

    From Generative Artificial Intelligence in Healthcare: Automation Bias, Deskilling, and Cognitive Implications – A Systematic Review (2026) · cites this paper as [50]

    The Introduction and Discussion describe technical explainability methods that can help assure ML systems, including before and after deployment. They do not propose an architecture that embeds constraint validation or ethical guardrails directly into AI systems; the Conclusion instead places XAI within a wider framework of human accountability. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 55

  1. [2]Bhatt U et al. 2020 Explainable machine learning in deployment. In Proc. of the 2020 Conf. on Fairness, Accountability, and Transparency, pp. 648–657.
  2. [3]McDermid JA. 2010 Safety Critical Software. In Encyclopedia of Aerospace Engineering (eds R Blockley, W Shyy). 10.1002/9780470686652.eae506. · DOI
  3. [4]Zhang GP. 2000 Neural networks for classification: a survey. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 30, 451- 462. (10.1109/5326.897072) · DOI
  4. [5]Wang G. 2008 A survey on training algorithms for support vector machine classifiers. In 2008 Fourth Int. Conf. on Networked Computing and Advanced Information Management, vol. 1, Gyeongju, South Korea, 2–4 September 2008, pp. 123–128. New York, NY: IEEE.
  5. [6]Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA. 2017 Deep reinforcement learning: a brief survey. IEEE Signal Process Mag. 34, 26- 38. (10.1109/MSP.2017.2743240) · DOI · Paper page
  6. [10]Ashmore R, Calinescu R, Paterson C. 2019 Assuring the machine learning lifecycle: desiderata, methods, and challenges. http://arxiv.org/abs/190504223.
  7. [13]Burton S, Habli I, Lawton T, McDermid J, Morgan P, Porter Z. 2020 Mind the gaps: assuring the safety of autonomous systems from an engineering, ethical, and legal perspective. Artif. Intell. 279, 103201. (10.1016/j.artint.2019.103201) · DOI
  8. [15]Mittelstadt B, Russell C, Wachter S. 2019 Explaining explanations in AI. In Proc. of the Conf. on fairness, accountability, and transparency, Atlanta, GA, 29–31 January 2019, pp. 279–288. New York, NY: ACM.
  9. [16]Raz J. 2011 From normativity to responsibility. Oxford, UK: Oxford University Press.
  10. [18]Biran O, McKeown K. 2014 Justification narratives for individual classifications. In Proc. of the AutoML workshop at ICML, vol. 2014, pp. 1–7.
  11. [19]Ward FR, Habli I. 2020 An assurance case pattern for the interpretability of machine learning in safety-critical systems. In Int. Conf. on Computer Safety, Reliability, and Security, Lisbon, Portugal, 15–18 September 2020, pp. 395–407. Berlin, Germany: Springer.
  12. [22]Rudin C. 2019 Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1, 206- 215. (10.1038/s42256-019-0048-x) PMC9122117 35603010 · DOI · Paper page
  13. [23]Weller A. 2019 Transparency: motivations and challenges. In Explainable AI: interpreting, explaining and visualizing deep learning (eds W Samek, G Montavon, A Vedaldi, LK Hansen, K-R Müller), pp. 23–40. Berlin, Germany: Springer.
  14. [24]Langer M, Oster D, Speith T, Hermanns H, Kästner L, Schmidt E, Sesing A, Baum K. 2021 What do we want from Explainable Artificial Intelligence (XAI)? A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research. Artif. Intel. 296, 103473. (10.1016/j.artint.2021.103473) · DOI · Paper page
  15. [29]Hastie T, Tibshirani R, Friedman J. 2009 The elements of statistical learning. Berlin, Germany: Springer.
  16. [30]Louppe G. 2014 Understanding random forests: from theory to practice. http://arxiv.org/abs/14077502.
  17. [31]Breiman L. 2001 Random forests. Mach. Learn. 45, 5- 32. (10.1023/A:1010933404324) · DOI · Paper page
  18. [34]Gilpin LH, Bau D, Yuan BZ, Bajwa A, Specter M, Kagal L. 2018 Explaining explanations: an overview of interpretability of machine learning. In 2018 IEEE 5th Int. Conf. on data science and advanced analytics (DSAA), Turin, Italy, 1–3 October 2018, pp. 80–89. New York, NY: IEEE.
  19. [35]Montano J, Palmer A. 2003 Numeric sensitivity analysis applied to feedforward neural networks. Neural Comput. Appl. 12, 119- 125. (10.1007/s00521-003-0377-9) · DOI
  20. [36]Liang B, Li H, Su M, Bian P, Li X, Shi W. 2017 Deep text classification can be fooled. http://arxiv.org/abs/170408006.
  21. [37]Zeiler MD, Fergus R. 2014 Visualizing and understanding convolutional networks. In European Conf. on computer vision, Zürich, Switzerland, 6–12 September 2014, pp. 818–833. Berlin, Germany: Springer.
  22. [38]Ribeiro MT, Singh S, Guestrin C. 2016 ‘Why should I trust you?’ Explaining the predictions of any classifier. In Proc. of the 22nd ACM SIGKDD Int. Conf. on knowledge discovery and data mining, San Francisco, CA, 13–17 August 2016, pp. 1135–1144. New York, NY: ACM.
  23. [40]Bhatt U, Ravikumar P, Moura JM. 2019 Towards aggregating weighted feature attributions. http://arxiv.org/abs/190110040.
  24. [41]Štrumbelj E, Kononenko I. 2014 Explaining prediction models and individual predictions with feature contributions. Knowl. Inf. Syst. 41, 647- 665. (10.1007/s10115-013-0679-x) · DOI · Paper page
  25. [42]Chen J, Song L, Wainwright MJ, Jordan MI. 2018 L-shapley and c-shapley: efficient model interpretation for structured data. http://arxiv.org/abs/180802610.
  26. [45]Zintgraf LM, Cohen TS, Adel T, Welling M. 2017 Visualizing deep neural network decisions: prediction difference analysis. http://arxiv.org/abs/170204595.
  27. [47]Shrikumar A, Greenside P, Shcherbina A, Kundaje A. 2016 Not just a black box: learning important features through propagating activation differences. http://arxiv.org/abs/160501713.
  28. [48]Sundararajan M, Taly A, Yan Q. 2017 Axiomatic attribution for deep networks. http://arxiv.org/abs/170301365.
  29. [49]Shrikumar A, Greenside P, Kundaje A. 2017 Learning important features through propagating activation differences. http://arxiv.org/abs/170402685.
  30. [50]Ancona M, Ceolini E, Öztireli C, Gross M. 2017 Towards better understanding of gradient-based attribution methods for deep neural networks. http://arxiv.org/abs/171106104.
  31. [52]Kolodner JL. 1992 An introduction to case-based reasoning. Artif. Intell. Rev. 6, 3- 34. (10.1007/BF00155578) · DOI
  32. [53]Richter MM, Weber RO. 2016 Case-based reasoning. Berlin, Germany: Springer.
  33. [54]Jia Y, Kaul C, Lawton T, Murray-Smith R, Habli I. 2020 Prediction of weaning from mechanical ventilation using convolutional neural networks. Artif. Intell. Med. 117, 102087. 34127233
  34. [55]Wachter S, Mittelstadt B, Russell C. 2017 Counterfactual explanations without opening the black box: automated decisions and the GDPR. Harv. JL & Tech. 31, 841. (10.2139/ssrn.3063289) · DOI · Paper page
  35. [56]Mothilal RK, Sharma A, Tan C. 2020 Explaining machine learning classifiers through diverse counterfactual explanations. In Proc. of the 2020 Conf. on Fairness, Accountability, and Transparency, Barcelona, Spain, 27–30 January 2020, pp. 607–617.
  36. [57]Sharma S, Henderson J, Ghosh J. 2019 Certifai: Counterfactual explanations for robustness, transparency, interpretability, and fairness of artificial intelligence models. http://arxiv.org/abs/190507857.
  37. [58]Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R. 2013 Intriguing properties of neural networks. http://arxiv.org/abs/13126199.
  38. [59]Xie C, Tan M, Gong B, Wang J, Yuille AL, Le QV. 2020 Adversarial examples improve image recognition. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition, Seattle, WA, 13–19 June 2020, pp. 819–828. New York, NY: IEEE.
  39. [60]Jia R, Liang P. 2017 Adversarial examples for evaluating reading comprehension systems. http://arxiv.org/abs/170707328.
  40. [61]Sato M, Suzuki J, Shindo H, Matsumoto Y. 2018 Interpretable adversarial perturbation in input embedding space for text. http://arxiv.org/abs/180502917.
  41. [62]Etmann C, Lunz S, Maass P, Schönlieb CB. 2019 On the connection between adversarial robustness and saliency map interpretability. http://arxiv.org/abs/190504172.
  42. [63]Koh PW, Liang P. 2017 Understanding black-box predictions via influence functions. http://arxiv.org/abs/170304730.
  43. [65]Ambrosino N, Gabbrielli L. 2010 The difficult-to-wean patient. Expert Rev. Respiratory Med. 4, 685- 692. (10.1586/ers.10.58) 20923345 · DOI
  44. [66]Kuo HJ, Chiu HW, Lee CN, Chen TT, Chang CC, Bien MY. 2015 Improvement in the prediction of ventilator weaning outcomes by an artificial neural network in a medical ICU. Respir. Care 60, 1560- 1569. (10.4187/respcare.03648) 26329358 · DOI
  45. [67]Johnson AE et al. 2016 MIMIC-III, a freely accessible critical care database. Sci. Data 3, 1- 9. (10.1038/sdata.2016.35) PMC4878278 27219127 · DOI · Paper page
  46. [69]Sessler CN, Gosnell MS, Grap MJ, Brophy GM, O’Neal PV, Keane KA, Tesoro EP, Elswick RK. 2002 The Richmond Agitation–Sedation Scale: validity and reliability in adult intensive care unit patients. Am. J. Respir. Crit. Care Med. 166, 1338- 1344. (10.1164/rccm.2107138) 12421743 · DOI · Paper page
  47. [70]Mehrabi N, Morstatter F, Saxena N, Lerman K, Galstyan A. 2019 A survey on bias and fairness in machine learning. http://arxiv.org/abs/190809635.
  48. [71]Eubanks VA. 2018 Automating inequality: how high-tech tools profile, police, and punish the poor. New York, NY: St. Martin’s Press.
  49. [73]Habli I, Lawton T, Porter Z. 2020 Artificial intelligence in health care: accountability and safety. Bull. World Health Organ. 98, 251. (10.2471/BLT.19.237487) 32284648 PMC7133468 · DOI · Paper page
  50. [74]Sullivan E. 2019 Understanding from machine learning models. Br. J. Philos. Sci. (10.1093/bjps/axz035) · DOI
  51. [75]Kelly TP. Arguing safety: a systematic approach to managing safety cases. PhD thesis, University of York.
  52. [76]Burton S, Gauerhof L, Heinzemann C. 2017 Making the case for safety of machine learning in highly automated driving. In Int. Conf. on Computer Safety, Reliability, and Security, Trento, Italy, 12–15 September 2017, pp. 5–16. Berlin, Germany: Springer.
  53. [77]Picardi C, Hawkins R, Paterson C, Habli I. 2019 A pattern for arguing the assurance of machine learning in medical diagnosis systems. In Int. Conf. on Computer Safety, Reliability, and Security, pp. 165–179. Berlin, Germany: Springer.
  54. [78]Denney E, Pai G, Habli I. 2015 Dynamic safety cases for through-life safety assurance. In 2015 IEEE/ACM 37th IEEE Int. Conf. on Software Engineering, Florence, Italy, 16–24 May 2015, vol. 2, pp. 587–590. New York, NY: IEEE.
  55. [79]Asaadi E, Denney E, Menzies J, Pai GJ, Petroff D. 2020 Dynamic assurance cases: a pathway to trusted autonomy. Computer 53, 35- 46. (10.1109/MC.2020.3022030) · DOI

Cited by

Sorted by citations

Authors
McDermid, J. A., Jia, Y., Porter, Z., & Habli, I.
Affiliations
  • University of York (GB)
  • Department of Computer Science, University of York, Deramore Lane, York YO10 5GH, UK
Corresponding author
McDermid, John A.
ORCID
Venue
Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
Publisher
The Royal Society
Published
2021-08-16
Volume
379
Issue
2207
Pages
20200363
ISSN
  • 1364-503X (ISSN-L)
  • 1471-2962
DOI
10.1098/rsta.2020.0363
PMC
PMC8366909
OpenAlex
W3195947859
Cited by
136
References
15
Citations by year
2026: 28 · 2025: 36 · 2024: 32 · 2023: 25 · 2022: 14 · 2021: 1
FWCI
6.36
Type
journal article
Language
en
Open access
Licence
Locations
  • Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
Funders
  • Assuring Autonomy International Programme
  • University of York
  • Lloyds Register Foundation
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
Keywords
explainable AI, model transparency, model interpretability, AI ethics, accountability, AI regulation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review