CoChat
Sign Up
PreprintCited by 10Explainable Artificial Intelligence (XAI)Ethics and Social Impacts of AIDigitalization, Law, and Regulation

The DOI matches its DataCite record; its reference list is not available.

Flexible and Context-Specific AI Explainability: A Multidisciplinary Approach

Open access
Beaudouin, Valérie, Bloch, Isabelle, Bounie, David, Clémençon, Stéphan, d'Alché-Buc, Florence, Eagan, James, Maxwell, Winston, Mozharovskyi, Pavlo, Parekh, Jayneel·2020·arXiv
DOI 10.48550/arxiv.2003.07703arXiv 2003.07703
Open this paper in CoChat — free account

Abstract

The recent enthusiasm for artificial intelligence (AI) is due principally to advances in deep learning. Deep learning methods are remarkably accurate, but also opaque, which limits their potential use in safety-critical applications. To achieve trust and accountability, designers and operators of machine learning algorithms must be able to explain the inner workings, the results and the causes of failures of algorithms to users, regulators, and citizens. The originality of this paper is to combine technical, legal and economic aspects of explainability to develop a framework for defining the "right" level of explain-ability in a given context. We propose three logical steps: First, define the main contextual factors, such as who the audience of the explanation is, the operational context, the level of harm that the system could cause, and the legal/regulatory framework. This step will help characterize the operational and legal needs for explanation, and the corresponding social benefits. Second, examine the technical tools available, including post hoc approaches (input perturbation, saliency maps...) and hybrid AI approaches. Third, as function of the first two steps, choose the right levels of global and local explanation outputs, taking into the account the costs involved. We identify seven kinds of costs and emphasize that explanations are socially useful only when total social benefits exceed costs.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2003.07703

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The second case involves the COMPAS predictive justice algorithm. The Wisconsin Supreme court did not require the release of the source code of the COMPAS algorithm but it enshrined an overall explainability requirement to allow judges to better assess the accuracy and weight to be given to the risk score in sentencing.

    “The second case involves the COMPAS predictive justice algorithm. If the Wisconsin Supreme court 13 did not require the release of the source code of the COMPAS algorithm it enshrined an overall explainability requirement to allow judges to better assess the accuracy and weight to be given to the risk score in sentencing (see [8] and [9]).”

    From Fairness and explainability in automatic decision-making systems. A challenge for computer science and law (2023) · cites this paper as [8]

  • For some authors, 'explainability' and 'interpretability' are interchangeable notions, while others make a distinction.

    “For some authors, 'explainability' and 'interpretability' are interchangeable notions ([8], p. 8), while others make a distinction.”

    From Fairness and explainability in automatic decision-making systems. A challenge for computer science and law (2023) · cites this paper as [8]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

Cited by 10

Research works printed but not linked: 47

  1. Al-Shedivat, M., Dubey, A., and Xing, E. P. (2017). Contextual explanation networks. arXiv preprint arXiv:1705.10301 . · arXiv
  2. Alvarez-Melis, D. and Jaakkola, T. S. (2017). A causal framework for explaining the predictions of black-box sequence-to-sequence models. arXiv preprint arXiv:1707.01943 . · arXiv
  3. Babic, B., Gerke, S., Evgeniou, T., and Cohen, I. G. (2019). Algorithms on regulatory lockdown in medicine. Science , 366(6470):1202-1204.
  4. Bang, S., Xie, P., Wu, W., and Xing, E. (2019). Explaining a black-box using deep variational information bottleneck approach. arXiv preprint arXiv:1902.06918 . · arXiv
  5. Bhattacharyya, S., Cofer, D., Musliner, D., Mueller, J., and Engstrom, E. (2015). Certification considerations for adaptive systems. In 2015 International Conference on Unmanned Aircraft Systems (ICUAS) , pages 270-279. IEEE.
  6. Borg, M., Englund, C., Wnuk, K., Duran, B., Levandowski, C., Gao, S., Tan, Y., Kaijser, H., L¨ onn, H., and T¨ ornqvist, J. (2019). Safely entering the deep: A review of verification and validation for machine learning and a challenge elicitation in the automotive industry. Journal of Automotive Software Engineering , 1(1):1-19.
  7. Breiman, L. (2001). Random forests. Machine Learning , 45(1):5-32.
  8. Burrell, J. (2016). How the machine 'thinks': Understanding opacity in machine learning algorithms. Big Data & Society , 3(1):2053951715622512.
  9. Chen, C., Li, O., Tao, D., Barnett, A., Rudin, C., and Su, J. K. (2019). This looks like that: deep learning for interpretable image recognition. In Advances in Neural Information Processing Systems , pages 8928-8939.
  10. Clancey, W. J. (1983). The epistemology of a rule-based expert system-a framework for explanation. Artificial intelligence , 20(3):215-251.
  11. Coglianese, C. and Lehr, D. (2018). Transparency and algorithmic governance. Administrative Law Review , 71:1.
  12. Cortes, C. and Vapnik, V. (1995). Support-vector networks. Machine learning , 20(3):273-297.
  13. Doshi-Velez, F., Kortz, M., Budish, R., Bavitz, C., Gershman, S., O'Brien, D., Schieber, S., Waldo, J., Weinberger, D., and Wood, A. (2017). Accountability of ai under the law: The role of explanation. arXiv preprint arXiv:1711.01134 . d'Alch´ e Buc, F., Andr´ es, V., and Nadal, J.-P. (1994). Rule extraction with fuzzy neural network. International journal of neural systems , 5(01):1-11. · arXiv
  14. Ernest, N., Carroll, D., Schumacher, C., Clark, M., Cohen, K., and Lee, G. (2016). Genetic fuzzy based artificial intelligence for unmanned combat aerial vehicle control in simulated air combat missions. Journal of Defense Management , 6(1):21670374.
  15. Felici, M., Koulouris, T., and Pearson, S. (2013). Accountability for data governance in cloud ecosystems. In 2013 IEEE 5th International Conference on Cloud Computing Technology and Science , volume 2, pages 327-332. IEEE.
  16. Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., and Pedreschi, D. (2018). A survey of methods for explaining black box models. ACM computing surveys (CSUR) , 51(5):93.
  17. IEEE (2019). Ethically aligned design: A vision for prioritizing human well-being with autonomous and intelligent systems. IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems .
  18. Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., and Sayres, R. (2017). Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). arXiv preprint arXiv:1711.11279 . · arXiv
  19. Kurd, Z. and Kelly, T. (2003). Safety lifecycle for developing safety critical artificial neural networks. In Anderson, S., Felici, M., and Littlewood, B., editors, Computer Safety, Reliability, and Security , pages 77-91, Berlin, Heidelberg. Springer Berlin Heidelberg.
  20. Lee, G.-H., Alvarez-Melis, D., and Jaakkola, T. S. (2019a). Towards robust, locally linear deep networks. In International Conference on Learning Representations .
  21. Lee, G.-H., Jin, W., Alvarez-Melis, D., and Jaakkola, T. S. (2019b). Functional transparency for structured data: a game-theoretic approach. arXiv preprint arXiv:1902.09737 . · arXiv
  22. Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems , pages 47654774.
  23. Mackay, W. E. (1990). Users and Customizable Software: A Co-Adaptive Phenomenon . PhD thesis, Massechusetts Institute of Technology.
  24. Mascarilla, L. (1997). Fuzzy Rules Extraction and Redunduncy Elimination: an Application to Remote Sensing Image Analysis. International Journal of Intelligent Systems , 12(11/12):793-818.
  25. Melis, D. A. and Jaakkola, T. (2018). Towards robust interpretability with selfexplaining neural networks. In Advances in Neural Information Processing Systems , pages 7775-7784.
  26. Minsky, M. and Papert, S. (1969). Perceptrons . M.I.T. Press.
  27. Peterson, G. E. (1993). Foundation for neural network verification and validation. In Science of Artificial Neural Networks II , volume 1966, pages 196-207. International Society for Optics and Photonics.
  28. Ribeiro, M. T., Singh, S., and Guestrin, C. (2016a). Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining , pages 1135-1144. ACM.
  29. Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986). Learning representations by back-propagating errors. nature , 323(6088):533-536.
  30. Schapire, R. E. (1990). The strength of weak learnability. Machine learning , 5(2):197-227.
  31. Schauer, F. (1995). Giving reasons. Stanford Law Review , pages 633-659.
  32. Selbst, A. and Barocas, S. (2018). The intuitive appeal of explainable machines. SSRN Electronic Journal , 87.
  33. Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision , pages 618-626.
  34. Shortliffe, E. H. and Buchanan, B. G. (1975). A model of inexact reasoning in medicine. Mathematical Biosciences , 23(3):351 - 379.
  35. Simonyan, K., Vedaldi, A., and Zisserman, A. (2013). Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 . · arXiv
  36. Smilkov, D., Thorat, N., Kim, B., Vi´ egas, F., and Wattenberg, M. (2017). Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825 . · arXiv
  37. Springenberg, J. T., Dosovitskiy, A., Brox, T., and Riedmiller, M. (2014). Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806 . · arXiv
  38. Stone, C. J. (1977). Consistent nonparametric regression. The Annals of Statistics , pages 595-620.
  39. Sundararajan, M., Taly, A., and Yan, Q. (2017). Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine LearningVolume 70 , pages 3319-3328. JMLR. org.
  40. Swartout, W., Paris, C., and Moore, J. (1991). Explanations in knowledge systems: Design for explainable expert systems. IEEE Expert , 6(3):58-64.
  41. Thomas, P. S., Castro da Silva, B., Barto, A. G., Giguere, S., Brun, Y., and Brunskill, E. (2019). Preventing undesirable behavior of intelligent machines. Science , 366(6468):999-1004.
  42. Tishby, N., Pereira, F. C., and Bialek, W. (2000). The information bottleneck method. arXiv preprint physics/0004057 . · arXiv
  43. Towell, G. G. and Shavlik, J. W. (1994). Knowledge-based artificial neural networks. Artificial intelligence , 70(1-2):119-165.
  44. Winfield, A. F. and Jirotka, M. (2017). The case for an ethical black box. In Annual Conference Towards Autonomous Robotic Systems , pages 262-273. Springer.
  45. Wortham, R. H., Theodorou, A., and Bryson, J. J. (2016). What does the robot think? transparency as a fundamental design requirement for intelligent systems. In Ijcai-2016 ethics for artificial intelligence workshop .
  46. Yeung, K., Howes, A., and Pogrebna, G. (2019). Ai governance by human rightscentred design, deliberation and oversight: An end to ethics washing. The Oxford Handbook of AI Ethics, Oxford University Press (2019) .
  47. Zhang, Q., Nian Wu, Y., and Zhu, S.-C. (2018). Interpretable convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 8827-8836.

Metadata

Authors
Beaudouin, V., Bloch, I., Bounie, D., Clémençon, S., d'Alché-Buc, F., Eagan, J., Maxwell, W., Mozharovskyi, P., & Parekh, J.
Affiliations
  • Institut Polytechnique de Paris (FR)
  • IMAGES
  • Laboratoire de Probabilités et Modèles Aléatoires (FR)
  • DIVA
  • Institut de recherche mathématique de Rennes (FR)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
DOI
10.48550/arxiv.2003.07703
arXiv
2003.07703
OpenAlex
W4287827273
Cited by
10
References
0
Citations by year
2024: 1 · 2023: 6 · 2022: 1 · 2021: 1 · 2020: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • HAL (submittedVersion, other-oa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Law › Digitalization, Law, and Regulation
Keywords
explainable AI, post-hoc explanation, saliency maps, input perturbation, hybrid AI, legal and regulatory requirements, local and global explanations, trust in machine learning, safety-critical systems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review