CoChat
Sign Up
PreprintCited by 5,338Explainable Artificial Intelligence (XAI)Ethics and Social Impacts of AIPhilosophy and History of Science

Explanation in Artificial Intelligence: Insights from the Social Sciences

Open access
Miller, Tim·2017·arXiv
DOI 10.48550/arxiv.1706.07269arXiv 1706.07269
Open this paper in CoChat — free accountOpen PDF

Abstract

There has been a recent resurgence in the area of explainable artificial intelligence as researchers and practitioners seek to make their algorithms more understandable. Much of this research is focused on explicitly explaining decisions or actions to a human observer, and it should not be controversial to say that looking at how humans explain to each other can serve as a useful starting point for explanation in artificial intelligence. However, it is fair to say that most work in explainable artificial intelligence uses only the researchers' intuition of what constitutes a `good' explanation. There exists vast and valuable bodies of research in philosophy, psychology, and cognitive science of how people define, generate, select, evaluate, and present explanations, which argues that people employ certain cognitive biases and social expectations towards the explanation process. This paper argues that the field of explainable artificial intelligence should build on this existing research, and reviews relevant papers from philosophy, cognitive psychology/science, and social psychology, which study these topics. It draws out some important findings, and discusses ways that these can be infused with work on explainable artificial intelligence.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1706.07269

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Greater transparency potentially increases end user control and improves acceptance of complex algorithmic systems.

    “Greater transparency potentially increases end user control and improves acceptance of complex algorithmic systems [31,44,57,85].”

    From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [57]

    The Introduction says that more transparent or explainable systems may help users understand and trust intelligent agents, and links limited uptake to a lack of trust. It does not say transparency increases end-user control. Section 4.6.5 discusses an agent’s control over its own actions, not a user’s control over a system. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 86

  1. D. Allemang, M.C. Tanner, T. Bylander, J.R. Josephson, Computational complexity of hypothesis assembly, in: IJCAI, vol. 87, 1987, pp. 1112-1117.
  2. A. Arioua, M. Croitoru, Formalizing explanatory dialogues, in: International Conference on Scalable Uncertainty Management, Springer, 2015, pp. 282-297.
  3. E. Bekele, W.E. Lawson, Z. Horne, S. Khemlani, Human-level explanatory biases for person re-identification, in: HRI Workshop on Explainable Robotic Systems, 2018.
  4. P. Besnard, A. Hunter, Elements of Argumentation, vol. 47, MIT Press, Cambridge, 2008.
  5. O. Biran, C. Cotton, Explanation and justification in machine learning: a survey, in: IJCAI 2017 Workshop on Explainable Artificial Intelligence (XAI), 2017, pp. 8-13.
  6. R.I. Brafman, C. Domshlak, From one to many: planning for loosely coupled multi-agent systems, in: International Conference on Automated Planning and Scheduling, 2008, pp. 28-35.
  7. J. Broekens, M. Harbers, K. Hindriks, K. Van Den Bosch, C. Jonker, J.-J. Meyer, Do you get it? User-evaluated explainable BDI agents, in: German Conference on Multiagent System Technologies, Springer, 2010, pp. 28-39.
  8. S. Bromberger, Why-questions, in: R.G. Colodny (Ed.), Mind and Cosmos: Essays in Contemporary Science and Philosophy, Pittsburgh University Press, Pittsburgh, 1966, pp. 68-111.
  9. B. Buchanan, E. Shortliffe, Rule-based Expert Systems: the MYCIN Experiments the Stanford Heuristic Programming Project, Addison-Wesley, 1984.
  10. A. Burguet, D. Hilton, Effets de contexte sur l'explication causale, in: M. Bromberg, A. Trognon (Eds.), Psychologie Sociale et Communication, Dunod, Paris, 2004, pp. 219-228.
  11. R.M. Byrne, The construction of explanations, in: AI and Cognitive Science'90, Springer, 1991, pp. 337-351.
  12. A. Cawsey, Generating interactive explanations, in: AAAI, 1991, pp. 86-91.
  13. A. Cawsey, Explanation and Interaction: The Computer Generation of Explanatory Dialogues, MIT Press, 1992.
  14. T. Chakraborti, S. Sreedharan, Y. Zhang, S. Kambhampati, Plan explanations as model reconciliation: moving beyond explanation as soliloquy, in: Proceedings of IJCAI, 2017, https://www.ijcai .org /proceedings /2017 /0023 .pdf.
  15. K. Chan, T.-W. Lee, P.A. Sample, M.H. Goldbaum, R.N. Weinreb, T.J. Sejnowski, Comparison of machine learning and traditional classifiers in glaucoma diagnosis, IEEE Trans. Biomed. Eng. 49 (9) (2002) 963-974.
  16. B. Chandrasekaran, M.C. Tanner, J.R. Josephson, Explaining control strategies in problem solving, IEEE Expert 4 (1) (1989) 9-15.
  17. E. Charniak, R. Goldman, A probabilistic model of plan recognition, in: Proceedings of the Ninth National Conference on Artificial Intelligence-Volume 1, AAAI Press, 1991, pp. 160-165.
  18. Y. Chevaleyre, U. Endriss, J. Lang, N. Maudet, A short introduction to computational social choice, in: International Conference on Current Trends in Theory and Practice of Computer Science, Springer, 2007, pp. 51-69.
  19. M.M. de Graaf, B.F. Malle, How people explain action (and autonomous intelligent systems should too), in: AAAI Fall Symposium on Artificial Intelli- gence for Human-Robot Interaction, 2017.
  20. D.C. Dennett, The Intentional Stance, MIT Press, 1989.
  21. F. Dignum, R. Prada, G.J. Hofstede, From autistic to social agents, in: Proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems, IFAAMAS, 2014, pp. 1161-1164.
  22. R. Fagin, J. Halpern, Y. Moses, M. Vardi, Reasoning About Knowledge, Vol. 4, MIT Press, Cambridge, 1995.
  23. J. Fox, D. Glasspool, D. Grecu, S. Modgil, M. South, V. Patkar, Argumentation-based inference and decision making-a medical perspective, IEEE Intell. Syst. 22 (6) (2007) 34-41.
  24. M. Fox, D. Long, D. Magazzeni, Explainable planning, in: IJCAI 2017 Workshop on Explainable Artificial Intelligence (XAI), 2017, https://arxiv.org /pdf / 1709 .10256.
  25. N. Frosst, G. Hinton, Distilling a neural network into a soft decision tree, arXiv e-prints 1711.09784, https://arxiv.org /abs /1711.09784. · arXiv
  26. M. Ghallab, D. Nau, P. Traverso, Automated Planning: Theory and Practice, Elsevier, 2004.
  27. M. Greaves, H. Holmback, J. Bradshaw, What is a conversation policy? in: Issues in Agent Communication, Springer, 2000, pp. 118-131.
  28. H.P. Grice, Logic and conversation, in: Syntax and Semantics 3: Speech Arts, Academic Press, New York, 1975, pp. 41-58.
  29. R.J. Hankinson, Cause and Explanation in Ancient Greek Thought, Oxford University Press, 2001.
  30. M. Harradon, J. Druce, B. Ruttenberg, Causal learning and explanation of deep neural networks via autoencoded activations, arXiv e-prints 1802.00541, https://arxiv.org /abs /1802 .00541. · arXiv
  31. H.L.A. Hart, T. Honoré, Causation in the Law, OUP, Oxford, 1985.
  32. B. Hayes, J.A. Shah, Improving robot controller transparency through autonomous policy explanation, in: Proceedings of the 12th ACM/IEEE Interna- tional Conference on Human-Robot Interaction (HRI 2017), p. 2017.
  33. F. Heider, The Psychology of Interpersonal Relations, Wiley, New York, 1958.
  34. G. Hesslow, The problem of causal selection, in: Contemporary Science and Natural Explanation: Commonsense Conceptions of Causality, 1988, pp. 11-32.
  35. D. Hilton, Social attribution and explanation, in: Oxford Handbook of Causal Reasoning, Oxford University Press, 2017, pp. 645-676.
  36. D.J. Hilton, Logic and causal attribution, in: Contemporary Science and Natural Explanation: Commonsense Conceptions of Causality, New York Uni- versity Press, 1988, pp. 33-65.
  37. D.J. Hilton, J. McClure, B. Slugoski, Counterfactuals, conditionals and causality: a social psychological perspective, in: D.R. Mande, D.J. Hilton, P. Catellani (Eds.), The Psychology of Counterfactual Thinking, Routledge, London, 2005, pp. 44-60.
  38. D.J. Hilton, J.L. McClure, R. Ben Slugoski, The course of events: counterfactuals, causal sequences and explanation, in: D.R. Mandel, D.J. Hilton, P. Catellani (Eds.), The Psychology of Counterfactual Thinking, Routledge, 2005.
  39. R.R. Hoffman, G. Klein, Explaining explanation, part 1: theoretical foundations, IEEE Intell. Syst. 32 (3) (2017) 68-73.
  40. D. Hume, An Enquiry Concerning Human Understanding: A Critical Edition, vol. 3, Oxford University Press, 2000.
  41. J.M. Jaspars, D.J. Hilton, Mental models of causal reasoning, in: The Social Psychology of Knowledge, Cambridge University Press, 1988, pp. 335-358.
  42. J.R. Josephson, S.G. Josephson, Abductive Inference: Computation, Philosophy, Technology, Cambridge University Press, 1996.
  43. D. Kahneman, A. Tversky, The simulation heuristic, in: P.S.D. Kahneman, A. Tversky (Eds.), Judgment Under Uncertainty: Heuristics and Biases, Cam- bridge University Press, New York, 1982.
  44. H.H. Kelley, Attribution Theory in Social Psychology, in: Nebraska Symposium on Motivation, University of Nebraska Press, 1967, pp. 192-238.
  45. H.H. Kelley, Causal Schemata and the Attribution Process, General Learning Press, Morristown, NJ, 1972.
  46. T. Kulesza, M. Burnett, W.-K. Wong, S. Stumpf, Principles of explanatory debugging to personalize interactive machine learning, in: Proceedings of the 20th International Conference on Intelligent User Interfaces, ACM, 2015, pp. 126-137.
  47. T. Kulesza, S. Stumpf, M. Burnett, S. Yang, I. Kwan, W.-K. Wong, Too much, too little, or just right? Ways explanations impact end users' mental models, in: 2013 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), IEEE, 2013, pp. 3-10.
  48. T. Kulesza, S. Stumpf, W.-K. Wong, M.M. Burnett, S. Perona, A. Ko, I. Oberst, Why-oriented end-user debugging of naive Bayes text classification, ACM Trans. Interact. Intell. Syst. (TiiS) 1 (1) (2011) 2.
  49. P. Langley, B. Meadows, M. Sridharan, D. Choi, Explainable agency for intelligent autonomous systems, in: Proceedings of the Twenty-Ninth Annual Conference on Innovative Applications of Artificial Intelligence, AAAI Press, 2017.
  50. H.J. Levesque, A knowledge-level account of abduction, in: IJCAI, 1989, pp. 1061-1067.
  51. B.Y. Lim, A.K. Dey, Assessing demand for intelligibility in context-aware applications, in: Proceedings of the 11th International Conference on Ubiqui- tous Computing, ACM, 2009, pp. 195-204.
  52. M.P. Linegang, H.A. Stoner, M.J. Patterson, B.D. Seppelt, J.D. Hoffman, Z.B. Crittendon, J.D. Lee, Human-automation collaboration in dynamic mission planning: a challenge requiring an ecological approach, Proc. Human Factors Ergonom. Soc. Annual Meeting 50 (23) (2006) 2482-2486.
  53. Z.C. Lipton, The mythos of model interpretability, arXiv preprint arXiv:1606 .03490.
  54. T. Lombrozo, Explanation and abductive inference, in: Oxford Handbook of Thinking and Reasoning, 2012, pp. 260-276.
  55. J.L. Mackie, The Cement of the Universe, Oxford, 1980.
  56. B.F. Malle, How the Mind Explains Behavior: Folk Explanations, Meaning, and Social Interaction, MIT Press, 2004.
  57. B.F. Malle, Attribution theories: how people make sense of behavior, in: Theories in Social Psychology, 2011, pp. 72-95.
  58. J.L. McClure, R.M. Sutton, D.J. Hilton, The role of goal-based explanations, in: Social Judgments: Implicit and Explicit Processes, vol. 5, Cambridge University Press, 2003, p. 306.
  59. T. Miller, P. Howe, L. Sonenberg, Explainable AI: beware of inmates running the asylum, in: IJCAI 2017 Workshop on Explainable Artificial Intelligence (XAI), 2017, pp. 36-42, http://people .eng .unimelb .edu .au /tmiller /pubs /explanation -inmates .pdf.
  60. C. Muise, V. Belle, P. Felli, S. McIlraith, T. Miller, A.R. Pearce, L. Sonenberg, Planning over multi-agent epistemic states: a classical planning approach, in: B. Bonet, S. Koenig (Eds.), Proceedings of AAAI 2015, 2015, pp. 1-8.
  61. J. Overton, Scientific explanation and computation, in: D.B.L. Thomas Roth-Berghofer, Nava Tintarev (Eds.), Proceedings of the 6th International Explanation-Aware Computing (ExaCt) Workshop, 2011, pp. 41-50.
  62. J.A. Overton, Explanation in Science, Ph.D. thesis, The University of Western, Ontario, 2012.
  63. C.S. Peirce, Harvard lectures on pragmatism, in: Collected Papers, vol. 5, 1903.
  64. R. Petrick, M.E. Foster, Using general-purpose planning for action selection in human-robot interaction, in: AAAI 2016 Fall Symposium on Artificial Intelligence for Human-Robot Interaction, 2016.
  65. D. Poole, Normality and faults in logic-based diagnosis, in: IJCAI, vol. 89, 1989, pp. 1304-1310.
  66. H.E. Pople, On the mechanization of abductive logic, in: IJCAI, vol. 73, 1973, pp. 147-152.
  67. K. Popper, The Logic of Scientific Discovery, Routledge, 2005.
  68. M. Ranney, P. Thagard, Explanatory coherence and belief revision in naive physics, in: Proceedings of the Tenth Annual Conference of the Cognitive Science Society, 1988, pp. 426-432.
  69. A.S. Rao, M.P. Georgeff, BDI agents: from theory to practice, in: ICMAS, vol. 95, 1995, pp. 312-319.
  70. M.T. Ribeiro, S. Singh, C. Guestrin, Why should I trust you?: explaining the predictions of any classifier, in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2016, pp. 1135-1144.
  71. M. Robnik-Šikonja, I. Kononenko, Explaining classifications for individual instances, IEEE Trans. Knowl. Data Eng. 20 (5) (2008) 589-600.
  72. W.C. Salmon, Four Decades of Scientific Explanation, University of Pittsburgh Press, 2006.
  73. J. Samland, M.R. Waldmann, Do social norms influence causal inferences? in: P. Bello, M. Guarini, M. McShane, B. Scassellati (Eds.), Proceedings of the 36th Annual Conference of the Cognitive Science Society, Cognitive Science Society, 2014, pp. 1359-1364.
  74. M. Scriven, The concept of comprehension: from semantics to software, in: J.B. Carroll, R.O. Freedle (Eds.), Language Comprehension and the Acquisi- tion of Knowledge, W. H. Winston & Sons, Washington, 1972, pp. 31-39.
  75. Z. Shams, M. de Vos, N. Oren, J. Padget, Normative practical reasoning via argumentation and dialogue, in: Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI-16), AAAI Press, 2016.
  76. R. Singh, T. Miller, J. Newn, L. Sonenberg, E. Velloso, F. Vetere, Combining planning with gaze for online human intention recognition, in: Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems, 2018.
  77. K. Stubbs, P. Hinds, D. Wettergreen, Autonomy and common ground in human-robot interaction: a field study, IEEE Intell. Syst. 22 (2) (2007) 42-50.
  78. W.R. Swartout, J.D. Moore, Explanation in second generation expert systems, in: Second Generation Expert Systems, Springer, 1993, pp. 543-585.
  79. N. Vasilyeva, D.A. Wilkenfeld, T. Lombrozo, Goals affect the perceived quality of explanations, in: D.C. Noelle, R. Dale, A.S. Warlaumont, J. Yoshimi, T. Matlock, C.D. Jennings, P.P. Maglio (Eds.), Proceedings of the 37th Annual Conference of the Cognitive Science Society, Cognitive Science Society, 2015, pp. 2469-2474.
  80. F.B. von der Osten, M. Kirley, T. Miller, The minds of many: opponent modelling in a stochastic game, in: Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI), AAAI Press, 2017, pp. 3845-3851.
  81. G.H. Von Wright, Explanation and Understanding, Cornell University Press, 1971.
  82. D. Walton, Dialogical models of explanation, in: Proceedings of the International Explanation-Aware Computing (ExaCt) Workshop, 2007, pp. 1-9.
  83. D.N. Walton, Logical Dialogue -Games and Fallacies, University Press of America, Lanham, Maryland, 1984.
  84. D.S. Weld, G. Bansal, Intelligible Artificial Intelligence, arXiv e-prints, arXiv:1803 .04263, https://arxiv.org /pdf /1803 .04263 .pdf.
  85. M. Winikoff, Debugging agent programs with why?: questions, in: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Sys- tems, AAMAS '17, IFAAMAS, 2017, pp. 251-259.
  86. J. Woodward, Making Things Happen: A Theory of Causal Explanation, Oxford University Press, 2005.

Cited by

Sorted by citations

Authors
Miller, T.
Affiliations
The University of Melbourne (AU)
Corresponding author
Miller, Tim
ORCID
Miller, Tim
Venue
arXiv
Publisher
arXiv
Published
2017
Volume
267
Pages
1-38
ISSN
0004-3702 (ISSN-L)
DOI
10.48550/arxiv.1706.07269
arXiv
1706.07269
OpenAlex
W2670253439
Cited by
5,338
References
0
Citations by year
2026: 863 · 2025: 855 · 2024: 768 · 2023: 792 · 2022: 674 · 2021: 693 · 2020: 420 · 2019: 214 · 2018: 52 · 2017: 4 · 2012: 2
FWCI
191.50
Type
journal article
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • Artificial Intelligence (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Université de Toulouse
  • State Government of Victoria
  • Defence Science Institute
  • Australian Research Council (DP160104083)
  • Defence Science and Technology Group
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy and History of Science
Keywords
explainable AI, human explanations, cognitive biases, social expectations, explanation evaluation, explanation generation, philosophy of explanation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review