CoChat
Sign Up
Cited by 34Explainable Artificial Intelligence (XAI)Ethics and Social Impacts of AIPhilosophy and History of Science

Explainable Artificial Intelligence in Data Science: From Foundational Issues Towards Socio-technical Considerations

Open access
Borrego-Díaz, Joaquín, Galán-Páez, Juan·2022·Minds and Machines
DOI 10.1007/s11023-022-09603-z
Open this paper in CoChat — free accountOpen PDF

Abstract

A widespread need to explain the behavior and outcomes of AI-based systems has emerged, due to their ubiquitous presence. Thus, providing renewed momentum to the relatively new research area of eXplainable AI (XAI). Nowadays, the importance of XAI lies in the fact that the increasing control transference to this kind of system for decision making -or, at least, its use for assisting executive stakeholders- already affects many sensitive realms (as in Politics, Social Sciences, or Law). The decision-making power handover to opaque AI systems makes mandatory explaining those, primarily in application scenarios where the stakeholders are unaware of both the high technology applied and the basic principles governing the technological solutions. The issue should not be reduced to a merely technical problem; the explainer would be compelled to transmit richer knowledge about the system (including its role within the informational ecosystem where he/she works). To achieve such an aim, the explainer could exploit, if necessary, practices from other scientific and humanistic areas. The first aim of the paper is to emphasize and justify the need for a multidisciplinary approach that is beneficiated from part of the scientific and philosophical corpus on Explaining, underscoring the particular nuances of the issue within the field of Data Science. The second objective is to develop some arguments justifying the authors’ bet by a more relevant role of ideas inspired by, on the one hand, formal techniques from Knowledge Representation and Reasoning, and on the other hand, the modeling of human reasoning when facing the explanation. This way, explaining modeling practices would seek a sound balance between the pure technical justification and the explainer-explainee agreement.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s11023-022-09603-z

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • For people who are not AI specialists, it may be better to employ natural language, narrative, explanation by example, visualisations, and so forth.

    “For her, as for other people who are not AI specialists, it may be better to employ natural language, narrative, explanation by example, visualisations, and so forth (Barredo Arrieta et al., 2020;Borrego-Díaz & Galán-Páez, 2022;Lipton, 2018).”

    From Hidden Risks: Artificial Intelligence and Hermeneutic Harm (2025) · cites this paper as Borrego-Díaz & Galán-Páez, 2022

    The Introduction (§1) says explanations should be tailored for laypeople, and §3 recommends narratives, metaphors, and exemplary models; §6.7 also mentions a visual metaphor. These passages support the general advice, but our copy does not specifically recommend natural language as an explanatory format. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Further complications are introduced when AI systems produce outputs in interaction with humans or other AI or complex sociotechnical systems.

    “Moreover, further complications are introduced when AI systems produce outputs in interaction with humans or other AI or complex sociotechnical systems (Borrego-Díaz & Galán-Páez, 2022;Miller, 2019).”

    From Hidden Risks: Artificial Intelligence and Hermeneutic Harm (2025) · cites this paper as Borrego-Díaz & Galán-Páez, 2022

    The Introduction (§1; page not provided) says AI operates within broader sociotechnical systems that include humans, and that explaining its outcomes can become an interactive, sociotechnical challenge. Our copy does not clearly address the added claim that outputs produced through interaction with other AI introduce complications. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 101

  1. AA, V. (2015). The Field Guide to Data Science (2nd ed.). Booz Allen Hamilton.
  2. Addis, T. (2014). Natural and artificial reasoning-an exploration of modelling human thinking. Advanced information and knowledge processing. Springer.
  3. Alonso-Jiménez, J. A., Borrego-Daz, J., Chávez-González, A. M., & Martín-Mateos, F. J. (2006). Foun- dational challenges in automated semantic web data and ontology cleaning. IEEE Intelligent Sys- tems, 21(1), 42-52.
  4. Alrøe, H. F., & Noe, E. (2014). Second-order science of interdisciplinary research: A polyocular frame- work for wicked problems. Constructivist Foundations, 10(1), 65-76.
  5. Anderson, J. D., & Perona, P. (2014). Toward a science of computational ethology. Neuron, 84(1), 18-31.
  6. Aranda-Corral, G. A. & Borrego-Díaz, J. (2010). Mereotopological analysis of formal concepts in secu- rity ontologies. In Herrero, Á., Corchado, E., Redondo, C., & Alonso, Á (Eds.), Computational Intelligence in Security for Information Systems 2010-Proceedings of the 3rd International Conference on Computational Intelligence in Security for Information Systems (CISIS'10), León, Spain, November 11-12, 2010, Vol. 85 of Advances in Intelligent and Soft Computing (pp. 33-40).
  7. Aranda-Corral, G. A., Borrego-Díaz, J., & Galán-Páez, J. (2013a). Qualitative reasoning on complex sys- tems from observations. In Hybrid Artificial Intelligent Systems (pp. 202-211). Springer .
  8. Aranda-Corral, G. A., Borrego-Díaz, J., & Giráldez-Cru, J. (2013b). Agent-mediated shared conceptual- izations in tagging services. Multimedia Tools Applications, 65(1), 5-28.
  9. Aranda-Corral, G. A., Borrego-Díaz, J., & Galán-Páez, J. (2018). Synthetizing qualitative (logical) pat- terns for pedestrian simulation from data. In Bi, Y., Kapoor, S., & Bhatia, R., (Eds.), Proceedings of SAI Intelligent Systems Conference (IntelliSys) 2016 (pp. 243-260). Springer.
  10. Aronson, D. R. (2011). The illusory validity of subjective technical analysis, chapter 2 (pp. 33-101).
  11. Awad, E., Dsouza, S., Bonnefon, J.-F., Shariff, A., & Rahwan, I. (2020). Crowdsourcing moral machines. Communications of ACM, 63(3), 48-55.
  12. Barberis, S. D. (2012). Un análisis crítico de la concepción mecanicista de la explicación. Revista Lati- noamericana de Filosofia, 38(2), 233-265.
  13. Barrowman, N. (2014). Correlation, causation, and confusion. The New Atlantis, 1(43), 23-44.
  14. Bas, C. V. F. (1980). The Scientific Image. Oxford University Press.
  15. Booth, S., Muise, C., & Shah, J. (2019). Evaluating the interpretability of the knowledge compilation map: Communicating logical statements effectively. In Kraus, S., (Eds.), Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019 (pp. 5801-5807).
  16. Borenstein, J., Herkert, J. R., & Miller, K. W. (2019). Self-driving cars and engineering ethics: The need for a system level analysis. Science and Engineering Ethics, 25(2), 383-398.
  17. Borrego-Díaz, J., & Chávez-González, A. M. (2006). Visual ontology cleaning: Cognitive principles and applicability. Lecture Notes in Computer Science. In Y. Sure & J. Domingue (Eds.), The Semantic Web: Research and Applications, 3rd European Semantic Web Conference, ESWC 2006, Budva, Montenegro, June 11-14, 2006, Proceedings (Vol. 4011, pp. 317-331). Springer.
  18. Bradshaw, J. M., Hoffman, R. R., Woods, D. D., & Johnson, M. (2013). The seven deadly myths of auton- omous systems. IEEE Intelligent Systems, 28(3), 54-61.
  19. Cambria, E., Schuller, B., Xia, Y., & Havasi, C. (2013). New avenues in opinion mining and sentiment analysis. IEEE Intelligent Systems, 28(2), 15-21.
  20. Craver, C. (2007). Explaining the Brain: Mechanisms and the Mosaic Unity of Neuroscience. Oxford scholarship online: Philosophy module. Clarendon Press.
  21. Craver, C. F. (2006). When mechanistic models explain. Synthese, 153(3), 355-376.
  22. Darden, L. (2006). Reasoning in biological discoveries: Essays on mechanisms, interfield relations, and anomaly resolution. Cambridge Studies in Philosophy and Biology. Cambridge University Press.
  23. Davis, R., Shrobe, H., & Szolovits, P. (1993). What is a knowledge representation? AI Magazine, 14(1), 17.
  24. de Fine Licht, K., & de Fine Licht, J. (2020). Artificial intelligence, transparency, and public decision- making. AI Society, 35(4), 917-926.
  25. de Regt, H. (2017). Understanding Scientific Understanding. Oxford Studies in Philosophy of Science. Oxford University Press.
  26. Dick, S. (2015). Of models and machines: Implementing bounded rationality. Isis, 106(3), 623-634.
  27. Dimitrijević, D. R. (2019). Causal closure of the physical, mental causation, and physics. European Jour- nal for Philosophy of Science, 10(1), 1.
  28. Doran, D., Schulz, S., & Besold, T. R. (2017). What does explainable AI really mean? A new conceptu- alization of perspectives. In T. R. Besold, & O. Kutz, (Eds.), Proc. First Int. Workshop on Com- prehensibility and Explanation in AI and ML, Volume 2071 of CEUR Workshop Proceedings (pp. 1-8). CEUR-WS.org.
  29. Dudai, Y., & Evers, K. (2014). To simulate or not to simulate: What are the questions? Neuron, 84(2), 254-261.
  30. Findl, J., & Suárez, J. (2021). Descriptive understanding and prediction in Covid-19 modelling. History and Philosophy of the Life Sciences, 43(4), 1-31.
  31. Främling, K. (2020). Decision theory meets explainable AI. In D. Calvaresi, A. Najjar, M. Winikoff, & K. Främling (Eds.), Explainable, transparent autonomous agents and multi-agent systems (pp. 57-74). Springer.
  32. Füllsack, M. (2014). The circular conditions of second-order science sporadically illustrated with agent- based experiments at the roots of observation. Constructivist Foundations, 10(1), 46-54.
  33. Gabbay, D. M., & Woods, J. (2003). Chapter 3-logic as a description of a logical agent. In D. M. Gab- bay & J. Woods (Eds.), Agenda Relevance, Volume 1 of A Practical Logic of Cognitive Systems (pp. 41-68). Elsevier.
  34. Gajdoš, P., & Snášel, V. (2014). A new FCA algorithm enabling analyzing of complex and dynamic data sets. Soft Computing, 18(4), 683-694.
  35. Gerbaudo, P. (2020). From data analytics to data hermeneutics. Online political discussions, digital meth- ods and the continuing rel-evance of interpretative approaches. Digital Culture & Society, 2(2), 95-112.
  36. Gigerenzer, G., & Goldstein, D. G. (1996). Reasoning the fast and frugal way: Models of bounded ration- ality. Psychological Review, 103(4), 650-669.
  37. Gigerenzer, G., Martignon, L., Hoffrage, U., Rieskamp, J., Czerlinski, J., & Goldstein, D. G. (2008). One- reason decision making, Chapter 108, (Vol. 1, pp. 1004-1017). Elsevier.
  38. Gigerenzer, G., & Selten, R. (2002). Bounded rationality: The adaptive toolbox. MIT Press.
  39. Glazer, J., & Rubinstein, A. (2012). A model of persuasion with boundedly rational agents. Journal of Political Economy, 120(6), 1057-1082.
  40. Graham, P. J. (2010). Theorizing justification. In Knowledge and skepticism (pp. 45-71). MIT Press.
  41. Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., & Yang, G.-Z. (2019). Xai-explainable artificial intelligence. Science Robotics, 4(37), 7120.
  42. Guttinger, S. (2020). The limits of replicability. European Journal for Philosophy of Science, 10(2), 10.
  43. Hedström, P., & Ylikoski, P. (2010). Causal mechanisms in the social sciences. Annual Review of Sociol- ogy, 36(1), 49-67.
  44. Hempel, C. (1970). Aspects of scientific explanation: And other essays in the philosophy of science. Number v. 2 in Aspects of Scientific Explanation: And Other Essays in the Philosophy of Science. Free Press.
  45. Hernandez, J., & Ortega, R. (2019). Bounded rationality in decision-making. MOJ Research Review, 2(1), 1-8.
  46. Hinsen, K. (2014). Computational science: Shifting the focus from tools to models. F1000Research, 3(101), 1-15.
  47. Hofman, J., Watts, D. J., Athey, S., Garip, F., Griffiths, T. L., Kleinberg, J., et al. (2021). Integrating explanation and prediction in computational social science. Nature, 595(7866), 181-188.
  48. Huneman, P. (2018). Outlines of a theory of structural explanations. Philosophical Studies, 175(3), 665-702.
  49. Hvoreckỳ, J., Šimúth, J., & Lichardus, B. (2013). Managing rational and not-fully-rational knowledge. Acta Polytechnica Hungarica, 10(2), 121-132.
  50. Ihde, D. (2010). Heidegger's technologies: Postphenomenological perspectives. Fordham University Press.
  51. Janssen, M., & Kuk, G. (2016). Big and open linked data (bold) in research, policy, and practice. Journal of Organizational Computing and Electronic Commerce, 26(1-2), 3-13.
  52. Jarke, J., & Macgilchrist, F. (2021). Dashboard stories: How narratives told by predictive analytics recon- figure roles, risk and sociality in education. Big Data & Society, 8(1), 20539517211025560.
  53. Johnson, N. F., Restrepo, E. M., & Johnson, D. E. (2015). Modeling human conflict and terrorism across geographic scales, Chapter 10 (pp. 209-233). Springer.
  54. Jones, A. J., Artikis, A., & Pitt, J. (2013). The design of intelligent socio-technical systems. Artificial Intelligence Review, 39(1), 5-20.
  55. Kim, J. (2005). Physicalism, or something near enough. Princeton University Press.
  56. King, M. (2020). Explanations and candidate explanations in physics. European Journal for Philosophy of Science, 10(1), 7.
  57. Klein, G., Moon, B., & Hoffman, R. (2006). Making sense of sensemaking 2: A macrocognitive model. IEEE Intelligent Systems, 21, 88-92.
  58. Kliegr, T., Bahník, Štěpán, & Fürnkranz, J. (2021). A review of possible effects of cognitive biases on interpretation of rule-based machine learning models. Artificial Intelligence, 295, 103458.
  59. Koehler, D. (1991). Explanation, imagination, and confidence in judgment. Psychological Bulletin, 110, 499-519.
  60. Korotayev, A. (2014). The Arab spring: A quantitative analysis. Arab Studies Quarterly, 36, 149-169.
  61. Kroes, P., & Verbeek, P. (2014). The moral status of technical artefacts. Philosophy of Engineering and Technology. Springer.
  62. Landes, J. (2020). Variety of evidence and the elimination of hypotheses. European Journal for Philoso- phy of Science, 10(2), 12.
  63. Leonelli, S. (2016). Data-centric biology: A philosophical study. University of Chicago Press.
  64. Lewis, R. L., Howes, A. D., & Singh, S. (2014). Computational rationality: Linking mechanism and behavior through bounded utility maximization. Topics in Cognitive Science, 6(2), 279-311.
  65. Li, J., & Liu, H. (2017). Challenges of feature selection for big data analytics. IEEE Intelligent Systems, 32(2), 9-15.
  66. Lipman, B. L. (1999). Decision theory without logical omniscience: Toward an axiomatic framework for bounded rationality. The Review of Economic Studies, 66(2), 339-361.
  67. Lipton, Z. C. (2018). The mythos of model interpretability: In machine learning, the concept of interpret- ability is both important and slippery. Queue, 16(3), 31-57.
  68. Lombrozo, T. (2007). Simplicity and probability in causal explanation. Cognitive Psychology, 55(3), 232-257.
  69. Longino, H. E. (2006). Theoretical pluralism and the scientific study of behavior, Chapter 6 (Vol. 19, pp. 102-131). University of Minnesota Press, ned-new edition edition.
  70. Lundberg, S. M. & Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Proceed- ings of the 31st International Conference on Neural Information Processing Systems, NIPS'17 (pp. 4768-4777).
  71. Margolis, J. (1983). The logic and structures of fictional narrative. Philosophy and Literature, 7(2), 162-181.
  72. Meier, P. (2015). Digital humanitarians: How big data is changing the face of humanitarian response. CRC Press Inc.
  73. Moreira, C. (2019). Unifying decision-making: A review on evolutionary theories on rationality and cog- nitive biases, , Chapter 19 (pp. 235-248). Springer.
  74. Müller, K. H., & Riegler, A. (2014). Second-order science: A vast and largely unexplored science fron- tier. Constructivist Foundations, 10(1), 7-15.
  75. National Academies of Sciences, E. and Medicine. (2016). In Refining the Concept of Scientific Inference When Working with Big Data: Proceedings of a Workshop-in Brief. The National Academies Press.
  76. Newell, A. (1982). The knowledge level. Artificial Intelligence, 18(1), 87-127.
  77. Nickel, M., Murphy, K., Tresp, V., & Gabrilovich, E. (2016). A review of relational machine learning for knowledge graphs. Proceedings of the IEEE, 104(1), 11-33.
  78. Pachur, T., & Biele, G. (2007). Forecasting from ignorance: The use and usefulness of recognition in lay predictions of sports events. Acta Psychologica, 125(1), 99-116.
  79. Páez, A. (2009). Artificial explanations: The epistemological interpretation of explanation in AI. Syn- these, 170(1), 131-146.
  80. Papineau, D. (2001). The rise of physicalism, Chapter 1 (pp. 3-36).
  81. Pirolli, P. & Card, S. (2005). The sensemaking process and leverage points for analyst technology as iden- tified through cognitive task analysis. In Proceedings of International Conference on Intelligence Analysis (pp. 2-4).
  82. Price, M., Walker, S., & Wiley, W. (2018). The machine beneath: Implications of artificial intelligence in strategic decision making. PRISM, 7(4), 92-105.
  83. Reutlinger, A. (2014). Why is there universal macrobehavior? Renormalization group explanation as non- causal explanation. Philosophy of Science, 81(5), 1157-1170.
  84. Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "why should i trust you?": explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16 (pp. 1135-1144). Association for Computing Machinery.
  85. Ricker, B. (2017). Reflexivity, positionality and rigor in the context of big data research, Chapter 4 (pp. 96-118). University of Iowa Press.
  86. Russell, S. J., & Norvig, P. (2003). Artificial Intelligence: A modern approach (2nd ed.). Pearson Education.
  87. Salmon, W., & Press, P. U. (1984). Scientific explanation and the causal structure of the world. LPE Lim- ited Paperback Editions. Princeton University Press.
  88. Schupbach, J. N. (2019). Conjunctive explanations and inference to the best explanation. Teorema: Revista Internacional de Filosofía, 38(3), 143-162.
  89. Simon, H. (1957a). A behavioural model of rational choice. In H. Simon (Ed.), Models of man: Social and rational; mathematical essays on rational human behavior in a social setting (pp. 241-260).
  90. Slack, D., Hilgard, S., Singh, S., & Lakkaraju, H. (2021). Feature attributions and counterfactual explana- tions can be manipulated. CoRR. 1 3 Explainable Artificial Intelligence in Data Science
  91. Stern, L. (2005). Interpretive reasoning. Cornell University Press.
  92. Subrahmanian, V. S., & Kumar, S. (2017). Predicting human behavior: The next frontiers. Science, 355(6324), 489-489.
  93. Suchman, L. A. (1987). Plans and situated actions: The problem of human-machine communication. Cambridge University Press.
  94. Sullivan, E. (2019). Universality caused: The case of renormalization group explanation. European Jour- nal for Philosophy of Science, 9(3), 36.
  95. Summerfield, C., & Tsetsos, K. (2015). Do humans make good decisions? Trends in Cognitive Sciences, 19(1), 27-34.
  96. Todd, P. M. (2007). How much information do we need? The European Journal of Operational Research, 177(3), 1317-1332.
  97. Townsend, J., Chaton, T., & Monteiro, J. M. (2019). Extracting relational explanations from deep neural networks: a survey from a neural-symbolic perspective. IEEE Transactions on Neural Networks and Learning Systems (pp. 1-15).
  98. Vassiliades, A., Bassiliades, N., & Patkos, T. (2021). Argumentation and explainable artificial intelli- gence: A survey. The Knowledge Engineering Review, 36, e5.
  99. Wang, X., Wang, D., Xu, C., He, X., Cao, Y., & Chua, T. (2019). Explainable reasoning over knowledge graphs for recommendation. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019 (pp. 5329-5336). AAAI Press.
  100. Weld, D. S. & Bansal, G. (2018). Intelligible artificial intelligence. CoRR.
  101. Woodward, J. (2019). Scientific explanation. In Zalta, E. N., (Eds.) The Stanford Encyclopedia of Phi- losophy. Metaphysics Research Lab, Stanford University, winter 2019 edition.

Cited by

Sorted by citations

Authors
Borrego-Díaz, J., & Galán-Páez, J.
ORCID
Borrego-Díaz, Joaquín
Venue
Minds and Machines
Publisher
Springer Science and Business Media LLC
Published
2022-05-12
Volume
32
Issue
3
Pages
485-531
ISSN
  • 0924-6495 (ISSN-L)
  • 1572-8641
DOI
10.1007/s11023-022-09603-z
OpenAlex
W4280566055
Cited by
34
References
12
Citations by year
2026: 6 · 2025: 12 · 2024: 9 · 2023: 5 · 2022: 2
FWCI
2.30
Type
journal article
Language
en
Open access
Licence
Locations
  • Minds and Machines (publishedVersion, cc-by)
  • Deposito de Investigacion Universidad de Sevilla (University of Seville) (submittedVersion, cc-by-nc-nd)
Funders
  • Agencia Estatal de Investigación (PID2019-109152GB-I00/AEI/10.13039/501100011033)
  • Universidad de Sevilla
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy and History of Science
Keywords
explainable AI, explanation, knowledge representation and reasoning, AI decision-making, explainability, philosophy of explanation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review