Scientific Exploration and Explainable Artificial Intelligence
Open accessAbstract
Models developed using machine learning are increasingly prevalent in scientific research. At the same time, these models are notoriously opaque. Explainable AI aims to mitigate the impact of opacity by rendering opaque models transparent. More than being just the solution to a problem, however, Explainable AI can also play an invaluable role in scientific exploration. This paper describes how post-hoc analytic techniques from Explainable AI can be used to refine target phenomena in medical science, to identify starting points for future investigations of (potentially) causal relationships, and to generate possible explanations of target phenomena in cognitive science. In this way, this paper describes how Explainable AI—over and above machine learning itself—contributes to the efficiency and scope of data-driven scientific research.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
XAI is unlikely to fully resolve these epistemic difficulties in all cases and is therefore unlikely (for now at least) to fully minimise the risk of hermeneutic harm.
“We have suggested that XAI-for all its other social, ethical, or scientific benefits (Zednik & Boelsen, 2022)-is unlikely to fully resolve these epistemic difficulties in all cases and is therefore unlikely (for now at least) to fully minimise the risk of hermeneutic harm.”
From Hidden Risks: Artificial Intelligence and Hermeneutic Harm (2025) · cites this paper as Zednik & Boelsen, 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 2× · see citing works
“Detailed analyses have revealed that there are in fact many different sources (Burrell, 2016) and kinds of opacity (Zednik, 2019), and that different stakeholders are impacted by opacity in different ways (Tomsett et al., 2018).”
BackgroundEuropean Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · see citing works
Miotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016) · Scientific Reports · doi:10.1038/srep26094 · Cited by 1,710 · see citing works
Cichy, R. M., Khosla, A., Pantazis, D., Torralba, A., & Oliva, A. (2016) · Scientific Reports · doi:10.1038/srep27755 · Cited by 716 · see citing works
Solving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence
Zednik, C. (2021) · Philosophy & Technology · doi:10.1007/s13347-019-00382-7 · Cited by 419 · see citing works
Self-citationThe philosophical novelty of computer simulation methods
Humphreys, P. (2009) · Synthese · doi:10.1007/s11229-008-9435-2 · Cited by 362 · see citing works
Beyond Sparsity: Tree Regularization of Deep Models for Interpretability
Wu, M., Hughes, M., Parbhoo, S., Zazzi, M., Roth, V., & Doshi-Velez, F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11501 · Cited by 213 · see citing works
Flagged first, then by citations
Research works printed but not linked: 34
- Bau, D., Zhou, B., Khosla, A., Oliva, A., & Torralba, A. (2017). Network dissection: Quantifying inter- pretability of deep visual representations. IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), 3319-3327.
- Beisbart, C. (2021). Opacity thought through: On the intransparency of computer simulations. Synthese. https:// doi. org/ 10. 1007/ s11229-021-03305-2 · DOI
- Boon, M. (2020). "How Scientists are Brought Back into Science: The Error of Empiricism". In: Ber- tolaso M., Sterpetti F. (eds.) A Critical Reflection on Automated Science. Human Perspectives in Health Sciences and Technology, vol 1. Springer, Cham.
- Bühlmann, P. (2013). Causal statistical inference in high dimensions. Mathematical Methods in Opera- tions Research, 77(3), 357-370.
- Cichy, R. M., & Kaiser, D. (2019). Deep neural networks as scientific models. Trends in Cognitive Sci- ences, 23(4), 305-317.
- Confalonieri, R., Coba, L., Wagner, B., & Besold, T. R. (2021). A historical perspective of explainable Artificial Intelligence. WIREs Data Mining and Knowledge Discovery, 11(1), e1391.
- Craver, C., & Darden, L. (2013). In Search of Mechanisms: Discoveries Across the Life Sciences. Chicago University Press.
- Dattilo, A., Vanderburg, A., Shallue, C. J., Mayo, A. W., Berlind, P., Bieryla, A., Calkins, M. L., Esquerdo, G. A., Everett, M. E., Howell, S. B., Latham, D. W., Scott, N. J., & Yu, L. (2019). Identi- fying exoplanets with deep learning II: Two new super-earths uncovered by a neural network in K2 data. arXiv, 1903.10507. · arXiv
- Doran, D., Schulz, S., & Besold, T. R. (2017). What does explainable AI really mean? A new conceptual- ization of perspectives. arXiv, 1710.00794 · arXiv
- Hohman, F. M., Kahng, M., Pienta, R., & Chau, D. H. (2018). Visual analytics in deep learning: An inter- rogative survey for the next frontiers. IEEE Transactions on Visualization and Computer Graphics.
- Kriegeskorte, N., Mur, M., & Bandettini, P. (2008). Representational similarity analysis -connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2(4), 1-28.
- Li, X., Wu, J., Chen, E. Z., & Jiang, H. (2019). What evidence does deep learning model use to classify skin lesions? arXiv, 1811.01051v2. · arXiv
- Lipton, Z.C. (2016). The mythos of model interpretability. arXiv, 1606.03490v3. · arXiv
- Lundberg, S. M. & Lee, S. (2017). A unified approach to interpreting model predictions. arXiv, 1705.07874v2. · arXiv
- Ma, W., Qiu, Z., Song, J., Li, J., Cheng, Q., Zhai, J., & Ma, C. (2018). A deep convolutional neural net- work approach for predicting phenotypes from genotypes. Planta, 248(5), 1307-1318.
- Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. MIT Press.
- Massimi, M. (2019). Two kinds of exploratory models. Philosophy of Science, 86(5), 869-881.
- Pearl, J. (2000). Causality: Models, Reasoning, and Inference. Cambridge University Press.
- Ratti, E. (2015). Big Data Biology: Between Eliminative Inferences and Exploratory Experiments. Phi- losophy of Science, 82(2), 198-218.
- Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why Should I Trust You?": Explaining the Predictions of Any Classifier. arXiv, 1602.04938v3. · arXiv
- Ritchie, J. B., Kaplan, D.M., & Klein, C. (2019). Decoding the brain: neural representation and the limits of multivariate pattern analysis in cognitive neuroscience. British Journal for the Philosophy of Sci- ence, 70(2, 581-607.
- Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206-215.
- Salmon, W. C. (1989). Four decades of scientific explanation. University of Minnesota Press.
- Shagrir, O. (2006). Why we view the brain as a computer. Synthese, 153(3), 393-416.
- Scientific Exploration and Explainable Artificial… Steinle, F. (1997). Entering New Fields: Exploratory Uses of Experimentation. Philosophy of Science 64 (Proceedings): S64-S74.
- Sullivan, E. (2019). Understanding from machine learning models. The British Journal for the Philoso- phy of Science, axz035.
- Tomsett, R., Braines, D., Harborne, D., Preece, A., & Chakraborty, S. (2018). Interpretable to whom? A role-based model for analyzing interpretable machine learning systems. arXiv, 1806.07552. · arXiv
- Wachter, S., Mittelstadt, B., & Russell, C. (2018). Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law & Technology, 31(2).
- Wu, Y., Ding, Y., Tanaka, Y., & Zhang, W. (2014). Risk factors contributing to type 2 diabetes and recent advances in the treatment and prevention. International Journal of Medical Sciences, 11(11), 1185-1200.
- Zednik, C. (2018). Will machine learning yield machine intelligence? In V. Müller (ed.), Philosophy and Theory of Artificial Intelligence 2017. PT-AI 2017. Studies in Applied Philosophy, Epistemology and Rational Ethics, 44. Springer: Cham.
- Zerilli, J., Knott, A., Maclaurin, J., & Gavaghan, C. (2018). Transparency in algorithmic and human deci- sion-making: Is there a double standard? Philosophy & Technology, 32(4), 661-683.
- Zhavoronkov, A. (2018). Artificial intelligence for drug discovery, biomarker development, and genera- tion of novel chemistry. Molecular Pharmaceutics, 15(10), 4311-4313.
- Zilke, J. R., Mencia, E. L., & Janssen, F. (2016). DeepRED -Rule extraction from deep neural networks. In T. Calders, M. Ceci, & D. Malerba (eds.), Discovery Science 19th International Conference (pp. 457-473).
- Zintgraf, L. M., Cohen, T. S., Adel, T., & Welling, M. (2017). Visualizing deep neural network decisions: Prediction difference analysis. arXiv, 1702.04595. · arXiv
Cited by
Sorted by citations
Hidden Risks: Artificial Intelligence and Hermeneutic Harm
Rebera, A. P., Lauwaert, L., & Oimann, A.-K. (2025) · Minds and Machines · doi:10.1007/s11023-025-09733-0 · Cited by 6 · see citing works
“We have suggested that XAI—for all its other social, ethical, or scientific benefits (Zednik & Boelsen, 2022 )—is unlikely to fully resolve these epistemic difficulties in all cases and is therefore unlikely (for now at least) to fully minimise the risk of hermeneutic harm.”
- Authors
- Zednik, C., & Boelsen, H.
- Affiliations
- Eindhoven University of Technology (NL)
- Otto-von-Guericke-Universität Magdeburg (DE)
- Corresponding author
- Zednik, Carlos
- ORCID
- Zednik, Carlos
- Venue
- Minds and Machines
- Published
- 2022-03
- Volume
- 32
- Issue
- 1
- Pages
- 219-239
- ISSN
- 0924-6495 (ISSN-L)
- 1572-8641
- OpenAlex
- W4220941667
- Cited by
- 80
- References
- 7
- Citations by year
- 2026: 17 · 2025: 22 · 2024: 27 · 2023: 9 · 2022: 5
- FWCI
- 5.18
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Minds and Machines (publishedVersion, cc-by)
- TU/e Research Portal (publishedVersion, other-oa)
- TU/e Research Portal (submittedVersion, other-oa)
- Funders
- Deutsche Forschungsgemeinschaft (ZE-1062/4-1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Philosophy and History of Science
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- explainable AI, scientific exploration, model opacity, post-hoc explanation, causal discovery, medical sciences, cognitive science, data-driven research, scientific explanation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
