Transparency and the Black Box Problem: Why We Do Not Trust AI
The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“In contrast, those who valued deep technical immersion expressed greater ambivalence or resistance to AI tools, especially when outputs felt like”
From Redefining the Programmer: Human-AI Collaboration, LLMs, and Security in Modern Software Engineering (2025) · cites this paper as [30]
Experimental — accuracy not yet measuredIn the citing paper
“However, AI based on deep learning algorithms is not transparent, making clinicians uncertain about the signs of diagnosis.”
From Survey of Explainable AI Techniques in Healthcare (2023) · cites this paper as [2]
In the citing paper
“Additionally, generative AI holds significant implications for intellectual property rights.65”
From Balancing innovation and regulation in the age of generative artificial intelligence (2024) · cites this paper as [65]
Citations
Cites
Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor
Lebovits, H. (2019) · Public Integrity · doi:10.1080/10999922.2018.1511671 · Cited by 3,279 · cited 1×
“…moral hazards associated with the increasing prevalence of algorithms and their use as a substitute for human judgment in decisions within the criminal justice system, consumer credit ratings, finance services, college admissions, and job applications, among others (Eubanks, 2018; O’Neill, 2020).”
BackgroundHow the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 2×
“Certainly, comparisons between DL’s neural networks and human brains can be made, and, in many ways, human cognition can be viewed as a black box as well (Burrell, 2016; Castelvecchi, 2016).”
BackgroundBaier, A. (1986) · Ethics · doi:10.1086/292745 · Cited by 2,567 · cited 5×
“Responses to violations of trust typically entail feelings of betrayal, not just dis-appointment, because the trustee not only has failed to be competent but also has failed to consider, honor, or respect the trustor’s interests or goods (Baier, 1986).”
BackgroundInfluentialCan we open the black box of AI?
Castelvecchi, D. (2016) · Nature · doi:10.1038/538020a · Cited by 1,773 · cited 1×
“Certainly, comparisons between DL’s neural networks and human brains can be made, and, in many ways, human cognition can be viewed as a black box as well (Burrell, 2016; Castelvecchi, 2016).”
BackgroundMiotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016) · Scientific Reports · doi:10.1038/srep26094 · Cited by 1,710 · cited 1×
“Deep Patient proved to be better able to predict future disease and disabilities, especially diabetes, schizophrenia, and cancer, using information such as previous diagnoses, medications, lab results, and procedures than simpler algorithms or human counterparts (Miotto et al., 2016).”
Background(2002) · Choice Reviews Online · doi:10.5860/choice.40-1460 · Cited by 1,617
Explainable AI: From black box to glass box
Rai, A. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-019-00710-5 · Cited by 1,317 · cited 3×
“An important question, however, is whether XAI gives us reasons to trust the system that is being modeled or what seems more likely, the model itself (Rai, 2020, 139).”
BackgroundMethodTrust as an Affective Attitude
Jones, K. (1996) · Ethics · doi:10.1086/233694 · Cited by 1,227 · cited 1×
“What counts as good reasons for trusting is a matter of considerable debate, but at the very least that B is capable or competent to do X is necessary for A to trust B (Baier, 1986; Jones, 1996, 2012; Simpson, 2012).”
BackgroundOn the Morality of Artificial Agents
Floridi, L., & Sanders, J. W. (2004) · Minds and Machines · doi:10.1023/b:mind.0000035461.63578.9d · Cited by 1,218 · cited 1×
“Trust typically is thought to be relational where A trusts B to do X (Flores & Solomon, 1998).”
Background(2018) · Cited by 1,167
Fisher, A., Rudin, C., & Dominici, F. (2018) · arXiv · doi:10.48550/arxiv.1801.01489 · Cited by 1,165
The accuracy, fairness, and limits of predicting recidivism
Dressel, J., & Farid, H. (2018) · Science Advances · doi:10.1126/sciadv.aao5580 · Cited by 1,071
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · cited 3×
“It should be noted that we are not seeking a full and comprehensive interpretation of the entire logic and workings of the system or model, what some call “global interpretability,” but rather reasons for the specific decision or outcome, or “local interpretability” (Guidotti et al., 2018, 6).”
BackgroundThe threat of algocracy: reality, resistance and accommodation
Danaher, J. (2016) · Philosophy & Technology · doi:10.1007/s13347-015-0211-1 · Cited by 519 · cited 1×
“…some scholars even have argued that the growing dependence on these systems gives rise to “the threat of algocracy—a situation in which algorithm-based systems structure and constrain the opportunities for human participation in, and comprehension of, public deci-sion-making” (Danaher, 2016, 246).”
BackgroundTransparency in Algorithmic and Human Decision-Making: Is There a Double Standard?
Zerilli, J., Knott, A., Maclaurin, J., & Gavaghan, C. (2019) · Philosophy & Technology · doi:10.1007/s13347-018-0330-6 · Cited by 420 · cited 2×
“…standard in which machine tools must be transparent to a degree that is in some cases unattainable, in order to be considered transparent at all, while human decision-making can get by with reasons satisfying the comparatively undemanding standards of practical reason” (Zerilli et al., 2019, 668).”
BackgroundPETTIT, P. (1995) · Philosophy & Public Affairs · doi:10.1111/j.1088-4963.1995.tb00029.x · Cited by 419 · cited 2×
“…with little knowledge of the trustee’s competence, but these cases typically are either the first stage of an iterative process to gather information about the person’s competence or as a means to induce the kind of responsiveness associated with the second sense of trustworthiness (Pettit, 1995).”
BackgroundSolving the Black Box Problem: A Normative Framework for Explainable Artificial Intelligence
Zednik, C. (2021) · Philosophy & Technology · doi:10.1007/s13347-019-00382-7 · Cited by 419 · cited 5×
“In addition to end users and the technical device itself, we should consider the designers, programmers, “data subjects,” operators, “decision subjects,” and examiners, or those tasked with auditing and inspecting a system (Zednik, 2019).”
BackgroundInfluentialThe philosophical novelty of computer simulation methods
Humphreys, P. (2009) · Synthese · doi:10.1007/s11229-008-9435-2 · Cited by 362 · cited 1×
“Systems can be opaque to the extent that epistemi-cally relevant elements are unknown to an agent and so opacity might vary depending on the agent (Humphreys, 2009).”
BackgroundEthics-Based Auditing to Develop Trustworthy AI
Mökander, J., & Floridi, L. (2021) · Minds and Machines · doi:10.1007/s11023-021-09557-8 · Cited by 268 · cited 1×
“Making black box technology more transparent will remain an essential part of this process, but other avenues of inquiry, such as ethics-based auditing of AI, also are critical to this project and should be explored further (Mökander & Floridi, 2021).”
BackgroundJones, K. (2012) · Ethics · doi:10.1086/667838 · Cited by 244 · cited 2×
“What counts as good reasons for trusting is a matter of considerable debate, but at the very least that B is capable or competent to do X is necessary for A to trust B (Baier, 1986; Jones, 1996, 2012; Simpson, 2012).”
BackgroundThe Pragmatic Turn in Explainable Artificial Intelligence (XAI)
Páez, A. (2019) · Minds and Machines · doi:10.1007/s11023-019-09502-w · Cited by 230 · cited 5×
“This can be achieved by developing interpretative models that approximate DL processes, such as linear approximations or decision trees, or by using post hoc interpretative methods, such as natural language explanations or functional models (Páez, 2019).”
BackgroundMethodInfluentialThe age of secrecy and unfairness in recidivism prediction
Rudin, C., Wang, C., & Coker, B. (2020) · Harvard Data Science Review · doi:10.1162/99608f92.6ed64b30 · Cited by 177
A Misdirected Principle with a Catch: Explicability for AI
Robbins, S. (2019) · Minds and Machines · doi:10.1007/s11023-019-09509-3 · Cited by 174 · cited 1×
“Decisions that require explanation, the argument concludes, therefore should not be made by AI (Robbins, 2019).”
BackgroundSimpson, T. W. (2012) · Pacific Philosophical Quarterly · doi:10.1111/j.1468-0114.2012.01438.x · Cited by 167 · cited 1×
“What counts as good reasons for trusting is a matter of considerable debate, but at the very least that B is capable or competent to do X is necessary for A to trust B (Baier, 1986; Jones, 1996, 2012; Simpson, 2012).”
BackgroundFlores, F., & Solomon, R. C. (1998) · Business Ethics Quarterly · doi:10.2307/3857326 · Cited by 148 · cited 1×
“Trust typically is thought to be relational where A trusts B to do X (Flores & Solomon, 1998).”
BackgroundTrusting Our Selves to Technology
Kiran, A. H., & Verbeek, P.-P. (2010) · Knowledge, Technology & Policy · doi:10.1007/s12130-010-9123-7 · Cited by 141 · cited 1×
“A more fruitful route for establishing trust in AI is to reject the binary distinction between humans and technology and acknowledge the mediating role that technology plays in human life and that it is interwoven in our lives (Kiran & Verbeek, 2010).”
BackgroundIntroduction: the Governance of Algorithms
D’Agostino, M., & Durante, M. (2018) · Philosophy & Technology · doi:10.1007/s13347-018-0337-z · Cited by 138 · cited 1×
“Issues about by what standards we measure or assess the function of algorithms or how we govern their use become more acute (D’Agostino & Durante, 2018).”
BackgroundThe Pragmatic Turn in Explainable Artificial Intelligence (XAI)
Páez, A. (2019) · PhilPapers (PhilPapers Foundation) · Cited by 80
Can We Make Sense of the Notion of Trustworthy Technology?
Nickel, P. J., Franssen, M., & Kroes, P. (2010) · Knowledge, Technology & Policy · doi:10.1007/s12130-010-9124-6 · Cited by 76 · cited 2×
“A socio-technical system sees technology as more than “a collection of devices intermediating between their designers on one hand and their users on the other” but is understood to be a hybrid between the technical and social (Nickel et al., 2010).”
BackgroundTaddeo, M., & Floridi, L. (2011) · Ethics and Information Technology · doi:10.1007/s10676-010-9263-1 · Cited by 68
Scheman, N. (2020) · The Routledge Handbook of Trust and Philosophy · doi:10.4324/9781315542294-2 · Cited by 57
Appraising Black-Boxed Technology: the Positive Prospects
Dahl, E. S. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0275-1 · Cited by 41 · cited 5×
“Merely inspecting the inner workings of these devices and systems, however, might not be possible for practical reasons and falls short of the kind of transparency required for trust (Dahl, 2018).”
BackgroundInfluentialPitt, J. C. (2010) · Knowledge, technology & policy/Knowledge in society/Knowledge, technology, & policy · doi:10.1007/s12130-010-9125-5 · Cited by 40 · cited 1×
“For some, to ask whether we can trust technology is akin to making a category mis-take because trust can occur only between people or moral agents (Pitt, 2010).”
BackgroundTrust and Accountability in a Digital Age
O’Neill, O. (2019) · Philosophy · doi:10.1017/s0031819119000457 · Cited by 36 · cited 2×
“Algorithms used as alternatives to credit agency scores to assess loan applications have come under scrutiny because of the use of proxies, such as zip codes, which disproportionately discriminate against minority communities (O’Neill, 2020).”
BackgroundMethodDurante, M. (2010) · Knowledge, technology & policy/Knowledge in society/Knowledge, technology, & policy · doi:10.1007/s12130-010-9118-4 · Cited by 23 · cited 1×
“Thus, trust in AI requires what Durante calls a “web of trust” that “is founded on circular informational causality ” oriented toward a shared goal or purpose (Durante, 2010, 355).”
BackgroundVirtues in the Public Sphere: Citizenship, Civic Friendship and Duty
(2018) · Routledge eBooks · doi:10.4324/9780429505096 · Cited by 13
von Eschenbach, W. J. (2018) · Routledge eBooks · doi:10.4324/9780429505096-10 · Cited by 1 · cited 1×
“Trust is a relationship of reciprocal duties, obligations, and expectations, and so feelings of betrayal or outrage are apt should trust be violated (von Eschenbach 2019).”
Background
Cited by
Survey of Explainable AI Techniques in Healthcare
Chaddad, A., Peng, J., Xu, J., & Bouridane, A. (2023) · Sensors · doi:10.3390/s23020634 · Cited by 591
Trust in AI: progress, challenges, and future directions
Afroogh, S., Akbari, A., Malone, E., Kargar, M., & Alambeigi, H. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04044-8 · Cited by 507
“These black-box networks are complex and opaque in terms of operation (von Eschenbach, 2021), and even sometimes, the developer has limited access to how they operate.”
Petersson, L., Larsson, I., Nygren, J. M., Nilsen, P., Neher, M., Reed, J. E., Tyskbo, D., & Svedberg, P. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-08215-8 · Cited by 434
“the “black box” problem) might yield resistance to some AI systems [61].”
BackgroundEsmaeilzadeh, P. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102861 · Cited by 377
Hariyanto, H., Kristianingsih, F. X. D., & Maharani, R. (2025) · Discover Education · doi:10.1007/s44217-025-00908-6 · Cited by 330
“These “black box” systems often lack transparency, making it difficult for educators and students to understand or validate AI-generated decisions [64, 65].”
Metadata
- Authors
- von Eschenbach, W. J.
- Venue
- Philosophy & Technology
- Published
- 2021-09-01
- Volume
- 34
- Issue
- 4
- Pages
- 1607-1622
- ISSN
- 2210-5433 (ISSN-L)
- 2210-5441
- OpenAlex
- W3198525581
- Cited by
- 786
- References
- 37
- Citations by year
- 2026: 239 · 2025: 271 · 2024: 176 · 2023: 68 · 2022: 30 · 2021: 2
- FWCI
- 24.38
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Philosophy & Technology (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Keywords
- black box problem, transparency, trust in AI, explainable AI, deep learning, opacity, automated decision-making, socio-technical systems, trustworthiness, philosophy of trust
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.