It is Not “Accuracy vs. Explainability”—We Need Both for Trustworthy AI Systems
The DOI matches its Crossref record; 26 of the 27 references it cites were found in a registry or an index.
Abstract
We are witnessing the emergence of an “AI economy and society” where AI technologies and applications are increasingly impacting health care, business, transportation, defense and many aspects of everyday life. Many successes have been reported where AI systems even surpassed the accuracy of human experts. However, AI systems may produce errors, can exhibit bias, may be sensitive to noise in the data, and often lack technical and judicial transparency resulting in reduction in trust and challenges to their adoption. These recent shortcomings and concerns have been documented in both the scientific and general press such as accidents with self-driving cars, biases in healthcare or hiring and face recognition systems for people of color, and seemingly correct decisions later found to be made due to wrong reasons etc. This has resulted in the emergence of many government and regulatory initiatives requiring trustworthy and ethical AI to provide accuracy and robustness, some form of explainability, human control and oversight, elimination of bias, judicial transparency and safety. The challenges in delivery of trustworthy AI systems have motivated intense research on explainable AI systems (XAI). The original aim of XAI is to provide human understandable information of how AI systems make their decisions in order to increase user trust. In this paper we first very briefly summarize current XAI work and then challenge the recent arguments that present “accuracy vs. explainability” as being mutually exclusive and for focusing mainly on deep learning with its limited XAI capabilities. We then present our recommendations for the broad use of XAI in all stages of delivery of high stakes trustworthy AI systems, e.g., development; validation/certification; and trustworthy production and maintenance.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For safety-critical systems, explainability is not enough for a system to be trustworthy.Various requirements such as robustness, reliability, traceability, data quality, and other factors must be demonstrated [25,43,62,63].”
From A requirements model for AI algorithms in functional safety-critical systems with an explainable self-enforcing network from a developer perspective (2024) · cites this paper as [63]
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · cited 1×
“(or feature importance) method was developed for Random Forest classifiers [38] and leveraged by many XAI researchers, for example in [28], [29].”
MethodA Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 1×
“tion methods were also developed, like Shapley one based on game theory [40], which has been applied for explaining”
MethodDermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · cited 1×
“cancer reported in [32], where in addition to classic accuracy”
BackgroundRudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087 · cited 2×
“Vector Machines (SVM) and ensemble tree AI algorithms respectively, as well as for deep learning systems [43].”
BackgroundExplainable AI: a review of machine learning interpretability methods
Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020) · Entropy · doi:10.3390/e23010018 · Cited by 2,922 · cited 4×
“XAI techniques then aim to explain how AI systems make their decisions, and aim to do so in easily understandable way to humans (17, 18, 19, 23, 25, 26, 27).”
BackgroundInfluentialModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 3×
“More concrete best practices based on these regulations are being developed as well (10, 11, 12, 13).”
BackgroundThe role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · cited 3×
“takes made on the data/entities they understand [15], [18], [26], [27].”
BackgroundArtificial Intelligence and Black‐Box Medical Decisions: Accuracy versus Explainability
London, A. J. (2019) · Hastings Center Report · doi:10.1002/hast.973 · Cited by 801 · cited 2×
“its perils and that as long as non-explainable black box systems are accurate and well tested they should be used [23], [24].”
Background
Research works printed but not linked: 20
- Szabo, 2019, Artificial intelligence is rushing into patient care—And could raise risks
- Kaushal, 2020, Toward fairness in health care training data
- 2017, Can A.I. be taught to explain itself
- Goodman, 2018, Why Amazon’s automated hiring tool discriminated against women
- Harwell, 2019, Federal study confirms bias in facial recognition systems
- 2021, Proposal for a regulation of the european parliament and of the council laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain union legislative acts, COM/2021/206 final
- 2017, Asilomar ethical AI principles
- 2022, The EU general data protection regulation 2016/679 (GDPR), recital 71
- 2021, How does FDA regulate AI/ML in medical devices and products
- Ford, 2021, Rule of the Robots: How Artificial Intelligence Will Transform Everything
- McCausland, 2019, Self-driving Uber car that hit and killed woman did not recognize that pedestrians jaywalk
- Turek, 2018, Explainable artificial intelligence (XAI)
- Petkovic, 2023, Explainable Deep Learning AI: Methods and Challenges
- 2022, Occam’s razor, Britannica
- Dong, Towards interpretable deep neural networks by leveraging adversarial examples, 975
- Mitsa, 2022, How do you know you have enough training data?
- 10.1007/978-1-4899-7687-1_192 · DOI · Paper page
- Guszcza, 2018, AI needs human-centered design
- Lundberg, 2018, Interpretable machine learning with XGBoost
- Bhattacharya, 2021, Top 10 massive failures of artifficial inteliogence till date
Cited by
Interpretability research of deep learning: A literature survey
Xu, B., & Yang, G. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102721 · Cited by 215
Jedličková, A. (2024) · AI & Society · doi:10.1007/s00146-024-02040-9 · Cited by 158
“Despite potential tensions among requirements, values, or principles, such as the accuracy-explainability tradeoff, trustworthy AI systems must meet all ethical requirements (Petkovic 2023).”
Exploring the potential and limitations of artificial intelligence in animal anatomy
Choudhary, O. P., Infant, S. S., As, V., Chopra, H., & Manuta, N. (2024) · Annals of Anatomy - Anatomischer Anzeiger · doi:10.1016/j.aanat.2024.152366 · Cited by 135
The role of AI in emergency department triage: An integrative systematic review
Arab, R. A. E., & Moosa, O. A. A. (2025) · Intensive and Critical Care Nursing · doi:10.1016/j.iccn.2025.104058 · Cited by 97
Tech Giants’ Responsible Innovation and Technology Strategy: An International Policy Review
Li, W., Yiğitcanlar, T., Nili, A., & Browne, W. N. (2023) · Smart Cities · doi:10.3390/smartcities6060153 · Cited by 45
Metadata
- Authors
- Petkovic, D.
- Affiliations
- Computer Science Department, San Francisco State University, San Francisco, CA, USA
- ORCID
- Petkovic, Dragutin
- Venue
- IEEE Transactions on Technology and Society
- Published
- 2023-03
- Volume
- 4
- Issue
- 1
- Pages
- 46-53
- ISSN
- 2637-6415 (ISSN-L)
- OpenAlex
- W4319863522
- Cited by
- 63
- References
- 27
- Citations by year
- 2026: 13 · 2025: 26 · 2024: 20 · 2023: 4
- FWCI
- 5.58
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Technology and Society (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
- Keywords
- explainable AI, trustworthy AI, trust in AI, algorithmic bias, deep learning limitations, AI regulation, AI ethics, human oversight of AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: It is Not “Accuracy vs. Explainability”—We Need Both for Trustworthy AI Systems. https://app.cochat.ai/papers/10.1109/tts.2023.3239921
- The paper
- Petkovic, D. (2023). It is Not “Accuracy vs. Explainability”—We Need Both for Trustworthy AI Systems. IEEE Transactions on Technology and Society. https://doi.org/10.1109/tts.2023.3239921
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.