The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Should AI models be explainable to clinicians?
Open accessAbstract
In the high-stakes realm of critical care, where daily decisions are crucial and clear communication is paramount, comprehending the rationale behind Artificial Intelligence (AI)-driven decisions appears essential. While AI has the potential to improve decision-making, its complexity can hinder comprehension and adherence to its recommendations. “Explainable AI” (XAI) aims to bridge this gap, enhancing confidence among patients and doctors. It also helps to meet regulatory transparency requirements, offers actionable insights, and promotes fairness and safety. Yet, defining explainability and standardising assessments are ongoing challenges and balancing performance and explainability can be needed, even if XAI is a growing field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 4×
“In medical AI, where models are typically based on detailed, structured data grounded in physiopathology, the performance difference between interpretable and more complex models often turns out to be minimal [34].”
BackgroundMethodInfluentialAn overview of clinical decision support systems: Benefits, risks, and strategies for success
Sutton, R. T., Pincock, D., Baumgart, D. C., Sadowski, D. C., Fedorak, R. N., & Kroeker, K. I. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0221-y · Cited by 3,019 · cited 1×
“In the high-pressure environment of the ICU, doctors need clarity when making decisions, especially when using AI-based support systems [13].”
BackgroundThe false hope of current approaches to explainable artificial intelligence in health care
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00208-9 · Cited by 1,724 · cited 2×
“Moreover, identifying an AI model’s key features does not ensure their effective or expected use, particularly from a clinical perspective [28].”
BackgroundVasey, B., Nagendran, M., Campbell, B., Clifton, D. A., Collins, G. S., Denaxas, S., Denniston, A. K., Faes, L., Geerts, B., Ibrahim, M., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01772-9 · Cited by 1,039 · cited 1×
“These caveats partly explain why there is still no consensus on whether AI models, as seen in decision support systems, should inherently possess explainability as a core attribute [35].”
BackgroundMarkus, A. F., Kors, J. A., & Rijnbeek, P. R. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103655 · Cited by 811
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 1×
“Authors have suggested that this overconfidence may, in part, arise from an "information overload" effect [32, 41], which might also induce data fatigue.”
BackgroundAI for radiographic COVID-19 detection selects shortcuts over signal
DeGrave, A. J., Janizek, J. D., & Lee, S.-I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00338-7 · Cited by 631
Hatib, F., Jian, Z., Buddi, S., Lee, C., Settels, J., Sibert, K., Rinehart, J., & Cannesson, M. (2018) · Anesthesiology · doi:10.1097/aln.0000000000002300 · Cited by 614
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594
Lauritsen, S. M., Kristensen, M., Olsen, M. V., Larsen, M. S., Lauritsen, K. M., Jørgensen, M. J., Lange, J., & Thiesson, B. (2020) · Nature Communications · doi:10.1038/s41467-020-17431-x · Cited by 448
Research works printed but not linked: 6
- Grote T. Allure of simplicity: on interpretable machine learning models in healthcare. Philod Med. 2023;4:1.
- Goodman B, Flaxman S. European Union regulations on algorithmic decision-making and a ‘right to explanation’. 2016 [cited 2023 Oct 26]; Available from: https://arxiv.org/abs/1606.08813 · arXiv
- Chung NC, Chung H, Lee H, Chung H, Brocki L, Dyer G. False Sense of Security in Explainable Artificial Intelligence (XAI) [Internet]. arXiv; 2024 [cited 2024 May 23]. Available from: https://arxiv.org/abs/2405.03820 · arXiv
- Schneider J. Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agenda [Internet]. arXiv; 2024 [cited 2024 May 23]. Available from: http://arxiv.org/abs/2404.09554 · arXiv
- European Commission. Directorate General for Communications Networks, Content and Technology., High Level Expert Group on Artificial Intelligence. Ethics guidelines for trustworthy AI. [Internet]. LU: Publications Office; 2019 [cited 2023 Oct 26]. Available from: https://data.europa.eu/doi/https://doi.org/10.2759/346720 · doi:10.2759/346720
- Jacovi A. Trends in Explainable AI (XAI) Literature [Internet]. arXiv; 2023 [cited 2024 May 23]. Available from: https://arxiv.org/abs/2301.05433 · arXiv
Cited by
Abbas, Q., Jeong, W., & Lee, S. W. (2025) · Healthcare · doi:10.3390/healthcare13172154 · Cited by 236
“As AI continues to evolve, the emphasis on explainability will be pivotal for its responsible and sustainable integration into routine clinical workflows [45,46].”
Leivaditis, V., Maniatopoulos, A. A., Lausberg, H., Mulita, F., Papatriantafyllou, A., Liolis, E., Beltsios, E., Adamou, A., Kontodimopoulos, N., & Dahm, M. (2025) · Journal of Clinical Medicine · doi:10.3390/jcm14082729 · Cited by 76
Artificial intelligence to revolutionize IBD clinical trials: a comprehensive review
Sedaño, R., Solitano, V., Vuyyuru, S. K., Yuan, Y., Hanžel, J., Ma, C., Nardone, O. M., & Jairath, V. (2025) · Therapeutic Advances in Gastroenterology · doi:10.1177/17562848251321915 · Cited by 67
Machine learning for extracellular vesicles enables diagnostic and therapeutic nanobiotechnology
Tiwari, A. K., Widodo, Krisnawati, D. I., Tzou, K.-Y., & Kuo, T. (2026) · Journal of Nanobiotechnology · doi:10.1186/s12951-025-03952-4 · Cited by 60
Saeidnia, H. R., Firuzpour, F., Kozak, M., & majd, H. S. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11117-w · Cited by 50
Metadata
- Authors
- Abgrall, G., Holder, A. L., Chelly Dagdia, Z., Zeitouni, K., & Monnet, X.
- Affiliations
- Inserm (FR)
- Emory University (US)
- Université de Versailles Saint-Quentin-en-Yvelines (FR)
- Corresponding author
- Abgrall, Gwénolé
- Venue
- Critical Care
- Published
- 2024-09-12
- Volume
- 28
- Issue
- 1
- Pages
- 301-301
- ISSN
- 1364-8535 (ISSN-L)
- PMC
- PMC11391805
- OpenAlex
- W4402492681
- Cited by
- 175
- References
- 39
- Citations by year
- 2026: 87 · 2025: 82 · 2024: 6
- FWCI
- 25.83
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Critical Care (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- HAL (submittedVersion, other-oa)
- Funders
- Fondation pour la Recherche Médicale (FDM202306017126)
- Société de Réanimation de Langue Française
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- explainable AI, clinical decision support, AI decision transparency, clinician trust in AI, regulatory transparency
- MeSH terms
- Clinical Decision-Making, Artificial Intelligence, Critical Care, Humans, Physicians
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Should AI models be explainable to clinicians? https://app.cochat.ai/papers/10.1186/s13054-024-05005-y
- The paper
- Abgrall, G., Holder, A. L., Chelly Dagdia, Z., Zeitouni, K., & Monnet, X. (2024). Should AI models be explainable to clinicians? Critical Care. https://doi.org/10.1186/s13054-024-05005-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
