Believing in black boxes: machine learning for healthcare does not need explainability to be evidence-based
Open accessAbstract
Objective To examine the role of explainability in machine learning for healthcare (MLHC), and its necessity and significance with respect to effective and ethical MLHC application. Study design and setting This commentary engages with the growing and dynamic corpus of literature on the use of MLHC and artificial intelligence (AI) in medicine, which provide the context for a focused narrative review of arguments presented in favour of and opposition to explainability in MLHC. Results We find that concerns regarding explainability are not limited to MLHC, but rather extend to numerous well-validated treatment interventions as well as to human clinical judgment itself. We examine the role of evidence-based medicine in evaluating inexplicable treatments and technologies, and highlight the analogy between the concept of explainability in MLHC and the related concept of mechanistic reasoning in evidence-based medicine. Conclusion Ultimately, we conclude that the value of explainability in MLHC is not intrinsic, but is instead instrumental to achieving greater imperatives such as performance and trust. We caution against the uncompromising pursuit of explainability, and advocate instead for the development of robust empirical methods to successfully evaluate increasingly inexplicable algorithmic systems.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Generative AI models in health care, particularly those based on deep learning, are often seen as "black boxes".
“Technically speaking, similar to all other AI models, generative AI models in health care, particularly those based on deep learning, are often seen as “black boxes” [78].”
From Generative AI in medical practice: in-depth exploration of privacy and security challenges (2024) · cites this paper as [78]
Section 4, “Empirically validating the inexplicable” (page unavailable), describes deep-learning architectures in health care as presently inscrutable and discusses “opening the black box.” That supports the characterization of some deep-learning health-care models, but the passage does not address generative AI models specifically. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
Evidence based medicine: what it is and what it isn't
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996) · BMJ · doi:10.1136/bmj.312.7023.71 · Cited by 15,769 · see citing works
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · see citing works
The environment and disease: association or causation?
Hill, A. B. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076814562718 · Cited by 7,302 · see citing works
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1× · see citing works
“In-deed, prominent case studies have demonstrated precisely how explainable (and not intentionally biased) models can nonetheless deliver biased results that are only apparent upon empirical analysis [ 57 , 58 ].”
BackgroundThe Environment and Disease: Association or Causation?
Hill, A. B. (1965) · Proceedings of the Royal Society of Medicine · doi:10.1177/003591576505800503 · Cited by 6,974 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909 · see citing works
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Yeh, M. (2014) · Journal of Nursing Research · doi:10.1097/jnr.0000000000000065 · Cited by 3,836 · see citing works
Artificial intelligence in radiology
Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H., & Aerts, H. J. W. L. (2018) · Nature Reviews Cancer · doi:10.1038/s41568-018-0016-5 · Cited by 3,803 · see citing works
Mortality and Morbidity in Patients Receiving Encainide, Flecainide, or Placebo
Echt, D. S., Liebson, P. R., Mitchell, L. B., Peters, R. W., Obias-Manno, D., Barker, A. H., Arensberg, D., Baker, A., Friedman, L., Greene, H. L., et al. (1991) · New England Journal of Medicine · doi:10.1056/nejm199103213241201 · Cited by 3,244 · see citing works
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524 · see citing works
The mechanisms of action of metformin
Rena, G., Hardie, D. G., & Pearson, E. R. (2017) · Diabetologia · doi:10.1007/s00125-017-4342-z · Cited by 2,287 · see citing works
Ethics Guidelines for Trustworthy AI
Cannarsa, M. (2021) · The Cambridge Handbook of Lawyering in the Digital Age · doi:10.1017/9781108936040.022 · Cited by 2,095 · see citing works
Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685 · see citing works
The ethics of AI in health care: A mapping review
Morley, J., Machado, C. C. V., Burr, C., Cowls, J., Joshi, I., Taddeo, M., & Floridi, L. (2020) · Social Science & Medicine · doi:10.1016/j.socscimed.2020.113172 · Cited by 1,105 · cited 1× · see citing works
“Sub-sequently, the pursuit of explainability has become an area of significant importance in contemporary MLHC research, emphasized by academics in computer science [ 4 , 8–12 ] medicine [12–14] , and bioethics [7] , as well as major regulatory bodies [15] and healthcare technology companies [16] .”
BackgroundArtificial Intelligence and Black‐Box Medical Decisions: <i>Accuracy versus Explainability</i>
London, A. J. (2019) · Hastings Center Report · doi:10.1002/hast.973 · Cited by 801 · see citing works
Linguistic Features Identify Alzheimer’s Disease in Narrative Speech
Fraser, K. C., Meltzer, J. A., & Rudzicz, F. (2015) · Journal of Alzheimer's Disease · doi:10.3233/jad-150520 · Cited by 797 · see citing works
Explainable machine learning in deployment
Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J. M. F., & Eckersley, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375624 · Cited by 633 · see citing works
Potential liability for physicians using artificial intelligence
Price, W. N., Gerke, S., & Cohen, I. G. (2019) · JAMA · doi:10.1001/jama.2019.15064 · Cited by 579 · see citing works
Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare
Cirillo, D., Catuara-Solarz, S., Morey, C., Guney, E., Subirats, L., Mellino, S., Gigante, A., Valencia, A., Rementeria, M. J., Chadha, A. S., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0288-5 · Cited by 553 · see citing works
Interpreting Causality in the Health Sciences
Russo, F., & Williamson, J. (2007) · International Studies in the Philosophy of Science · doi:10.1080/02698590701498084 · Cited by 549 · see citing works
Transparency in Algorithmic and Human Decision-Making: Is There a Double Standard?
Zerilli, J., Knott, A., Maclaurin, J., & Gavaghan, C. (2019) · Philosophy & Technology · doi:10.1007/s13347-018-0330-6 · Cited by 420 · see citing works
Identifying Ethical Considerations for Machine Learning Healthcare Applications
Char, D. S., Abràmoff, M. D., & Feudtner, C. (2020) · The American Journal of Bioethics · doi:10.1080/15265161.2020.1819469 · Cited by 415 · see citing works
Hidden stratification causes clinically meaningful failures in machine learning for medical imaging
Oakden-Rayner, L., Dunnmon, J., Carneiro, G., & Re, C. (2020) · Proceedings of the ACM Conference on Health, Inference, and Learning · doi:10.1145/3368555.3384468 · Cited by 411 · see citing works
Assessing risk, automating racism
Benjamin, R. (2019) · Science · doi:10.1126/science.aaz3873 · Cited by 378 · see citing works
Cutillo, C. M., Sharma, K. R., Foschini, L., Kundu, S., Mackintosh, M., Mandl, K. D., MI in Healthcare Workshop Working Group, Beck, T., Collier, E., Colvis, C., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0254-2 · Cited by 337 · see citing works
Translation of highly promising basic science research into clinical applications
Contopoulos-Ioannidis, D. G., Ntzani, E. E., & Ioannidis, J. P. A. (2003) · The American Journal of Medicine · doi:10.1016/s0002-9343(03)00013-5 · Cited by 323 · see citing works
What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019) · arXiv · doi:10.48550/arxiv.1905.05134 · Cited by 318 · see citing works
Desai, R. J., Wang, S. V., Vaduganathan, M., Evers, T., & Schneeweiß, S. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.18962 · Cited by 294 · see citing works
Mechanisms underlying the weight loss effects of RYGB and SG: similar, yet different
Pucci, A. B. F., & Batterham, R. L. (2018) · Journal of Endocrinological Investigation · doi:10.1007/s40618-018-0892-2 · Cited by 222 · see citing works
The European artificial intelligence strategy: implications and challenges for digital health
Cohen, I. G., Evgeniou, T., Gerke, S., & Minssen, T. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30112-6 · Cited by 171 · see citing works
Mechanistic Evidence: Disambiguating the Russo–Williamson Thesis
Illari, P. M. (2011) · International Studies in the Philosophy of Science · doi:10.1080/02698595.2011.574856 · Cited by 171 · see citing works
The Routledge Companion to Philosophy of Medicine
Schües, C. (2016) · Routledge eBooks · doi:10.4324/9781315720739 · Cited by 164 · see citing works
Algorithmic Decision-Making and the Control Problem
Zerilli, J., Knott, A., Maclaurin, J., & Gavaghan, C. (2019) · Minds and Machines · doi:10.1007/s11023-019-09513-7 · Cited by 145 · see citing works
Enslaving the Algorithm: From a “Right to an Explanation” to a “Right to Better Decisions”?
Edwards, L., & Veale, M. (2018) · IEEE Security & Privacy · doi:10.1109/msp.2018.2701152 · Cited by 141 · see citing works
Explainable AI for Healthcare: From Black Box to Interpretable Models
Adadi, A., & Berrada, M. (2020) · Advances in Intelligent Systems and Computing · doi:10.1007/978-981-15-0947-6_31 · Cited by 137 · see citing works
Toussaint, K., Yang, X. C., Zielinski, M. A., Reigle, K. L., Sacavage, S. D., Nagar, S., & Raffa, R. B. (2010) · Journal of Clinical Pharmacy and Therapeutics · doi:10.1111/j.1365-2710.2009.01143.x · Cited by 135 · see citing works
Gillon, R. (2015) · Journal of Medical Ethics · doi:10.1136/medethics-2014-102282 · Cited by 124 · see citing works
Ethical considerations about artificial intelligence for prognostication in intensive care
Beil, M., Proft, I., van Heerden, D., Sviri, S., & van Heerden, P. V. (2019) · Intensive Care Medicine Experimental · doi:10.1186/s40635-019-0286-6 · Cited by 124 · see citing works
Evidence-based mechanistic reasoning
Howick, J., Glasziou, P., & Aronson, J. K. (2010) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.2010.100146 · Cited by 121 · see citing works
Nanayakkara, S., Fogarty, S., Tremeer, M., Ross, K. J., Richards, B., Bergmeir, C., Xu, S. Y., Stub, D. A., Smith, K. L., TACEY, M. A., et al. (2018) · PLoS Medicine · doi:10.1371/journal.pmed.1002709 · Cited by 117 · see citing works
Deep learning: definition and perspectives for thoracic imaging
Chassagnon, G., Vakalopolou, M., Nikos Paragios, A., & Revel, M. (2019) · European Radiology · doi:10.1007/s00330-019-06564-3 · Cited by 116 · see citing works
The new medical model: a renewed challenge for biomedicine
Fuller, J. (2017) · Canadian Medical Association Journal · doi:10.1503/cmaj.160627 · Cited by 113 · see citing works
The evidence that evidence-based medicine omits
Clarke, B., Gillies, D. A., Illari, P. K., Federica Russo, B., & Williamson, J. (2012) · Preventive Medicine · doi:10.1016/j.ypmed.2012.10.020 · Cited by 107 · see citing works
Mechanisms responsible for excess weight loss after bariatric surgery
Ionut, V., & Bergman, R. N. (2011) · Journal of Diabetes Science and Technology · doi:10.1177/193229681100500536 · Cited by 105 · see citing works
Meske, C., & Bunde, E. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50334-5_4 · Cited by 99 · see citing works
Explainable Artificial Intelligence for Safe Intraoperative Decision Support
Gordon, L., Grantcharov, T., & Rudzicz, F. (2019) · JAMA Surgery · doi:10.1001/jamasurg.2019.2821 · Cited by 94 · see citing works
Explaining deep neural networks for knowledge discovery in electrocardiogram analysis
Hicks, S. A., Isaksen, J. L., Thambawita, V. L., Ghouse, J., Ahlberg, G., Linneberg, A. R., Grarup, N., Strümke, I., Ellervik, C., OLESEN, M. S., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-90285-5 · Cited by 78 · see citing works
Nestor, B. A., McDermott, M. B. A., Boag, W., Berner, G., Naumann, T., Hughes, M. C., Goldenberg, A., & Ghassemi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.00690 · Cited by 78 · see citing works
Mechanisms Underlying Type 2 Diabetes Remission After Metabolic Surgery
Pérez‐Pevida, B., Escalada, J., Miras, A. D., & Frühbeck, G. (2019) · Frontiers in Endocrinology · doi:10.3389/fendo.2019.00641 · Cited by 67 · see citing works
Ethics and Learning Health Care: The Essential roles of engagement, transparency, and accountability
KASS, N. E., & Faden, R. (2018) · Learning Health Systems · doi:10.1002/lrh2.10066 · Cited by 63 · see citing works
What is mechanistic evidence, and why do we need it for evidence-based policy?
Marchionni, C., & Reijula, S. (2018) · Studies in History and Philosophy of Science Part A · doi:10.1016/j.shpsa.2018.08.003 · Cited by 55 · see citing works
Enslaving the Algorithm: From a 'Right to an Explanation' to a 'Right to Better Decisions'?
Edwards, L., & Veale, M. (2018) · Strathprints: The University of Strathclyde institutional repository (University of Strathclyde) · Cited by 53 · see citing works
Explaining Machine Learning Decisions
Zerilli, J. (2022) · Philosophy of Science · doi:10.1017/psa.2021.13 · Cited by 50 · see citing works
The use of mechanistic evidence in drug approval
Aronson, J. K., Caze, A. L., Kelly, M. P., Parkkinen, V., & Williamson, J. (2018) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12960 · Cited by 45 · see citing works
Nardini, C., Annoni, M., & Schiavone, G. (2012) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2012.01907.x · Cited by 17 · see citing works
LeBlang, T. R. (1995) · PubMed · Cited by 13 · see citing works
Lemoine, M. (2016) · HAL · doi:10.4324/9781315720739.ch27 · Cited by 7 · see citing works
Clinical Diagnosis—Is There Any Other Type?
Dhaliwal, G. (2020) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2020.3048 · Cited by 7 · see citing works
"Deep" Visual Patterns Are Informative to Practicing Radiologists in Mammograms in Diagnostic Tasks
Sevilla, J., & Hegdé, J. (2017) · Journal of Vision · doi:10.1167/17.10.90 · Cited by 6 · see citing works
Evidence, discovery and justification: the case of evidence‐based medicine
Gaeta, R., & Gentile, N. (2015) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12419 · Cited by 6 · see citing works
Hegdé, J., & Bart, E. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00670 · Cited by 5 · see citing works
Mayer, D. M. (2006) · Epilepsia · doi:10.1111/j.1528-1167.2006.00651.x · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Holzinger A, Biemann C, Pattichis CS, Kell DB. What do we need to build explainable AI systems for the medical do- main? arXiv:171209923 [cs, stat] . Published online December 28, 2017. Accessed November 24, 2020. http:// arxiv.org/ abs/ 1712.09923
- Lenox RH , Frazer A . Mechanism of action of antidepressants and mood stablizers. Neuropsychopharmacology: The Fifth Generation of Progress. Philadelphia: Lippincott Williams & Wilkins Citeseer; 2002 .
Cited by
Sorted by citations
Generative AI in medical practice: in-depth exploration of privacy and security challenges
Chen, Y., & Esmaeilzadeh, P. (2024) · Journal of Medical Internet Research · doi:10.2196/53008 · Cited by 353 · see citing works
“Technically speaking, similar to all other AI models, generative AI models in health care, particularly those based on deep learning, are often seen as “black boxes” [78].”
Chen, H., Gomez, C., Huang, C.-M., & Unberath, M. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00699-2 · Cited by 300 · see citing works
Medical deep learning—A systematic meta-review
Egger, J., Gsaxner, C., Pepe, A., Pomykala, K. L., Jonske, F., Kurz, M., Li, J., & Kleesiek, J. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106874 · Cited by 270 · see citing works
Rahman, A., Debnath, T., Kundu, D., Khan, M. S. I., Aishi, A. A., Sazzad, S., Sayduzzaman, M., & Band, S. S. (2024) · AIMS Public Health · doi:10.3934/publichealth.2024004 · Cited by 251 · see citing works
Allgaier, J., Mulansky, L., Draelos, R. L., & Pryss, R. C. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102616 · Cited by 190 · see citing works
Hogg, H. D. J., Al-Zubaidy, M., Technology Enhanced Macular Services Study Reference Group, Talks, J., Denniston, A. K., Kelly, C. J., Malawana, J., Papoutsi, C., Teare, M. D., Keane, P. A., et al. (2023) · Journal of Medical Internet Research · doi:10.2196/39742 · Cited by 163 · see citing works
Wang, H., Wu, W., Dou, Z., He, L., & Yang, L. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105173 · Cited by 156 · see citing works
REFORMS: Consensus-based Recommendations for Machine-learning-based Science
Kapoor, S., Cantrell, E. M., Peng, K., Pham, T. H., Bail, C. A., Gundersen, O. E., Hofman, J. M., Hullman, J. R., Lones, M. A., Malik, M. M., et al. (2024) · Science Advances · doi:10.1126/sciadv.adk3452 · Cited by 192 · see citing works
A Comparative Analysis of LIME and SHAP Interpreters With Explainable ML-Based Diabetes Predictions
Ahmed, S., Kaiser, M. S., Hossain, M. S., & Andersson, K. (2024) · IEEE Access · doi:10.1109/access.2024.3422319 · Cited by 168 · see citing works
Review of interpretable machine learning for process industries
Carter, A., Imtiaz, S. A., & Naterer, G. F. (2022) · Process Safety and Environmental Protection · doi:10.1016/j.psep.2022.12.018 · Cited by 115 · see citing works
Javaid, A., Zghyer, F., Kim, C., Spaulding, E. M., Isakadze, N., Ding, J., Kargilis, D., Gao, Y., Rahman, F., Brown, D. E., et al. (2022) · American Journal of Preventive Cardiology · doi:10.1016/j.ajpc.2022.100379 · Cited by 109 · see citing works
Evidence, ethics and the promise of artificial intelligence in psychiatry
McCradden, M., Hui, K., & Buchman, D. Z. (2023) · Journal of Medical Ethics · doi:10.1136/jme-2022-108447 · Cited by 97 · see citing works
Artificial intelligence applications in histopathology
Bahadir, C. D., Omar, M., Rosenthal, J., Marchionni, L., Liechty, B., Pisapia, D. J., & Sabuncu, M. R. (2024) · Nature Reviews Electrical Engineering · doi:10.1038/s44287-023-00012-7 · Cited by 84 · see citing works
Five critical quality criteria for artificial intelligence-based prediction models
van Royen, F. S.-A., Asselbergs, F. W., Alfonso, F., Vardas, P. E., & van Smeden, M. (2023) · European Heart Journal · doi:10.1093/eurheartj/ehad727 · Cited by 77 · see citing works
Artificial Intelligence Interpretation of the Electrocardiogram: A State-of-the-Art Review
Ose, B. M., Sattar, Z., Gupta, A., Gahona, C. C. T., Harvey, C., & Noheria, A. (2024) · Current Cardiology Reports · doi:10.1007/s11886-024-02062-1 · Cited by 67 · see citing works
The ethical requirement of explainability for AI-DSS in healthcare: a systematic review of reasons
Freyer, N., Groß, D., & Lipprandt, M. (2024) · BMC Medical Ethics · doi:10.1186/s12910-024-01103-2 · Cited by 67 · see citing works
Tree-based local explanations of machine learning model predictions, AraucanaXAI
Parimbelli, E., Buonocore, T. M., Nicora, G., Michalowski, W., Wilk, S., & Bellazzi, R. (2021) · Artif. Intell. Medicine · doi:10.1016/j.artmed.2022.102471 · Cited by 46 · see citing works
Patino, G. A., Amiel, J. M., Brown, M., Lypson, M. L., & Chan, T. M. (2024) · Academic Medicine · doi:10.1097/acm.0000000000005636 · Cited by 43 · see citing works
Clinical deployment environments: Five pillars of translational machine learning for health
Harris, S. K., Bonnici, T., Keen, T., Lilaonitkul, W., White, M. J., & Swanepoel, N. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.939292 · Cited by 43 · see citing works
A flexible symbolic regression method for constructing interpretable clinical prediction models
Cava, W. L., Lee, P. C., Ajmal, I., Ding, X., Solanki, P., Cohen, J. B., Moore, J. H., & Herman, D. S. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00833-8 · Cited by 42 · see citing works
On the interpretability of quantum neural networks
Pira, L., & Ferrie, C. (2024) · Quantum Machine Intelligence · doi:10.1007/s42484-024-00191-y · Cited by 42 · see citing works
McCoy, L. G., Ng, F. Y. C., Sauer, C. M., Legaspi, K. E. Y., Jain, B., Gallifant, J., McClurkin, M., Hammond, A., Goode, D., Gichoya, J. W., et al. (2024) · BMC Medical Education · doi:10.1186/s12909-024-06048-z · Cited by 48 · see citing works
Self-citationGonzález-Alday, R., García-Cuesta, E., Maojo, & Kulikowski, C. (2023) · Applied Sciences · doi:10.3390/app131910778 · Cited by 38 · see citing works
Li, L. T., Haley, L. C., Boyd, A. K., & Bernstam, E. V. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104531 · Cited by 45 · see citing works
Blackman, J., & Veerapen, R. J. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-02891-2 · Cited by 38 · see citing works
Carreras, J., Nakamura, N., & Hamoudi, R. (2022) · Healthcare · doi:10.3390/healthcare10010155 · Cited by 36 · see citing works
A Framework for Interpretability in Machine Learning for Medical Imaging
Wang, A. Q., Karaman, B. K., Kim, H., Rosenthal, J., Saluja, R., Young, S. I., & Sabuncu, M. R. (2024) · IEEE Access · doi:10.1109/access.2024.3387702 · Cited by 37 · see citing works
Oberste, L., & Heinzl, A. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3227225 · Cited by 35 · see citing works
Why we do need explainable AI for healthcare
Cinà, G., Röber, T. E., Goedhart, R., & Birbil, Ş. İ. (2025) · Diagnostic and Prognostic Research · doi:10.1186/s41512-025-00209-4 · Cited by 39 · see citing works
DeepFixCX : Explainable privacy‐preserving image compression for medical image analysis
Gaudio, A., Smailagic, A., Faloutsos, C., Mohan, S., Johnson, E., Liu, Y., Costa, P., & Campilho, A. (2023) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1495 · Cited by 31 · see citing works
Recommendations for using artificial intelligence in clinical flow cytometry
Ng, D. P., Simonson, P. D., Tárnok, A., Lucas, F. M., Kern, W., Rolf, N., Bogdanoski, G., Green, C., Brinkman, R. R., & Czechowska, K. (2024) · Cytometry Part B Clinical Cytometry · doi:10.1002/cyto.b.22166 · Cited by 31 · see citing works
Prognosticating the outcome of intensive care in older patients—a narrative review
Beil, M., Moreno, R. P., Fronczek, J., Kogan, Y., Moreno, R. P., Flaatten, H. K., Guidet, B., de Lange, D. W., Leaver, S. K., Nachshon, A., et al. (2024) · Annals of Intensive Care · doi:10.1186/s13613-024-01330-1 · Cited by 29 · see citing works
Against explainability requirements for ethical artificial intelligence in health care
Kawamleh, S. (2022) · AI and Ethics · doi:10.1007/s43681-022-00212-1 · Cited by 27 · see citing works
Ursin, F., Lindner, F., Ropinski, T., Salloch, S., & Timmermann, C. (2023) · Ethik in der Medizin · doi:10.1007/s00481-023-00761-x · Cited by 28 · see citing works
Gashi, M., Vuković, M., Jekic, N., Thalmann, S., Holzinger, A., Jean-Quartier, C., & Jeanquartier, F. (2022) · BioMedInformatics · doi:10.3390/biomedinformatics2010009 · Cited by 25 · see citing works
Fahmi, A., Wong, D., Walker, L. E., Buchan, I. E., Pirmohamed, M., Sharma, A. K., Cant, H. E. O., Ashcroft, D. M., & van Staa, T. P. (2023) · PLoS ONE · doi:10.1371/journal.pone.0281466 · Cited by 24 · see citing works
González, R., Saha, A., Campbell, C. J. V., Nejat, P., Lokker, C., & Norgan, A. P. (2023) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2023.100347 · Cited by 21 · see citing works
Jennings, A. M., & Cox, D. J. (2023) · Behavior Analysis in Practice · doi:10.1007/s40617-023-00868-z · Cited by 19 · see citing works
Lans, A., Pierik, R., Bales, J. R., Fourman, M. S., Shin, D., Kanbier, L. N., Rifkin, J., DiGiovanni, W., Chopra, R. R., Moeinzad, R., et al. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102396 · Cited by 19 · see citing works
Machine Learning for Precision Agriculture and Crop Yield Optimization
Roy, P., Padhiary, M., Hoque, A., Sahu, B., Roy, D., & Kumar, K. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-9208-9.ch007 · Cited by 18 · see citing works
In humble defense of unexplainable black box prediction models in healthcare
van Royen, F. S.-A., Weerts, H., de Hond, A., Geersing, G., Rutten, F. H., Moons, K. G. M., & van Smeden, M. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.112013 · Cited by 18 · see citing works
The emerging role of artificial intelligence enabled electrocardiograms in healthcare
Sau, A., & Ng, F. S. (2023) · BMJ Medicine · doi:10.1136/bmjmed-2022-000193 · Cited by 16 · see citing works
Sun, Z., Yuan, Y., Farrahi, V., Herold, F., Xia, Z., Xiong, X., Qiao, Z., Shi, Y., Yang, Y., Qi, K., et al. (2024) · BMC Public Health · doi:10.1186/s12889-024-20510-z · Cited by 16 · see citing works
Li, F., Li, J., Lu, Q., You, Y., Xu, Z., Liu, L., Fu, L., Peng, G., Yi, J., & Li, C. (2025) · Materials & Design · doi:10.1016/j.matdes.2025.113948 · Cited by 15 · see citing works
Prediction and explainability in AI: Striking a new balance?
Raz, A. E., Heinrichs, B., Avnoon, N., Eyal, G., & Inbar, Y. (2024) · Big Data & Society · doi:10.1177/20539517241235871 · Cited by 15 · see citing works
Graph Neural Networks for Parkinson’s Disease Monitoring and Alerting
Zafeiropoulos, N., Bitilis, P., Tsekouras, G. E., & Kotis, K. (2023) · Sensors · doi:10.3390/s23218936 · Cited by 14 · see citing works
A prediction framework for pharmaceutical drug consumption using short time-series
Bertolotti, F., Schettini, F., Ferrario, L. B., Bellavia, D., & Foglia, E. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124265 · Cited by 14 · see citing works
McCradden, M. D., Thai, K., Assadi, A., Tonekaboni, S., Stedman, I., Joshi, S. D., Zhang, M., Chevalier, F., & Goldenberg, A. (2025) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2024-112919 · Cited by 17 · see citing works
Machine Learning in Computer Aided Engineering
Montáns, F. J., Cueto, E., & Bathe, K. (2023) · Computational methods in engineering & the sciences · doi:10.1007/978-3-031-36644-4_1 · Cited by 13 · see citing works
Early Warning Systems for Critical Illness Outside the Intensive Care Unit
Henry, K. E., & Giannini, H. M. (2024) · Critical Care Clinics · doi:10.1016/j.ccc.2024.03.007 · Cited by 13 · see citing works
Prinzi, F., Militello, C., Zarcaro, C., Bartolotta, T. V., Gaglio, S., & Vitabile, S. (2025) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2024.108576 · Cited by 11 · see citing works
Mohanraj, P., Raman, V., & Ramanathan, S. (2024) · Diagnostics · doi:10.3390/diagnostics14192181 · Cited by 10 · see citing works
Phillips, V. J. (2025) · AI & Society · doi:10.1007/s00146-025-02337-3 · Cited by 11 · see citing works
Acoustic and Natural Language Markers for Bipolar Disorder: A Pilot, mHealth Cross-Sectional Study
Crocamo, C., Cioni, R. M., Canestro, A., Nasti, C., Palpella, D., Piacenti, S., Bartoccetti, A., Re, M., Simonetti, V., Pietro, C. B. D. S., et al. (2025) · JMIR Formative Research · doi:10.2196/65555 · Cited by 10 · see citing works
Applications of Machine Learning in Healthcare: An Overview
Samarpita, S., & Satpathy, R. (2022) · IEEE International Conference on Industrial Electronics: Developments & Applications (ICIDeA) · doi:10.1109/icidea53933.2022.9970177 · Cited by 10 · see citing works
Alarcon, G., & Willis, S. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.134 · Cited by 10 · see citing works
A time series algorithm to predict surgery in neonatal necrotizing enterocolitis
Cui, C., Qiu, L., Li, L., Chen, F., Liu, X., Sun, H. W., Liu, X.-C., Bao, L., & Li, L.-Q. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02695-w · Cited by 9 · see citing works
Ratkevičiūtė, K., & Aliukonis, V. (2025) · Healthcare · doi:10.3390/healthcare13121429 · Cited by 11 · see citing works
Nowrozy, R., Ahmed, K. E. U., & Wang, H. (2025) · PLoS ONE · doi:10.1371/journal.pone.0310553 · Cited by 12 · see citing works
Ma, F.-Q., He, C., Yang, H.-R., Hu, Z., Mao, H.-R., Fan, C.-Y., Qi, Y., Zhang, J., & Xu, B. (2023) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-023-02192-6 · Cited by 8 · see citing works
A review of applications of artificial intelligence in cardiorespiratory rehabilitation
Raja, M. A., Loughran, R., & McCaffery, F. (2023) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2023.101327 · Cited by 8 · see citing works
Automatic Assessment of Speech Intelligibility using Consonant Similarity for Head and Neck Cancer
Quintas, S., Mauclair, J., Woisard, V., & Pinquier, J. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-182 · Cited by 8 · see citing works
Gallardo‐Pizarro, A., Peyrony, O., Chumbita, M., Monzó, P., Aiello, T. F., Teijón‐Lumbreras, C., Gras, E., Mensa, J., Soriano, À., & García‐Vidal, C. (2024) · Expert Review of Anti-infective Therapy · doi:10.1080/14787210.2024.2322445 · Cited by 8 · see citing works
Artificial intelligence analysis of the single-lead ECG predicts long-term clinical outcomes
Alrumayh, A., Bächtiger, P., Sau, A., Mansell, J., Almonte, M. T., Chhatwal, K., Ng, F. S., Kelshiker, M. A., & Peters, N. S. (2025) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztaf057 · Cited by 7 · see citing works
Cross-platform multi-cancer histopathology classification using local-window vision transformers
Nayeem, M. D., Nisita, N. J., Islam, M. M., Rahman, M. S., & Ali, A. B. M. S. (2025) · Scientific Reports · doi:10.1038/s41598-025-24791-1 · Cited by 7 · see citing works
ChatGPT and beyond with artificial intelligence (AI) in health: Lessons to be learned
Thiébaut, R., Hejblum, B. P., Fleur Mougin, A., Tzourio, C., & Richert, L. E. (2023) · Joint Bone Spine · doi:10.1016/j.jbspin.2023.105607 · Cited by 6 · see citing works
Rahman, M. M., Calhoun, V. D., & Plis, S. (2026) · Imaging Neuroscience · doi:10.1162/imag.a.1129 · Cited by 14 · see citing works
Arends, B. K. O., McCormick, J. M., van der Harst, P., Heus, P., & van Es, R. (2025) · European Journal of Clinical Investigation · doi:10.1111/eci.14387 · Cited by 5 · see citing works
Great debate: artificial intelligence will replace much of what cardiologists do
Averbuch, T., Asselbergs, F. W., Vardas, P. E., & Spall, H. G. C. V. (2025) · European Heart Journal · doi:10.1093/eurheartj/ehaf305 · Cited by 5 · see citing works
Moral Values in Medical AI: A Scoping ReviewPreprint
Victor, G., Barbu, A., & Bélisle‐Pipon, J. (2024) · Research Square · doi:10.21203/rs.3.rs-4391239/v1 · Cited by 5 · see citing works
Hlede, V., Valanci, S., D’Antuono, G., Dow, H., Wiggins, R. H., Dorman, T., & O’Beirne, R. (2026) · JMIR Medical Education · doi:10.2196/69156 · Cited by 4 · see citing works
Using AI explainable models and handwriting/drawing tasks for psychological well-being
Prinzi, F., Barbiero, P., Greco, C., Amorese, T., Cordasco, G., Lió, P., Vitabile, S., & Esposito, A. A. (2024) · Information Systems · doi:10.1016/j.is.2024.102465 · Cited by 4 · see citing works
Ardahanlı, İ., Aslan, R., Akdoğan, M., Akgün, O., & Ardahanlı, A. (2025) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2025.153926 · Cited by 4 · see citing works
Patiño, G. A., & Roberts, L. W. (2024) · Academic Medicine · doi:10.1097/acm.0000000000005793 · Cited by 4 · see citing works
Imperfect physics-guided neural networks
Carter, A., Imtiaz, S. A., & Naterer, G. F. (2024) · Chemical Engineering Science · doi:10.1016/j.ces.2024.121153 · Cited by 3 · see citing works
Zhao, Y., Ren, B., Yu, W., Zhang, H., Zhao, D., Lv, J., Xie, Z., Jiang, K., Shang, L., Yao, H., et al. (2022) · Neurology and Therapy · doi:10.1007/s40120-022-00355-7 · Cited by 3 · see citing works
New Era’s of Artificial Intelligence in Pharmaceutical Industries
Dubey, A., & Yadav, A. P. (2024) · Asian Journal of Pharmaceutical Research and Development · doi:10.22270/ajprd.v12i2.1362 · Cited by 3 · see citing works
State-space-guided neural networks for fault detection
Carter, A., Rezaei, A., Imtiaz, S. A., & Naterer, G. F. (2025) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2025.109260 · Cited by 2 · see citing works
Kassem, A., Folkes, J., Mukherjee, S., & Rosengarten, J. A. (2025) · The Egyptian Heart Journal · doi:10.1186/s43044-025-00689-1 · Cited by 2 · see citing works
Wu, W., Mei, B., Wu, Y., Liu, L., Wang, Y., Lombardo, M., & Yang, H. (2026) · Food Research International · doi:10.1016/j.foodres.2026.118499 · Cited by 2 · see citing works
Hajikarimloo, B., Mohammadzadeh, I., Tos, S. M., Mortezaei, A., & Habibi, M. A. (2026) · Acta Neurochirurgica · doi:10.1007/s00701-026-06775-w · Cited by 2 · see citing works
Artificial Intelligence in Healthcare
Choudhary, R., Kaushik, A., & Igulu, K. T. (2022) · Advances in business information systems and analytics book series · doi:10.4018/978-1-6684-4950-9.ch001 · Cited by 2 · see citing works
ChatGPT et au-delà avec l’intelligence artificielle en santé : des leçons à tirer
Thiébaut, R., Hejblum, B. P., Fleur Mougin, A., Tzourio, C., & Richert, L. E. (2023) · Revue du Rhumatisme · doi:10.1016/j.rhum.2023.07.007 · Cited by 2 · see citing works
Inter-Pol: An Interpretable Machine Learning Framework for Solvent Polarity Prediction
Ugurlu, S. Y. (2025) · Journal of Solution Chemistry · doi:10.1007/s10953-025-01508-6 · Cited by 1 · see citing works
Sleptsova, E. E., Vershinina, O., Ivanchenko, M., & Turubanova, V. D. (2026) · International Journal of Molecular Sciences · doi:10.3390/ijms27020667 · Cited by 1 · see citing works
Dave, A., Dave, A., & Moussa, I. D. (2026) · Bioengineering · doi:10.3390/bioengineering13020167 · Cited by 1 · see citing works
Machine Learning aplicado à atenção domiciliar para predição de condição de óbito
Silva, D. H. C., & Timo, E. M. D. N. (2022) · Research Society and Development · doi:10.33448/rsd-v11i14.36078 · Cited by 1 · see citing works
Cui, C., Chen, F., & Li, L.-Q. (2023) · PubMed Central · doi:10.7499/j.issn.1008-8830.2302165 · Cited by 1 · see citing works
The fundamentals of AI ethics in medical imaging
Amann, J., Bürger, V. K., Livne, M., Bui, C. K. T., & Madai, V. I. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-323761-4.00010-9 · Cited by 3 · see citing works
Algorithmic Transparency and Algorithmic Accountability in Organizations
Mastrogiorgio, A., Nauta, P. D., & Martinez, M. (2025) · Lecture notes in information systems and organisation · doi:10.1007/978-3-032-01396-5_8 · Cited by 1 · see citing works
KI in der Medizin: Ethische Aspekte
Wágner, R. (2023) · Zeitschrift für medizinische Ethik · doi:10.30965/29498570-20230010 · Cited by 1 · see citing works
Prosperi, M., Bian, J., Harle, C. A., Wang, M., Lee, G., Li, Y., Štiglic, G., Kleesiek, J., Pongtriang, P., & Veltri, P. (2023) · ACM SIGBioinformatics Record · doi:10.1145/3615362.3615363
HealSmart Homes: Integrating Technology for Enhanced Convalescent Care
Kanimozhi, T. (2024) · Journal of Electrical Systems · doi:10.52783/jes.3537
Santos, F., Suárez, F., & Sallán, J. G. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-97-3305-7_15
Machine learning in predicting postoperative complications in Crohn’s disease
Zhang, L., Chen, L.-X., Yang, W.-J., & Hu, B. (2024) · World Journal of Gastrointestinal Surgery · doi:10.4240/wjgs.v16.i8.2745
Hlede, V., Valanci, S., D’Antuono, G., Dow, H., Wiggins, R. H., Dorman, T., & O'Beirne, R. (2024) · doi:10.2196/preprints.69156
Patiño, G. A., Amiel, J. M., Brown, M. E. L., Lypson, M. L., & Chan, T. M. (2024) · doi:10.32920/27926490.v1
Patiño, G. A., Amiel, J. M., Brown, M. E. L., Lypson, M. L., & Chan, T. M. (2024) · doi:10.32920/27926490
Validation of a Novel Data-Driven Algorithm to Detect Atypical Prescriptions in Radiation Therapy
Thropp, C., Hepel, J. T., Leech, T., Klein, E., & Li, Q. (2025) · Advances in Radiation Oncology · doi:10.1016/j.adro.2025.101804
Pervaiz, K., & Bawa, S. S. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-93601-2_6 · Cited by 2 · see citing works
- Authors
- McCoy, L. G., Brenna, C. T. A., Chen, S. S., Vold, K., & Das, S.
- Affiliations
- University of Toronto (CA)
- Corresponding author
- McCoy, Liam G.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2022-02
- Volume
- 142
- Pages
- 252-257
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W3213880547
- Cited by
- 118
- References
- 64
- Citations by year
- 2026: 17 · 2025: 28 · 2024: 31 · 2023: 23 · 2022: 19
- FWCI
- 1.32
- Type
- journal article
- Language
- en
- Open access
- green
- philpapers.org
- Licence
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- PhilPapers (PhilPapers Foundation) (submittedVersion)
- Funders
- Canadian Institutes of Health Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- explainability, machine learning in healthcare, artificial intelligence in healthcare, evidence-based medicine, mechanistic reasoning, algorithm evaluation, trust in algorithms, clinical judgment, ethical use of AI, black-box models
- MeSH terms
- Machine Learning, Artificial Intelligence, Delivery of Health Care, Humans, Technology, Trust
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
