The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Practical guide to SHAP analysis: Explaining supervised machine learning model predictions in drug development
Open accessAbstract
Despite increasing interest in using Artificial Intelligence (AI) and Machine Learning (ML) models for drug development, effectively interpreting their predictions remains a challenge, which limits their impact on clinical decisions. We address this issue by providing a practical guide to SHapley Additive exPlanations (SHAP), a popular feature‐based interpretability method, which can be seamlessly integrated into supervised ML models to gain a deeper understanding of their predictions, thereby enhancing their transparency and trustworthiness. This tutorial focuses on the application of SHAP analysis to standard ML black‐box models for regression and classification problems. We provide an overview of various visualization plots and their interpretation, available software for implementing SHAP, and highlight best practices, as well as special considerations, when dealing with binary endpoints and time‐series models. To enhance the reader's understanding for the method, we also apply it to inherently explainable regression models. Finally, we discuss the limitations and ongoing advancements aimed at tackling the current drawbacks of the method.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551
17. A Value for n-Person Games
Shapley, L. S. (1953) · Contributions to the Theory of Games (AM-28), Volume II · doi:10.1515/9781400881970-018 · Cited by 6,950
Machine Learning Interpretability: A Survey on Methods and Metrics
Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019) · Electronics · doi:10.3390/electronics8080832 · Cited by 1,842
Fisher, A., Rudin, C., & Dominici, F. (2018) · arXiv · doi:10.48550/arxiv.1801.01489 · Cited by 1,165
National Center for Health Statistics (NCHS)
(2008) · Encyclopedia of Global Health · doi:10.4135/9781412963855.n815 · Cited by 1,104
Aas, K., Jullum, M., & Løland, A. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103502 · Cited by 1,023
Nuclear feature extraction for breast tumor diagnosis:
Street, W. N., Wolberg, W. H., & Mangasarian, O. L. (1993) · SPIE Proceedings · doi:10.1117/12.148698 · Cited by 526
Marques, L., Costa, B., Pereira, M., Silva, A., Santos, J., Saldanha, L., Silva, I., Magalhães, P., Schmidt, S., & Vale, N. (2024) · Pharmaceutics · doi:10.3390/pharmaceutics16030332 · Cited by 390
An Efficient Explanation of Individual Classifications using Game Theory
StrumbeljErik, & KononenkoIgor (2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1756007 · Cited by 380
Benchmarking Deep Learning Interpretability in Time Series PredictionsPreprint
Ismail, A. A., Gunady, M. K., Bravo, H. C., & Feizi, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.13924 · Cited by 297
Early lung cancer diagnostic biomarker discovery by machine learning methods
Xie, Y., Meng, W.-Y., Li, R.-Z., Wang, Y.-W., Qian, X., Chan, C., Yu, Z.-F., Fan, X.-X., Pan, H.-D., Xie, C., et al. (2021) · Translational Oncology · doi:10.1016/j.tranon.2020.100907 · Cited by 269
Machine learning for comprehensive forecasting of Alzheimer’s Disease progression
Fisher, C. K., Smith, A. M., Walsh, J. R., Diseases, C. A. M., Simon, A. J., Edgar, C., Jack, C. R., Holtzman, D., Russell, D., Hill, D., et al. (2019) · Scientific Reports · doi:10.1038/s41598-019-49656-2 · Cited by 223
TimeSHAP: Explaining Recurrent Models through Sequence Perturbations
Bento, J., Saleiro, P., Cruz, A. F., Figueiredo, M. A. T., & Bizarro, P. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467166 · Cited by 168
Evaluation of post-hoc interpretability methods in time-series classification
Turbé, H., Bjelogrlic, M., Lovis, C., & Mengaldo, G. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00620-w · Cited by 160
SurvSHAP(t): Time-dependent explanations of machine learning survival models
Krzyziński, M., Spytek, M., Baniecki, H., & Biecek, P. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110234 · Cited by 147
Ayano, Y. M., Schwenker, F., Dufera, B. D., & Debelee, T. G. (2022) · Diagnostics · doi:10.3390/diagnostics13010111 · Cited by 142
Prediction of disease progression and outcomes in multiple sclerosis with machine learning
Pinto, M. F., Oliveira, H., Batista, S., Cruz, L., Pinto, M., Correia, I., Martins, P., & Teixeira, C. (2020) · Scientific Reports · doi:10.1038/s41598-020-78212-6 · Cited by 129
Opportunities and challenges using artificial intelligence in ADME/Tox
Bhhatarai, B., Walters, W. P., Hop, C. E. C. A., Lanza, G., & Ekins, S. (2019) · Nature Materials · doi:10.1038/s41563-019-0332-5 · Cited by 121
Disease Progression Modeling in Chronic Obstructive Pulmonary Disease
Young, A. L., Bragman, F., Rangelov, B., Han, M. K., Galbán, C. J., Lynch, D. A., Hawkes, D. J., Alexander, D. C., & Hurst, J. R. (2019) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201908-1600oc · Cited by 110
GraphSVX: Shapley Value Explanations for Graph Neural Networks
Duval, A., & Malliaros, F. D. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86520-7_19 · Cited by 107
Keutzer, L., You, H., Farnoud, A., Nyberg, J., Wicha, S. G., Maher‐Edwards, G., Vlasakakis, G., Moghaddam, G. K., Svensson, E. M., Menden, M. P., et al. (2022) · Pharmaceutics · doi:10.3390/pharmaceutics14081530 · Cited by 104
Lu, J. J.-Q., Bender, B. C., Jin, J. Y., & Guan, Y. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00357-4 · Cited by 85
WindowSHAP: An efficient framework for explaining time-series classifiers based on Shapley values
Nayebi, A., Tipirneni, S., Reddy, C. K. A., Foreman, B. P., & Subbian, V. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104438 · Cited by 83
Terranova, N., Renard, D., Shahin, M. H., Menon, S., Cao, Y., Hop, C. E. C. A., Hayes, S., Madrasi, K., Stodtmann, S., Tensfeldt, T., et al. (2024) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.3053 · Cited by 78
Self-citationFang, C., Xu, D. H., Su, J., Dry, J. R., & Linghu, B. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00381-z · Cited by 53
Zhang, W., Roy Burman, S. S., Chen, J., Donovan, K. A., Cao, Y., Shu, C., Zhang, B., Zeng, Z., Gu, S., Zhang, Y., et al. (2022) · Genomics, Proteomics & Bioinformatics · doi:10.1016/j.gpb.2022.11.008 · Cited by 51
Multiple stakeholders drive diverse interpretability requirements for machine learning in healthcare
Imrie, F., Davis, R. I., & van der Schaar, M. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00698-2 · Cited by 50
Liu, Q., Zhu, H., Liu, C., Jean, D., Huang, S. M., ElZarrad, M. K., Blumenthal, G. M., & Wang, Y. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1771 · Cited by 49
Application of Deep Neural Network Models in Drug Discovery Programs
Grebner, C., Matter, H., Kofink, D., Wenzel, J., Schmidt, F., & Heßler, G. (2021) · ChemMedChem · doi:10.1002/cmdc.202100418 · Cited by 49
Janßen, A., Hoogendoorn, M., Cnossen, M. H., & Mathôt, R. A. A. (2022) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12828 · Cited by 39
Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progression
Qian, Z., Zame, W. R., Fleuren, L. M., Elbers, P., & van der Schaar, M. (2021) · Pure Amsterdam UMC · doi:10.48550/arxiv.2106.02875 · Cited by 35
Basu, S., Munafo, A., Ben-Amor, A.-F., Roy, S., Girard, P., & Terranova, N. (2022) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1002/psp4.12796 · Cited by 32
Zhu, X., Zhang, M., Wen, Y., & Shang, D. (2022) · Frontiers in Pharmacology · doi:10.3389/fphar.2022.994665 · Cited by 27
Salih, A. M., Galazzo, I. B., Cruciani, F., Brusini, L., & Radeva, P. I. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897253 · Cited by 20
Harun, R., Lu, J. J.-Q., Kassir, N., & Zhang, W. (2023) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.3076 · Cited by 17
Harun, R., Yang, E., Kassir, N., Zhang, W., & Lu, J. J.-Q. (2023) · Pharmaceutics · doi:10.3390/pharmaceutics15051381 · Cited by 12
Cui, Z. L., KADZIOLA, Z. A., Lipkovich, I. A., Faries, D. E., Sheffield, K. M., & Carter, G. C. (2021) · Journal of Comparative Effectiveness Research · doi:10.2217/cer-2020-0230 · Cited by 10
Cited by
Kabir, S., Hossain, M. S., & Andersson, K. (2025) · Algorithms · doi:10.3390/a18090556 · Cited by 44
Celepli, S., Bigat, I., Karakas, B., Tezcan, H. M., Yar, M. D., Celepli, P., Aksahin, M. F., Hancerliogullari, O., Yilmaz, Y. F., & Erogul, O. (2025) · Diagnostics · doi:10.3390/diagnostics15162065 · Cited by 6
Wu, L., Mai, L., Wang, H., Huang, J., He, X., Zhan, X., Khalemsky, A., Kumar, V. A., Paxton, J. H., Tsilimingras, D., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02522-8 · Cited by 8
Metadata
- Authors
- Ponce‐Bobadilla, A. V., Schmitt, V., Maier, C. S., Mensing, S., & Stodtmann, S.
- Venue
- Clinical and Translational Science
- Publisher
- Wiley
- Published
- 2024-10-28
- Volume
- 17
- Issue
- 11
- Pages
- e70056-e70056
- ISSN
- 1752-8054 (ISSN-L)
- 1752-8062
- OpenAlex
- W4403845965
- Cited by
- 826
- References
- 40
- FWCI
- 122.93
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.