Interpretation of machine learning models using shapley values: application to compound potency and multi-target activity predictions
Open accessAbstract
Difficulties in interpreting machine learning (ML) models and their predictions limit the practical applicability of and confidence in ML in pharmaceutical research. There is a need for agnostic approaches aiding in the interpretation of ML models regardless of their complexity that is also applicable to deep neural network (DNN) architectures and model ensembles. To these ends, the SHapley Additive exPlanations (SHAP) methodology has recently been introduced. The SHAP approach enables the identification and prioritization of features that determine compound classification and activity prediction using any ML model. Herein, we further extend the evaluation of the SHAP methodology by investigating a variant for exact calculation of Shapley values for decision tree methods and systematically compare this variant in compound activity and potency value predictions with the model-independent SHAP method. Moreover, new applications of the SHAP analysis approach are presented including interpretation of DNN models for the generation of multi-target activity profiles and ensemble regression models for potency prediction.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Statistical methods to quantify contribution of certain chemical features to the predicted outcome have been suggested, borrowing concepts from game theory.
“Similarly, statistical methods to quantify contribution of certain chemical features to the predicted outcome have been suggested, borrowing concepts from “game theory”.275”
From Advancing Targeted Protein Degradation via Multiomics Profiling and Artificial Intelligence (2023) · cites this paper as [275]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works
The Nature of Statistical Learning Theory
Vapnik, V. (2000) · Statistics for Engineering and Information Science · doi:10.1007/978-1-4757-3264-1 · Cited by 46,387 · see citing works
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138 · see citing works
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · see citing works
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
Seel, N. M. (2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5195 · Cited by 19,525 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
TensorFlow: A system for large-scale machine learningPreprint
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.5555/3026877.3026899 · Cited by 16,369 · see citing works
A tutorial on support vector regression
Smola, A. J., & Schölkopf, B. (2004) · Statistics and Computing · doi:10.1023/b:stco.0000035301.49549.88 · Cited by 13,482 · see citing works
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551 · see citing works
Geurts, P., Ernst, D., & Wehenkel, L. (2006) · Machine Learning · doi:10.1007/s10994-006-6226-1 · Cited by 9,300 · see citing works
Extended-Connectivity Fingerprints
Rogers, D. O. W., & Hahn, M. (2010) · Journal of Chemical Information and Modeling · doi:10.1021/ci100050t · Cited by 7,597 · see citing works
Friedman, J. H. (2002) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(01)00065-2 · Cited by 7,067 · see citing works
(1994) · Cited by 5,513 · see citing works
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399 · see citing works
ChEMBL: a large-scale bioactivity database for drug discovery
Gaulton, A., Bellis, L. J., Bento, A. P., Chambers, J., Davies, M., Hersey, A., Light, Y., McGlinchey, S., Michalovich, D., Al-Lazikani, B., et al. (2012) · Nucleic Acids Research · doi:10.1093/nar/gkr777 · Cited by 4,382 · see citing works
Abe, S. (2001) · Springer eBooks · doi:10.1007/978-1-4471-0285-4 · Cited by 3,740 · see citing works
ZINC 15 – Ligand Discovery for Everyone
Sterling, T., & Irwin, J. J. (2015) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00559 · Cited by 3,224 · see citing works
Pattern recognition and machine learning
(2007) · Choice Reviews Online · doi:10.5860/choice.44-5091 · Cited by 2,782 · see citing works
Advances in Neural Information Processing Systems 27 (NIPS 2014)
Zhong, M., Goddard, N., & Sutton, C. A. (2014) · Cited by 2,440 · see citing works
QSAR Modeling: Where have you been? Where are you going to?
Cherkasov, A., Muratov, E. N., Fourches, D., Varnek, A., Baskin, I. I., Cronin, M., Dearden, J., Gramatica, P., Martin, Y. C., Todeschini, R., et al. (2014) · Journal of Medicinal Chemistry · doi:10.1021/jm4004285 · Cited by 2,164 · see citing works
The Balanced Accuracy and Its Posterior Distribution
Brodersen, K. H., Ong, C. S., Stephan, K. E., & Buhmann, J. M. (2010) · 2010 20th International Conference on Pattern Recognition · doi:10.1109/icpr.2010.764 · Cited by 1,816 · see citing works
Machine learning in chemoinformatics and drug discovery
Lo, Y.-C., Rensi, S. E., Torng, W., & Altman, R. B. (2018) · Drug Discovery Today · doi:10.1016/j.drudis.2018.05.010 · Cited by 960 · see citing works
Neural Networks and Deep Learning
Aggarwal, C. (2018) · Cambridge International Law Journal · doi:10.1007/978-3-319-94463-0 · Cited by 932 · see citing works
Monotonic solutions of cooperative games
Young, H. P. (1985) · International Journal of Game Theory · doi:10.1007/bf01769885 · Cited by 914 · see citing works
Machine-learning approaches in drug discovery: methods and applications
Lavecchia, A. (2014) · Drug Discovery Today · doi:10.1016/j.drudis.2014.10.012 · Cited by 801 · see citing works
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 636 · see citing works
Handbook of Uncertainty Quantification
Ghanem, R., Higdon, D., & Owhadi, H. (2016) · Springer eBooks · doi:10.1007/978-3-319-11259-6 · Cited by 594 · see citing works
Rodríguez-Pérez, R., & Bajorath, J. (2019) · Journal of Medicinal Chemistry · doi:10.1021/acs.jmedchem.9b01101 · Cited by 520 · see citing works
Self-citationMachine Learning Methods for Property Prediction in Chemoinformatics:Quo Vadis?
Varnek, A. A., & Baskin, I. I. (2012) · Journal of Chemical Information and Modeling · doi:10.1021/ci200409x · Cited by 245 · see citing works
Interpretation of Quantitative Structure-Activity Relationship Models: Past, Present, and Future
Polishchuk, P. (2017) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.7b00274 · Cited by 236 · see citing works
Introduction to Sensitivity Analysis
Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., & Tarantola, S. (2007) · doi:10.1002/9780470725184.ch1 · Cited by 178 · see citing works
So, S., & Richards, W. G. (1992) · Journal of Medicinal Chemistry · doi:10.1021/jm00095a016 · Cited by 138 · see citing works
Rodríguez-Pérez, R., Vogt, M., & Bajorath, J. (2017) · ACS Omega · doi:10.1021/acsomega.7b01079 · Cited by 113 · see citing works
Self-citationTrade-Off Between Accuracy and Interpretability for PredictiveIn SilicoModeling
Johansson, U., Sönströd, C., Norinder, U., & Boström, H. (2011) · Future Medicinal Chemistry · doi:10.4155/fmc.11.23 · Cited by 89 · see citing works
Multitask Machine Learning for Classifying Highly and Weakly Potent Kinase Inhibitors
Rodríguez-Pérez, R., & Bajorath, J. (2019) · ACS Omega · doi:10.1021/acsomega.9b00298 · Cited by 66 · see citing works
Self-citationStumpfe, D., Dimova, D., & Bajorath, J. (2016) · Journal of Medicinal Chemistry · doi:10.1021/acs.jmedchem.6b00906 · Cited by 61 · see citing works
Self-citationAn approach to the interpretation of backpropagation neural network models in QSAR studies
Baskin, I. I., Айт, А. О., Halberstam, N. M., Palyulin, V. A., & Зефиров, Н. С. (2002) · SAR and QSAR in environmental research · doi:10.1080/10629360290002073 · Cited by 55 · see citing works
Visual Interpretation of Kernel‐Based Prediction Models
Hansen, K., Baehrens, D., Schroeter, T. S., Rupp, M., & Müller, K. (2011) · Molecular Informatics · doi:10.1002/minf.201100059 · Cited by 55 · see citing works
Visualization and Interpretation of Support Vector Machine Activity Predictions
Balfer, J., & Bajorath, J. (2015) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00175 · Cited by 51 · see citing works
Self-citationDimova, D., & Bajorath, J. (2017) · Molecules · doi:10.3390/molecules22050730 · Cited by 16 · see citing works
Self-citationBalfer, J., & Bajorath, J. (2014) · Journal of Chemical Information and Modeling · doi:10.1021/ci500410g · Cited by 14 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- [21]Shapley LS Kuhn HW Tucker AW A value for N-person games. Contributions to the theory of games Annals of mathematical studies 1953 Princeton Princeton University Press 307 317
Cited by
Sorted by citations
Drug discovery with explainable artificial intelligence
Jiménez-Luna, J., Grisoni, F., & Schneider, G. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00236-4 · Cited by 1,063 · see citing works
Recent advances in AI-based toxicity prediction for drug discovery
Lee, H., Kim, J., Kim, J.-W., & Lee, Y. (2025) · Frontiers in Chemistry · doi:10.3389/fchem.2025.1632046 · Cited by 76 · see citing works
Advancing Targeted Protein Degradation via Multiomics Profiling and Artificial Intelligence
Duran-Frigola, M., Cigler, M., & Winter, G. E. (2023) · Journal of the American Chemical Society · doi:10.1021/jacs.2c11098 · Cited by 50 · see citing works
- Authors
- Rodríguez-Pérez, R., & Bajorath, J.
- Affiliations
- University of Bonn (DE)
- Corresponding author
- Bajorath, Jürgen
- Venue
- Journal of Computer-Aided Molecular Design
- Published
- 2020-05-02
- Volume
- 34
- Issue
- 10
- Pages
- 1013-1026
- ISSN
- 0920-654X (ISSN-L)
- 1573-4951
- PMC
- PMC7449951
- OpenAlex
- W3022298391
- Cited by
- 718
- References
- 43
- Citations by year
- 2026: 126 · 2025: 208 · 2024: 150 · 2023: 95 · 2022: 91 · 2021: 42 · 2020: 6
- FWCI
- 29.00
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Journal of Computer-Aided Molecular Design (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
- Keywords
- SHAP, Shapley value, model interpretability, decision tree, deep neural network, ensemble regression, feature importance, model-agnostic explanation
- MeSH terms
- Machine Learning, Pharmaceutical Preparations, Humans, Models, Molecular, Structure-Activity Relationship, Therapeutic Equivalency, Neural Networks, Computer, Drug Discovery
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
