Benchmarking Molecular Feature Attribution Methods with Activity Cliffs
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Feature attribution techniques are popular choices within the explainable artificial intelligence toolbox, as they can help elucidate which parts of the provided inputs used by an underlying supervised-learning method are considered relevant for a specific prediction. In the context of molecular design, these approaches typically involve the coloring of molecular graphs, whose presentation to medicinal chemists can be useful for making a decision of which compounds to synthesize or prioritize. The consistency of the highlighted moieties alongside expert background knowledge is expected to contribute to the understanding of machine-learning models in drug design. Quantitative evaluation of such coloring approaches, however, has so far been limited to substructure identification tasks. We here present an approach that is based on maximum common substructure algorithms applied to experimentally-determined activity cliffs. Using the proposed benchmark, we found that molecule coloring approaches in conjunction with classical machine-learning models tend to outperform more modern, graph-neural-network alternatives. The provided benchmark data are fully open sourced, which we hope will facilitate the testing of newly developed molecular feature attribution techniques.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
ChEMBL: a large-scale bioactivity database for drug discovery
Gaulton, A., Bellis, L. J., Bento, A. P., Chambers, J., Davies, M., Hersey, A., Light, Y., McGlinchey, S., Michalovich, D., Al-Lazikani, B., et al. (2012) · Nucleic Acids Research · doi:10.1093/nar/gkr777 · Cited by 4,382
Explainable AI: a review of machine learning interpretability methods
Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020) · Entropy · doi:10.3390/e23010018 · Cited by 2,922
Convolutional Networks on Graphs for Learning Molecular Fingerprints
Duvenaud, D., Maclaurin, D., Aguilera-Iparraguirre, J., Gómez-Bombarelli, R., Hirzel, T., Aspuru-Guzik, A., & Adams, R. P. (2015) · arXiv · doi:10.48550/arxiv.1509.09292 · Cited by 2,628
The rise of deep learning in drug discovery
Chen, H., Engkvist, O., Wang, Y., Olivecrona, M., & Blaschke, T. (2018) · Drug Discovery Today · doi:10.1016/j.drudis.2018.01.039 · Cited by 1,770
Explainable AI: Interpreting, Explaining and Visualizing Deep Learning
(2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6 · Cited by 1,377
Drug discovery with explainable artificial intelligence
Jiménez-Luna, J., Grisoni, F., & Schneider, G. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00236-4 · Cited by 1,063
Self-citationUnmasking Clever Hans predictors and assessing what machines really learn
Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K.-R. (2019) · Nature Communications · doi:10.1038/s41467-019-08987-4 · Cited by 1,047
On Outliers and Activity CliffsWhy QSAR Often Disappoints
Maggiora, G. M. (2006) · Journal of Chemical Information and Modeling · doi:10.1021/ci060117s · Cited by 728
A graph-convolutional neural network model for the prediction of chemical reactivity
Coley, C. W., Jin, W., Rogers, L., Jamison, T. F., Jaakkola, T. S., Green, W. H., Barzilay, R., & Jensen, K. F. (2019) · Chemical Science · doi:10.1039/c8sc04228d · Cited by 679
Extreme Gradient Boosting as a Method for Quantitative Structure–Activity Relationships
Sheridan, R. P., Wang, W. M., Liaw, A., Ma, J., & Gifford, E. M. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00591 · Cited by 544
Matched Molecular Pairs as a Medicinal Chemistry Tool: Miniperspective
Griffen, E., Leach, A. G., Robb, G. R., & Warner, D. J. (2011) · Journal of Medicinal Chemistry · doi:10.1021/jm200452d · Cited by 282
Marchese Robinson, R. L., Palczewska, A., Palczewski, J., & Kidley, N. (2017) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00753 · Cited by 167
Coloring Molecules with Explainable Artificial Intelligence for Preclinical Relevance Assessment
Jiménez-Luna, J., Skalic, M., Weskamp, N., & Schneider, G. (2021) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.0c01344 · Cited by 96
Self-citationLiu, R., Yu, X., & Wallqvist, A. (2015) · Journal of Cheminformatics · doi:10.1186/s13321-015-0053-y · Cited by 92
Sheridan, R. P. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00825 · Cited by 84
Universal Approach for Structural Interpretation of QSAR/QSPR Models
Polishchuk, P. G., Kuz'min, V. E., Artemenko, A. G., & Muratov, E. N. (2013) · Molecular Informatics · doi:10.1002/minf.201300029 · Cited by 76
Cited by
Exposing the Limitations of Molecular Machine Learning with Activity Cliffs
van Tilborg, D., Alenicheva, A., & Grisoni, F. (2022) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.2c01073 · Cited by 337
Unleashing the power of generative AI in drug discovery
Gangwal, A., & Lavecchia, A. (2024) · Drug Discovery Today · doi:10.1016/j.drudis.2024.103992 · Cited by 195
Quantitative evaluation of explainable graph neural networks for molecular property prediction
Rao, J., Zheng, S., Lu, Y., & Yang, Y. (2022) · Patterns · doi:10.1016/j.patter.2022.100628 · Cited by 90
Metadata
- Authors
- Jiménez-Luna, J., Skalic, M., & Weskamp, N.
- Venue
- Journal of Chemical Information and Modeling
- Publisher
- American Chemical Society (ACS)
- Published
- 2022-01-12
- Volume
- 62
- Issue
- 2
- Pages
- 274-283
- ISSN
- 1549-9596 (ISSN-L)
- 1549-960X
- OpenAlex
- W4220790374
- Cited by
- 47
- References
- 17
- FWCI
- 4.69
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.