Deep Learning-Based Conformal Prediction of Toxicity
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Predictive modeling for toxicity can help reduce risks in a range of applications and potentially serve as the basis for regulatory decisions. However, the utility of these predictions can be limited if the associated uncertainty is not adequately quantified. With recent studies showing great promise for deep learning-based models also for toxicity predictions, we investigate the combination of deep learning-based predictors with the conformal prediction framework to generate highly predictive models with well-defined uncertainties. We use a range of deep feedforward neural networks and graph neural networks in a conformal prediction setting and evaluate their performance on data from the Tox21 challenge. We also compare the results from the conformal predictors to those of the underlying machine learning models. The results indicate that highly predictive models can be obtained that result in very efficient conformal predictors even at high confidence levels. Taken together, our results highlight the utility of conformal predictors as a convenient way to deliver toxicity predictions with confidence, adding both statistical guarantees on the model performance as well as better predictions of the minority class compared to the underlying models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Graph neural networks: A review of methods and applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2020) · AI Open · doi:10.1016/j.aiopen.2021.01.001 · Cited by 6,880
Graph Neural Networks: A Review of Methods and Applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2018) · arXiv · doi:10.48550/arxiv.1812.08434 · Cited by 6,880
MoleculeNet: a benchmark for molecular machine learning
Wu, Z., Ramsundar, B., Feinberg, E. N., Gomes, J., Geniesse, C., Pappu, A. S., Leswing, K., & Pande, V. (2018) · Chemical Science · doi:10.1039/c7sc02664a · Cited by 3,107 · cited 1×
“42,43 To help put the performance of the models in perspective, we also provide the results 46 Our best models are comparable to some of the top leaderboard submissions, although not the best performing.”
Survey on deep learning with class imbalance
Johnson, J. M., & Khoshgoftaar, T. M. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0192-5 · Cited by 3,032
Analyzing Learned Molecular Representations for Property Prediction
Yang, K., Swanson, K., Jin, W., Coley, C., Eiden, P., Gao, H., Guzman-Perez, A., Hopper, T., Kelley, B., Mathea, M., et al. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.9b00237 · Cited by 1,923
The rise of deep learning in drug discovery
Chen, H., Engkvist, O., Wang, Y., Olivecrona, M., & Blaschke, T. (2018) · Drug Discovery Today · doi:10.1016/j.drudis.2018.01.039 · Cited by 1,770
DeepTox: toxicity prediction using deep learning
Mayr, A., Klambauer, G., Unterthiner, T., & Hochreiter, S. (2016) · Frontiers in Environmental Science · doi:10.3389/fenvs.2015.00080 · Cited by 1,081 · cited 1×
“3 As discussed above, predictions of toxicity is a field where prediction confidence is paramount, 24 but most deep learning methods do not give accurate quantification of the prediction uncertainty.”
BackgroundTsubaki, M., Tomii, K., & Sese, J. (2019) · Bioinformatics · doi:10.1093/bioinformatics/bty535 · Cited by 688
The US Federal Tox21 Program: A strategic and operational plan for continued leadership
Thomas, R. (2018) · ALTEX · doi:10.14573/altex.1803011 · Cited by 229
Wittwehr, C., Aladjov, H., Ankley, G., Byrne, H. J., de Knecht, J., Heinzle, E., Klambauer, G., Landesmann, B., Luijten, M., MacKay, C., et al. (2017) · Toxicological Sciences · doi:10.1093/toxsci/kfw207 · Cited by 161 · cited 1×
“3 As discussed above, predictions of toxicity is a field where prediction confidence is paramount, 24 but most deep learning methods do not give accurate quantification of the prediction uncertainty.”
BackgroundReliable Prediction Errors for Deep Neural Networks Using Test-Time Dropout
Cortés-Ciriano, I., & Bender, A. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.9b00297 · Cited by 51 · cited 1×
“13 14 – 16 However, this combination has not previously been explored for toxicity predictions.”
Background
Cited by
ToxinPred 3.0: An improved method for predicting the toxicity of peptides
Rathore, A. S., Choudhury, S., Arora, A., Tijare, P., & Raghava, G. P. S. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108926 · Cited by 317
The evolving role of investigative toxicology in the pharmaceutical industry
Pognan, F., Beilmann, M., Boonen, H. C. M., Czich, A., Dear, G., Hewitt, P., Mow, T., Oinonen, T., Roth, A., Steger-Hartmann, T., et al. (2023) · Nature Reviews Drug Discovery · doi:10.1038/s41573-022-00633-x · Cited by 255
Seoni, S., Jahmunah, V., Salvi, M., Barua, P. D., Molinari, F., & Acharya, U. R. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107441 · Cited by 239
Uncertainty-aware deep learning in healthcare: A scoping review
Loftus, T. J., Shickel, B., Ruppert, M. M., Balch, J. A., Ozrazgat-Baslanti, T., Tighe, P. J., Efron, P. A., Hogan, W. R., Rashidi, P., Upchurch, G. R., et al. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000085 · Cited by 73
“Best results were demonstrated for deep neural networks rather than random forest or light gradient boosting machine models, and conformal prediction offered a controllable error rate and better recall for all three model types [26].”
BackgroundReliable machine learning models in genomic medicine using conformal prediction
Papangelou, C., Kyriakidis, K., Natsiavas, P., Chouvarda, I., & Malousi, A. (2025) · Frontiers in Bioinformatics · doi:10.3389/fbinf.2025.1507448 · Cited by 12
Metadata
- Authors
- Zhang, J., Norinder, U., & Svensson, F.
- Venue
- Journal of Chemical Information and Modeling
- Publisher
- American Chemical Society (ACS)
- Published
- 2021-05-27
- Volume
- 61
- Issue
- 6
- Pages
- 2648-2657
- ISSN
- 1549-9596 (ISSN-L)
- 1549-960X
- OpenAlex
- W3165163721
- Cited by
- 65
- References
- 12
- FWCI
- 4.87
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.