The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Principles of QSAR models validation: internal and external
Open accessAbstract
The recent REACH Policy of the European Union has led to scientists and regulators to focus their attention on establishing general validation principles for QSAR models in the context of chemical regulation (previously known as the Setubal, nowadays, the OECD principles). This paper gives a brief analysis of some principles: unambiguous algorithm, Applicability Domain (AD), and statistical validation. Some concerns related to QSAR algorithm reproducibility and an example of a fast check of the applicability domain for MLR models are presented. Common myths and misconceptions related to popular techniques for verifying internal predictivity, particularly for MLR models (for instance cross‐validation, bootstrap), are commented on and compared with commonly used statistical techniques for external validation. The differences in the two validating approaches are highlighted, and evidence is presented that only models that have been validated externally, after their internal validation, can be considered reliable and applicable for both external prediction and regulatory purposes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Golbraikh, A., & Tropsha, A. (2002) · Journal of Molecular Graphics and Modelling · doi:10.1016/s1093-3263(01)00123-1 · Cited by 3,735
Hawkins, D. M. (2004) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci0342472 · Cited by 2,522
Tropsha, A., Gramatica, P., & Gombar, V. K. (2003) · QSAR & Combinatorial Science · doi:10.1002/qsar.200390007 · Cited by 2,188
Self-citationLinear Model Selection by Cross-validation
Shao, J. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10476299 · Cited by 1,790
Eriksson, L., Jaworska, J., Worth, A. P., Cronin, M. T. D., McDowell, R. M., & Gramatica, P. (2003) · Environmental Health Perspectives · doi:10.1289/ehp.5758 · Cited by 1,325
Self-citationGenetic algorithms as a strategy for feature selection
Leardi, R., Boggia, R., & Terrile, M. (1992) · Journal of Chemometrics · doi:10.1002/cem.1180060506 · Cited by 828
Plots, Transformations, and Regression
Weisberg, S. (1987) · Technometrics · doi:10.1080/00401706.1987.10488260 · Cited by 800
Assessing Model Fit by Cross-Validation
Hawkins, D. M., Basak, S. C., & Mills, D. (2003) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci025626i · Cited by 757
Rational selection of training and test sets for the development of validated QSAR models
Golbraikh, A., Shen, M., Xiao, Z., Xiao, Y.-D., Lee, K.-H., & Tropsha, A. (2003) · Journal of Computer-Aided Molecular Design · doi:10.1023/a:1025386326946 · Cited by 712
Chemometric methods in molecular design
Waterbeemd, H. V. D. (1995) · Medical Entomology and Zoology · Cited by 561
Cited by
Best Practices for QSAR Model Development, Validation, and Exploitation
Tropsha, A. (2010) · Molecular Informatics · doi:10.1002/minf.201000061 · Cited by 2,024
Concepts of Artificial Intelligence for Computer-Assisted Drug Discovery
Yang, X., Wang, Y., Byrne, R., Schneider, G., & Yang, S. (2019) · Chemical Reviews · doi:10.1021/acs.chemrev.8b00728 · Cited by 1,022
Daina, A., Michielin, O., & Zoete, V. (2014) · Journal of Chemical Information and Modeling · doi:10.1021/ci500467k · Cited by 974
3D-QSAR in Drug Design - A Review
Verma, J., Khedkar, V., & Coutinho, E. (2010) · Current Topics in Medicinal Chemistry · doi:10.2174/156802610790232260 · Cited by 830
Deep learning for computational chemistry
Goh, G. B., Hodas, N. O., & Vishnu, A. (2017) · Journal of Computational Chemistry · doi:10.1002/jcc.24764 · Cited by 809
On a simple approach for determining applicability domain of QSAR models
Roy, K., Kar, S., & Ambure, P. (2015) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2015.04.013 · Cited by 801
QSARINS: A new software for the development, analysis, and validation of QSAR MLR models
Gramatica, P., Chirico, N., Papa, E., Cassani, S., & Kovarich, S. (2013) · Journal of Computational Chemistry · doi:10.1002/jcc.23361 · Cited by 780
Self-citationChirico, N., & Gramatica, P. (2011) · Journal of Chemical Information and Modeling · doi:10.1021/ci200211n · Cited by 779
Self-citationComputational methods in drug discovery
Leelananda, S. P., & Lindert, S. (2016) · Beilstein Journal of Organic Chemistry · doi:10.3762/bjoc.12.267 · Cited by 639
Big-Data Science in Porous Materials: Materials Genomics and Machine Learning
Jablonka, K. M., Ongari, D., Moosavi, S. M., & Smit, B. (2020) · Chemical Reviews · doi:10.1021/acs.chemrev.0c00004 · Cited by 599
Sorption of organic compounds by aged polystyrene microplastic particles
Hüffer, T., Weniger, A.-K., & Hofmann, T. (2018) · Environmental Pollution · doi:10.1016/j.envpol.2018.01.022 · Cited by 590
Comments on the Definition of the Q2 Parameter for QSAR Validation
Consonni, V., Ballabio, D., & Todeschini, R. (2009) · Journal of Chemical Information and Modeling · doi:10.1021/ci900115y · Cited by 585
On Two Novel Parameters for Validation of Predictive QSAR Models
Pratim Roy, P., Paul, S., Mitra, I., & Roy, K. (2009) · Molecules · doi:10.3390/molecules14051660 · Cited by 563
Loyola-Gonzalez, O. (2019) · IEEE Access · doi:10.1109/access.2019.2949286 · Cited by 559
Further exploring r metrics for validation of QSPR models
Ojha, P. K., Mitra, I., Das, R. N., & Roy, K. (2011) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2011.03.011 · Cited by 554
Chirico, N., & Gramatica, P. (2012) · Journal of Chemical Information and Modeling · doi:10.1021/ci300084j · Cited by 536
Self-citationHow not to develop a quantitative structure–activity or structure–property relationship (QSAR/QSPR)
Dearden, J. C., Cronin, M. T. D., & Kaiser, K. L. E. (2009) · SAR and QSAR in Environmental Research · doi:10.1080/10629360902949567 · Cited by 468
Big data and artificial intelligence modeling for drug discovery
Zhu, H. (2020) · Annual Review of Pharmacology and Toxicology · doi:10.1146/annurev-pharmtox-010919-023324 · Cited by 455
“Instead of only showing self-correlations, the developed models using these new machine learning approaches were always validated using cross-validations, external validations, and/or experimental validations (14, 63, 82, 83).”
MethodGramatica, P., & Sangion, A. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00088 · Cited by 420
Self-citationTetko, I. V., Sushko, I., Pandey, A. K., Zhu, H., Tropsha, A., Papa, E., Öberg, T., Todeschini, R., Fourches, D., & Varnek, A. (2008) · Journal of Chemical Information and Modeling · doi:10.1021/ci800151m · Cited by 416
Predictive and mechanistic multivariate linear regression models for reaction development
Santiago, C. B., Guo, J.-Y., & Sigman, M. S. (2018) · Chemical Science · doi:10.1039/c7sc04679k · Cited by 404
Ain, Q. U., Aleksandrova, A., Roessler, F. D., & Ballester, P. J. (2015) · WIREs Computational Molecular Science · doi:10.1002/wcms.1225 · Cited by 357
An index for characterization of nanomaterials in biological systems
Xia, X.-R., Monteiro-Riviere, N. A., & Riviere, J. E. (2010) · Nature Nanotechnology · doi:10.1038/nnano.2010.164 · Cited by 352
Exploring the impact of size of training sets for the development of predictive QSAR models
Roy, P. P., Leonard, J. T., & Roy, K. (2008) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2007.07.004 · Cited by 306
On some aspects of validation of predictive quantitative structure–activity relationship models
Roy, K. (2007) · Expert Opinion on Drug Discovery · doi:10.1517/17460441.2.12.1567 · Cited by 303
Applications of Polyparameter Linear Free Energy Relationships in Environmental Chemistry
Endo, S., & Goss, K.-U. (2014) · Environmental Science & Technology · doi:10.1021/es503369t · Cited by 302
Mitra, I., Saha, A., & Roy, K. (2010) · Molecular Simulation · doi:10.1080/08927022.2010.503326 · Cited by 295
Combinatorial QSAR Modeling of Chemical Toxicants Tested against Tetrahymena pyriformis
Zhu, H., Tropsha, A., Fourches, D., Varnek, A., Papa, E., Gramatica, P., Öberg, T., Dao, P., Cherkasov, A., & Tetko, I. V. (2008) · Journal of Chemical Information and Modeling · doi:10.1021/ci700443v · Cited by 287
Self-citationOn the Development and Validation of QSAR Models
Gramatica, P. (2013) · Methods in Molecular Biology · doi:10.1007/978-1-62703-059-5_21 · Cited by 274
Self-citationMachine and deep learning approaches for cancer drug repurposing
Issa, N. T., Stathias, V., Schürer, S., & Dakshanamurthy, S. (2021) · Seminars in Cancer Biology · doi:10.1016/j.semcancer.2019.12.011 · Cited by 274
Principles of QSAR Modeling: Comments and Suggestions From Personal Experience
Gramatica, P. (2020) · International Journal of Quantitative Structure-Property Relationships · doi:10.4018/ijqspr.20200701.oa1 · Cited by 258
Self-citationGramatica, P., Cassani, S., Roy, P. P., Kovarich, S., Yap, C. W., & Papa, E. (2012) · Molecular Informatics · doi:10.1002/minf.201200075 · Cited by 258
Self-citationPhytotoxicity of veterinary antibiotics to seed germination and root elongation of crops
Pan, M., & Chu, L. M. (2016) · Ecotoxicology and Environmental Safety · doi:10.1016/j.ecoenv.2015.12.027 · Cited by 256
On the evaluation of the viscosity of nanofluid systems: Modeling and data assessment
Hemmati-Sarapardeh, A., Varamesh, A., Husein, M. M., & Karan, K. (2018) · Renewable and Sustainable Energy Reviews · doi:10.1016/j.rser.2017.07.049 · Cited by 255
Use of machine learning approaches for novel drug discovery
Lima, A. N., Philot, E. A., Trossini, G. H. G., Scott, L. P. B., Maltarollo, V. G., & Honorio, K. M. (2016) · Expert Opinion on Drug Discovery · doi:10.1517/17460441.2016.1146250 · Cited by 254
Mamy, L., Patureau, D., Barriuso, E., Bedos, C., Bessac, F., Louchart, X., Martin-laurent, F., Miege, C., & Benoit, P. (2015) · Critical Reviews in Environmental Science and Technology · doi:10.1080/10643389.2014.955627 · Cited by 177
“In order to facilitate the broader application of the QSAR in organic compounds risks assessment, it is important to define criteria of acceptability of these predicting models, improving their validation, defining their accuracy, and defining and checking their applicability domain (range of cases for which predictio…”
BackgroundAssessment of predictive models for estimating the acute aquatic toxicity of organic chemicals
Melnikov, F., Kostal, J., Voutchkova-Kostal, A., Zimmerman, J. B., & T. Anastas, P. (2016) · Green Chemistry · doi:10.1039/c6gc00720a · Cited by 158
Bigdeli, A., Palchetti, S., Pozzi, D., Hormozi-Nezhad, M. R., Baldelli Bombelli, F., Caracciolo, G., & Mahmoudi, M. (2016) · ACS Nano · doi:10.1021/acsnano.6b00261 · Cited by 157
Rolinger, L., Rüdt, M., & Hubbuch, J. (2020) · Analytical and Bioanalytical Chemistry · doi:10.1007/s00216-020-02407-z · Cited by 126
Gajewicz, A. (2017) · Nanoscale · doi:10.1039/c7nr02211e · Cited by 71
Robust predictive models for estimating frost deposition on horizontal and parallel surfaces
Zendehboudi, A., & Li, X. (2017) · International Journal of Refrigeration · doi:10.1016/j.ijrefrig.2017.05.013 · Cited by 49
Wu, W., Zhang, C., Lin, W., Chen, Q., Guo, X., Qian, Y., & Zhang, L. (2015) · PLOS ONE · doi:10.1371/journal.pone.0119575 · Cited by 47
Endo, S. (2022) · Environmental Science & Technology · doi:10.1021/acs.est.2c00865 · Cited by 21
Modeling of frost thermal conductivity on parallel surface channels
Zendehboudi, A., Hosseini, S. H., & Ahmadi, G. (2019) · Measurement · doi:10.1016/j.measurement.2019.03.045 · Cited by 20
Wu, W., Zhang, R., Peng, S., Li, X., & Zhang, L. (2016) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2016.06.011 · Cited by 13
Metadata
- Authors
- Gramatica, P.
- Venue
- QSAR & Combinatorial Science
- Publisher
- Wiley
- Published
- 2007-05
- Volume
- 26
- Issue
- 5
- Pages
- 694-701
- ISSN
- 1611-020X (ISSN-L)
- 1611-0218
- OpenAlex
- W2054716083
- Cited by
- 2,208
- References
- 36
- FWCI
- 63.18
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.