Beware of q2!

Golbraikh, Alexander; Tropsha, Alexander

doi:10.1016/s1093-3263(01)00123-1

Cited by 3,735Computational Drug Discovery MethodsMachine Learning in Materials ScienceAdvanced Chemical Physics Studies

Beware of q2!

Golbraikh, Alexander, Tropsha, Alexander·2002·Journal of Molecular Graphics and Modelling

DOI 10.1016/s1093-3263(01)00123-1

Open this paper in CoChat — free account

Abstract

Validation is a crucial aspect of any quantitative structure-activity relationship (QSAR) modeling. This paper examines one of the most popular validation criteria, leave-one-out cross-validated R2 (LOO q2). Often, a high value of this statistical characteristic (q2 > 0.5) is considered as a proof of the high predictive ability of the model. In this paper, we show that this assumption is generally incorrect. In the case of 3D QSAR, the lack of the correlation between the high LOO q2 and the high predictive ability of a QSAR model has been established earlier [Pharm. Acta Helv. 70 (1995) 149; J. Chemomet. 10(1996)95; J. Med. Chem. 41 (1998) 2553]. In this paper, we use two-dimensional (2D) molecular descriptors and k nearest neighbors (kNN) QSAR method for the analysis of several datasets. No correlation between the values of q2 for the training set and predictive ability for the test set was found for any of the datasets. Thus, the high value of LOO q2 appears to be the necessary but not the sufficient condition for the model to have a high predictive power. We argue that this is the general property of QSAR models developed using LOO cross-validation. We emphasize that the external validation is the only way to establish a reliable QSAR model. We formulate a set of criteria for evaluation of predictive ability of QSAR models.

Integrity signals

Registry

Crossref · 10.1016/s1093-3263(01)00123-1

Cites

39 of 39 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

QSAR modeling was one of the first research fields that highlighted the importance of data curation, rigorous validation of developed models, and data reproducibility.
“Importantly, QSAR modeling was one of the first research fields that highlighted the importance of data curation,13 rigorous validation of developed models,14 and data reproducibility,15 that has recently become a significant concern to the scientific community.16”
From QSAR without borders (2020) · cites this paper as [14]
Rigorous model validation prior to prediction and the importance of rigor in modeling protocols have been articulated in seminal QSAR publications and have already been adopted as regulatory requirements.
“Similarly, rigorous model validation prior to prediction14 and the importance of rigor in modeling protocols291 have been articulated in several seminal publications in QSAR field292 and have already been adopted as regulatory requirements.84”
From QSAR without borders (2020) · cites this paper as [14]

Graph theory
Harary, F. (1969) · Cited by 10,739 · see citing works
Structural Determination of Paraffin Boiling Points
Wiener, H. (1947) · Journal of the American Chemical Society · doi:10.1021/ja01193a005 · Cited by 4,184 · see citing works
Comparative molecular field analysis (CoMFA). 1. Effect of shape on binding of steroids to carrier proteins
Cramer, R. D., Patterson, D. E., & Bunce, J. D. (1988) · Journal of the American Chemical Society · doi:10.1021/ja00226a005 · Cited by 3,931 · see citing works
Characterization of molecular branching
Randić, M. (1975) · Journal of the American Chemical Society · doi:10.1021/ja00856a001 · Cited by 3,664 · see citing works
Substituent constants for correlation analysis in chemistry and biology
Hansch, C. H., & Leo, A. J. (1979) · Medical Entomology and Zoology · Cited by 2,910 · see citing works
Molecular Connectivity in Chemistry and Drug Research
(1976) · Medicinal chemistry · doi:10.1016/b978-0-12-406560-4.x5001-6 · Cited by 1,628 · see citing works
Applied Statistics
Sachs, L. (1984) · Springer series in statistics · doi:10.1007/978-1-4612-5246-7 · Cited by 1,016 · see citing works
Graph theory and molecular orbitals. XII. Acyclic polyenes
Gutman, İ., ić, B. R., Trinajstić, N., & WILCOX, C. F. J. (1975) · The Journal of Chemical Physics · doi:10.1063/1.430994 · Cited by 829 · see citing works
Molecular Structure Description: The Electrotopological State
Kier, L. B., & Hall, L. H. (1999) · Medical Entomology and Zoology · Cited by 467 · see citing works
Novel Variable Selection Quantitative Structure−Property Relationship Approach Based on the k -Nearest-Neighbor Principle
Zheng, W., & Tropsha, A. (1999) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci980033m · Cited by 450 · see citing works
Self-citation
Substituent constants for correlation analysis
Hansch, C. H., Rockwell, S. D., Jow, P. Y. C., Leo, A. J., & Steller, E. E. (1977) · Journal of Medicinal Chemistry · doi:10.1021/jm00212a024 · Cited by 416 · see citing works
Statistical Validation of QSAR Results
Wold, S., Eriksson, L., & Clementi, S. (1995) · Methods and principles in medicinal chemistry · doi:10.1002/9783527615452.ch5 · Cited by 335 · see citing works
The electrotopological state: structure information at the atomic level for molecular graphs
Hall, L. H., Mohney, B. K., & Kier, L. B. (1991) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci00001a012 · Cited by 326 · see citing works
Three-Dimensional Quantitative Similarity−Activity Relationships (3D QSiAR) from SEAL Similarity Matrices
Kubinyi, H., Hamprecht, F. A., & Mietzner, T. (1998) · Journal of Medicinal Chemistry · doi:10.1021/jm970732a · Cited by 242 · see citing works
A Shape Index from Molecular Graphs
Kier, L. B. (1985) · Quantitative Structure-Activity Relationships · doi:10.1002/qsar.19850040303 · Cited by 234 · see citing works
Applications of the radius-diameter diagram to the classification of topological and geometrical shapes of chemical compounds
Petitjean, M. (1992) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci00008a012 · Cited by 166 · see citing works
The Electrotopological State: An Atom Index for QSAR
Hall, L. H., Mohney, B. K., & Kier, L. B. (1991) · Quantitative Structure-Activity Relationships · doi:10.1002/qsar.19910100108 · Cited by 164 · see citing works
Prediction of Isomeric Differences in Paraffin Properties
Platt, J. R. (1952) · The Journal of Physical Chemistry · doi:10.1021/j150495a009 · Cited by 163 · see citing works
Isomer discrimination by topological information approach
Bonchev, D., Mekenyan, O., & Trinajstić, N. (1981) · Journal of Computational Chemistry · doi:10.1002/jcc.540020202 · Cited by 155 · see citing works
SAR of 9-Amino-1,2,3,4-tetrahydroacridine-Based Acetylcholinesterase Inhibitors: Synthesis, Enzyme Inhibitory Activity, QSAR, and Structure-Based CoMFA of Tacrine Analogues
Recanatini, M., Cavalli, A., Belluti, F., Piazzi, L., Rampa, A., Bisi, A., Gobbi, S., Valenti, P., Andrisano, V., Bartolini, M., et al. (2000) · Journal of Medicinal Chemistry · doi:10.1021/jm990971t · Cited by 154 · see citing works
Evolutionary variable selection in regression and PLS analyses
Kubinyi, H. (1996) · Journal of Chemometrics · doi:10.1002/(sici)1099-128x(199603)10:2 · Cited by 152 · see citing works
Counts of all walks as atomic and molecular descriptors
Ruecker, G., & RUECKER, C. (1993) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci00015a005 · Cited by 144 · see citing works
An extensive ecdysteroid CoMFA
Dinan, L. N., Hormann, R. E., & Fujimoto, T. T. (1999) · Journal of Computer-Aided Molecular Design · doi:10.1023/a:1008052320014 · Cited by 137 · see citing works
Novel Chirality Descriptors Derived from Molecular Topology
Golbraikh, A., Bonchev, D., & Tropsha, A. (2001) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci000082a · Cited by 136 · see citing works
Self-citation
E-state fields: Applications to 3D QSAR
Kellogg, G. E., Kier, L. B., Gaillard, P., & Hall, L. H. (1996) · Journal of Computer-Aided Molecular Design · doi:10.1007/bf00134175 · Cited by 95 · see citing works
Rational Combinatorial Library Design. 2. Rational Design of Targeted Combinatorial Peptide Libraries Using Chemical Similarity Probe and the Inverse QSAR Approaches
Cho, S. J., Zheng, W., & Tropsha, A. (1998) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci9700945 · Cited by 83 · see citing works
Self-citation
Rational Combinatorial Library Design. 1. Focus-2D: A New Approach to the Design of Targeted Combinatorial Chemical Libraries
Zheng, W., Cho, S. J., & Tropsha, A. (1998) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci970095x · Cited by 82 · see citing works
Self-citation
Quantitative Structure−Activity Relationship Modeling of Dopamine D1Antagonists Using Comparative Molecular Field Analysis, Genetic Algorithms−Partial Least-Squares, and K Nearest Neighbor Methods
Hoffman, B. T., Cho, S. J., Zheng, W., Wyrick, S. D., Nichols, D. E., Mailman, R. B., & Tropsha, A. (1999) · Journal of Medicinal Chemistry · doi:10.1021/jm980415j · Cited by 79 · see citing works
Self-citation
Molecular Determinants of MAO Selectivity in a Series of Indolylmethylamine Derivatives: Biological Activities, 3D-QSAR/CoMFA Analysis, and Computational Simulation of Ligand Recognition
Morón, J. A., Campillo, M., Pérez, V., Unzeta, M., & Pardo, L. (2000) · Journal of Medicinal Chemistry · doi:10.1021/jm991164x · Cited by 66 · see citing works
Determination of Topological Equivalence in Molecular Graphs from the Topological State
Hall, L. H., & Kier, L. B. (1990) · Quantitative Structure-Activity Relationships · doi:10.1002/qsar.19900090207 · Cited by 65 · see citing works
A Unified Framework for Using Neural Networks To Build QSARs
Ajay (1993) · Journal of Medicinal Chemistry · doi:10.1021/jm00075a011 · Cited by 54 · see citing works
Inclusion of Symmetry as a Shape Attribute in Kappa Index Analysis
Kier, L. B. (1987) · Quantitative Structure-Activity Relationships · doi:10.1002/qsar.19870060103 · Cited by 54 · see citing works
A Differential Molecular Connectivity Index
Kier, L. B., & Hall, L. H. (1991) · Quantitative Structure-Activity Relationships · doi:10.1002/qsar.19910100208 · Cited by 49 · see citing works
Novel Indices for the Topological Complexity of Molecules
Bonchev, D. (1997) · SAR and QSAR in environmental research · doi:10.1080/10629369708039123 · Cited by 47 · see citing works
Synthesis, Evaluation, and Comparative Molecular Field Analysis of 1-Phenyl-3-amino-1,2,3,4-tetrahydronaphthalenes as Ligands for Histamine H1 Receptors
Bucholtz, E. C., Brown, R. L., Tropsha, A., Booth, R. G., & Wyrick, S. D. (1999) · Journal of Medicinal Chemistry · doi:10.1021/jm980428x · Cited by 43 · see citing works
Self-citation
Use of comparative molecular field analysis and cluster analysis in series design
Novellino, E., Fattorusso, C., & Greco, G. (1995) · Pharmaceutica Acta Helvetiae · doi:10.1016/0031-6865(95)00014-z · Cited by 43 · see citing works
The CoMFA Steroids as a Benchmark Dataset for Development of 3D QSAR Methods
Coats, E. A. (2005) · Kluwer Academic Publishers eBooks · doi:10.1007/0-306-46858-1_13 · Cited by 36 · see citing works
"New Tricks for an Old Dog": Development and Application of Novel QSAR Methods for Rational Design of Combinatorial Chemical Libraries and Database Mining
Tropsha, A., Cho, S.-J., & Zheng, W. (1999) · ACS symposium series · doi:10.1021/bk-1999-0719.ch013 · Cited by 22 · see citing works
Self-citation
Experimental Design in Synthesis Planning and Structure‐Property Correlations
Austel, V., Sjöstróm, M., Eriksson, L., Hall, L. H., Waterbeemd, V. D. H., Costantino, G., Clementi, S., Cruciani, G., & Valigi, R. (1995) · Methods and principles in medicinal chemistry · doi:10.1002/9783527615452.ch3 · Cited by 14 · see citing works

Flagged first, then by citations

Sorted by citations

Recent advances and applications of machine learning in solid-state materials science
Schmidt, J., Marques, M. R. G., Botti, S., & Marques, M. A. L. (2019) · npj Computational Materials · doi:10.1038/s41524-019-0221-0 · Cited by 2,655 · see citing works
Principles of QSAR models validation: internal and external
Gramatica, P. (2007) · QSAR & Combinatorial Science · doi:10.1002/qsar.200610151 · Cited by 2,229 · see citing works
The Importance of Being Earnest: Validation is the Absolute Essential for Successful Application and Interpretation of QSPR Models
Tropsha, A., Gramatica, P., & Gombar, V. K. (2003) · QSAR & Combinatorial Science · doi:10.1002/qsar.200390007 · Cited by 2,195 · see citing works
Self-citation
QSAR Modeling: Where have you been? Where are you going to?
Cherkasov, A., Muratov, E. N., Fourches, D., Varnek, A., Baskin, I. I., Cronin, M., Dearden, J., Gramatica, P., Martin, Y. C., Todeschini, R., et al. (2014) · Journal of Medicinal Chemistry · doi:10.1021/jm4004285 · Cited by 2,164 · see citing works
Self-citation
Best Practices for QSAR Model Development, Validation, and Exploitation
Tropsha, A. (2010) · Molecular Informatics · doi:10.1002/minf.201000061 · Cited by 2,042 · see citing works
Self-citation
Multivariate Analysis in Metabolomics
Worley, B., & Powers, R. (2012) · Current Metabolomics · doi:10.2174/2213235x11301010092 · Cited by 1,361 · see citing works
Methods for reliability and uncertainty assessment and for applicability evaluations of classification- and regression-based QSARs.
Eriksson, L., Jaworska, J., Worth, A. P., Cronin, M. T. D., McDowell, R. M., & Gramatica, P. (2003) · Environmental Health Perspectives · doi:10.1289/ehp.5758 · Cited by 1,325 · see citing works
Classification tools in chemistry. Part 1: linear models. PLS-DA
Ballabio, D., & Consonni, V. (2013) · Analytical Methods · doi:10.1039/c3ay40582f · Cited by 1,270 · see citing works
iLOGP: A Simple, Robust, and Efficient Description of n -Octanol/Water Partition Coefficient for Drug Design Using the GB/SA Approach
Daina, A., Michielin, O. A., & Zoete, V. (2014) · Journal of Chemical Information and Modeling · doi:10.1021/ci500467k · Cited by 1,004 · see citing works
y-Randomization and Its Variants in QSPR/QSAR
Rücker, C., Rücker, G., & Meringer, M. (2007) · Journal of Chemical Information and Modeling · doi:10.1021/ci700157b · Cited by 916 · see citing works
A tutorial review: Metabolomics and partial least squares-discriminant analysis – a marriage of convenience or a shotgun wedding
Gromski, P. S., Muhamadali, H., Ellis, D. I., Yun Xu, A., Correa, E. S., Turner, M. L., & Goodacre, R. (2015) · Analytica Chimica Acta · doi:10.1016/j.aca.2015.02.012 · Cited by 914 · see citing works
QSAR without borders
Muratov, E. N., Bajorath, J., Sheridan, R. P., Tetko, I. V., Filimonov, D., Poroikov, V., Oprea, T. I., Baskin, I. I., Varnek, A., Roitberg, A., et al. (2020) · Chemical Society Reviews · doi:10.1039/d0cs00098a · Cited by 913 · see citing works
Self-citation
Trust, But Verify: On the Importance of Chemical Structure Curation in Cheminformatics and QSAR Modeling Research
Fourches, D., Muratov, E., & Tropsha, A. (2010) · Journal of Chemical Information and Modeling · doi:10.1021/ci100176x · Cited by 832 · see citing works
Self-citation
Statistical strategies for avoiding false discoveries in metabolomics and related experiments
Broadhurst, D., & Kell, D. B. (2006) · Metabolomics · doi:10.1007/s11306-006-0037-z · Cited by 819 · see citing works
“, 2001; Brereton, 2003), where the dataset is randomly spilit into n mutually exclusive subsets (the folds) of approximately equal size (the special case where n = the number of samples is known as leave-one-out crossvalidation – though Golbraikh and Tropsha, in a wonderfully titled article, make a very clear case aga…”
Background
On Some Aspects of Variable Selection for Partial Least Squares Regression Models
Roy, P. P., & Roy, K. (2007) · QSAR & Combinatorial Science · doi:10.1002/qsar.200710043 · Cited by 793 · see citing works
Matrix metalloproteinases (MMPs): Chemical–biological functions and (Q)SARs
Verma, R. P., & Hansch, C. H. (2007) · Bioorganic & Medicinal Chemistry · doi:10.1016/j.bmc.2007.01.011 · Cited by 783 · see citing works
Real External Predictivity of QSAR Models: How To Evaluate It? Comparison of Different Validation Criteria and Proposal of Using the Concordance Correlation Coefficient
Chirico, N., & Gramatica, P. (2011) · Journal of Chemical Information and Modeling · doi:10.1021/ci200211n · Cited by 783 · see citing works
QSARINS: A new software for the development, analysis, and validation of QSAR MLR models
Gramatica, P., Chirico, N., Papa, E., Cassani, S., & Kovarich, S. (2013) · Journal of Computational Chemistry · doi:10.1002/jcc.23361 · Cited by 782 · see citing works
Assessing Model Fit by Cross-Validation
Hawkins, D. M., Basak, S. C., & Mills, D. R. (2003) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci025626i · Cited by 760 · see citing works
Beware of R 2 : Simple, Unambiguous Assessment of the Prediction Accuracy of QSAR and QSPR Models
Alexander, D. L., Tropsha, A., & Winkler, D. A. (2015) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00206 · Cited by 754 · see citing works
“The numerator in the fraction in (1) is the sum of squared residuals (SSR).”
Self-citation
Be aware of error measures. Further studies on validation of predictive QSAR models
Roy, K., Das, R. N., Ambure, P. S., & Aher, R. B. (2016) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2016.01.008 · Cited by 714 · see citing works
Rational selection of training and test sets for the development of validated QSAR models
Golbraikh, A., Shen, M., Xiao, Z., Xiao, Y.-D., Lee, K.-H., & Tropsha, A. (2003) · Journal of Computer-Aided Molecular Design · doi:10.1023/a:1025386326946 · Cited by 712 · see citing works
Self-citation
PLS/OPLS models in metabolomics: the impact of permutation of dataset rows on the K-fold cross-validation quality parameters
Triba, M. N., Moyec, L. L., Amathieu, R., Goossens, C., Bouchemal, N., Nahon, P., Rutledge, D. N., & Savarin, P. (2014) · Molecular BioSystems · doi:10.1039/c4mb00414k · Cited by 691 · see citing works
OPERA models for predicting physicochemical properties and environmental fate endpoints
Mansouri, K., Grulke, C., Judson, R., & Williams, A. J. (2018) · Journal of Cheminformatics · doi:10.1186/s13321-018-0263-1 · Cited by 635 · see citing works
“Different ways of calculating Q2 are available in the literature [50, 61, 62].”
Background
Comments on the Definition of the Q 2 Parameter for QSAR Validation
Consonni, V., Ballabio, D., & Todeschini, R. (2009) · Journal of Chemical Information and Modeling · doi:10.1021/ci900115y · Cited by 586 · see citing works
On Two Novel Parameters for Validation of Predictive QSAR Models
Roy, P. P., Paul, S., Mitra, I., & Roy, K. (2009) · Molecules · doi:10.3390/molecules14051660 · Cited by 570 · see citing works
“Only recently, validation of QSAR models has received considerable attention [7-19].”
Machine Learning for Electrocatalyst and Photocatalyst Design and Discovery
Mai, H., Le, T. C., Chen, D., Winkler, D. A., & Caruso, R. A. (2022) · Chemical Reviews · doi:10.1021/acs.chemrev.2c00061 · Cited by 574 · see citing works
Quantitative Structure–Property Relationship Modeling of Diverse Materials Properties
Le, T. C., Epa, V. C., Burden, F. R., & Winkler, D. A. (2012) · Chemical Reviews · doi:10.1021/cr200066h · Cited by 563 · see citing works
Chemometrics in metabolomics—A review in human disease diagnosis
Madsen, R., Lundstedt, T., & Trygg, J. (2009) · Analytica Chimica Acta · doi:10.1016/j.aca.2009.11.042 · Cited by 559 · see citing works
Further exploring r metrics for validation of QSPR models
Ojha, P. K., Mitra, I., Das, R. N., & Roy, K. (2011) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2011.03.011 · Cited by 556 · see citing works
Real External Predictivity of QSAR Models. Part 2. New Intercomparable Thresholds for Different Validation Criteria and the Need for Scatter Plot Inspection
Chirico, N., & Gramatica, P. (2012) · Journal of Chemical Information and Modeling · doi:10.1021/ci300084j · Cited by 537 · see citing works
Predictive QSAR modeling based on diversity sampling of experimental datasets for the training and test set selection
Golbraikh, A., & Tropsha, A. (2002) · Journal of Computer-Aided Molecular Design · doi:10.1023/a:1020869118689 · Cited by 504 · see citing works
Self-citation
How not to develop a quantitative structure–activity or structure–property relationship (QSAR/QSPR)
Dearden, J. C., Cronin, M. T. D., & Kaiser, K. L. E. (2009) · SAR and QSAR in Environmental Research · doi:10.1080/10629360902949567 · Cited by 468 · see citing works
“It should be noted, however, that internal cross-validation has recently been shown not to be a good indicator of the predictivity of a QSAR/QSPR [10,11,56].”
Background
Comparative Studies on Some Metrics for External Validation of QSPR Models
Roy, K., Mitra, I., Kar, S., Ojha, P. K., Das, R. N., & Kabir, H. (2011) · Journal of Chemical Information and Modeling · doi:10.1021/ci200520g · Cited by 463 · see citing works
Big data and artificial intelligence modeling for drug discovery
Zhu, H. (2020) · Annual Review of Pharmacology and Toxicology · doi:10.1146/annurev-pharmtox-010919-023324 · Cited by 455 · see citing works
“In the same period, model validation was emphasized and treated as a must-have component of modeling (81).”
Background
Proposed minimum reporting standards for data analysis in metabolomics
Goodacre, R., Broadhurst, D., Smilde, A. K., Kristal, B. S., Baker, J. D., Beger, R. D., Bessant, C., Connor, S. C., Capuani, G., Craig, A., et al. (2007) · Metabolomics · doi:10.1007/s11306-007-0081-3 · Cited by 441 · see citing works
“Such models are known to be extremely overoptimistic, especially for K = 1 (Golbraikh and Tropsha 2002 ).”
Background
A Historical Excursus on the Statistical Validation Parameters for QSAR Models: A Clarification Concerning Metrics and Terminology
Gramatica, P., & Sangion, A. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00088 · Cited by 420 · see citing works
Predictive and mechanistic multivariate linear regression models for reaction development
Santiago, C. B., Guo, J.-Y., & Sigman, M. S. (2018) · Chemical Science · doi:10.1039/c7sc04679k · Cited by 403 · see citing works
Assessment of artificial neural network and genetic programming as predictive tools
Gandomi, A. H., & Roke, D. A. (2015) · Advances in Engineering Software · doi:10.1016/j.advengsoft.2015.05.007 · Cited by 407 · see citing works
Prediction of mechanical properties of green concrete incorporating waste foundry sand based on gene expression programming
Iqbal, M. F., Liu, Q., Azim, I., Zhu, X., Yang, J., Javed, M. F., & Rauf, M. (2019) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2019.121322 · Cited by 401 · see citing works
Validation of chemometric models – A tutorial
Westad, F., & Marini, F. (2015) · Analytica Chimica Acta · doi:10.1016/j.aca.2015.06.056 · Cited by 392 · see citing works
Quantitative Nanostructure−Activity Relationship Modeling
Fourches, D., Pu, D., Tassa, C., Weissleder, R., Shaw, S. Y., Mumper, R. J., & Tropsha, A. (2010) · ACS Nano · doi:10.1021/nn1013484 · Cited by 388 · see citing works
Self-citation
Time-Split Cross-Validation as a Method for Estimating the Goodness of Prospective Prediction.
Sheridan, R. P. (2013) · Journal of Chemical Information and Modeling · doi:10.1021/ci400084k · Cited by 376 · see citing works
Evaluation of model predictive ability by external validation techniques
Consonni, V., Ballabio, D., & Todeschini, R. (2010) · Journal of Chemometrics · doi:10.1002/cem.1290 · Cited by 367 · see citing works
Pitfalls in QSAR
Cronin, M., & Schultz, T. W. (2003) · Journal of Molecular Structure THEOCHEM · doi:10.1016/s0166-1280(02)00616-4 · Cited by 350 · see citing works
Empirical scoring functions for structure-based virtual screening: applications, critical aspects, and challenges
Guedes, I. A., Pereira, F. S. S., & Dardenne, L. E. (2018) · Frontiers in Pharmacology · doi:10.3389/fphar.2018.01089 · Cited by 331 · see citing works
Quantitative Structure–Activity Relationship (QSAR) for the Oxidation of Trace Organic Contaminants by Sulfate Radical
Xiao, R., Ye, T., Wei, Z., Luo, S., Yang, Z., & Spinney, R. (2015) · Environmental Science & Technology · doi:10.1021/acs.est.5b03078 · Cited by 323 · see citing works
Good practices in LIBS analysis: Review and advices
Haddad, J. E., Canioni, L., & Bousquet, B. (2014) · Spectrochimica Acta Part B Atomic Spectroscopy · doi:10.1016/j.sab.2014.08.039 · Cited by 312 · see citing works
Nonlinear Genetic-Based Models for Prediction of Flow Number of Asphalt Mixtures
Gandomi, A. H., Alavi, A. H., Mirzahosseini, M. R., & Nejad, F. M. (2010) · Journal of Materials in Civil Engineering · doi:10.1061/(asce)mt.1943-5533.0000154 · Cited by 311 · see citing works
On some aspects of validation of predictive quantitative structure–activity relationship models
Roy, K. (2007) · Expert Opinion on Drug Discovery · doi:10.1517/17460441.2.12.1567 · Cited by 304 · see citing works
Some New Trends in Chemical Graph Theory
García‐Domenech, R., Gálvez, J., de Julián‐Ortiz, J. V., & Pogliani, L. (2008) · Chemical Reviews · doi:10.1021/cr0780006 · Cited by 302 · see citing works
A new multi-gene genetic programming approach to nonlinear system modeling. Part I: materials and structural engineering problems
Gandomi, A. H., & Alavi, A. H. (2011) · Neural Computing and Applications · doi:10.1007/s00521-011-0734-z · Cited by 300 · see citing works
“Furthermore, new criteria recommended by Golbraikh and Tropsha [43] are checked for external validation of the MGGP prediction models (Problems I, II, III) on the validation data sets.”
Method
Combinatorial QSAR Modeling of Chemical Toxicants Tested against Tetrahymena pyriformis
Zhu, H., Tropsha, A., Fourches, D., Varnek, A. A., Papa, E., Gramatica, P., Öberg, T., Dao, P., Cherkasov, A. R., & Tetko, I. V. (2008) · Journal of Chemical Information and Modeling · doi:10.1021/ci700443v · Cited by 288 · see citing works
Self-citation
On the Development and Validation of QSAR Models
Gramatica, P. (2012) · Methods in molecular biology · doi:10.1007/978-1-62703-059-5_21 · Cited by 286 · see citing works
A Comparative Study of Random Forest and Genetic Engineering Programming for the Prediction of Compressive Strength of High Strength Concrete (HSC)
Farooq, F., Amin, M. N., Khan, K., Sadiq, M. R., Javed, M. F., Aslam, F., & Alyousef, R. (2020) · Applied Sciences · doi:10.3390/app10207330 · Cited by 280 · see citing works
“The results show that all the variables considered in this study strongly affect the compressive strength property of HSC.”
Background
Applying machine learning techniques to predict the properties of energetic materials
Elton, D. C., Boukouvalas, Z., Butrico, M. S., Fuge, M., & Chung, P. W. (2018) · Scientific Reports · doi:10.1038/s41598-018-27344-x · Cited by 274 · see citing works
Predicting Small-Molecule Solvation Free Energies: An Informal Blind Test for Computational Chemistry
Nicholls, A. O., Mobley, D. L., Guthrie, J. P., Chodera, J. D., Bayly, C. I., Cooper, M. D., & Pande, V. S. (2008) · Journal of Medicinal Chemistry · doi:10.1021/jm070549+ · Cited by 275 · see citing works
Statistical external validation and consensus modeling: A QSPR case study for Koc prediction
Gramatica, P., Giani, E., & Papa, E. (2006) · Journal of Molecular Graphics and Modelling · doi:10.1016/j.jmgm.2006.06.005 · Cited by 261 · see citing works
Principles of QSAR Modeling
Gramatica, P. (2020) · International Journal of Quantitative Structure-Property Relationships · doi:10.4018/ijqspr.20200701.oa1 · Cited by 263 · see citing works
“…each developed model are: a) to carefully verify the statistical predictivity of QSAR models by their validation (as requested by OECD principle 4), also externally on new chemicals, b) to scramble the response (Y scrambling) (Eriksson et al, 2003; Golbraikh & Tropsha, 2002; Tropsha et al. 2003).”
BackgroundMethodInfluential
On Selection of Training and Test Sets for the Development of Predictive QSAR models
Leonard, J. T., & Roy, K. (2005) · QSAR & Combinatorial Science · doi:10.1002/qsar.200510161 · Cited by 258 · see citing works
Virtual Screening of Molecular Databases Using a Support Vector Machine
Jorissen, R. N., & Gilson, M. K. (2005) · Journal of Chemical Information and Modeling · doi:10.1021/ci049641u · Cited by 258 · see citing works
An evolutionary approach for modeling of shear strength of RC deep beams
Gandomi, A. H., Yun, G. J., & Alavi, A. H. (2013) · Materials and Structures · doi:10.1617/s11527-013-0039-z · Cited by 254 · see citing works
Comparative QSARs for antimalarial endochins: Importance of descriptor-thinning and noise reduction prior to feature selection
Ojha, P. K., & Roy, K. (2011) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2011.08.007 · Cited by 250 · see citing works
Predictive QSAR modeling based on diversity sampling of experimental datasets for the training and test set selection
Golbraikh, A., & Tropsha, A. (2000) · Molecular Diversity · doi:10.1023/a:1021372108686 · Cited by 250 · see citing works
Self-citation
Cross-validation as the objective function for variable-selection techniques
Baumann, K. (2003) · TrAC Trends in Analytical Chemistry · doi:10.1016/s0165-9936(03)00607-1 · Cited by 248 · see citing works
Toward the Development of “Nano‐QSARs”: Advances and Challenges
Puzyn, T., Leszczyńska, D., & Leszczyński, J. (2009) · Small · doi:10.1002/smll.200900179 · Cited by 247 · see citing works
A Novel Automated Lazy Learning QSAR (ALL-QSAR) Approach: Method Development, Applications, and Virtual Screening of Chemical Databases Using Validated ALL-QSAR Models
Zhang, S., Golbraikh, A., Oloff, S., Kohn, H., & Tropsha, A. (2006) · Journal of Chemical Information and Modeling · doi:10.1021/ci060132x · Cited by 246 · see citing works
Self-citation
Prediction of principal ground-motion parameters using a hybrid method coupling artificial neural networks and simulated annealing
Alavi, A. H., & Gandomi, A. H. (2011) · Computers & Structures · doi:10.1016/j.compstruc.2011.08.019 · Cited by 244 · see citing works
ADME Properties Evaluation in Drug Discovery: Prediction of Caco-2 Cell Permeability Using a Combination of NSGA-II and Boosting
Wang, N., Dong, J. N., Deng, Y.-H., Zhu, M.-F., Wen, M., Yao, Z., Lü, A., Wang, J., & Cao, D. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.5b00642 · Cited by 244 · see citing works
A new predictive model for compressive strength of HPC using gene expression programming
Mousavi, S. M., Aminian, P., Gandomi, A. H., Alavi, A. H., & Bolandi, H. (2011) · Advances in Engineering Software · doi:10.1016/j.advengsoft.2011.09.014 · Cited by 243 · see citing works
Prediction reliability of QSAR models: an overview of various validation tools
De, P., Kar, S., Ambure, P. S., & Roy, K. (2022) · Archives of Toxicology · doi:10.1007/s00204-022-03252-y · Cited by 235 · see citing works
“Furthermore, there are few scopes of further refining the present tools like inclusion of computation of Golbraikh and Tropsha’s (Golbraikh and Tropsha 2002) criteria in the Double Cross Validation tool and computation of leave-many-out cross-validation ( Q(2) LMO )”
BackgroundMethodInfluential
Multi-stage genetic programming: A new strategy to nonlinear system modeling
Gandomi, A. H., & Alavi, A. H. (2011) · Information Sciences · doi:10.1016/j.ins.2011.07.026 · Cited by 232 · see citing works
“Furthermore, new criteria recommended by Golbraikh and Tropsha [20] are checked for the external validation of the MSGP prediction models (Problems I and II) on the test data sets.”
Method
Advancing computer-aided drug discovery (CADD) by big data and data-driven machine learning modeling
Zhao, L., Ciallella, H. L., Aleksunes, L. M., & Zhu, H. (2020) · Drug Discovery Today · doi:10.1016/j.drudis.2020.07.005 · Cited by 237 · see citing works
“In the same period, model validation was emphasized and treated as a crucial component of modeling procedures [128].”
Method
A Comparison of Methods for Modeling Quantitative Structure−Activity Relationships
Sutherland, J. J., O’Brien, L. A., & Weaver, D. F. (2004) · Journal of Medicinal Chemistry · doi:10.1021/jm0497141 · Cited by 230 · see citing works
ADME Evaluation in Drug Discovery. 7. Prediction of Oral Absorption by Correlation and Classification
Hou, T., Wang, J., Zhang, W., & Xu, X. (2006) · Journal of Chemical Information and Modeling · doi:10.1021/ci600343x · Cited by 234 · see citing works
Hormone Activity of Hydroxylated Polybrominated Diphenyl Ethers on Human Thyroid Receptor-β: In Vitro and In Silico Investigations
Li, F., Xie, Q. L., Li, X., Li, N., Chi, P., Chen, J., Wang, Z., & Hao, C. (2009) · Environmental Health Perspectives · doi:10.1289/ehp.0901457 · Cited by 228 · see citing works
“3, indicating no over fitting in the model (Golbraikh and Tropsha 2002).”
BackgroundInfluential
A strength prediction model using artificial intelligence for recycling waste tailings as cemented paste backfill
Qi, C., Fourie, A. B., Chen, Q., & Zhang, Q. (2018) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2018.02.154 · Cited by 226 · see citing works
Predicting Drug-Induced Hepatotoxicity Using QSAR and Toxicogenomics Approaches
Low, Y., Uehara, T., Minowa, Y., Yamada, H., Ohno, Y., Urushidani, T., Sedykh, A., Muratov, E. N., Кузьмин, В. Е., Fourches, D., et al. (2011) · Chemical Research in Toxicology · doi:10.1021/tx200148a · Cited by 224 · see citing works
“, internal validation) since training set accuracy alone is insufficient to establish robust and externally predictive models (26).”
Self-citation
Descriptor Selection Methods in Quantitative Structure–Activity Relationship Studies: A Review Study
Shahlaei, M. (2013) · Chemical Reviews · doi:10.1021/cr3004339 · Cited by 222 · see citing works
Advances in computational methods to predict the biological activity of compounds
Nantasenamat, C., Isarankura‐Na‐Ayudhya, C., & Prachayasittikul, V. (2010) · Expert Opinion on Drug Discovery · doi:10.1517/17460441.2010.492827 · Cited by 219 · see citing works
“Such parameters are essential for the successful application and interpretation of the QSAR models [132].”
Method
Applications of Gene Expression Programming for Estimating Compressive Strength of High‐Strength Concrete
Aslam, F., Farooq, F., Amin, M. N., Khan, K., Waheed, A., Akbar, A., Javed, M. F., Alyousef, R., & Alabduljabbar, H. (2020) · Advances in Civil Engineering · doi:10.1155/2020/8850535 · Cited by 218 · see citing works
“Golbraikh and Tropsha [58] proposed a generalized relationship that the slope of line regression (k′ or k) in the model should approach to 1.”
Background
A new set of amino acid descriptors and its application in peptide QSARs
Hu, M. Y., Liao, Z., Zhou, Y., & Li, S. Z. (2005) · Biopolymers · doi:10.1002/bip.20296 · Cited by 214 · see citing works
Gaussian Processes: A Method for Automatic QSAR Modeling of ADME Properties
Obrezanova, O., Cśanyi, G., Gola, J. M. R., & Segall, M. (2007) · Journal of Chemical Information and Modeling · doi:10.1021/ci7000633 · Cited by 214 · see citing works
Quantitative Structure–Selectivity Relationships in Enantioselective Catalysis: Past, Present, and Future
Zahrt, A. F., Athavale, S. V., & Denmark, S. E. (2020) · Chemical Reviews · doi:10.1021/acs.chemrev.9b00425 · Cited by 212 · see citing works
“NanoBRIDGES” software: Open access tools to perform QSAR and nano-QSAR modeling
Ambure, P. S., Aher, R. B., Gajewicz, A., Puzyn, T., & Roy, K. (2015) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2015.07.007 · Cited by 210 · see citing works
Validated QSAR Prediction of OH Tropospheric Degradation of VOCs: Splitting into Training−Test Sets and Consensus Modeling
Gramatica, P., Pilutti, P., & Papa, E. (2004) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci049923u · Cited by 209 · see citing works
Formulation of flow number of asphalt mixes using a hybrid computational method
Alavi, A. H., Ameri, M., Gandomi, A. H., & Mirzahosseini, M. R. (2010) · Construction and Building Materials · doi:10.1016/j.conbuildmat.2010.09.010 · Cited by 204 · see citing works
“New criteria recommended by Golbraikh and Tropsha [60] were also checked for the external validation of the GP/SA model on the testing data sets.”
Method
MedusaScore: An Accurate Force Field-Based Scoring Function for Virtual Drug Screening
Yin, S., Biedermannová, L., Vondrášek, J., & Dokholyan, N. V. (2008) · Journal of Chemical Information and Modeling · doi:10.1021/ci8001167 · Cited by 204 · see citing works
QSAR Modeling of the Blood–Brain Barrier Permeability for Diverse Organic Compounds
Zhang, L., Zhu, H., Oprea, T. I., Golbraikh, A., & Tropsha, A. (2008) · Pharmaceutical Research · doi:10.1007/s11095-008-9609-0 · Cited by 201 · see citing works
Self-citation
ADME Evaluation in Drug Discovery. 5. Correlation of Caco-2 Permeation with Simple Molecular Properties
Hou, T., Zhang, W., Xia, K., Qiao, X. B., & Xu, X. J. (2004) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci049884m · Cited by 201 · see citing works
Time Series Analysis and Forecast of the COVID-19 Pandemic in India using Genetic Programming
Salgotra, R., Gandomi, M., & Gandomi, A. H. (2020) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2020.109945 · Cited by 198 · see citing works
A new multi-gene genetic programming approach to non-linear system modeling. Part II: geotechnical and earthquake engineering problems
Gandomi, A. H., & Alavi, A. H. (2011) · Neural Computing and Applications · doi:10.1007/s00521-011-0735-y · Cited by 194 · see citing works
“Furthermore, new criteria recommended by Golbraikh and Tropsha [43] are checked for external validation of the MGGP prediction models (Problems I, II, III) on the validation data sets.”
Method
Characterization of structure–antioxidant activity relationship of peptides in free radical systems using QSAR models: Key sequence positions and their amino acid properties
Li, Y., & Li, B. (2012) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2012.10.029 · Cited by 193 · see citing works
The History and Development of Quantitative Structure-Activity Relationships (QSARs)
Dearden, J. C. (2016) · International Journal of Quantitative Structure-Property Relationships · doi:10.4018/ijqspr.2016010101 · Cited by 192 · see citing works
Prediction of Compression Index of Fine-Grained Soils Using a Gene Expression Programming Model
Danial Mohammadzadeh, S., Kazemi, S.-F., Mosavi, A. H., Nasseralshariati, E., & Tah, J. H. M. (2019) · Infrastructures · doi:10.3390/infrastructures4020026 · Cited by 189 · see citing works
“h t h k     Golbraikh and Tropsha [43] 0.”
Structure activity relationships and quantitative structure activity relationships for the flavonoid-mediated inhibition of breast cancer resistance protein
Zhang, S., Yang, X., Coburn, R. A., & Morris, M. E. (2005) · Biochemical Pharmacology · doi:10.1016/j.bcp.2005.05.017 · Cited by 188 · see citing works
Critical review and advices on spectral-based normalization methods for LIBS quantitative analysis
Guezenoc, J., Gallet‐Budynek, A., & Bousquet, B. (2019) · Spectrochimica Acta Part B Atomic Spectroscopy · doi:10.1016/j.sab.2019.105688 · Cited by 185 · see citing works
Toxicity of 58 substituted anilines and phenols to algae Pseudokirchneriella subcapitata and bacteria Vibrio fischeri: Comparison with published data and QSARs
Aruoja, V., Sihtmäe, M., Dubourguier, H., & Kahru, A. (2011) · Chemosphere · doi:10.1016/j.chemosphere.2011.05.023 · Cited by 184 · see citing works
Evaluation guidelines for machine learning tools in the chemical sciences
Bender, A., Schneider, N., Segler, M., Walters, W. P., Engkvist, O., & Rodrigues, T. (2022) · Nature Reviews Chemistry · doi:10.1038/s41570-022-00391-9 · Cited by 186 · see citing works
The Trouble with QSAR (or How I Learned To Stop Worrying and Embrace Fallacy)
Johnson, S. R. (2007) · Journal of Chemical Information and Modeling · doi:10.1021/ci700332k · Cited by 183 · see citing works

Rational selection of training and test sets for the development of validated QSAR models
Golbraikh, A., Shen, M., Xiao, Z., Xiao, Y.-D., Lee, K.-H., & Tropsha, A. (2003) · Journal of Computer-Aided Molecular Design · doi:10.1023/a:1025386326946 · Cited by 712 · see citing works
Self-citation
Beware of Unreliable Q2! A Comparative Study of Regression Metrics for Predictivity Assessment of QSAR Models
Todeschini, R., Ballabio, D., & Grisoni, F. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00277 · Cited by 107 · see citing works
On some aspects of validation of predictive QSAR models
Roy, K., Roy, P., & Leonard, J. (2008) · Chemistry Central Journal · doi:10.1186/1752-153x-2-s1-p9 · Cited by 27 · see citing works
Validation of QSAR Models
Dastmalchi, S., Hamzeh‐Mivehroud, M., & Sokouti, B. (2018) · CRC Press eBooks · doi:10.1201/9781351113076-6 · Cited by 2 · see citing works
Beware of External Validation! - A Comparative Study of Several Validation Techniques used in QSAR Modelling
Majumdar, S., & Basak, S. C. (2018) · Current Computer - Aided Drug Design · doi:10.2174/1573409914666180426144304 · Cited by 31 · see citing works

Authors: Golbraikh, A., & Tropsha, A.
Affiliations: University of North Carolina at Chapel Hill (US)
Corresponding author: Tropsha, Alexander
ORCID: Golbraikh, Alexander
Tropsha, Alexander
Venue: Journal of Molecular Graphics and Modelling
Publisher: Elsevier BV
Published: 2002-01
Volume: 20
Issue: 4
Pages: 269-276
ISSN: 1093-3263 (ISSN-L)
DOI: 10.1016/s1093-3263(01)00123-1
OpenAlex: W2087661061
Cited by: 3,735
References: 39
Citations by year: 2026: 133 · 2025: 164 · 2024: 179 · 2023: 174 · 2022: 194 · 2021: 197 · 2020: 215 · 2019: 189 · 2018: 200 · 2017: 174 · 2016: 181 · 2015: 175 · 2014: 197 · 2013: 203 · 2012: 202
FWCI: 52.71
Type: journal article
Language: en
Open access: closed
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations: Journal of Molecular Graphics and Modelling (publishedVersion)
PubMed (publishedVersion)
Topics: Physical Sciences › Computer Science › Computational Theory and Mathematics › Computational Drug Discovery Methods
Physical Sciences › Materials Science › Materials Chemistry › Machine Learning in Materials Science
Physical Sciences › Physics and Astronomy › Atomic and Molecular Physics, and Optics › Advanced Chemical Physics Studies
Keywords: leave-one-out cross-validation, QSAR model validation, predictive power, external validation, 2D molecular descriptors, overfitting, model evaluation metrics
MeSH terms: Data Interpretation, Statistical, Ligands, Mathematical Computing, Models, Molecular, Receptors, Histamine H1, Steroids, Transcortin, Databases, Factual, Quantitative Structure-Activity Relationship, Ecdysteroids

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Beware of q2!

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks