The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
The Importance of Being Earnest: Validation is the Absolute Essential for Successful Application and Interpretation of QSPR Models
Abstract
This paper emphasizes the importance of rigorous validation as a crucial, integral component of Quantitative Structure Property Relationship (QSPR) model development. We consider some examples of published QSPR models, which in spite of their high fitted accuracy for the training sets and apparent mechanistic appeal, fail rigorous validation tests, and, thus, may lack practical utility as reliable screening tools. We present a set of simple guidelines for developing validated and predictive QSPR models. To this end, we discuss several validation strategies including (1) randomization of the modelled property, also called Y‐scrambling, (2) multiple leave‐many‐out cross‐validations, and (3) external validation using rational division of a dataset into training and test sets. We also highlight the need to establish the domain of model applicability in the chemical space to flag molecules for which predictions may be unreliable, and discuss some algorithms that can be used for this purpose. We advocate the broad use of these guidelines in the development of predictive QSPR models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
Regression Diagnostics -- Identifying Influential Data and Sources of Collinearity
Beaumont, C. D., Belsley, D. A., Kuh, E., & Welsch, R. E. (1981) · The Journal of the Operational Research Society · doi:10.2307/2581267 · Cited by 6,359
Robust Regression and Outlier Detection
Piepel, G. F. (1989) · Technometrics · doi:10.1080/00401706.1989.10488524 · Cited by 6,144
Cramer, R. D., Patterson, D. E., & Bunce, J. D. (1988) · Journal of the American Chemical Society · doi:10.1021/ja00226a005 · Cited by 3,931
Golbraikh, A., & Tropsha, A. (2002) · Journal of Molecular Graphics and Modelling · doi:10.1016/s1093-3263(01)00123-1 · Cited by 3,735
Self-citationComputer Aided Design of Experiments
Kennard, R. W., & Stone, L. A. (1969) · Technometrics · doi:10.1080/00401706.1969.10490666 · Cited by 3,648
Plots, Transformations, and Regression
Weisberg, S. (1987) · Technometrics · doi:10.1080/00401706.1987.10488260 · Cited by 800
Gasteiger, J., & Zupan, J. (1993) · Angewandte Chemie International Edition in English · doi:10.1002/anie.199305031 · Cited by 503
Zheng, W., & Tropsha, A. (2000) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci980033m · Cited by 450
Self-citationLindberg, W., Persson, J.-A., & Wold, S. (1983) · Analytical Chemistry · doi:10.1021/ac00255a014 · Cited by 430
An Algorithm for the Construction of "D-Optimal" Experimental Designs
Mitchell, T. J. (2000) · Technometrics · doi:10.2307/1271431 · Cited by 421
Statistical Validation of QSAR Results
Wold, S., Eriksson, L., & Clementi, S. (1995) · Methods and Principles in Medicinal Chemistry · doi:10.1002/9783527615452.ch5 · Cited by 334
Chemometric Methods in Molecular Design
(1995) · Methods and Principles in Medicinal Chemistry · doi:10.1002/9783527615452 · Cited by 330
Quantitative structure–permeability relationships (QSPRs) for percutaneous absorption
Moss, G. P., Dearden, J. C., Patel, H., & Cronin, M. T. D. (2002) · Toxicology in Vitro · doi:10.1016/s0887-2333(02)00003-6 · Cited by 268
An Algorithm for the Construction of "D-Optimal" Experimental Designs
Mitchell, T. J. (1974) · Technometrics · doi:10.2307/1267940 · Cited by 267
Robust QSAR Models Using Bayesian Regularized Neural Networks
Burden, F. R., & Winkler, D. A. (1999) · Journal of Medicinal Chemistry · doi:10.1021/jm980697n · Cited by 241
Kubinyi, H., Hamprecht, F. A., & Mietzner, T. (1998) · Journal of Medicinal Chemistry · doi:10.1021/jm970732a · Cited by 240
Artificial neural networks in classification of NIR spectral data: Design of the training set
Wu, W., Walczak, B., Massart, D. L., et al. (1996) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/0169-7439(95)00077-1 · Cited by 227
Comparison of algorithms for dissimilarity-based compound selection
Snarey, M., Terrett, N. K., Willett, P., & Wilton, D. J. (1997) · Journal of Molecular Graphics and Modelling · doi:10.1016/s1093-3263(98)00008-4 · Cited by 205
Gramatica, P., Corradi, M., & Consonni, V (2000) · Chemosphere · doi:10.1016/s0045-6535(99)00463-4 · Cited by 178
Self-citationToward an Optimal Procedure for Variable Selection and QSAR Model Building
Yasri, A., & Hartsough, D. (2001) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci010291a · Cited by 177
Fan, Y., Shi, L. M., Kohn, K. W., Pommier, Y., & Weinstein, J. N. (2001) · Journal of Medicinal Chemistry · doi:10.1021/jm0005151 · Cited by 162
Multivariate design and modeling in QSAR
Eriksson, L., & Johansson, E. (1996) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/0169-7439(96)00023-8 · Cited by 149
Taylor, R. (1995) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci00023a009 · Cited by 144
Use of Automatic Relevance Determination in QSAR Studies Using Bayesian Neural Networks
Burden, F. R., Ford, M. G., Whitley, D. C., & Winkler, D. A. (2000) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci000450a · Cited by 134
Kauffman, G. W., & Jurs, P. C. (2001) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci010073h · Cited by 126
Random or Rational Design? Evaluation of Diverse Compound Subsets from Chemical Structure Databases
Pötter, T., & Matter, H. (1998) · Journal of Medicinal Chemistry · doi:10.1021/jm9700878 · Cited by 113
A new algorithm for optimal, distance-based experimental design
Marengo, E. A., & Todeschini, R. (1992) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/0169-7439(92)80076-g · Cited by 109
Cho, S. J., Zheng, W., & Tropsha, A. (1998) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci9700945 · Cited by 83
Self-citationStructure–toxicity relationships for selected halogenated aliphatic chemicals
Akers, K. S., Sinks, G. D., & Schultz, T. W. (1999) · Environmental Toxicology and Pharmacology · doi:10.1016/s1382-6689(98)00048-9 · Cited by 82
Single and domain mode variable selection in 3D QSAR applications
Norinder, U. (1996) · Journal of Chemometrics · Cited by 77
QSAR study on the tropospheric degradation of organic compounds
Gramatica, P., Consonni, V., & Todeschini, R. (1999) · Chemosphere · doi:10.1016/s0045-6535(98)00539-6 · Cited by 73
Self-citationGironés, X., Gallegos, A., & Carbó‐Dorca, R. (2000) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci0004558 · Cited by 51
Use of comparative molecular field analysis and cluster analysis in series design
Novellino, E., Fattorusso, C., & Greco, G. (1995) · Pharmaceutica Acta Helvetiae · doi:10.1016/0031-6865(95)00014-z · Cited by 43
Suzuki, T., Ide, K., Ishida, M., & Shapiro, S. (2001) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci000333f · Cited by 42
Golbraikh, A. (2000) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci990437u · Cited by 38
The CoMFA Steroids as a Benchmark Dataset for Development of 3D QSAR Methods
Coats, E. A. (2005) · Kluwer Academic Publishers eBooks · doi:10.1007/0-306-46858-1_13 · Cited by 35
Construction of High-Quality Structure−Property−Activity Regressions: The Boiling Points of Sulfides
Randić, M., & Basak, S. C. (2000) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci990115q · Cited by 29
Bordás, B., Kőmíves, T., Szántó, Z., & Lopata, A. (2000) · Journal of Agricultural and Food Chemistry · doi:10.1021/jf990395+ · Cited by 26
QSAR Modeling of Large Heterogeneous Sets of Molecules
Devillers, J. (2001) · SAR and QSAR in environmental research · doi:10.1080/10629360108039832 · Cited by 22
Cited by
Recent advances and applications of machine learning in solid-state materials science
Schmidt, J., Marques, M. R. G., Botti, S., & Marques, M. A. L. (2019) · npj Computational Materials · doi:10.1038/s41524-019-0221-0 · Cited by 2,607
Principles of QSAR models validation: internal and external
Gramatica, P. (2007) · QSAR & Combinatorial Science · doi:10.1002/qsar.200610151 · Cited by 2,208
Self-citationBest Practices for QSAR Model Development, Validation, and Exploitation
Tropsha, A. (2010) · Molecular Informatics · doi:10.1002/minf.201000061 · Cited by 2,024
Self-citationEriksson, L., Jaworska, J., Worth, A. P., Cronin, M. T. D., McDowell, R. M., & Gramatica, P. (2003) · Environmental Health Perspectives · doi:10.1289/ehp.5758 · Cited by 1,325
Self-citationDaina, A., Michielin, O., & Zoete, V. (2014) · Journal of Chemical Information and Modeling · doi:10.1021/ci500467k · Cited by 974
Metadata
- Authors
- Tropsha, A., Gramatica, P., & Gombar, V. K.
- Venue
- QSAR & Combinatorial Science
- Publisher
- Wiley
- Published
- 2003-04
- Volume
- 22
- Issue
- 1
- Pages
- 69-77
- ISSN
- 1611-020X (ISSN-L)
- 1611-0218
- OpenAlex
- W2074681440
- Cited by
- 2,188
- References
- 40
- FWCI
- 33.87
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.