The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
External Validation and Prediction Employing the Predictive Squared Correlation Coefficient — Test Set Activity Mean vs Training Set Activity Mean
Abstract
The external prediction capability of quantitative structure-activity relationship (QSAR) models is often quantified using the predictive squared correlation coefficient, q (2). This index relates the predictive residual sum of squares, PRESS, to the activity sum of squares, SS, without postprocessing of the model output, the latter of which is automatically done when calculating the conventional squared correlation coefficient, r (2). According to the current OECD guidelines, q (2) for external validation should be calculated with SS referring to the training set activity mean. Our present findings including a mathematical proof demonstrate that this approach yields a systematic overestimation of the prediction capability that is triggered by the difference between the training and test set activity means. Example calculations with three regression models and data sets taken from literature show further that for external test sets, q (2) based on the training set activity mean may become even larger than r (2). As a consequence, we suggest to always use the test set activity mean when quantifying the external prediction capability through q (2) and to revise the respective OECD guidance document accordingly. The discussion includes a comparison between r (2) and q (2) value ranges and the q (2) statistics for cross-validation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Better Bootstrap Confidence Intervals
Efron, B. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478410 · Cited by 3,364
Hawkins, D. M. (2004) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci0342472 · Cited by 2,522
Estimating the error rate of a prediction rule: improvement on cross-validation
Efron, B. (1983) · Journal of the American Statistical Association · doi:10.1080/01621459.1983.10477973 · Cited by 2,300
Tropsha, A., Gramatica, P., & Gombar, V. K. (2003) · QSAR & Combinatorial Science · doi:10.1002/qsar.200390007 · Cited by 2,188
Linear Model Selection by Cross-validation
Shao, J. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10476299 · Cited by 1,790
The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction
Allen, D. M. (1974) · Technometrics · doi:10.1080/00401706.1974.10489157 · Cited by 1,469
Cramer, R. D., Bunce, J. D., Patterson, D. E., & Frank, I. E. (1988) · Quantitative Structure-Activity Relationships · doi:10.1002/qsar.19880070105 · Cited by 833
Similarity to Molecules in the Training Set Is a Good Discriminator for Prediction Accuracy in QSAR
Sheridan, R. P., Feuston, B. P., Maiorov, V. N., & Kearsley, S. K. (2004) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci049782w · Cited by 298
A Stepwise Approach for Defining the Applicability Domain of SAR and QSAR Models
Dimitrov, S., Dimitrova, G., Pavlov, T., Dimitrova, N., Patlewicz, G., Niemela, J., & Mekenyan, O. (2005) · Journal of Chemical Information and Modeling · doi:10.1021/ci0500381 · Cited by 291
Nguyen, T. H., Goss, K.-U., & Ball, W. P. (2005) · Environmental Science & Technology · doi:10.1021/es048839s · Cited by 276
Cited by
Comments on the Definition of the Q2 Parameter for QSAR Validation
Consonni, V., Ballabio, D., & Todeschini, R. (2009) · Journal of Chemical Information and Modeling · doi:10.1021/ci900115y · Cited by 585
Roy, K., Chakraborty, P., Mitra, I., Ojha, P. K., Kar, S., & Das, R. N. (2013) · Journal of Computational Chemistry · doi:10.1002/jcc.23231 · Cited by 466
Gramatica, P., & Sangion, A. (2016) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.6b00088 · Cited by 420
Energy-based descriptors to rapidly predict hydrogen storage in metal–organic frameworks
Bucior, B. J., Bobbitt, N. S., Islamoglu, T., Goswami, S., Gopalan, A., Yildirim, T., Farha, O. K., Bagheri, N., & Snurr, R. Q. (2019) · Molecular Systems Design & Engineering · doi:10.1039/c8me00050f · Cited by 308
Mitra, I., Saha, A., & Roy, K. (2010) · Molecular Simulation · doi:10.1080/08927022.2010.503326 · Cited by 295
Principles of QSAR Modeling: Comments and Suggestions From Personal Experience
Gramatica, P. (2020) · International Journal of Quantitative Structure-Property Relationships · doi:10.4018/ijqspr.20200701.oa1 · Cited by 258
Wu, W., Zhang, C., Lin, W., Chen, Q., Guo, X., Qian, Y., & Zhang, L. (2015) · PLOS ONE · doi:10.1371/journal.pone.0119575 · Cited by 47
Hosseini, R., Newlands, N., Dean, C., & Takemura, A. (2015) · Remote Sensing · doi:10.3390/rs70302752 · Cited by 31
Wu, W., Zhang, R., Peng, S., Li, X., & Zhang, L. (2016) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2016.06.011 · Cited by 13
Metadata
- Authors
- Schüürmann, G., Ebert, R.-U., Chen, J., Wang, B., & Kühne, R.
- Venue
- Journal of Chemical Information and Modeling
- Publisher
- American Chemical Society (ACS)
- Published
- 2008-10-28
- Volume
- 48
- Issue
- 11
- Pages
- 2140-2145
- ISSN
- 1549-9596 (ISSN-L)
- 1549-960X
- OpenAlex
- W2073150707
- Cited by
- 531
- References
- 22
- FWCI
- 21.77
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
