The problem of overfitting.
Abstract
ADVERTISEMENT RETURN TO ISSUEArticleNEXTThe Problem of OverfittingDouglas M. HawkinsView Author Information School of Statistics, University of Minnesota, Minneapolis, Minnesota 55455 Cite this: J. Chem. Inf. Comput. Sci. 2004, 44, 1, 1–12Publication Date (Web):December 2, 2003Publication History Received30 October 2003Published online2 December 2003Published inissue 1 January 2004https://pubs.acs.org/doi/10.1021/ci0342472https://doi.org/10.1021/ci0342472research-articleACS PublicationsCopyright © 2004 American Chemical SocietyRequest reuse permissionsArticle Views19627Altmetric-Citations1642LEARN ABOUT THESE METRICSArticle Views are the COUNTER-compliant sum of full text article downloads since November 2008 (both PDF and HTML) across all institutions and individuals. These metrics are regularly updated to reflect usage leading up to the last few days.Citations are the number of other articles citing this article, calculated by Crossref and updated daily. Find more information about Crossref citation counts.The Altmetric Attention Score is a quantitative measure of the attention that a research article has received online. Clicking on the donut icon will load a page at altmetric.com with additional details about the score and the social media presence for the given article. Find more information on the Altmetric Attention Score and how the score is calculated. Share Add toView InAdd Full Text with ReferenceAdd Description ExportRISCitationCitation and abstractCitation and referencesMore Options Share onFacebookTwitterWechatLinked InRedditEmail Other access options SUBJECTS:Calibration,Hydrocarbons,Quality management,Testing and assessment,Transition temperature Get e-Alerts
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Based upon ResNet, many studies have managed to improve the performance of the original ResNet, such as pre-activation ResNet, wide ResNet, stochastic depth ResNets (SDR), and ResNet in ResNet (RiR).
“Based upon ResNet, many studies have managed to improve the performance of the original ResNet, such as pre-activation ResNet [38], wide ResNet [39], stochastic depth ResNets (SDR) [40], and ResNet in ResNet (RiR) [41].”
From A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects (2022) · cites this paper as [38]
No matter how many layers a neural network has, if it uses linear activation, the output is always a linear combination of the input, meaning hidden layers have no effect.
“He et al. [38] verify no matter how many layers the neural network has, the output is always a linear combination of the input, which means hidden layers have no effect.”
From A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects (2022) · cites this paper as [38]
Nonlinear models, together with the use of a significant amount of descriptors lacking physical meaning, offer the risk of producing excellent performance indexes due to overfitting and/or bias to the training set construction.
“Together with the use of a significant amount of descriptors lacking physical meaning, nonlinear models offer the risk of producing excellent performance indexes due to overfitting and/or bias to the training set construction (e.g., capturing the rules adopted during the selection of active and decoy compounds) (Hawkins, 2004; Abu-Mostafa et al., 2012).”
From Empirical scoring functions for structure-based virtual screening: applications, critical aspects, and challenges (2018) · cites this paper as Hawkins, 2004
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162 · see citing works
The Jackknife, the Bootstrap and Other Resampling Plans
Efron, B. (1982) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611970319 · Cited by 8,120 · see citing works
A Statistical View of Some Chemometrics Regression Tools
Frank, L. E., & Friedman, J. H. (1993) · Technometrics · doi:10.1080/00401706.1993.10485033 · Cited by 2,217 · see citing works
Methods for Statistical Data Analysis of Multivariate Observations.
Sen, P. K., & Gnanadesikan, R. (1978) · Journal of the American Statistical Association · doi:10.2307/2286626 · Cited by 900 · see citing works
Applied Regression Including Computing and Graphics
Cook, R. D., & Weisberg, S. (1999) · Wiley series in probability and statistics · doi:10.1002/9780470316948 · Cited by 731 · see citing works
Methods for Statistical Data Analysis of Multivariate Observations
Gnanadesikan, R. (1997) · Wiley series in probability and statistics · doi:10.1002/9781118032671 · Cited by 295 · see citing works
Inflation of R 2 in Best Subset Regression
Rencher, A. C., & Pun, F. C. (1980) · Technometrics · doi:10.1080/00401706.1980.10486100 · Cited by 162 · see citing works
Selwood, D. L., Livingstone, D. J., Comley, J. C. W., et al. (1990) · Journal of Medicinal Chemistry · doi:10.1021/jm00163a023 · Cited by 128 · see citing works
QSAR with Few Compounds and Many Features
Hawkins, D. M., Basak, S. C., & Shi, X. (2001) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci0001177 · Cited by 77 · see citing works
Self-citationYaffe, D., Cohen, Y., Espinosa, G., Arenas, À., & Giralt, F. (2001) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci010323u · Cited by 60 · see citing works
Ziegler, C. B., Bitha, P., KUCK, N. A., et al. (1990) · Journal of Medicinal Chemistry · doi:10.1021/jm00163a024 · Cited by 45 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects
Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3084827 · Cited by 5,273 · see citing works
“After convolution, feature maps consist of a large number of features that is prone to causing overfitting problem [21].”
BackgroundApostolopoulos, I. D., & Mpesiana, T. A. (2020) · Physical and Engineering Sciences in Medicine · doi:10.1007/s13246-020-00865-4 · Cited by 2,338 · see citing works
Albumentations: fast and flexible image augmentations
Buslaev, A., Iglovikov, V. I., Khvedchenya, E., Parinov, A., Druzhinin, M., & Kalinin, A. A. (2020) · Information · doi:10.3390/info11020125 · Cited by 2,714 · see citing works
Improving imbalanced learning through a heuristic oversampling method based on k-means and SMOTE
Douzas, G., Bacao, F., & Last, F. (2018) · Information Sciences · doi:10.1016/j.ins.2018.06.056 · Cited by 1,277 · see citing works
Practitioner’s guide to latent class analysis: Methodological considerations and common pitfalls
Sinha, P., Calfee, C. S., & Delucchi, K. L. (2021) · Critical Care Medicine · doi:10.1097/ccm.0000000000004710 · Cited by 1,204 · see citing works
Luo, W., Phung, D., Tran, T., Gupta, S., Rana, S., Karmakar, C., Shilton, A., Yearwood, J., Dimitrova, N., Ho, T. B., et al. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5870 · Cited by 988 · see citing works
Machine Learning Testing: Survey, Landscapes and Horizons
Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897 · see citing works
Fame for sale: Efficient detection of fake Twitter followers
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2015) · Decision Support Systems · doi:10.1016/j.dss.2015.09.003 · Cited by 461 · see citing works
“It is well known that trained classifiers can be subject to “overfitting”, namely the problem of being too specialized on the training dataset and unable to generalize the classification to new and unseen data [45].”
BackgroundBéchet, Q., Shilton, A., & Guieysse, B. (2013) · Biotechnology Advances · doi:10.1016/j.biotechadv.2013.08.014 · Cited by 430 · see citing works
Performance of machine-learning scoring functions in structure-based virtual screening.
Wójcikowski, M., Ballester, P. J., & Siedlecki, P. (2017) · Scientific Reports · doi:10.1038/srep46710 · Cited by 364 · see citing works
Guedes, I. A., Pereira, F. S. S., & Dardenne, L. E. (2018) · Frontiers in Pharmacology · doi:10.3389/fphar.2018.01089 · Cited by 331 · see citing works
Ursu, O., Rayan, A., Goldblum, A., & Oprea, T. I. (2011) · WIREs Computational Molecular Science · doi:10.1002/wcms.52 · Cited by 276 · see citing works
Navigating the Complexity of Generative AI Adoption in Software Engineering
Russo, D. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3652154 · Cited by 217 · see citing works
“…high effect size observed between ‘Perception about the Technology’ and ‘Compatibil-ity Factors’ might raise concerns of potential overfitting, a scenario where the model inadvertently learns noise in the training data, thereby compromising its ability to accurately predict unseen data [47].”
BackgroundBoukerche, A., Tao, Y., & Sun, P. (2020) · Computer Networks · doi:10.1016/j.comnet.2020.107484 · Cited by 215 · see citing works
Bailey, D. H., Borwein, J. M., López de Prado, M., & Zhu, Q. J. (2014) · Notices of the American Mathematical Society · doi:10.1090/noti1105 · Cited by 221 · see citing works
Cross-Validation Visualized: A Narrative Guide to Advanced Methods
Allgaier, J., & Pryss, R. (2024) · Machine Learning and Knowledge Extraction · doi:10.3390/make6020065 · Cited by 176 · see citing works
van Westen, G. J. P., Wegner, J. K., IJzerman, A. P., van Vlijmen, H. W. T., & Bender, A. (2011) · MedChemComm · doi:10.1039/c0md00165a · Cited by 176 · see citing works
Quantification of biases in predictions of protein stability changes upon mutations
Pucci, F., Bernaerts, K. V., Kwasigroch, J. M., & Rooman, M. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty348 · Cited by 169 · see citing works
Overfitting in linear feature extraction for classification of high-dimensional image data
Liu, R., & Gillies, D. F. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2015.11.015 · Cited by 123 · see citing works
Challenges in the Development of Soft Sensors for Bioprocesses: A Critical Review
Brunner, V., Siegl, M., Geier, D., & Becker, T. (2021) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2021.722202 · Cited by 100 · see citing works
Passive acoustic monitoring of animal populations with transfer learning
Dufourq, E., Batist, C., Foquet, R., & Durbach, I. (2022) · Ecological Informatics · doi:10.1016/j.ecoinf.2022.101688 · Cited by 97 · see citing works
Duering, M., Gonik, M., Malik, R., Zieren, N., Reyes, S., Jouvent, E., Hervé, D., Gschwendtner, A., Opherk, C., Chabriat, H., et al. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2012.10.084 · Cited by 89 · see citing works
Selecting Representative Data Sets
Borovicka, T., Jirina, M., Kordik, P., & Jiri, M. (2012) · Advances in Data Mining Knowledge Discovery and Applications · doi:10.5772/50787 · Cited by 94 · see citing works
Ghasemzadeh, H., Hillman, R. E., & Mehta, D. D. (2024) · Journal of Speech, Language, and Hearing Research · doi:10.1044/2023_jslhr-23-00273 · Cited by 72 · see citing works
Bloch, L., Friedrich, C. M., & Initiative, F. T. A. D. N. (2021) · Alzheimer's Research & Therapy · doi:10.1186/s13195-021-00879-4 · Cited by 68 · see citing works
Laengsri, V., Shoombuatong, W., Adirojananon, W., Nantasenamat, C., Prachayasittikul, V., & Nuchnoi, P. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0929-2 · Cited by 66 · see citing works
Fernández-Torras, A., Duran-Frigola, M., Bertoni, M., Locatelli, M., & Aloy, P. (2022) · Nature Communications · doi:10.1038/s41467-022-33026-0 · Cited by 59 · see citing works
Robust scoring functions for protein-ligand interactions with quantum chemical charge models.
Wang, J.-C., Lin, J.-H., Chen, C.-M., Perryman, A. L., & Olson, A. J. (2011) · Journal of Chemical Information and Modeling · doi:10.1021/ci200220v · Cited by 54 · see citing works
Machine-learning models for combinatorial catalyst discovery
Landrum, G. A., Penzotti, J. E., & Putta, S. (2005) · Measurement Science and Technology · doi:10.1088/0957-0233/16/1/035 · Cited by 48 · see citing works
Naulaerts, S., Dang, C. C., & Ballester, P. J. (2017) · Oncotarget · doi:10.18632/oncotarget.20923 · Cited by 41 · see citing works
Statistical Modeling of a Ligand Knowledge Base
Mansson, R. A., Welsh, A. H., Fey, N., & Orpen, A. G. (2006) · Journal of Chemical Information and Modeling · doi:10.1021/ci600212t · Cited by 33 · see citing works
Trivedi, K., & Shroff, H. (2021) · 2021 ITU Kaleidoscope: Connecting Physical and Virtual Worlds (ITU K) · doi:10.23919/ituk53220.2021.9662116 · Cited by 21 · see citing works
Productivity of Nannochloropsis oceanica in an industrial closely spaced flat panel photobioreactor
Norsker, N. H., Michiels, M., Slegers, P. M., Swinkels, G. L. A. M., Barbosa, M. J., & Wijffels, R. H. (2019) · Algal Research · doi:10.1016/j.algal.2019.101632 · Cited by 17 · see citing works
Popp, J. L., Thiele, J. A., Faskowitz, J., Seguin, C., Sporns, O., & Hilger, K. (2025) · Communications Biology · doi:10.1038/s42003-025-08231-4 · Cited by 17 · see citing works
Generalization to New Actions in Reinforcement Learning
Jain, A., Szot, A., & Lim, J. J. (2020) · arXiv · doi:10.48550/arxiv.2011.01928 · Cited by 14 · see citing works
“Finally, the tasksolving policy architecture must be flexible to incorporate new actions and be trained through a procedure that avoids overfitting (Hawkins, 2004) to training actions.”
Background
- Authors
- Hawkins, D. M.
- Affiliations
- University of Minnesota (US)
- Corresponding author
- Hawkins, Douglas M.
- ORCID
- Hawkins, Douglas M.
- Venue
- Journal of Chemical Information and Computer Sciences
- Publisher
- American Chemical Society (ACS)
- Published
- 2003-12-02
- Volume
- 44
- Issue
- 1
- Pages
- 1-12
- ISSN
- 0095-2338 (ISSN-L)
- OpenAlex
- W2213612645
- Cited by
- 2,522
- References
- 11
- Citations by year
- 2026: 143 · 2025: 248 · 2024: 239 · 2023: 208 · 2022: 223 · 2021: 258 · 2020: 220 · 2019: 141 · 2018: 129 · 2017: 98 · 2016: 77 · 2015: 82 · 2014: 67 · 2013: 62 · 2012: 39
- FWCI
- 29.52
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Chemical Information and Computer Sciences (publishedVersion)
- PubMed (publishedVersion)
- Funders
- U.S. Air Force
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Physical Sciences › Chemistry › Analytical Chemistry › Spectroscopy and Chemometric Analyses
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Keywords
- overfitting, calibration, model validation, transition temperature
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
