The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
What You See May Not Be What You Get: A Brief, Nontechnical Introduction to Overfitting in Regression-Type Models
Open accessAbstract
OBJECTIVE: Statistical models, such as linear or logistic regression or survival analysis, are frequently used as a means to answer scientific questions in psychosomatic research. Many who use these techniques, however, apparently fail to appreciate fully the problem of overfitting, ie, capitalizing on the idiosyncrasies of the sample at hand. Overfitted models will fail to replicate in future samples, thus creating considerable uncertainty about the scientific merit of the finding. The present article is a nontechnical discussion of the concept of overfitting and is intended to be accessible to readers with varying levels of statistical expertise. The notion of overfitting is presented in terms of asking too much from the available data. Given a certain number of observations in a data set, there is an upper limit to the complexity of the model that can be derived with any acceptable degree of uncertainty. Complexity arises as a function of the number of degrees of freedom expended (the number of predictors including complex terms such as interactions and nonlinear terms) against the same data set during any stage of the data analysis. Theoretical and empirical evidence--with a special focus on the results of computer simulation studies--is presented to demonstrate the practical consequences of overfitting with respect to scientific inference. Three common practices--automated variable selection, pretesting of candidate predictors, and dichotomization of continuous variables--are shown to pose a considerable risk for spurious findings in models. The dilemma between overfitting and exploring candidate confounders is also discussed. Alternative means of guarding against overfitting are discussed, including variable aggregation and the fixing of coefficients a priori. Techniques that account and correct for complexity, including shrinkage and penalization, also are introduced.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The finding that the changes in the mindfulness facets observation, acting with awareness, and non-reactivity to inner experience were no longer significant after including an additional, third, covariate could also be a consequence of a reduction in power or overfitting of the model as in general, more observations per predictor lead to more reliable estimates (41).”
From Trait mindfulness and personality characteristics in a microdosing ADHD sample: a naturalistic prospective survey study (2023) · cites this paper as [41]
In the citing paper
“While our consideration of multiple potential confounders is a strength, regression models are increasingly prone to overfitting as the number of covariates increases (Babyak, 2004).”
From Can mild traumatic brain injury alter cognition chronically? A LIMBIC-CENC multicenter study. (2023) · cites this paper as Babyak, 2004
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
Regression Models and Life-Tables
Cox, D. R. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_37 · Cited by 26,297
Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences
Cohen, P., Cohen, P., West, S. G., & Aiken, L. S. (2014) · Psychology Press eBooks · doi:10.4324/9781410606266 · Cited by 20,737
(2005) · Statistics for Biology and Health · doi:10.1007/0-387-27255-0_9 · Cited by 20,133
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Harrell, F. E. (2001) · Springer Series in Statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,195
How many subjects does it take to do a regression analysis
Green, S. B. (1991) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2603_7 · Cited by 3,916 · cited 3×
“As Green (7) points out, however, a much larger number of observations may be needed if the effect size is small or the predictors are highly correlated.”
BackgroundInfluentialOn the practice of dichotomization of quantitative variables.
MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.19 · Cited by 3,538 · cited 1×
“4 For a thorough and damning review of dichotomization in general, see MacCallum et al. ( 27 ).”
BackgroundInternal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746
Cited by
Cumulative risk and child development.
Evans, G. W., Li, D., & Whipple, S. S. (2013) · Psychological Bulletin · doi:10.1037/a0031808 · Cited by 2,024
Briel, M., Meade, M., Mercat, A., Brower, R. G., Talmor, D., Walter, S. D., Slutsky, A. S., Pullenayegum, E., Zhou, Q., Cook, D., et al. (2010) · JAMA · doi:10.1001/jama.2010.218 · Cited by 1,467
Eeftens, M., Beelen, R., de Hoogh, K., Bellander, T., Cesaroni, G., Cirach, M., Declercq, C., Dėdelė, A., Dons, E., de Nazelle, A., et al. (2012) · Environmental Science & Technology · doi:10.1021/es301948k · Cited by 1,214
Luo, W., Phung, D., Tran, T., Gupta, S., Rana, S., Karmakar, C., Shilton, A., Yearwood, J., Dimitrova, N., Ho, T. B., et al. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5870 · Cited by 988
Beelen, R., Hoek, G., Vienneau, D., Eeftens, M., Dimakopoulou, K., Pedeli, X., Tsai, M.-Y., Künzli, N., Schikowski, T., Marcon, A., et al. (2013) · Atmospheric Environment · doi:10.1016/j.atmosenv.2013.02.037 · Cited by 1,014
Metadata
- Authors
- Babyak, M. A.
- Affiliations
- Duke University (US)
- Corresponding author
- Babyak, M. A.
- Venue
- Psychosomatic Medicine
- Published
- 2004-05-01
- Volume
- 66
- Issue
- 3
- Pages
- 411-421
- ISSN
- 0033-3174 (ISSN-L)
- 1534-7796
- OpenAlex
- W2165786903
- Cited by
- 2,061
- References
- 35
- Citations by year
- 2026: 109 · 2025: 139 · 2024: 110 · 2023: 98 · 2022: 110 · 2021: 167 · 2020: 144 · 2019: 149 · 2018: 122 · 2017: 139 · 2016: 110 · 2015: 100 · 2014: 105 · 2013: 92 · 2012: 92
- FWCI
- 16.88
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Psychosomatic Medicine (publishedVersion)
- Psychosomatic Medicine (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Keywords
- overfitting, regression models, model complexity, degrees of freedom, replication, variable aggregation, shrinkage, penalty method, spurious findings, psychosomatic research
- MeSH terms
- Computer Simulation, Data Interpretation, Statistical, Humans, Psychophysiology, Psychosomatic Medicine, Regression Analysis, Research Design, Statistics as Topic, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: What You See May Not Be What You Get: A Brief, Nontechnical Introduction to Overfitting in Regression-Type Models. https://app.cochat.ai/papers/10.1097/01.psy.0000127692.23278.a9
- The paper
- Babyak, M. A. (2004). What You See May Not Be What You Get: A Brief, Nontechnical Introduction to Overfitting in Regression-Type Models. Psychosomatic Medicine. https://doi.org/10.1097/01.psy.0000127692.23278.a9
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.