What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models

Babyak, M. A.

doi:10.1097/01.psy.0000127692.23278.a9

Cited by 2,061Statistical Methods in EpidemiologyStatistical Methods and InferenceStatistical Methods and Applications

What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models

Open access

Babyak, M. A.·2004·Psychosomatic Medicine

DOI 10.1097/01.psy.0000127692.23278.a9

Open this paper in CoChat — free account

Abstract

OBJECTIVE: Statistical models, such as linear or logistic regression or survival analysis, are frequently used as a means to answer scientific questions in psychosomatic research. Many who use these techniques, however, apparently fail to appreciate fully the problem of overfitting, ie, capitalizing on the idiosyncrasies of the sample at hand. Overfitted models will fail to replicate in future samples, thus creating considerable uncertainty about the scientific merit of the finding. The present article is a nontechnical discussion of the concept of overfitting and is intended to be accessible to readers with varying levels of statistical expertise. The notion of overfitting is presented in terms of asking too much from the available data. Given a certain number of observations in a data set, there is an upper limit to the complexity of the model that can be derived with any acceptable degree of uncertainty. Complexity arises as a function of the number of degrees of freedom expended (the number of predictors including complex terms such as interactions and nonlinear terms) against the same data set during any stage of the data analysis. Theoretical and empirical evidence--with a special focus on the results of computer simulation studies--is presented to demonstrate the practical consequences of overfitting with respect to scientific inference. Three common practices--automated variable selection, pretesting of candidate predictors, and dichotomization of continuous variables--are shown to pose a considerable risk for spurious findings in models. The dilemma between overfitting and exploring candidate confounders is also discussed. Alternative means of guarding against overfitting are discussed, including variable aggregation and the fixing of coefficients a priori. Techniques that account and correct for complexity, including shrinkage and penalization, also are introduced.

Integrity signals

Registry

Crossref · 10.1097/01.psy.0000127692.23278.a9

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The finding that the changes in the mindfulness facets observation, acting with awareness, and non-reactivity to inner experience were no longer significant after including an additional, third, covariate could also be a consequence of a reduction in power or overfitting of the model.
“The finding that the changes in the mindfulness facets observation, acting with awareness, and non-reactivity to inner experience were no longer significant after including an additional, third, covariate could also be a consequence of a reduction in power or overfitting of the model as in general, more observations per predictor lead to more reliable estimates (41).”
From Trait mindfulness and personality characteristics in a microdosing ADHD sample: a naturalistic prospective survey study (2023) · cites this paper as [41]
Regression models are increasingly prone to overfitting as the number of covariates increases.
“While our consideration of multiple potential confounders is a strength, regression models are increasingly prone to overfitting as the number of covariates increases (Babyak, 2004).”
From Can mild traumatic brain injury alter cognition chronically? A LIMBIC-CENC multicenter study. (2023) · cites this paper as Babyak, 2004

Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · see citing works
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500 · see citing works
Regression Models and Life-Tables
Cox, D. R. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_37 · Cited by 26,297 · see citing works
Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences
Cohen, P., Cohen, P., West, S. G., & Aiken, L. S. (2014) · Psychology Press eBooks · doi:10.4324/9781410606266 · Cited by 20,819 · see citing works
Generalized Linear Models
(2005) · Springer eBooks · doi:10.1007/0-387-27255-0_9 · Cited by 20,333 · see citing works
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · see citing works
Regression Modeling Strategies
Harrell, F. E. (2001) · Springer series in statistics · doi:10.1007/978-1-4757-3462-1 · Cited by 9,270 · see citing works
How many subjects does it take to do a regression analysis
Green, S. B. (1991) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2603_7 · Cited by 3,916 · cited 3× · see citing works
“As Green (7) points out, however, a much larger number of observations may be needed if the effect size is small or the predictors are highly correlated.”
BackgroundInfluential
On the practice of dichotomization of quantitative variables.
MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.19 · Cited by 3,538 · cited 1× · see citing works
“4 For a thorough and damning review of dichotomization in general, see MacCallum et al. ( 27 ).”
Background
Internal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746 · see citing works
Generalized Linear Models
Schuenemeyer, J. H., McCullagh, P. J., & Nelder, J. A. (1992) · Technometrics · doi:10.2307/1269238 · Cited by 2,702 · see citing works
Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates
Peduzzi, P., Concato, J., Feinstein, A. R., & Holford, T. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00048-8 · Cited by 2,154 · see citing works
Things I have learned (so far).
Cohen, J. (1990) · American Psychologist · doi:10.1037/0003-066x.45.12.1304 · Cited by 1,781 · see citing works
Generalized Estimating Equations
Hardin, J. W., & Hilbe, J. M. (2002) · doi:10.1201/9781420035285 · Cited by 1,766 · see citing works
Backward, forward and stepwise automated subset selection algorithms: Frequency of obtaining authentic and noise variables
Derksen, S. A., & Keselman, H. J. (1992) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1992.tb00992.x · Cited by 767 · see citing works
Bivariate median splits and spurious statistical significance.
Maxwell, S. E., & Delaney, H. D. (1993) · Psychological Bulletin · doi:10.1037/0033-2909.113.1.181 · Cited by 661 · see citing works
Prognostic Modeling with Logistic Regression Analysis
Steyerberg, E. W., Eijkemans, M. J. C., Harrell, F. E., & Habbema, J. D. F. (2001) · Medical Decision Making · doi:10.1177/0272989x0102100106 · Cited by 559 · see citing works
Things I have learned (so far).
Cohen, J. (1992) · American Psychological Association eBooks · doi:10.1037/10109-028 · Cited by 553 · see citing works
Risk Adjustment in Outcome Assessment: the Charlson Comorbidity Index
Sicotte, C., Tilquin, C., & D’Hoore, W. (1993) · Methods of Information in Medicine · doi:10.1055/s-0038-1634956 · Cited by 545 · see citing works
Stepwise Regression and Stepwise Discriminant Analysis Need Not Apply here: A Guidelines Editorial
Thompson, B. (1995) · Educational and Psychological Measurement · doi:10.1177/0013164495055004001 · Cited by 434 · see citing works
Negative Consequences of Dichotomizing Continuous Predictor Variables
Irwin, J. R., & McClelland, G. H. (2003) · Journal of Marketing Research · doi:10.1509/jmkr.40.3.366.19237 · Cited by 425 · see citing works
Bootstrap investigation of the stability of a cox regression model
Altman, D. G., & Andersen, P. K. (1989) · Statistics in Medicine · doi:10.1002/sim.4780080702 · Cited by 311 · see citing works
A possible explanation for the probabilities of quantum mechanics
Aerts, D. (1986) · Journal of Mathematical Physics · doi:10.1063/1.527362 · Cited by 218 · see citing works
Prediction Error and Its Estimation for Subset-Selected Models
Roecker, E. B. (1991) · Technometrics · doi:10.2307/1269417 · Cited by 123 · see citing works
On the Cost of Data Analysis
Faraway, J. J. (1992) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1992.10474582 · Cited by 80 · see citing works
Science, technology, and social change
(1989) · Choice Reviews Online · doi:10.5860/choice.26-5636 · Cited by 78 · see citing works
The Origin of the Non-Classical Character of the Quantum Probability Model
Aerts, D. (1987) · Springer eBooks · doi:10.1007/978-3-7091-2971-5_5 · Cited by 47 · see citing works
An attempt to imagine parts of the reality of the micro-world
Aerts, D. (1990) · VUBIR (Vrije Universiteit Brussel) · Cited by 32 · see citing works
The effects of transformations and preliminary tests for non‐linearity in regression
Grambsch, P. M., & O’Brien, P. C. (1991) · Statistics in Medicine · doi:10.1002/sim.4780100504 · Cited by 31 · see citing works
Relativity theory: What is reality?
Aerts, D. (1996) · Foundations of Physics · doi:10.1007/bf02282126 · Cited by 19 · see citing works
The Metaphoric Origins of Objectivity, Subjectivity, and Consciousness in The Direct Perception of Reality
Mulaik, S. A. (1995) · Philosophy of Science · doi:10.1086/289857 · Cited by 14 · see citing works
On the problem of non-locality in quantum mechanics
Aerts, D., & Reignier, J. (1991) · VUBIR (Vrije Universiteit Brussel) · doi:10.5169/seals-116311 · Cited by 13 · see citing works
Participating in the World: Research and Education in a Changing Society
Aerts, D. (1999) · Springer eBooks · doi:10.1007/978-94-011-4706-4_1 · Cited by 5 · see citing works

Flagged first, then by citations

Sorted by citations

Higher vs Lower Positive End-Expiratory Pressure in Patients With Acute Lung Injury and Acute Respiratory Distress Syndrome: Systematic Review and Meta-analysis
Briel, M., Meade, M., Mercat, A., Brower, R. G., Talmor, D., Walter, S. D., Slutsky, A. S., Pullenayegum, E., Zhou, Q., Cook, D., et al. (2010) · JAMA · doi:10.1001/jama.2010.218 · Cited by 1,467 · see citing works
Guidelines for Developing and Reporting Machine Learning Predictive Models in Biomedical Research: A Multidisciplinary View
Luo, W., Phung, D., Tran, T., Gupta, S., Rana, S., Karmakar, C., Shilton, A., Yearwood, J., Dimitrova, N., Ho, T. B., et al. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5870 · Cited by 988 · see citing works
Variable selection strategies and its importance in clinical prediction modelling.
Chowdhury, M. Z. I., & Turin, T. C. (2020) · Family Medicine and Community Health · doi:10.1136/fmch-2019-000262 · Cited by 1,011 · see citing works
Why do we respond so differently? Reviewing determinants of human salivary cortisol responses to challenge
Kudielka, B. M., Hellhammer, D. H., & Wüst, S. (2009) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2008.10.004 · Cited by 963 · see citing works
Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses
Sun, X., Briel, M., Walter, S. D., & Guyatt, G. H. (2010) · BMJ · doi:10.1136/bmj.c117 · Cited by 808 · see citing works
Distress in couples coping with cancer: A meta-analysis and critical review of role and gender effects.
Hagedoorn, M., Sanderman, R., Bolks, H. N., Tuinstra, J., & Coyne, J. C. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.1.1 · Cited by 777 · see citing works
Depression and coronary heart disease
Carney, R. M., & Freedland, K. E. (2017) · Nature Reviews Cardiology · doi:10.1038/nrcardio.2016.181 · Cited by 705 · see citing works
Depression and anxiety as predictors of 2-year cardiac events in patients with stable coronary artery disease
Frasure-Smith, N., & Lespérance, F. (2008) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2007.4 · Cited by 571 · see citing works
Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting
Collins, G. S., Mallett, S., Omar, O., & Yu, L.-M. (2011) · BMC Medicine · doi:10.1186/1741-7015-9-103 · Cited by 484 · see citing works
To assess, to control, to exclude: Effects of biobehavioral factors on circulating inflammatory markers
O’Connor, M.-F., Bower, J. E., Cho, H. J., Creswell, J. D., Dimitrov, S., Hamby, M. E., Hoyt, M. A., Martin, J. L., Robles, T. F., Sloan, E. K., et al. (2009) · Brain, Behavior, and Immunity · doi:10.1016/j.bbi.2009.04.005 · Cited by 500 · see citing works
Association between change in high density lipoprotein cholesterol and cardiovascular disease morbidity and mortality: systematic review and meta-regression analysis
Briel, M., Ferreira-Gonzalez, I., You, J. J., Karanicolas, P. J., Akl, E. A., Wu, P., Blechacz, B., Bassler, D., Wei, X., Sharman, A., et al. (2009) · BMJ · doi:10.1136/bmj.b92 · Cited by 491 · see citing works
Evidence for low-grade systemic proinflammatory activity in patients with posttraumatic stress disorder
von Känel, R., Hepp, U., Kraemer, B., Traber, R., Keel, M., Mica, L., & Schnyder, U. (2007) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2006.06.009 · Cited by 408 · see citing works
Prenatal Depression, Prenatal Anxiety, and Spontaneous Preterm Birth: A Prospective Cohort Study Among Women With Early and Regular Care
Dayan, J., Creveuil, C., Marks, M. N., Conroy, S., Herlicoviez, M., Dreyfus, M., & Tordjman, S. (2006) · Psychosomatic Medicine · doi:10.1097/01.psy.0000244025.20549.bd · Cited by 356 · see citing works
The association between burnout, depression, anxiety, and inflammation biomarkers: C-reactive protein and fibrinogen in men and women
Toker, S., Shirom, A., Shapira, I., Berliner, S., & Melamed, S. (2005) · Journal of Occupational Health Psychology · doi:10.1037/1076-8998.10.4.344 · Cited by 340 · see citing works
"Sleepiness" is serious in adolescence: Two surveys of 3235 Canadian students
Gibson, E. S., Powles, A. P., Thabane, L., O'Brien, S., Molnar, D. S., Trajanovic, N., Ogilvie, R., Shapiro, C., Yan, M., & Chilcott-Tanser, L. (2006) · BMC Public Health · doi:10.1186/1471-2458-6-116 · Cited by 298 · see citing works
Untitled work
Bogner (2007) · doi:10.2337/dc07-0974 · Cited by 209 · see citing works
Accounting for heterogeneity via random-effects models and moderator analyses in meta-analysis.
Viechtbauer, W. (2007) · Zeitschrift für Psychologie / Journal of Psychology · doi:10.1027/0044-3409.215.2.104 · Cited by 193 · see citing works
Diabetes, Common Mental Disorders, and Disability: Findings From the UK National Psychiatric Morbidity Survey
Das-Munshi, J., Stewart, R., Ismail, K., Bebbington, P. E., Jenkins, R., & Prince, M. J. (2007) · Psychosomatic Medicine · doi:10.1097/psy.0b013e3180cc3062 · Cited by 183 · see citing works
Long‐term psychological benefits of cognitive‐behavioral stress management for women with breast cancer: 11‐year follow‐up of a randomized controlled trial
Stagl, J. M., Bouchard, L. C., Lechner, S. C., Blomberg, B. B., Gudenkauf, L. M., Jutagir, D. R., Glück, S., Derhagopian, R. P., Carver, C. S., & Antoni, M. H. (2015) · Cancer · doi:10.1002/cncr.29076 · Cited by 174 · see citing works
Children’s externalizing and internalizing symptoms over time: The role of individual differences in patterns of RSA responding
Hinnant, J. B., & El-Sheikh, M. (2009) · Journal of Abnormal Child Psychology · doi:10.1007/s10802-009-9341-1 · Cited by 167 · see citing works
Greater inflammatory activity and blunted glucocorticoid signaling in monocytes of chronically stressed caregivers
Miller, G. E., Murphy, M. L. M., Cashman, R., Ma, R., Ma, J., Arevalo, J. M. G., Kobor, M. S., & Cole, S. W. (2014) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2014.05.016 · Cited by 166 · see citing works
Accuracy of the MacArthur Competence Assessment Tool for Clinical Research (MacCAT-CR) for Measuring Children’s Competence to Consent to Clinical Research
Hein, I. M., Troost, P. W., Lindeboom, R., Benninga, M. A., Zwaan, C. M., van Goudoever, J. B., & Lindauer, R. J. L. (2014) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2014.1694 · Cited by 148 · see citing works
Family caregivers' distress levels related to quality of life, burden, and preparedness: Family caregiver distress in NSCLC
Fujinami, R., Sun, V., Zachariah, F., Uman, G., Grant, M., & Ferrell, B. (2015) · Psycho-Oncology · doi:10.1002/pon.3562 · Cited by 140 · see citing works
A randomized controlled trial of cognitive-behavioral stress management in breast cancer: survival and recurrence at 11-year follow-up
Stagl, J. M., Lechner, S. C., Carver, C. S., Bouchard, L. C., Gudenkauf, L. M., Jutagir, D. R., Diaz, A., Yu, Q., Blomberg, B. B., Ironson, G., et al. (2015) · Breast Cancer Research and Treatment · doi:10.1007/s10549-015-3626-6 · Cited by 141 · see citing works
Regression shrinkage methods for clinical prediction models do not guarantee improved performance: Simulation study
Van Calster, B., van Smeden, M., De Cock, B., & Steyerberg, E. W. (2020) · Statistical Methods in Medical Research · doi:10.1177/0962280220921415 · Cited by 120 · see citing works
Overfitting in linear feature extraction for classification of high-dimensional image data
Liu, R., & Gillies, D. F. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2015.11.015 · Cited by 123 · see citing works
The Role of Spirituality in the Relationship Between Religiosity and Depression in Prostate Cancer Patients
Nelson, C., Jacobson, C. M., Weinberger, M. I., Bhaskaran, V., Rosenfeld, B., Breitbart, W., & Roth, A. J. (2009) · Annals of Behavioral Medicine · doi:10.1007/s12160-009-9139-y · Cited by 118 · see citing works
Increased pro-inflammatory milieu in combat related PTSD – A new cohort replication study
Lindqvist, D., Dhabhar, F. S., Mellon, S. H., Yehuda, R., Grenon, S. M., Flory, J. D., Bierer, L. M., Abu-Amara, D., Coy, M., Makotkine, I., et al. (2017) · Brain, Behavior, and Immunity · doi:10.1016/j.bbi.2016.09.012 · Cited by 114 · see citing works
Mindfulness meditation for symptom reduction in fibromyalgia: psychophysiological correlates.
Lush, E., Salmon, P., Floyd, A., Studts, J. L., Weissbecker, I., & Sephton, S. E. (2009) · Journal of Clinical Psychology in Medical Settings · doi:10.1007/s10880-009-9153-z · Cited by 112 · see citing works
The Role of Psychological Stress in Inflammatory Bowel Disease: Quality Assessment of Methods of 18 Prospective Studies and Suggestions for Future Research
Cámara, R. J. A., Ziegler, R., Begré, S., Schoepfer, A. M., & von Känel, R. (2009) · Digestion · doi:10.1159/000226087 · Cited by 106 · see citing works
Serum chemokine levels in major depressive disorder
Lehto, S. M., Niskanen, L., Herzig, K.-H., Tolmunen, T., Huotari, A., Viinamäki, H., Koivumaa-Honkanen, H., Honkalampi, K., Ruotsalainen, H., & Hintikka, J. (2010) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2009.06.007 · Cited by 100 · see citing works
Pain sensitivity and pain catastrophizing are associated with persistent pain and disability after lumbar spine surgery
Coronado, R. A., George, S. Z., Devin, C. J., Wegener, S. T., & Archer, K. R. (2015) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2015.06.003 · Cited by 97 · see citing works
Anxiety, depression, and stress as risk factors for atrial fibrillation after cardiac surgery.
Tully, P. J., Bennetts, J. S., Baker, R. A., McGavigan, A. D., Turnbull, D. A., & Winefield, H. R. (2011) · Heart & Lung · doi:10.1016/j.hrtlng.2009.12.010 · Cited by 96 · see citing works
Psychological interventions to reduce suicidality in high-risk patients with major depression: a randomized controlled trial
Celano, C. M., Beale, E. E., Mastromauro, C. A., Stewart, J. G., Millstein, R. A., Auerbach, R. P., Bedoya, C. A., & Huffman, J. C. (2017) · Psychological Medicine · doi:10.1017/s0033291716002798 · Cited by 89 · see citing works
Stress management, leukocyte transcriptional changes and breast cancer recurrence in a randomized trial: An exploratory analysis
Antoni, M. H., Bouchard, L. C., Jacobs, J. M., Lechner, S. C., Jutagir, D. R., Gudenkauf, L. M., Carver, C. S., Lutgendorf, S., Cole, S. W., Lippman, M., et al. (2016) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2016.09.012 · Cited by 89 · see citing works
Systematic Evaluation of the Patient-Reported Outcome (PRO) Content of Clinical Trial Protocols
Kyte, D., Duffy, H., Fletcher, B., Gheorghe, A., Mercieca-Bebber, R., King, M., Draper, H., Ives, J., Brundage, M., Blazeby, J., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0110229 · Cited by 82 · see citing works
Altered Blood Coagulation in Patients With Posttraumatic Stress Disorder
von Känel, R., Hepp, U., Buddeberg, C., Keel, M., Mica, L., Aschbacher, K., & Schnyder, U. (2006) · Psychosomatic Medicine · doi:10.1097/01.psy.0000221229.43272.9d · Cited by 86 · see citing works
Insomnia increases risk for cardiovascular events in women and in men with low socioeconomic status: A longitudinal, register-based study
Canivet, C., Nilsson, P. M., Lindeberg, S. I., Karasek, R., & Östergren, P.-O. (2014) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2014.02.001 · Cited by 78 · see citing works
Depressive symptoms in persons with acute coronary syndrome: specific symptom scales and prognosis
Doyle, F., Conroy, R., McGee, H., & Delaney, M. (2010) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2009.07.013 · Cited by 74 · see citing works
Barriers to effective diabetes management–a survey of people with severe mental illness
Mulligan, K., McBain, H., Lamontagne-Godwin, F., Chapman, J., Flood, C., Haddad, M., Jones, J., & Simpson, A. (2018) · BMC Psychiatry · doi:10.1186/s12888-018-1744-5 · Cited by 68 · see citing works
Mood and nonmood components of perceived stress and exacerbation of Crohnʼs disease
Cámara, R. J. A., Schoepfer, A. M., Pittet, V., Begré, S., & von Känel, R. (2011) · Inflammatory Bowel Diseases · doi:10.1002/ibd.21623 · Cited by 67 · see citing works
Fatigue, not self-rated motor symptom severity, affects quality of life in functional motor disorders
Gelauff, J. M., Kingma, E. M., Kalkman, J. S., Bezemer, R., van Engelen, B. G. M., Stone, J., Tijssen, M. A. J., & Rosmalen, J. G. M. (2018) · Journal of Neurology · doi:10.1007/s00415-018-8915-7 · Cited by 67 · see citing works
Neuroticism and Cardiovascular Disease Mortality: Socioeconomic Status Modifies the Risk in Women (UK Health and Lifestyle Survey)
Hagger-Johnson, G., Roberts, B., Boniface, D., Sabia, S., Batty, G. D., Elbaz, A., Singh-Manoux, A., & Deary, I. J. (2012) · Psychosomatic Medicine · doi:10.1097/psy.0b013e31825c85ca · Cited by 63 · see citing works
Intergenerational effects of witnessing domestic violence: Health of the witnesses and their children
Forke, C. M., Catallozzi, M., Localio, A. R., Grisso, J. A., Wiebe, D. J., & Fein, J. A. (2019) · Preventive Medicine Reports · doi:10.1016/j.pmedr.2019.100942 · Cited by 63 · see citing works
Higher body mass index (BMI) is associated with reduced glucocorticoid inhibition of inflammatory cytokine production following acute psychosocial stress in men
Wirtz, P. H., Ehlert, U., Emini, L., & Suter, T. (2008) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2008.05.002 · Cited by 56 · see citing works
Developing and Evaluating Prediction Models in Rehabilitation Populations
Seel, R. T., Steyerberg, E. W., Malec, J. F., Sherer, M., & Macciocchi, S. N. (2012) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2012.04.021 · Cited by 49 · see citing works
Effect of Chronic Dementia Caregiving and Major Transitions in the Caregiving Situation on Kidney Function: A Longitudinal Study
von Känel, R., Mausbach, B. T., Dimsdale, J. E., Mills, P. J., Patterson, T. L., Ancoli-Israel, S., Ziegler, M. G., Roepke, S. K., Chattillion, E. A., Allison, M., et al. (2012) · Psychosomatic Medicine · doi:10.1097/psy.0b013e3182408c14 · Cited by 49 · see citing works
Adding a smartphone app to internet-based self-help for social anxiety: A randomized controlled trial
Boettcher, J., Magnusson, K., Marklund, A., Berglund, E., Blomdahl, R., Braun, U., Delin, L., Lundén, C., Sjöblom, K., Sommer, D., et al. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.04.052 · Cited by 47 · see citing works
Multivariate regression: the pitfalls of automated variable selection
Sainani, K. L. (2013) · PM&R · doi:10.1016/j.pmrj.2013.07.007 · Cited by 43 · see citing works
Patient Health Questionnaire-9 score and adverse cardiac outcomes in patients hospitalized for acute cardiac disease
Beach, S. R., Januzzi, J. L., Mastromauro, C. A., Healy, B. C., Beale, E. E., Celano, C. M., & Huffman, J. C. (2013) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2013.08.001 · Cited by 40 · see citing works
Depression and anxiety following psychosis: Associations with mindfulness and psychological flexibility
White, R. G., Gumley, A. I., McTaggart, J., Rattrie, L., McConville, D., Cleare, S., & Mitchell, G. (2013) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465812000239 · Cited by 37 · see citing works
Peripheral antioxidant markers are associated with total hippocampal and CA3/dentate gyrus volume in MDD and healthy controls—Preliminary findings
Lindqvist, D., Mueller, S., Mellon, S. H., Su, Y., Epel, E. S., Reus, V. I., Rosser, R., Mahan, L., Mackin, R. S., Yang, T. T., et al. (2014) · Psychiatry Research: Neuroimaging · doi:10.1016/j.pscychresns.2014.09.002 · Cited by 33 · see citing works
New onset anxiety and depression in patients with an implantable cardioverter defibrillator during 24 months of follow-up (data from the national DEFIB-WOMEN study)
Pedersen, S. S., Nielsen, J. C., Wehberg, S., Jørgensen, O. D., Riahi, S., Haarbo, J., Philbert, B. T., Larsen, M. L., & Johansen, J. B. (2021) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2021.07.003 · Cited by 32 · see citing works
Embracing complex models: Exploratory network analyses of psychological (In)Flexibility processes and unique associations with psychiatric symptomology and quality of life
Baker, L. D., & Berghoff, C. R. (2022) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2021.12.002 · Cited by 28 · see citing works
More Than the Sum of Its Parts: Meta-Analysis and Its Potential to Discover Sources of Heterogeneity in Psychosomatic Medicine
Tak, L. M., Meijer, A., Manoharan, A., de Jonge, P., & Rosmalen, J. G. M. (2010) · Psychosomatic Medicine · doi:10.1097/psy.0b013e3181d714e1 · Cited by 23 · see citing works
Reporting quality of clinical trial protocols: a repeated cross-sectional study about the Adherence to SPIrit Recommendations in Switzerland, CAnada and GErmany (ASPIRE-SCAGE)
Gryaznov, D., von Niederhäusern, B., Speich, B., Kasenda, B., Ojeda-Ruiz, E., Blümle, A., Schandelmaier, S., Mertz, D., Odutayo, A., Tomonaga, Y., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-053417 · Cited by 22 · see citing works
Can mild traumatic brain injury alter cognition chronically? A LIMBIC-CENC multicenter study.
Walker, W. C., O'Neil, M. E., Ou, Z., Pogoda, T. K., Belanger, H. G., Scheibel, R. S., Presson, A. P., Miles, S. R., Wilde, E. A., Tate, D. F., et al. (2023) · Neuropsychology · doi:10.1037/neu0000855 · Cited by 22 · see citing works
Untitled work
Schermelleh-Engel (2003) · doi:10.1016/j.psyneuen.2012.07.017 · Cited by 18 · see citing works
Models of alien species richness show moderate predictive accuracy and poor transferability
Capinha, C., Essl, F., Seebens, H., Pereira, H. M., & Kühn, I. (2018) · NeoBiota · doi:10.3897/neobiota.38.23518 · Cited by 16 · see citing works
Influencing walking behavior can increase the physical activity of patients with chronic pain hospitalized for multidisciplinary rehabilitation: an observational study
Terrier, P., Praz, C., Le Carré, J., Vuistiner, P., Léger, B., & Luthi, F. (2019) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-019-2561-9 · Cited by 14 · see citing works
Trait mindfulness and personality characteristics in a microdosing ADHD sample: a naturalistic prospective survey study
Haijen, E. C. H. M., Hurks, P. P. M., & Kuypers, K. P. C. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1233585 · Cited by 7 · see citing works
Application of predictive modeling tools for the identification of Ocimum spp. herbal products
Abraham, E. J., Chamberlain, S. J., Perera, W. H., Jordan, R. T., & Kellogg, J. J. (2025) · Analytical and Bioanalytical Chemistry · doi:10.1007/s00216-025-05735-0 · Cited by 2 · see citing works
“The use of PLS-discriminant analysis (PLS-DA) in herbal authentication studies has been thoroughly reviewed [8–10]; however, linear models like PLS tend to overfit data containing more variables than samples (as is the case with untargeted metabolomics) [5, 11, 12].”
Inferring catalysis in biological systems
Kondofersky, I., Theis, F. J., & Fuchs, C. (2016) · IET Systems Biology · doi:10.1049/iet-syb.2015.0087 · Cited by 1 · see citing works

Authors: Babyak, M. A.
Affiliations: Duke University (US)
Corresponding author: Babyak, M. A.
Venue: Psychosomatic Medicine
Publisher: Ovid Technologies (Wolters Kluwer Health)
Published: 2004-05-01
Volume: 66
Issue: 3
Pages: 411-421
ISSN: 0033-3174 (ISSN-L)
1534-7796
DOI: 10.1097/01.psy.0000127692.23278.a9
OpenAlex: W2165786903
Cited by: 2,061
References: 33
Citations by year: 2026: 109 · 2025: 139 · 2024: 110 · 2023: 98 · 2022: 110 · 2021: 167 · 2020: 144 · 2019: 149 · 2018: 122 · 2017: 139 · 2016: 110 · 2015: 100 · 2014: 105 · 2013: 92 · 2012: 92
FWCI: 16.88
Type: journal article
Language: en
Open access: hybrid
doi.org
Licence: cc-by-nc-nd
Locations: Psychosomatic Medicine (publishedVersion)
Psychosomatic Medicine (publishedVersion, cc-by-nc-nd)
PubMed (publishedVersion)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
Keywords: overfitting, regression models, model complexity, degrees of freedom, replication, variable aggregation, shrinkage, penalty method, spurious findings, psychosomatic research
MeSH terms: Computer Simulation, Data Interpretation, Statistical, Humans, Psychophysiology, Psychosomatic Medicine, Regression Analysis, Research Design, Statistics as Topic, Models, Statistical

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks