CoChat
Sign Up
PreprintCited by 26Statistics Education and MethodologiesStatistical Methods in EpidemiologyAdvanced Causal Inference Techniques

On the uses and abuses of regression models: a call for reform of statistical practice and teaching

Open access
Carlin, John B., Moreno-Betancur, Margarita·2023·arXiv
DOI 10.48550/arxiv.2309.06668arXiv 2309.06668PMC PMC12186762
Open this paper in CoChat — free accountOpen PDF

Abstract

Regression methods dominate the practice of biostatistical analysis, but biostatistical training emphasises the details of regression models and methods ahead of the purposes for which such modelling might be useful. More broadly, statistics is widely understood to provide a body of techniques for "modelling data", underpinned by what we describe as the "true model myth": that the task of the statistician/data analyst is to build a model that closely approximates the true data generating process. By way of our own historical examples and a brief review of mainstream clinical research journals, we describe how this perspective has led to a range of problems in the application of regression methods, including misguided "adjustment" for covariates, misinterpretation of regression coefficients and the widespread fitting of regression models without a clear purpose. We then outline a new approach to the teaching and application of biostatistical methods, which situates them within a framework that first requires clear definition of the substantive research question at hand within one of three categories: descriptive, predictive, or causal. Within this approach, the development and application of (multivariable) regression models, as well as other advanced biostatistical methods, should proceed differently according to the type of question. Regression methods will no doubt remain central to statistical practice as they provide a powerful tool for representing variation in a response or outcome variable as a function of "input" variables, but their conceptualisation and usage should follow from the purpose at hand.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2309.06668

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 69

  1. M. A. Hernán, J. Hsu, and B. Healy, "A Second Chance to Get Causal Inference Right: A Classification of Data Science Tasks," Chance 32, no. 1 (2019): 42-49.
  2. C. R. Lesko, M. P. Fox, and J. K. Edwards, "A Framework for Descrip- tive Epidemiology," American Journal of Epidemiology 191, no. 12 (2022): 2063-2070.
  3. E. W. Steyerberg, Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating, 2nd ed. (Switzerland: Springer Nature, 2019).
  4. M. A. Hernán and J. M. Robins, "Estimating Causal Effects From Epi- demiological Data," Journal of Epidemiology and Community Health 60 (2006): 578-586.
  5. D. Westreich and S. Greenland, "The Table 2 Fallacy: Presenting and Interpreting Confounder and Modifier Coefficients," American Journal of Epidemiology 177, no. 4 (2013): 292-298.
  6. C. M. Schooling and H. E. Jones, "Clarifying Questions About "Risk Factors": Predictors Versus Explanation," Emerging Themes in Epidemi- ology 15, no. 1 (2018): 10.
  7. A. Huitfeldt, "Is Caviar a Risk Factor for Being a Millionaire?," BMJ 355 (2016): i6536.
  8. G. E. P. Box, "Science and Statistics," Journal of the American Statistical Association 71, no. 356 (1976): 791-799.
  9. F. E. Pickworth, J. B. Carlin, M. R. Ditchfield, et al., "Sonographic Mea- surement of Renal Enlargement in Children With Acute Pyelonephritis and Time Needed for Resolution: Implications for Renal Growth Assess- ment," American Journal of Roentgenology 165, no. 2 (1995): 405-408.
  10. M. Katz, E. Phelan, J. B. Carlin, and S. W. Beasley, "Gas Enema for the Reduction of Intussusception: Relationship Between Clinical Signs and Symptoms and Outcome," American Journal of Roentgenology 160 (1993): 363-366.
  11. V. Amrhein, S. Greenland, and B. McShane, "Scientists Rise Up Against Statistical Significance," Nature 567 (2019): 305-307.
  12. S. Greenland, S. J. Senn, K. J. Rothman, et al., "Statistical Tests, P Val- ues, Confidence Intervals, and Power: A Guide to Misinterpretations," European Journal of Epidemiology 31, no. 4 (2016): 337-350.
  13. M. A. Jenkins, J. L. Hopper, L. B. Flander, J. B. Carlin, and G. G. Giles, "The Associations Between Childhood Asthma and Atopy, and Parental Asthma, Hay Fever and Smoking," Paediatric and Perinatal Epidemiology 7 (1993): 67-76.
  14. D. W. Hosmer, Jr., S. Lemeshow, and R. X. Sturdivant, Applied Logistic Regression, 3rd ed. (New York: John Wiley & Sons, 2013).
  15. M. Moreno-Betancur, "The Target Trial: A Powerful Device Beyond Well-Defined Interventions," Epidemiology 32, no. 2 (2021): 291-294.
  16. S. Galea and M. A. Hernán, "Win-Win: Reconciling Social Epidemiol- ogy and Causal Inference," American Journal of Epidemiology 189, no. 3 (2019): 167-170.
  17. N. J. Horton and S. S. Switzer, "Statistical Methods in the Journal," New England Journal of Medicine 353, no. 18 (2005): 1977-1979.
  18. P. Nieminen and J. Kaur, "Reporting of Data Analysis Methods in Psy- chiatric Journals: Trends From 1996 to 2018," International Journal of Methods in Psychiatric Research 28, no. 3 (2019): e1784.
  19. D. Jemielniak, G. Masukume, and M. Wilamowski, "The Most Influ- ential Medical Journals According to Wikipedia: Quantitative Analysis," Journal of Medical Internet Research 21, no. 1 (2019): e11429.
  20. H. Reese, A. D. Iuliano, N. N. Patel, et al., "Estimated Inci- dence of Coronavirus Disease 2019 (COVID-19) Illness and Hospitalization-United States, February-September 2020," Clinical Infectious Diseases 72, no. 12 (2020): e1010-e1017.
  21. J. M. Jones, M. Stone, H. Sulaeman, et al., "Estimated US Infection- and Vaccine-Induced SARS-CoV-2 Seroprevalence Based on Blood Dona- tions, July 2020-May 2021," Journal of the American Medical Association 326, no. 14 (2021): 1400-1409.
  22. P. Royston and D. G. Altman, "Regression Using Fractional Polyno- mials of Continuous Covariates: Parsimonious Parametric Modelling," Applied Statistics 43, no. 3 (1994): 429-467.
  23. D. Ruppert, M. P. Wand, and R. J. Carroll, Semiparametric Regression (Cambridge: Cambridge University Press, 2003).
  24. J. A. Barber and S. G. Thompson, "Analysis of Cost Data in Random- ized Trials: An Application of the Non-parametric Bootstrap," Statistics in Medicine 19 (2000): 3219-3236.
  25. D. H. Freeman and T. R. Holford, "Summary Rates," Biometrics 36, no. 2 (1980): 195-205.
  26. N. Keiding and D. Clayton, "Standardization and Control for Con- founding in Observational Studies: A Historical Perspective," Statistical Science 29, no. 4 (2014): 529-558.
  27. M. Moreno-Betancur, J. J. Koplin, A. L. Ponsonby, J. Lynch, and J. B. Carlin, "Measuring the Impact of Differences in Risk Factor Dis- tributions on Cross-Population Differences in Disease Occurrence: A Causal Approach," International Journal of Epidemiology 47, no. 1 (2018): 217-225.
  28. M. Downes and J. B. Carlin, "Multilevel Regression and Poststratifi- cation as a Modeling Approach for Estimating Population Quantities in Large Population Health Studies: A Simulation Study," Biometrical Jour- nal 62, no. 2 (2020): 479-491.
  29. A. Sjölander, "Regression Standardization With the R Package stdReg," European Journal of Epidemiology 31, no. 6 (2016): 563-574.
  30. J. S. Kaufman, "Statistics, Adjusted Statistics, and Maladjusted Statis- tics," American Journal of Law & Medicine 43, no. 2-3 (2017): 193-208.
  31. R. Tibshirani, "Regression Shrinkage and Selection Via the Lasso," Journal of the Royal Statistical Society: Series B: Methodological 58, no. 1 (1996): 267-288.
  32. T. Hastie, R. Tibshirani, and R. Tibshirani, "Best Subset, Forward Step- wise or Lasso? Analysis and Recommendations Based on Extensive Com- parisons," Statistical Science 35, no. 4 (2020): 579-592, 514.
  33. E. Christodoulou, J. Ma, G. S. Collins, E. W. Steyerberg, J. Y. Verbakel, and B. Van Calster, "A Systematic Review Shows no Performance Bene- fit of Machine Learning Over Logistic Regression for Clinical Prediction Models," Journal of Clinical Epidemiology 110 (2019): 12-22.
  34. M. A. Hernán, "Does Water Kill? A Call for Less Casual Causal Infer- ences," Annals of Epidemiology 26, no. 10 (2016): 674-680.
  35. M. A. Hernán, "The C-Word: Scientific Euphemisms Do Not Improve Causal Inference From Observational Data," American Journal of Public Health 108, no. 5 (2018): 616-619.
  36. L. E. Dang, S. Gruber, H. Lee, et al., "A Causal Roadmap for Generat- ing High-Quality Real-World Evidence," Journal of Clinical and Transla- tional Science 7, no. 1 (2023): e212.
  37. D. B. Rubin, "Estimating Causal Effects of Treatments in Random- ized and Nonrandomized Studies," Journal of Educational Psychology 66 (1974): 688-701.
  38. M. A. Hernán and J. M. Robins, "Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available," American Journal of Epidemiology 183, no. 8 (2016): 758-764.
  39. M. Moreno-Betancur, R. Wijesuriya, and J. B. Carlin, "The Case for Specifying the "Ideal Trials, Target Trials and Actual Randomized Trials" (2024), https://doi.org/10.48550/arXiv.2405.10026. · DOI
  40. T. J. VanderWeele, "Principles of Confounder Selection," European Journal of Epidemiology 34, no. 3 (2019): 211-219.
  41. K. F. Arnold, V. Davies, M. de Kamps, P. W. G. Tennant, J. Mbotwa, and M. S. Gilthorpe, "Reflections on Modern Methods: Generalized Linear Models for Prognosis and Intervention-Theory, Practice and Implica- tions for Machine Learning," International Journal of Epidemiology 49, no. 6 (2020): 2074-2082.
  42. S. R. Cole, R. W. Platt, E. F. Schisterman, H. Chu, and D. Westreich, "Il- lustrating Bias due to Conditioning on a Collider," International Journal of Epidemiology 39 (2010): 417-420.
  43. J. C. Digitale, J. N. Martin, and M. M. Glymour, "Tutorial on Directed Acyclic Graphs," Journal of Clinical Epidemiology 142 (2022): 264-267.
  44. M. Moreno-Betancur, K. J. Lee, F. P. Leacy, I. R. White, J. A. Simpson, and J. B. Carlin, "Canonical Causal Diagrams to Guide the Treatment of Missing Data in Epidemiologic Studies," American Journal of Epidemiol- ogy 187, no. 12 (2018): 2705-2715.
  45. J. Zhang, S. G. Dashti, J. B. Carlin, K. J. Lee, and M. Moreno-Betancur, "Recoverability and Estimation of Causal Effects Under Typical Multi- variable Missingness Mechanisms," Biometrical Journal 66, no. 3 (2024): 2200326.
  46. E. F. Schisterman, S. R. Cole, and R. W. Platt, "Overadjustment Bias and Unnecessary Adjustment in Epidemiologic Studies," Epidemiology 20, no. 4 (2009): 488-495.
  47. M. J. Smith, M. A. Mansournia, C. Maringe, et al., "Introduction to Computational Causal Inference Using Reproducible Stata, R, and Python Code: A Tutorial," Statistics in Medicine 41, no. 2 (2022): 407-432.
  48. S. Greenland, "Interactions in Epidemiology: Relevance, Identifica- tion and Estimation," Epidemiology 20, no. 1 (2009): 14-17.
  49. S. Greenland, J. M. Robins, and J. Pearl, "Confounding and Collapsi- bility in Causal Inference," Statistical Science 14, no. 1 (1999): 29-46.
  50. S. Greenland, M. A. Mansournia, and D. G. Altman, "Sparse Data Bias: A Problem Hiding in Plain Sight," BMJ 353 (2016): i1981.
  51. S. Greenland and M. A. Mansournia, "Penalization, Bias Reduction, and Default Priors in Logistic and Related Categorical and Survival Regressions," Statistics in Medicine 34, no. 23 (2015): 3133-3143.
  52. P. Cummings, "Methods for Estimating Adjusted Risk Ratios," Stata Journal 9, no. 2 (2009): 175-196.
  53. S. Ellul, J. B. Carlin, S. Vansteelandt, and M. Moreno-Betancur, "Causal Machine Learning Methods and Use of Sample Splitting in Set- tings With High-Dimensional Confounding" (2024), https://doi.org/10 .48550/arXiv.2405.15242. · DOI
  54. J. Levy, M. van der Laan, A. Hubbard, and R. Pirracchio, "A Funda- mental Measure of Treatment Effect Heterogeneity," Journal of Causal Inference 9, no. 1 (2021): 83-108.
  55. L. Breiman, "Statistical Modeling: The Two Cultures," Statistical Sci- ence 16, no. 3 (2001): 199-231.
  56. G. Shmueli, "To Explain or to Predict?," Statistical Science 25, no. 3 (2010): 289-310.
  57. M. A. Hernán, "A Definition of Causal Effect for Epidemiological Research," Journal of Epidemiology and Community Health 58, no. 4 (2004): 265-271.
  58. D. S. Small, "Introduction to Observational Studies and the Reprint of Cochran's Paper "Observational Studies" and Comments," Observational Studies 1 (2015): 124-125.
  59. S. Vansteelandt, "Statistical Modelling in the Age of Data Science," Observational Studies 7, no. 1 (2021): 217-228.
  60. E. Vittinghoff, D. V. Glidden, S. C. Shiboski, and C. E. McCulloch, Regression Methods in Biostatistics: Linear, Logistic, Survival, and Repeated Measures Models, 2nd ed. (New York: Springer, 2012).
  61. A. Gelman, J. Hill, and A. Vehtari, Regression and Other Stories (Cam- bridge: Cambridge University Press, 2020).
  62. D. G. Altman, "Statistics in Medical Journals," Statistics in Medicine 1, no. 1 (1982): 59-71.
  63. D. G. Altman, "Statistics in Medical Journals: Some Recent Trends," Statistics in Medicine 19 (2000): 3275-3289.
  64. W. Sauerbrei, M. Abrahamowicz, D. G. Altman, S. Le Cessie, J. Carpenter, and STRATOS initiative, "STRengthening Analytical Think- ing for Observational Studies: The STRATOS Initiative," Statistics in Medicine 33, no. 30 (2014): 5413-5432.
  65. K. F. Schulz, D. G. Altman, and D. Moher, "CONSORT 2010 State- ment: Updated Guidelines for Reporting Parallel Group Randomised Tri- als," BMJ 340 (2010): c332.
  66. K. G. M. Moons, D. G. Altman, J. B. Reitsma, et al., "Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): Explanation and Elaboration," Annals of Inter- nal Medicine 162, no. 1 (2015): W1-W73.
  67. H. J. Hansford, A. G. Cashin, M. D. Jones, et al., "Development of the TrAnsparent ReportinG of Observational Studies Emulating a Target Trial (TARGET) Guideline," BMJ Open 13, no. 9 (2023): e074626.
  68. B. A. Nosek, G. Alter, G. C. Banks, et al., "Promoting an Open Research Culture," Science 348, no. 6242 (2015): 1422-1425.
  69. D. van Ravenzwaaij, M. Bakker, R. Heesen, et al., "Perspectives on Sci- entific Error," Royal Society Open Science 10, no. 7 (2023): 230448.

Cited by

Sorted by citations

Authors
Carlin, J. B., & Moreno-Betancur, M.
Affiliations
The University of Melbourne (AU)
Corresponding author
Carlin, John B.
ORCID
Carlin, John B.
Venue
arXiv
Publisher
arXiv
Published
2023
Volume
44
Issue
13-14
Pages
e10244-e10244
ISSN
0277-6715 (ISSN-L)
DOI
10.48550/arxiv.2309.06668
arXiv
2309.06668
PMC
PMC12186762
OpenAlex
W4411606088
Cited by
26
References
0
Citations by year
2026: 12 · 2025: 11 · 2024: 3
FWCI
19.41
Type
journal article
Language
en
Open access
Licence
Locations
  • Statistics in Medicine (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • Minerva Access (University of Melbourne) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • Children’s Hospital of Wisconsin Research Institute
  • Murdoch Children's Research Institute
  • University of Melbourne
  • State Government of Victoria
  • Victorian Centre for Biostatistics
  • Medical Research Council
  • National Health and Medical Research Council (2009572)
Topics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords
regression models, covariate adjustment, research question formulation, statistical reform
MeSH terms
Data Interpretation, Statistical, Humans, Regression Analysis, Models, Statistical, Biostatistics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review