Relaxing the Rule of Ten Events per Variable in Logistic and Cox Regression
The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
The rule of thumb that logistic and Cox models should be used with a minimum of 10 outcome events per predictor variable (EPV), based on two simulation studies, may be too conservative. The authors conducted a large simulation study of other influences on confidence interval coverage, type I error, relative bias, and other model performance measures. They found a range of circumstances in which coverage and bias were within acceptable levels despite less than 10 EPV, as well as other factors that were as influential as or more influential than EPV. They conclude that this rule can be relaxed, in particular for sensitivity analyses undertaken to demonstrate adequate control of confounding.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Following the generally accepted rule of thumb that 5–10 events are required per covariate in logit or Cox regression models (Peduzzi et al., 1995; Peduzzi et al., 1996; Vittinghoff & McCulloch, 2007), we assessed whether or not each paper could have supported fully saturated regression models with product terms between all of the psychosocial problems considered.”
From Syndemics of psychosocial problems and HIV risk: A systematic review of empirical tests of the disease interaction concept (2015) · cites this paper as Vittinghoff & McCulloch, 2007
In the citing paper
“Because so few students used psychotherapy, we limited the number of explanatory variables to the number of observed events divided by five (14).”
From Mental health treatment seeking among veteran and civilian community college students (2017) · cites this paper as [14]
In the citing paper
“This sample size was chosen taking into account approximate sample size estimation criteria [38,39]. We worked with the criteria that 10 subjects would be needed per parameter included in a statistical model used to adjust for confounding.”
From Relationships, love and sexuality: what the Filipino teens think and feel (2009) · cites this paper as [38]
In the citing paper
“As Vittinghoff and McCulloch 27 recommended, the 10 outcome events per predictor variable rule was relaxed.”
From Mental health services for parents who lost a child to cancer: if we build them, Will they come? (2015) · cites this paper as [27]
In the citing paper
“The requirements regarding events per variable were fulfilled owing to a considerable number of injuries in the present dataset.34”
From How much running is too much? Identifying high-risk running sessions in a 5200-person cohort study (2025) · cites this paper as [34]
Citations
Cites
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · cited 1×
“(2, 3) more closely, we also examined models with all binary predictors.”
BackgroundModeling and variable selection in epidemiologic analysis
Greenland, S. (1989) · American Journal of Public Health · doi:10.2105/ajph.79.3.340 · Cited by 2,208
Peduzzi, P., Concato, J., Feinstein, A. R., & Holford, T. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00048-8 · Cited by 2,154 · cited 2×
“(2, 3) more closely, we also examined models with all binary predictors.”
BackgroundResultThe Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis
Hoenig, J. M., & Heisey, D. M. (2001) · The American Statistician · doi:10.1198/000313001300339897 · Cited by 2,046
Sample-Size Formula for the Proportional-Hazards Regression Model
Schoenfeld, D. A. (1983) · Biometrics · doi:10.2307/2531021 · Cited by 1,096
Concato, J., Peduzzi, P., Holford, T. R., & Feinstein, A. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00510-2 · Cited by 797 · cited 1×
“The rule of thumb that logistic and Cox models should be used with a minimum of 10 events per predictor variable (EPV) is based on two simulation studies (1–3).”
MethodSection 4 Long-term survival results in medically and surgically randomized patients
Detre, K. M., Peduzzi, P. N., Scott, S. M., & Davies, B. (1985) · Progress in Cardiovascular Diseases · doi:10.1016/0033-0620(85)90019-2 · Cited by 24
Section 2 Design and baseline characteristics
Peduzzi, P. N., Detre, K. M., & Gage, A. A. (1985) · Progress in Cardiovascular Diseases · doi:10.1016/0033-0620(85)90017-9 · Cited by 15
Harrell, F., Jr. (1996) · doi:10.1002/(sici)1097-0258(19960229)15:4<361::aid-sim168>3.0.co;2-4
Cited by
Items 101–200 of 1,018
Han, K., Song, K., & Choi, B. W. (2016) · Korean Journal of Radiology · doi:10.3348/kjr.2016.17.3.339 · Cited by 162
“From some empirical simulation studies, a rule of thumb for sample size has been suggested (13, 14) in which at least 10 events (or non-events depending on which side is smaller) are required per candidate predictor, although other investigations have found the value of 10 to be too strict (15) or, conversely, too lax…”
BackgroundIncreases in Intimate Partner Violence During COVID-19: Prevalence and Correlates
Peitzmeier, S. M., Fedina, L., Ashwell, L., Herrenkohl, T. I., & Tolman, R. (2022) · Journal of Interpersonal Violence · doi:10.1177/08862605211052586 · Cited by 160
“…a guideline that the number of variables in a multivariable logistic regression should not exceed 20% of the number of events in the dataset (Vittinghoff & McCulloch, 2007), we then selected a parsimonious multivariable model and collapsed variable categories based on our conceptual…”
MethodCytomegalovirus reactivation and associated outcome of critically ill patients with severe sepsis
Heininger, A., Haeberle, H., Fischer, I., Beck, R., Riessen, R., Rohde, F., Meisner, C., Jahn, G., Koenigsrainer, A., Unertl, K., et al. (2011) · Critical Care · doi:10.1186/cc10069 · Cited by 160
National and subnational prevalence and burden of glaucoma in China: A systematic analysis
Song, P., Wang, J., Bucan, K., Theodoratou, E., Rudan, I., & Chan, K. Y. (2017) · Journal of Global Health · doi:10.7189/jogh.07.020705 · Cited by 159
Disentangling Cognitive-Frailty: Results From the Gait and Brain Study
Montero-Odasso, M. M., Barnes, B., Speechley, M., Muir Hunter, S. W., Doherty, T. J., Duque, G., Gopaul, K., Sposato, L. A., Casas-Herrero, A., Borrie, M. J., et al. (2016) · The Journals of Gerontology Series A: Biological Sciences and Medical Sciences · doi:10.1093/gerona/glw044 · Cited by 158
ten Brinke, L., & Porter, S. (2012) · Law and Human Behavior · doi:10.1037/h0093929 · Cited by 194
“…and genuine pleaders, we conducted a direct binary logistic regression analysis with all four significant direct appeal cues (presence of upper face sadness and lower face happiness, word count, and percentage of tentative words) entered as predictors (Hypothesis 5; Vittinghoff & McCulloch, 2007).”
Anderson, W. G., Chase, R., Pantilat, S. Z., Tulsky, J. A., & Auerbach, A. D. (2011) · Journal of General Internal Medicine · doi:10.1007/s11606-010-1568-6 · Cited by 150
“Due to the small sample size and event rate, we limited eachmodel to one predictor in addition to a fixed effect for hospital.(26) All statistical analyses were performed with Stata 11 (StataCorp LP, College Station, TX).”
MethodDulai, P. S., Boland, B. S., Singh, S., Chaudrey, K., Koliani‐Pace, J. L., Kochhar, G. S., Parikh, M. P., Shmidt, E., Hartke, J. G., Chilukuri, P., et al. (2018) · Gastroenterology · doi:10.1053/j.gastro.2018.05.039 · Cited by 164
Sikkema, M., Kerkhof, M., Steyerberg, E. W., Kusters, J. G., van Strien, P. M. H., Looman, C. W., van Dekken, H., Siersema, P. D., & Kuipers, E. J. (2009) · The American Journal of Gastroenterology · doi:10.1038/ajg.2009.437 · Cited by 148
“However, lower ratios than one predictor for each of the 10 pairs have been previously used in etiological research and are sometimes required, given the frequency of the condition and event under study and the exploratory character of the research question (”
Five-Year Follow-up of Harms and Benefits of Behavioral Infant Sleep Intervention: Randomized Trial
Price, A. M. H., Wake, M., Ukoumunne, O. C., & Hiscock, H. (2012) · Pediatrics · doi:10.1542/peds.2011-3467 · Cited by 146
Sample size considerations and predictive performance of multinomial logistic prediction models
de Jong, V. M. T., Eijkemans, M. J. C., CALSTER, B. V., Timmerman, D., Moons, K. G. M., Steyerberg, E. W., & van Smeden, M. (2019) · Statistics in Medicine · doi:10.1002/sim.8063 · Cited by 186
Khalifeh, A., Kaouk, J. H., Bhayani, S. B., ROGERS, C. G., STIFELMAN, M. D., Tanagho, Y. S., Kumar, R. K., Gorin, M. A., Sivarajan, G., Samarasekera, D., et al. (2013) · The Journal of Urology · doi:10.1016/j.juro.2013.05.110 · Cited by 146
Smith, R. A., Ghaneh, P., Sutton, R., Raraty, M. G. T., Campbell, F., & Neoptolemos, J. P. (2008) · Journal of Gastrointestinal Surgery · doi:10.1007/s11605-008-0554-3 · Cited by 142
“157) and the limited number of deaths that were seen (24 out of 52) in the patient group analyzed.(24) 0 .”
BackgroundScheffer, H. J., Vroomen, L. G. P. H., de Jong, M. C., Melenhorst, M. C. A. M., Zonderhuis, B. M., Daams, F. D., Vogel, J. A., Besselink, M. G., van Kuijk, C., Nierop, J. E. W.-V., et al. (2016) · Radiology · doi:10.1148/radiol.2016152835 · Cited by 165
Choi, S., Kim, S. H., Park, C. K., Min, J. H., Lee, J. E., Choi, Y., & Lee, B. (2017) · Radiology · doi:10.1148/radiol.2017162846 · Cited by 144
Giacobbe, D. R., Battaglini, D., Enrile, E. M., Dentone, C., Vena, A., Robba, C., Ball, L., Bartoletti, M., Coloretti, I., Bella, S. D., et al. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10040555 · Cited by 157
“Fourth, there was a possible inclusion of too many variables in our multivariable models, although we ultimately preferred not to remove potential explanatory variables on the basis of stepwise selection in view of the purely exploratory nature of our analysis [31,32].”
MethodEarly clinical predictors of 5-year outcome after concussive blast traumatic brain injury
Mac Donald, C. L., Barber, J., Jordan, M., Johnson, A. M., Dikmen, S., Fann, J. R., & Temkin, N. (2017) · JAMA Neurology · doi:10.1001/jamaneurol.2017.0143 · Cited by 142
Chelu, M. G., King, J. B., Kholmovski, E. G., Ma, J., Gal, P., Marashly, Q., Aljuaid, M., Kaur, G., Silver, M. A., Johnson, K. A., et al. (2018) · Journal of the American Heart Association · doi:10.1161/jaha.117.006313 · Cited by 147
“However, this number may be overly conservative, and may be relaxed to 5 to 9 outcomes, particularly to demonstrate adequate control of confounding.(13) Because model 4 includes <10 outcomes per predictor variable, the results of this model should be interpreted with caution and compared with the results from themore…”
Diabetic foot ulcer incidence in relation to plantar pressure magnitude and measurement location
Ledoux, W. R., Shofer, J. B., Cowley, M. S., Ahroni, J. H., Cohen, V., & Boyko, E. J. (2013) · Journal of Diabetes and its Complications · doi:10.1016/j.jdiacomp.2013.07.004 · Cited by 141
“This rule of thumb, though, has been found to be too conservative in more recent simulation studies (Vittinghoff & McCulloch, 2007).”
ResultFritzsching, B., Fellenberg, J., Moskovszky, L., Sápi, Z., Krenacs, T., Machado, I., Poeschl, J., Lehner, B., Szendrõi, M., Bosch, A. L., et al. (2015) · OncoImmunology · doi:10.4161/2162402x.2014.990800 · Cited by 140
Halabi, Y., Xu, H., Long, D., Chen, Y., Yu, Z., Alhaek, F., & Alhaddad, W. (2021) · Safety Science · doi:10.1016/j.ssci.2021.105537 · Cited by 171
Neufeld, S. A. S., Dunn, V. J., Jones, P. B., Croudace, T. J., & Goodyer, I. M. (2017) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30002-0 · Cited by 139
Estimate of hepatocellular carcinoma incidence in patients with alcoholic cirrhosis
Ganne-Carrié, N., Chaffaut, C., Bourcier, V., Archambeaud, I., Perarnau, J.-M., Oberti, F., Roulot, D., Moreno, C., Louvet, A., Dao, T., et al. (2018) · Journal of Hepatology · doi:10.1016/j.jhep.2018.07.022 · Cited by 137
Executive Function and Depression as Independent Risk Factors for Postoperative Delirium
Smith, P. J., Attix, D. K., Weldon, B. C., Greene, N. H., & Monk, T. G. (2009) · Anesthesiology · doi:10.1097/aln.0b013e31819b5bc2 · Cited by 137
“and recent simulation studies have demonstrated that this reduced number of events per variable provides adequate model estimates in some circumstances.(51) In summary, poorer executive function and greater levels of depressive symptoms were associated with a greater incidence of postoperative delirium after major non…”
Nix, J., & Pickett, J. T. (2017) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2017.05.016 · Cited by 171
“None of the officers in our sample felt that the 7 Although there is no consensus on the minimum number of events required per predictor variable in logistic models, the most common recommendation is ten events per variable (see Vittinghoff & McCulloch, 2006).”
BackgroundCombined FET PET/MRI radiomics differentiates radiation injury from recurrent brain metastasis
Lohmann, P., Köcher, M., Ceccon, G. S., Bauer, E. K., Stoffels, G., Viswanathan, S., Ruge, M. I., Neumaier, B., Shah, N. J., Fink, G. R., et al. (2018) · NeuroImage Clinical · doi:10.1016/j.nicl.2018.08.024 · Cited by 137
Statistical analysis of latency outcomes in behavioral experiments
Jahn-Eimermacher, A., Lasarzik, I., & Raber, J. (2011) · Behavioural Brain Research · doi:10.1016/j.bbr.2011.03.007 · Cited by 135
“For single trial experiments Vittinghoff and McCulloch [7] demonstrated, that a minimum of five animals locating the target within the trial time per factor of the Cox model is sufficient to limit the risk of severely invalid results.”
BackgroundJeppesen, K., Coyle, J. D., & Miser, W. F. (2009) · The Annals of Family Medicine · doi:10.1370/afm.919 · Cited by 156
Morris, M. R., Zolyomi, A., Yao, C., Bahram, S., Bigham, J. P., & Kane, S. K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858116 · Cited by 131
“A conservative rule of thumb to avoid overfitting is to have at least ten examples per factor [29], which we do.”
BackgroundEstimating logit models with small samples
Rainey, C., & McCaskey, K. (2021) · Political Science Research and Methods · doi:10.1017/psrm.2021.9 · Cited by 151
Legemaat, M., Carr, P. J., van Rens, R. M., van Dijk, M., Poslawsky, I. E., & van den Hoogen, A. (2016) · The Journal of Vascular Access · doi:10.5301/jva.5000558 · Cited by 132
“We calculated that a sample of at least 200 PIVCs was clinically pragmatic and sufficient to provide a heterogeneous sample that contains all important factors that contribute to complications and is large enough to deal with potential biases (20).”
Watkins, D. J., Sánchez, B. N., Téllez‐Rojo, M. M., Lee, J. M., Mercado‐García, A., Blank-Goldenberg, C., Peterson, K. E., & Meeker, J. D. (2017) · Environmental Research · doi:10.1016/j.envres.2017.07.051 · Cited by 147
“…estimates for slopes were unstable (infinite 95% confidence intervals), possibly due to the small number of girls who had undergone menarche or had Tanner stages> 1, which limited the number of covariates we could reliably enter into models (Peduzzi et al., 1996; Vittinghoff and McCulloch, 2007).”
BackgroundOgawa, E., Furusyo, N., Kajiwara, E., Takahashi, K., Nomura, H., Maruyama, T., Tanabe, Y., Satoh, T., Nakamuta, M., Kotoh, K., et al. (2012) · Journal of Hepatology · doi:10.1016/j.jhep.2012.10.017 · Cited by 135
“Thus, our sample size and 5 to 9 EPV might be sufficient to insure the robustness of our model.”
BackgroundInfluentialDew, M. A., DiMartini, A. F., DeVito Dabbs, A. J., Fox, K. R., Myaskovsky, L., Posluszny, D. M., Switzer, G. E., Zomak, R. A., Kormos, R. L., & Toyoda, Y. (2012) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2011.11.009 · Cited by 130
“Thus, it has been recommended that when the cases to predictors ratio are small (here, with nine predictors in a model, the ratio is under 5 for each of these disorders), the Cox models should be reestimated, with weaker predictors excluded in order to determine the stability of any significant results [65,66].”
BackgroundFood Insecurity and Academic Disruption Among College Students
Phillips, E. L., McDaniel, A. E., & Croft, A. (2018) · NASPA Journal · doi:10.1080/19496591.2018.1470003 · Cited by 167
“Agresti (2007) recommended having 10 events per independent variable, though Vittinghoff and McCulloch (2007) found through a simulation study that it is possible to relax the 10 event assumption for logistic regression.”
BackgroundIndredavik, M. S., Vik, T., Evensen, K. A. I., Skranes, J., Taraldsen, G., & Brubakk, A.-M. (2010) · Journal of Developmental & Behavioral Pediatrics · doi:10.1097/dbp.0b013e3181d7b1d3 · Cited by 129
Lee, H., Park, J., Yoon, Y. E., Park, E., Kim, H., Lee, W., Kim, Y.-J., Cho, G., Sohn, D., Greiser, A., et al. (2017) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2017.09.005 · Cited by 139
“To avoid overfitting (21), we included only EuroSCORE II, prior use of diuretics, the presence or extent of LGE, and native T1 in the multivariate analysis.”
MethodCarus, A., Ladekarl, M. B., Hager, H., Nedergaard, B. S., & Donskov, F. N. (2013) · British Journal of Cancer · doi:10.1038/bjc.2013.167 · Cited by 149
“On the basis of the number of patients in the cohort and the number of events we included a maximum of five variables in the multivariate analysis (Vittinghoff and McCulloch, 2007): peritumoral CD66bþ neutrophils above median, peritumoral CD163þ macrophages above median, peritumoral CD8þ lymphocytes below median, peri…”
MethodOzawa, M., Ohara, T., Ninomiya, T., Hata, J., Yoshida, D., Mukai, N., Nagata, M., Uchida, K., Shirota, T., Kitazono, T., et al. (2014) · Journal of the American Geriatrics Society · doi:10.1111/jgs.12887 · Cited by 127
“In the multivariable-adjusted model, 15 covariates known to be potential risk or protective factors for dementia were selected: age; sex; low education; history of stroke; hypertension; diabetes mellitus; total cholesterol; body mass index; smoking habits; regular exercise; and energy, vegetable, fruit, fish, and meat…”
MethodAlos, N., Grant, R. M., Ramsay, T., Halton, J., Cummings, E. A., Miettunen, P., Abish, S. B., Atkinson, S. A., Barr, R. D., Cabral, D. A., et al. (2012) · Journal of Clinical Oncology · doi:10.1200/jco.2011.40.4830 · Cited by 141
Treurniet, K. M., Berkhemer, O. A., Immink, R. V., Lingsma, H. F., der Stam, V. M. C. W.-V., Hollmann, M. W., Vuyk, J., van Zwam, W. H., van der Lugt, A., van Oostenbrugge, R. J., et al. (2017) · Journal of NeuroInterventional Surgery · doi:10.1136/neurintsurg-2017-012988 · Cited by 127
Mouratidis, K. (2022) · Sustainable Cities and Society · doi:10.1016/j.scs.2022.104161 · Cited by 150
Mortality prediction models in the general trauma population: A systematic review
de Munter, L., Polinder, S., Lansink, K. W. W., Cnossen, M. C., Steyerberg, E. W., & de Jongh, M. A. C. (2016) · Injury · doi:10.1016/j.injury.2016.12.009 · Cited by 141
Six weeks of antibiotic treatment is sufficient following surgery for septic arthroplasty
Bernard, L., Legout, L., Zürcher, L., STERN, R. E., Rohner, P., Peter, R. E., Assal, M., Lew, D., Hoffmeyer, P. J., & Uçkay, İ. (2010) · Journal of Infection · doi:10.1016/j.jinf.2010.05.005 · Cited by 137
Measures of Executive Function and Depression Identify Patients at Risk for Postoperative Delirium
Greene, N. H., Attix, D. K., Weldon, B. C., Smith, P. J., McDonagh, D. L., & Monk, T. G. (2009) · Anesthesiology · doi:10.1097/aln.0b013e31819b5ba6 · Cited by 142
Bachoura, A., Guitton, T. G., Smith, M. R., Vrahas, M. S., Zurakowski, D., & Ring, D. (2010) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-010-1737-2 · Cited by 124
“2%), 13 variables were tested by multivariable analysis [18] to achieve a prediction algorithm with reliable odds ratios and confidence intervals for measuring infection risk [44].”
MethodShah, A. J. (2011) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.125 · Cited by 124
“We used the FRS in place of the individual risk factors (age, total and high-density lipoprotein cholesterol levels, diabetes, systolic/diastolic blood pressure, and smoking) to minimize statistical bias due to the small number of IHD deaths.(22) As a test of robustness of this substitution, analyses that adjusted for…”
Carr, P. J., Rippey, J. C. R., Budgeon, C. A., Cooke, M. L., Higgins, N., & Rickard, C. M. (2016) · The Journal of Vascular Access · doi:10.5301/jva.5000487 · Cited by 122
“Peduzzi et al provide a “rule of thumb” recommendation of ten events per variable analysed (42) and there others who recommend that the number required could be reduced (43).”
BackgroundWildes, T. M., Ruwe, A. P., Fournier, C., Gao, F., Carson, K. R., Piccirillo, J. F., Tan, B. R., & Colditz, G. A. (2013) · Journal of Geriatric Oncology · doi:10.1016/j.jgo.2013.02.002 · Cited by 131
“(26) However, recent simulation studies suggest that the dogmatic “rule of 10s”, that is, that there must be 10 events per predictor variable in multivariate analysis, may be relaxed, with 5–9 events per predictor variable performing similar to 10 or more events per predictor variable in the model.(27)”
BackgroundRisk factors for acute knee injury in female youth football
Hägglund, M., & Waldén, M. (2015) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-015-3922-z · Cited by 154
“As a rule of thumb, 10 outcome events per predictor variable (EPV) is often recommended for Cox regression, although it has also been suggested that models with 5–9 EPV could be sufficient [40].”
MethodBuchner, A. M., Shahid, M. W., Heckman, M. G., McNeil, R. B., Cleveland, P. W., Gill, K. R., Schore, A. N., Ghabril, M., Raimondo, M., Gross, S. A., et al. (2009) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2009.11.009 · Cited by 121
Obradović, A., Dallos, M. C., ZAHURAK, M. L., Partin, A. W., Schaeffer, E. M., Ross, A. E., Allaf, M. E., Nirschl, T. R., Liu, D., Chapman, C. G., et al. (2020) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-19-3372 · Cited by 128
Polzer, I., Schwahn, C., Völzke, H., Mündt, T., & Biffar, R. (2011) · Clinical Oral Investigations · doi:10.1007/s00784-011-0625-9 · Cited by 135
Cochet, H., Dubois, R., Yamashita, S., Al-Jefairi, N., Berte, B., Sellal, J., Hooks, D. A., Frontera, A., Amraoui, S., Zemoura, A., et al. (2017) · JACC. Clinical electrophysiology · doi:10.1016/j.jacep.2017.07.019 · Cited by 127
“Assuming an expected response rate of about two-thirds on the basis of a prior publication (13), the study was populated to enable the inclusion of 2 variables in predictive models (27).”
Movrin, I., Vengust, R., & Komadina, R. (2010) · Archives of Orthopaedic and Trauma Surgery · doi:10.1007/s00402-010-1106-3 · Cited by 118
“events per variables in the logistic regression model, EPV) possible predictors were analyzed separately in the logistic regression model [ 41 ].”
MethodThe BRAAFF checklist: a new dermoscopic algorithm for diagnosing acral melanoma
Lallas, A., Kyrgidis, Α., Koga, H., Moscarella, E., Tschandl, P., Apalla, Z., Stefani, A. D., Ioannides, D., Kittler, H., Kobayashi, K., et al. (2015) · British Journal of Dermatology · doi:10.1111/bjd.14045 · Cited by 134
“For that, we included in the multivariate analyses more variables than recommended.(30) To overcome this limitation the proposed BRAAFF checklist requires external validation in future studies.”
MethodBurns, K. E. A., Zubrinich, C., Tan, W., Raptis, S., Xiong, W., Smith, O., McDonald, E., Marshall, J. C., Saginur, R., Heslegrave, R., et al. (2013) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201208-1537oc · Cited by 115
Bieniek, K. F., Blessing, M. M., Heckman, M. G., Diehl, N. N., Serie, A. M., Paolini, M. A., Boeve, B. F., Savica, R., Reichard, R. R., & Dickson, D. W. (2020) · Brain Pathology · doi:10.1111/bpa.12757 · Cited by 115
“Considering commonly used guidelines regarding the number of variables that can be included in a given model (25, 32), multivariable models for the primary outcome of combined CTE pathology were adjusted for the subset of pre-specified potential CTE risk factors (age at death and documented head injury subsequent to a…”
MethodAn Evaluation of Risk Factors for Major Adverse Cardiovascular Events During Tocilizumab Therapy
Rao, V. U., Pavlov, A., Klearman, M., Musselman, D., Giles, J. T., Bathon, J. M., Sattar, N., & Lee, J. S. (2015) · Arthritis & Rheumatology · doi:10.1002/art.38920 · Cited by 113
Dunkel, N., Pittet, D., Tovmirzaeva, L., Suvà, D., Bernard, L., Lew, D., Hoffmeyer, P. J., & Uçkay, İ. (2013) · The Bone & Joint Journal · doi:10.1302/0301-620x.95b6.30114 · Cited by 113
Abdelaziz, R. B., Hafsi, H., Hajji, H., Boudabous, H., Chehida, A. B., Mrabet, A., Boussetta, K., Barsaoui, S., Sammoud, A., Hamzaoui, M., et al. (2017) · BMC Pediatrics · doi:10.1186/s12887-017-0965-y · Cited by 114
“enough to overcome potential biases [27].”
Trainee participation is associated with increased small adenoma detection
Buchner, A. M., Shahid, M. W., Heckman, M. G., Diehl, N. N., McNeil, R. B., Cleveland, P. W., Gill, K. R., Schore, A. N., Ghabril, M., Raimondo, M., et al. (2011) · Gastrointestinal Endoscopy · doi:10.1016/j.gie.2011.01.060 · Cited by 116
Evolution and early government responses to COVID-19 in South America
González‐Bustamante, B. (2020) · World Development · doi:10.1016/j.worlddev.2020.105180 · Cited by 121
A Twin Study of the Association Between PTSD Symptoms and Rheumatoid Arthritis
Boscarino, J. A., Forsberg, C. W., & Goldberg, J. (2010) · Psychosomatic Medicine · doi:10.1097/psy.0b013e3181d9a80c · Cited by 113
“We do not present data stratified by zygosity, because the number of RA cases was generally considered too small to obtain reliable statistical estimates separately for MZ and DZ pairs given the number of predictors used (30).”
MethodCoarctation of the Aorta and Coronary Artery Disease: Fact or Fiction?
Roifman, I., Therrien, J., Ionescu-Ittu, R., Pilote, L., Guo, L., Kotowycz, M. A., Martucci, G., & Marelli, A. J. (2012) · Circulation · doi:10.1161/circulationaha.111.088294 · Cited by 112
Mental health services for parents who lost a child to cancer: if we build them, Will they come?
Lichtenthal, W. G., Corner, G. W., Sweeney, C. R., Wiener, L., Roberts, K. E., Baser, R. E., Li, Y., Breitbart, W., Kissane, D. W., & Prigerson, H. G. (2015) · Journal of Clinical Oncology · doi:10.1200/jco.2014.59.0406 · Cited by 111
Kristensen, T. S., Kofoed, K. F., Kühl, J. T., Nielsen, W. B., Nielsen, M. B., & Kelbæk, H. (2011) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2011.01.058 · Cited by 116
“However, this study should be considered a proof-ofconcept study, and the a priori hypothesized predictor was highly significant, allowing a less conservative strategy for inclusion of established clinical covariates in the multivariate model (32).”
MethodGendron, T. F., Daughrity, L. M., Heckman, M. G., Diehl, N. N., Wuu, J., Miller, T. M., Pastor, P., Trojanowski, J. Q., Grossman, M., Berry, J. D., et al. (2017) · Annals of Neurology · doi:10.1002/ana.24980 · Cited by 112
“carriers, we adjusted only for age at disease onset and onset site due to the smaller number of deaths in this subgroup.(26) We addi-”
BackgroundDetermining behavioural differences of Y and Z generational cohorts in online shopping
Agrawal, D. K. (2022) · International Journal of Retail & Distribution Management · doi:10.1108/ijrdm-12-2020-0527 · Cited by 172
Positioning Injuries Associated with Robotic Assisted Urological Surgery
Mills, J. T., Burris, M. B., Warburton, D. J., Conaway, M. R., Schenkman, N. S., & Krupski, T. L. (2013) · The Journal of Urology · doi:10.1016/j.juro.2013.02.3185 · Cited by 113
Stiell, I. G., Clement, C. M., Brison, R. J., Rowe, B. H., Borgundvaag, B., Aaron, S. D., Lang, E., Calder, L. A., Perry, J. J., Forster, A. J., et al. (2013) · Academic Emergency Medicine · doi:10.1111/acem.12056 · Cited by 127
“While we had relatively few outcomes relative to the number of predictors, internal validation with bootstrapping demonstrated that the estimate and classification performance for risk scores were stable.(52) We chose not to adjust the analysis despite the fact that some patients were enrolled more than once.”
ResultReiseter, S., Gunnarsson, R., Aaløkken, T. M., Lund, M. B., Mynarek, G. K., Corander, J., Haydon, J., & Molberg, Ø. (2017) · Lara D. Veeken · doi:10.1093/rheumatology/kex077 · Cited by 109
An evaluation of penalised survival methods for developing prognostic models with rare events
Ambler, G., Seaman, S., & Omar, R. Z. (2012) · Statistics in Medicine · doi:10.1002/sim.4371 · Cited by 107
“If the model involves estimation of a large number of parameters relative to the number of events, there is a risk of the model being over-fitted [5, 6].”
MethodXhaard, A., Rocha, V., Bueno, B., de Latour, R. P., Lenglet, J., Petropoulou, A. D., Rodríguez‐Otero, P., Ribaud, P., Porcher, R., Socié, G., et al. (2011) · Transplantation and Cellular Therapy · doi:10.1016/j.bbmt.2011.06.012 · Cited by 107
“The variable selection for consideration in the model was used to avoid too many variables per event, although we chose not to conform to the empirical ‘‘rule of thumb’’ [21].”
MethodGoode, E. C., Clark, A. B., Mells, G. F., Srivastava, B., Spiess, K., Gelson, W. T. H., Trivedi, P. J., Lynch, K. D., Castren, E., Vesterhus, M. N., et al. (2019) · Hepatology · doi:10.1002/hep.30479 · Cited by 106
“Time-to-event analysis was conducted using Cox’s proportional hazards model, ensuring at least 10 events per risk factor would be included in the model.(26) To facilitate accurate risk prediction, events were truncated at 10 years of follow-up.”
MethodJeffery, N. D., Barker, A. K., Hu, H. Z., Alcott, C. J., Kraus, K. H., Scanlin, E. M., Granger, N. P., & Levine, J. M. (2016) · Journal of the American Veterinary Medical Association · doi:10.2460/javma.248.4.386 · Cited by 122
Observational Studies: Matching or Regression?
Brazauskas, R., & Logan, B. R. (2015) · Transplantation and Cellular Therapy · doi:10.1016/j.bbmt.2015.12.005 · Cited by 107
Patterns and predictors of first and subsequent recurrence in women with early breast cancer
Geurts, Y. M. G., Witteveen, A., Bretveld, R. W., Poortmans, P. M., Sonke, G. S., Strobbe, L. J. A., & Siesling, S. (2017) · Breast Cancer Research and Treatment · doi:10.1007/s10549-017-4340-3 · Cited by 110
“However, there are others suggesting that this rule is too conservative [32].”
BackgroundPredictors of laminitis development in a cohort of nonlaminitic ponies
Knowles, E. J., Elliott, J., Harris, P. A., Chang, Y., & Menzies‐Gow, N. J. (2022) · Equine Veterinary Journal · doi:10.1111/evj.13572 · Cited by 104
Factors predictive of ten-year mortality in severe anorexia nervosa patients
Huas, C., Caille, A., Godart, N., Foulon, C., Pham-Scottez, A., Divac, S., Dechartres, A., Lavoisy, G., Guelfi, J. D., Rouillon, F., et al. (2011) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2010.01627.x · Cited by 103
“As the lifetime predictor model presented a rather large number of covariables (n = 9) given the number of events (death, n = 40), results were validated by bootstrap procedure (26).”
MethodLai, T. Y. Y., Luk, F., Lee, G. K. Y., & Lam, D. S. C. (2012) · Eye · doi:10.1038/eye.2012.97 · Cited by 103
Leruez‐Ville, M., Sellier, Y., Salomon, L. J., Stirnemann, J., Jacquemard, F., & Ville, Y. G. (2013) · Clinical Infectious Diseases · doi:10.1093/cid/cit059 · Cited by 102
“To avoid overfitting, we aimed at obtaining multivariate models with at least 1 explanatory variable per 8–10 events, as suggested by the study of Vittinghoff et al [18].”
MethodGu, J., Remzi, F. H., Shen, B., Vogel, J. D., & Kiran, R. P. (2013) · Diseases of the Colon & Rectum · doi:10.1097/dcr.0b013e3182a0e702 · Cited by 101
“23 Pads usage (night) 10 (30) 21 (27) 0.”
Tateuchi, H., Koyama, Y., Akiyama, H., Goto, K., So, K., Kuroda, Y., & Ichihashi, N. (2017) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2017.02.796 · Cited by 105
Late Cardiovascular Complications after Hematopoietic Cell Transplantation
Chow, E. J., Wong, K., Lee, S. J., Cushing‐Haugen, K. L., Flowers, M. E. D., Friedman, D. L., Leisenring, W. M., Martin, P. J., Mueller, B. A., & Baker, K. S. (2014) · Transplantation and Cellular Therapy · doi:10.1016/j.bbmt.2014.02.012 · Cited by 103
“Given the possibility of introducing bias and losing precision with overly adjusted models, we also examined simpler models with at least 5 events per covariate, with overall similar results [23].”
Hombach, V., Merkle, N., Torzewski, J., Kraus, J. M., Kunze, M., Zimmermann, O., Kestler, H. A., & Wöhrle, J. (2009) · European Heart Journal · doi:10.1093/eurheartj/ehp293 · Cited by 109
Statistical review: frequently given comments
Lydersen, S. (2014) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2014-206186 · Cited by 118
Schoenenberger, A. W., Moser, A., Bertschi, D., Wenaweser, P., Windecker, S., Carrel, T. P., Stuck, A. E., & Stortecky, S. (2018) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2017.11.012 · Cited by 110
Spirituality, Social Support, and Survival in Hemodialysis Patients
Spinale, J. M., Cohen, S. D., Khetpal, P., Peterson, R. A., Clougherty, B., Puchalski, C. M., Patel, S. S., & Kimmel, P. L. (2008) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.01790408 · Cited by 117
“The relatively small sample size of this study and the number of events preclude the use of a large number of standard covariates in Cox regression analyses (46).”
BackgroundIba, K., Shinozaki, T., Maruo, K., & Noma, H. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01201-w · Cited by 124
Influence of midsole hardness of standard cushioned shoes on running-related injury risk
Theisen, D., Malisoux, L., Genin, J., Delattre, N., Seil, R., & Urhausen, A. (2014) · British Journal of Sports Medicine · doi:10.1136/bjsports-2013-092613 · Cited by 75
Borz, B., Durand, É., Godin, M., Tron, C., Canville, A., Litzler, P., Bessou, J., Cribier, A. G., & Eltchaninoff, H. (2012) · Heart · doi:10.1136/heartjnl-2012-303095 · Cited by 97
Surgical site infection after open reduction and internal fixation of tibial plateau fractures
Lin, S., Mauffrey, C. P., Hammerberg, E. M., Stahel, P. F., & Hak, D. J. (2013) · European Journal of Orthopaedic Surgery & Traumatology · doi:10.1007/s00590-013-1252-8 · Cited by 97
Vandenput, I., CALSTER, B. V., Capoen, A., Leunen, K., Berteloot, P., Neven, P., Moerman, P., Vergote, I. B., & Amant, F. C. (2009) · British Journal of Cancer · doi:10.1038/sj.bjc.6605157 · Cited by 97
“The limited sample size and number of events prohibited the investigation of more covariates (Vittinghoff and McCulloch, 2007).”
BackgroundFallah, A., Rodgers, S. D., Weil, A. G., Vadera, S., Mansouri, A., Connolly, M. B., Major, P., Ma, T., Devinsky, O., Weiner, H. L., et al. (2015) · Neurosurgery · doi:10.1227/neu.0000000000000875 · Cited by 97
“This equates to 10 events per variable (for 4 variables), which is the rule of thumb for a multivariate regression model.(18,19) Using an a value of .”
BackgroundEdiebah, D. E., Coens, C., Zikos, E., Quinten, C., Ringash, J. G., King, M. T., Koch, J. S.-V., Gotay, C. C., Greimel, E., Flechtner, H., et al. (2014) · British Journal of Cancer · doi:10.1038/bjc.2014.208 · Cited by 96
“Given the rule of thumb that CPHM should be used with a minimum of 10 or less outcome events per predictor (Concato et al, 1995; Peduzzi et al, 1995, 1996; Vittinghoff and McCulloch, 2007), this allows for an adequately robust analysis.”
MethodDuwe, G. (2012) · Sexual Abuse · doi:10.1177/1079063212453942 · Cited by 137
“It is important to emphasize, however, that even in models that had EPV values less than five, Vittinghoff and McCulloch (2007) found that most were not unduly affected by these problems.”
BackgroundMethodMissing, unreplaced teeth and risk of all-cause and cardiovascular mortality
Schwahn, C., Polzer, I., Haring, R., Dörr, M., Wallaschofski, H., Kocher, T., Mündt, T., Holtfreter, B., Samietz, S., Völzke, H., et al. (2012) · International Journal of Cardiology · doi:10.1016/j.ijcard.2012.04.061 · Cited by 102
“5 [36,44].”
BackgroundLohmann, P., Elahmadawy, M. A., Gutsche, R., Werner, J., Bauer, E. K., Ceccon, G. S., Köcher, M., Lerche, C. W., Rapp, M., Fink, G. R., et al. (2020) · Cancers · doi:10.3390/cancers12123835 · Cited by 108
“To further reduce the risk of overfitting, 5-fold cross-validation was performed during model generation, and the maximum number of features for a model generation was restricted to 4, according to published recommendations [64,65].”
MethodFactors affecting site occupancy by woodland bird species of conservation concern
Montague‐Drake, R., Lindenmayer, D. B., & Cunningham, R. B. (2009) · Biological Conservation · doi:10.1016/j.biocon.2009.07.009 · Cited by 102
Metadata
- Authors
- Vittinghoff, E., & McCulloch, C. E.
- Affiliations
- University of California, San Francisco (US)
- Venue
- American Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2007-01-12
- Volume
- 165
- Issue
- 6
- Pages
- 710-718
- ISSN
- 0002-9262 (ISSN-L)
- 1476-6256
- OpenAlex
- W2130373985
- Cited by
- 4,359
- References
- 9
- Citations by year
- 2026: 642 · 2025: 394 · 2024: 313 · 2023: 315 · 2022: 306 · 2021: 351 · 2020: 332 · 2019: 281 · 2018: 282 · 2017: 205 · 2016: 194 · 2015: 171 · 2014: 146 · 2013: 132 · 2012: 94
- FWCI
- 20.12
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- sample size, logistic regression, Cox proportional hazards model, confidence interval coverage, type I error, relative bias, simulation study, sensitivity analysis, confounding adjustment, small sample size
- MeSH terms
- Data Interpretation, Statistical, Factor Analysis, Statistical, Humans, Predictive Value of Tests, Sensitivity and Specificity, Bias, Confounding Factors, Epidemiologic, Prevalence, Multivariate Analysis, Confidence Intervals, Logistic Models, Proportional Hazards Models, Case-Control Studies, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.