A critical look at methods for handling missing covariates in epidemiologic regression analyses.
Abstract
Epidemiologic studies often encounter missing covariate values. While simple methods such as stratification on missing-data status, conditional-mean imputation, and complete-subject analysis are commonly employed for handling this problem, several studies have shown that these methods can be biased under reasonable circumstances. The authors review these results in the context of logistic regression and present simulation experiments showing the limitations of the methods. The method based on missing-data indicators can exhibit severe bias even when the data are missing completely at random, and regression (conditional-mean) imputation can be inordinately sensitive to model misspecification. Even complete-subject analysis can outperform these methods. More sophisticated methods, such as maximum likelihood, multiple imputation, and weighted estimating equations, have been given extensive attention in the statistics literature. While these methods are superior to simple methods, they are not commonly used in epidemiology, no doubt due to their complexity and the lack of packaged software to apply these methods. The authors contrast the results of multiple imputation to simple methods in the analysis of a case-control study of endometrial cancer, and they find a meaningful difference in results for age at menarche. In general, the authors recommend that epidemiologists avoid using the missing-indicator method and use more sophisticated methods whenever a large proportion of data are missing.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Commonly missing values are related to observed participant or disease characteristics.
“Commonly missing values are related to observed participant or disease characteristics.”
From Reporting and Methods in Clinical Prediction Research: A Systematic Review (2012) · cites this paper as [24]
Missing values in clinical study data rarely occur completely at random.
“Missing values in clinical study data rarely occur completely at random.”
From Reporting and Methods in Clinical Prediction Research: A Systematic Review (2012) · cites this paper as [24]
Exclusion of participants with missing values often leads to loss of statistical power and biased results.
“Exclusion of participants with missing values will therefore not only lead to loss of statistical power, but often also to biased results 10,12,13,23,24,57,58.”
From Reporting and Methods in Clinical Prediction Research: A Systematic Review (2012) · cites this paper as [24]
Cited by
Sorted by citations
Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence
Singer, J. D., & Willett, J. B. (2003) · doi:10.1093/acprof:oso/9780195152968.001.0001 · Cited by 10,647 · see citing works
Matching Methods for Causal Inference: A Review and a Look Forward
Stuart, E. A. (2010) · Statistical Science · doi:10.1214/09-sts313 · Cited by 5,478 · see citing works
Multiple imputation by chained equations: what is it and how does it work?
Azur, M. J., Stuart, E. A., Frangakis, C., & Leaf, P. J. (2011) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.329 · Cited by 3,542 · see citing works
“In contrast, if the observed data are highly predictive of the missing values the imputations will be more consistent across imputations, resulting in smaller, but still accurate, standard errors (Greenland and Finkle, 1995).”
ResultMultiple imputation after 18+ years
Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476908 · Cited by 4,072 · see citing works
Multiple imputation of discrete and continuous data by fully conditional specification
van Buuren, S. (2007) · Statistical Methods in Medical Research · doi:10.1177/0962280206074463 · Cited by 2,827 · see citing works
Multiple Imputation After 18+ Years
Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.2307/2291635 · Cited by 2,686 · see citing works
Review: a gentle introduction to imputation of missing values.
Donders, A. R. T., van der Heijden, G. J. M. G., Stijnen, T., & Moons, K. G. M. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.014 · Cited by 2,582 · see citing works
How can I deal with missing data in my study?
Bennett, D. A. (2001) · Australian and New Zealand Journal of Public Health · doi:10.1111/j.1467-842x.2001.tb00294.x · Cited by 1,656 · see citing works
A Multidimensional Index and Staging System for Idiopathic Pulmonary Fibrosis
Ley, B., Ryerson, C. J., Vittinghoff, E., Ryu, J. H., Tomassetti, S., Lee, J. S., Poletti, V., Buccioli, M., Elicker, B. M., Jones, K. D., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-156-10-201205150-00004 · Cited by 1,308 · see citing works
Hashibe, M., Brennan, P., Chuang, S., Boccia, S., Castellsagué, X., Chen, C., Curado, M. P., Maso, L. D., Daudt, A. W., Fabiánová, E., et al. (2009) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-08-0347 · Cited by 1,258 · see citing works
“We assumed that the education data were missing at random with respect to unmeasured covariates; whether education level was missing did not depend on any other unobserved or missing values (22).”
BackgroundApplied Longitudinal Data Analysis for Epidemiology: A Practical Guide
Twisk, J. W. R. (2003) · Cited by 1,244 · see citing works
Hashibe, M., Brennan, P., Benhamou, S., Castellsagué, X., Chen, C., Curado, M. P., Maso, L. D., Daudt, A. W., Fabiánová, E., Wunsch-Filho, V., et al. (2007) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djk179 · Cited by 1,097 · see citing works
“We assumed that the education data were missing at random, i.e., whether or not education level was missing did not depend on any other unobserved or missing values ( 35 ).”
BackgroundUsing the outcome for imputation of missing predictor values was preferred
Moons, K. G. M., Donders, R. A. R. T., Stijnen, T., & Harrell, F. E. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.009 · Cited by 994 · see citing works
Missing data and multiple imputation in clinical epidemiological research
Pedersen, A. B., Mikkelsen, E. M., Cronin-Fenton, D. P., Kristensen, N. R., Pham, T. M., Pedersen, L. H., & Petersen, I. (2017) · Clinical Epidemiology · doi:10.2147/clep.s129785 · Cited by 982 · see citing works
“Therefore, if the data are MCAR, the estimate of the mean remains unbiased.(4,12) Under MCAR, if our aim is to estimate means (which is rarely the main focus of research studies), mean imputation will not bias the estimates; it will only bias the standard errors as mentioned previously.”
InfluentialRegistries for Evaluating Patient Outcomes: A User’s Guide
(2020) · doi:10.23970/ahrqepcregistries4 · Cited by 968 · see citing works
Manjunath, G., Tighiouart, H., Ibrahim, H. N., MacLeod, B., Salem, D. N., Griffith, J. L., Coresh, J., Levey, A. S., & Sarnak, M. J. (2002) · Journal of the American College of Cardiology · doi:10.1016/s0735-1097(02)02663-3 · Cited by 916 · see citing works
“For de novo disease, because there were missing baseline data on cerebrovascular disease in 3,093 subjects, we performed analyses both by using the data available as well as using multiple imputation techniques for missing data (16).”
MethodSocioeconomic Disparities in Adverse Birth Outcomes
Blumenshine, P., Egerter, S. A., Barclay, C., Cubbin, C., & Braveman, P. (2010) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2010.05.012 · Cited by 853 · see citing works
Horton, N., & Kleinman, K. (2007) · American Statistician · doi:10.1198/000313007x172556 · Cited by 476 · see citing works
“These ad-hoc approaches have the potential to induce bias and are not recommended (Jones 1996; Greenland and Finkle 1995).”
BackgroundRecent trends in mortality and causes of death among persons with spinal cord injury
Vivo, M. J. D., Krause, J. S., & Lammertse, D. P. (1999) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/s0003-9993(99)90252-6 · Cited by 778 · see citing works
Schonewille, W. J., Wijman, C. A. C., Michel, P., Rueckert, C. M., Weimar, C., Mattle, H. P., Engelter, S. T., Tanné, D., Muir, K. W., Molina, C. A., et al. (2009) · The Lancet Neurology · doi:10.1016/s1474-4422(09)70173-5 · Cited by 778 · see citing works
Has Mortality from Acute Respiratory Distress Syndrome Decreased over Time?: A Systematic Review
Phua, J., Badia, J., Adhikari, N. K. J., Friedrich, J. O., Fowler, R., Singh, J. M., Scales, D. C., Stather, D. R., Li, A., Jones, A., et al. (2008) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200805-722oc · Cited by 744 · see citing works
External validation is necessary in prediction research:
Bleeker, S. E., Moll, H. A., Steyerberg, E. W., Donders, A. R. T., Derksen-Lubsen, G., Grobbee, D. E., & Moons, K. G. M. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00207-5 · Cited by 710 · see citing works
“Furthermore, we only applied single imputation instead of the statistically more appropriate multiple imputation technique [37].”
BackgroundMethodNearest neighbor imputation algorithms: a critical evaluation
Beretta, L., & Santaniello, A. (2016) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0318-z · Cited by 679 · see citing works
“Compared to the selection by Table 2 Correlation between expected and actual values of the dependent variable Y as calculated from equation [1], average of 500 samples of n = 400, k = 5, 15 % or 30 % of missing data”
BackgroundMethodWeight, Weight Gain, and Survival After Breast Cancer Diagnosis
Kroenke, C. H., Chen, W. Y., Rosner, B., & Holmes, M. D. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.01.079 · Cited by 670 · see citing works
van der Heijden, G. V. D., Donders, A., Stijnen, T., & Moons, K. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.015 · Cited by 664 · see citing works
Reporting and Methods in Clinical Prediction Research: A Systematic Review
Bouwmeester, W., Zuithoff, N. P. A., Mallett, S., Geerlings, M. I., Vergouwe, Y., Steyerberg, E. W., Altman, D. G., & Moons, K. G. M. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001221 · Cited by 600 · see citing works
“Exclusion of participants with missing values will therefore not only lead to loss of statistical power, but often also to biased results [10,12,13,23,24,57,58].”
MethodWood, A., White, I. R., & Thompson, S. G. (2004) · Clinical Trials · doi:10.1191/1740774504cn032oa · Cited by 546 · see citing works
“These methods may introduce bias in observational studies [36] but not in randomized controlled trials [2].”
MethodGroenwold, R. H. H., White, I. R., Donders, A. R. T., Carpenter, J. R., Altman, D. G., & Moons, K. G. M. (2012) · Canadian Medical Association Journal · doi:10.1503/cmaj.110977 · Cited by 538 · see citing works
The Effect of Multidisciplinary Care Teams on Intensive Care Unit Mortality
Kim, M. M., Barnato, A. E., Angus, D. C., Fleisher, L., & Kahn, J. M. (2010) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.521 · Cited by 531 · see citing works
Shapiro, N. I., Wolfe, R. E., Moore, R. B., Smith, E., Burdick, E., & Bates, D. W. (2003) · Critical Care Medicine · doi:10.1097/01.ccm.0000054867.01688.d1 · Cited by 494 · see citing works
“6% complete), these variables were dummy-coded and entered into the regression model as missing to control for the influence of missing data based on previously described methods (18).”
MethodUnderstanding infidelity: Correlates in a national random sample.
Atkins, D. C., Baucom, D. H., & Jacobson, N. S. (2001) · Journal of Family Psychology · doi:10.1037/0893-3200.15.4.735 · Cited by 458 · see citing works
Prevalence of Mammographically Dense Breasts in the United States
Sprague, B. L., Gangnon, R. E., Burt, V., Trentham‐Dietz, A., Hampton, J. M., Wellman, R., Kerlikowske, K. M., & Miglioretti, D. L. (2014) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/dju255 · Cited by 401 · see citing works
“duced by exclusion of exams with incomplete data (24).”
Dealing With Missing Outcome Data in Randomized Trials and Observational Studies
Groenwold, R. H. H., Donders, A. R. T., Roes, K. C. B., Harrell, F. E., & Moons, K. G. M. (2012) · American Journal of Epidemiology · doi:10.1093/aje/kwr302 · Cited by 398 · see citing works
Dizziness among Older Adults: A Possible Geriatric Syndrome
Tinetti, M. E., Williams, C. S., & Gill, T. M. (2000) · Annals of Internal Medicine · doi:10.7326/0003-4819-132-5-200003070-00002 · Cited by 389 · see citing works
Big Data in Organ Transplantation: Registries and Administrative Claims
Massie, A. B., Kuricka, L. M., & Segev, D. L. (2014) · American Journal of Transplantation · doi:10.1111/ajt.12777 · Cited by 387 · see citing works
“Strategies for dealing with missingness include (in order of robustness): case-wise deletion (complete-case analyses), missing indicator variables and imputation (42).”
MethodAdjusting for partially missing baseline measurements in randomized trials
White, I. R., & Thompson, S. G. (2005) · Statistics in Medicine · doi:10.1002/sim.1981 · Cited by 378 · see citing works
“For use in epidemiology with partly missing confounders, this method has been severely criticized because it does not completely adjust for confounding [30, 31].”
MethodUpdating methods improved the performance of a clinical prediction model in new patients
Janssen, K. J. M., Moons, K. G. M., Kalkman, C. J., Grobbee, D. E., & Vergouwe, Y. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.04.018 · Cited by 381 · see citing works
Mother-to-child transmission of hepatitis C virus: evidence for preventable peripartum transmission
Gibb, D. M., Goodall, R. L., Dunn, D., et al. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02681-7 · Cited by 375 · see citing works
Procedure-specific Risk Factor Analysis for the Development of Severe Postoperative Pain
Gerbershagen, H. J., Pogatzki-Zahn, E., Aduckathil, S., Peelen, L. M., Kappen, T. H., van Wijck, A. J. M., Kalkman, C. J., & Meissner, W. (2014) · Anesthesiology · doi:10.1097/aln.0000000000000108 · Cited by 372 · see citing works
Behavior and Development in Children and Age at the Time of First Anesthetic Exposure
Kalkman, C. J., Peelen, L. M., Moons, K. G. M., Veenhuizen, M., Bruens, M., Sinnema, G., & de Jong, T. P. V. M. (2009) · Anesthesiology · doi:10.1097/aln.0b013e31819c7124 · Cited by 346 · see citing works
Cowie, M., Michel Komajda, N., Murray‐Thomas, T., Underwood, J., & Ticho, B. (2006) · European Heart Journal · doi:10.1093/eurheartj/ehi859 · Cited by 335 · see citing works
Antibiotics for acute otitis media: a meta-analysis with individual patient data
Rovers, M. M., Glasziou, P., Appelman, C. L., Burke, P., McCormick, D. P., Damoiseaux, R. A. M. J., Gaboury, I., Little, P. S., & Hoes, A. W. (2006) · The Lancet · doi:10.1016/s0140-6736(06)69606-2 · Cited by 328 · see citing works
Obesity and Type 2 Diabetes Risk in Midadult Life: The Role of Childhood Adversity
Thomas, C., Hyppönen, E., & Power, C. (2008) · Pediatrics · doi:10.1542/peds.2007-2403 · Cited by 330 · see citing works
Time perspective, personality and smoking, body mass, and physical activity: An empirical study
Adams, J. M., & Nettle, D. (2008) · British Journal of Health Psychology · doi:10.1348/135910708x299664 · Cited by 313 · see citing works
“In ordert om ake best use of the data available, a'complete-subject' approachtoanalysis was taken (Greenland &Finkle, 1995; Rothman &Greenland, 1998).”
MethodHoffman, S., & HATCH, M. C. (2000) · Health Psychology · doi:10.1037/0278-6133.19.6.535 · Cited by 316 · see citing works
Rutten, F. H. (2010) · Archives of Internal Medicine · doi:10.1001/archinternmed.2010.112 · Cited by 307 · see citing works
“As the exclusion of patients with a missing value (so-called complete case analysis) may lead to biased results and loss of power, we imputed missing values for smoking status by using a regression method with the addition of a random error term.(24) Imputation was based on the correlation between this missing variabl…”
MethodIncluding all individuals is not enough: lessons for intention-to-treat analysis
White, I., Carpenter, J., & Horton, N. (2012) · Clinical Trials · doi:10.1177/1740774512450098 · Cited by 299 · see citing works
“Thus, mean imputation methods, which are inappropriate for missing covariates in nonrandomised studies [59], are appropriate for missing baseline values in randomised trials, provided that the imputed values respect the independence of the baseline values and randomised group [55].”
MethodZwanenburg, A. (2019) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-019-04391-8 · Cited by 306 · see citing works
Multivariable Analysis: A Practical Guide for Clinicians and Public Health Researchers
Katz, M. H. (2011) · Cited by 298 · see citing works
Henkel, D. M., Redfield, M. M., Weston, S. A., Gerber, Y., & Roger, V. L. (2008) · Circulation Heart Failure · doi:10.1161/circheartfailure.107.743146 · Cited by 287 · see citing works
What is the difference between missing completely at random and missing at random?
Bhaskaran, K., & Smeeth, L. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyu080 · Cited by 286 · see citing works
“Bhaskaran, K; Smeeth, L (2014) What is the difference between missing completely at random and missing at random? International journal of epidemiology, 43 (4).”
BackgroundCessation of alcohol drinking, tobacco smoking and the reversal of head and neck cancer risk
Marron, M., Boffetta, P., Zhang, Z., Zaridze, D. G., Wunsch-Filho, V., Winn, D. M., Wei, Q., Talamini, R., Szeszenia‐Dąbrowska, N., Sturgis, E. M., et al. (2009) · International Journal of Epidemiology · doi:10.1093/ije/dyp291 · Cited by 282 · see citing works
Gorter, P. M., van Lindert, A. S. R., de Vos, A. M., Meijs, M. F. L., van der Graaf, Y., Doevendans, P. A., Prokop, M., & Visseren, F. L. J. (2007) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2007.08.016 · Cited by 273 · see citing works
Gorter, P. M., de Vos, A. M., van der Graaf, Y., Stella, P. R., Doevendans, P. A., Meijs, M. F. L., Prokop, M., & Visseren, F. L. J. (2008) · The American Journal of Cardiology · doi:10.1016/j.amjcard.2008.04.002 · Cited by 269 · see citing works
Multiple Imputation With Large Data Sets: A Case Study of the Children's Mental Health Initiative
Stuart, E. A., Azur, M. J., Frangakis, C., & Leaf, P. J. (2009) · American Journal of Epidemiology · doi:10.1093/aje/kwp026 · Cited by 267 · see citing works
“Doing so results in correct standard error estimates and coverage rates, as compared with single imputation methods or simply including a missing data indicator for each variable in the model (17, 18).”
Barzi, F. (2004) · American Journal of Epidemiology · doi:10.1093/aje/kwh175 · Cited by 263 · see citing works
Advanced Statistics: Missing Data in Clinical Research--Part 2: Multiple Imputation
Newgard, C. D., & Haukoos, J. S. (2007) · Academic Emergency Medicine · doi:10.1197/j.aem.2006.11.038 · Cited by 261 · see citing works
Multiple-imputation for measurement-error correction
Cole, S. R., Chu, H., & Greenland, S. (2006) · International Journal of Epidemiology · doi:10.1093/ije/dyl097 · Cited by 256 · see citing works
Self-citationKim, E. S., Sun, J. K., Park, N., Kubzansky, L. D., & Peterson, C. (2013) · Journal of Behavioral Medicine · doi:10.1007/s10865-012-9406-4 · Cited by 244 · see citing works
Marshall, A., Altman, D. G., Royston, P. J., & HOLDER, R. L. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-7 · Cited by 248 · see citing works
“[11,16]).”
Imputation of missing data in time series for air pollutants
Junger, W. L., & de León, A. P. (2014) · Atmospheric Environment · doi:10.1016/j.atmosenv.2014.11.049 · Cited by 274 · see citing works
Rutten, F. H., Moons, K. G. M., Cramer, M. J., Grobbee, D. E., Zuithoff, N. P. A., Lammers, A.-W. J., & Hoes, A. W. (2005) · BMJ · doi:10.1136/bmj.38664.661181.55 · Cited by 238 · see citing works
Use of Multiple Imputation in the Epidemiologic Literature
Klebanoff, M. A., & Cole, S. R. (2008) · American Journal of Epidemiology · doi:10.1093/aje/kwn071 · Cited by 238 · see citing works
“Simple approaches, such as including an indicator variable for missingness, do not in general correct this bias (1).”
BackgroundMethodEekhout, I. E., de Boer, M. R., Twisk, J. W. R., de Vet, H. C. W., & Heymans, M. W. (2012) · Epidemiology · doi:10.1097/ede.0b013e3182576cdb · Cited by 235 · see citing works
Ackerstaff, R. G. A., Moons, K. G. M., van de Vlasakker, C. J. W., Moll, F. L., Vermeulen, F. E. E., Algra, A., & Spencer, M. P. (2000) · Stroke · doi:10.1161/01.str.31.8.1817 · Cited by 219 · see citing works
Burton, A., & Altman, D. G. (2004) · British Journal of Cancer · doi:10.1038/sj.bjc.6601907 · Cited by 219 · see citing works
“Other techniques, utilising all cases, include analysing the missing data as a separate category (Greenland and Finkle, 1995), single imputation (Little and Rubin, 1987), in which a single value is substituted for each missing value, and multiple imputation (Rubin, 1987; Schafer, 1997), where more than one independent…”
Zuithoff, N. P., Vergouwe, Y., King, M., Nazareth, I., van Wezep, M. J., Moons, K. G., & Geerlings, M. I. (2010) · BMC Family Practice · doi:10.1186/1471-2296-11-98 · Cited by 218 · see citing works
“complete case analysis) leads to biased results[23-26].”
Thiébaut, A. C. M., Kipnis, V., Chang, S., Subar, A. F., Thompson, F. E., Rosenberg, P. S., Hollenbeck, A. R., Leitzmann, M., & Schatzkin, A. G. (2007) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djk094 · Cited by 213 · see citing works
Toh, S., Reichman, M. E., Houstoun, M., Southworth, M. R., Ding, X. X., Hernandez, A. F., Levenson, M. S., Li, L., McCloskey, C. A., Shoaibi, A., et al. (2012) · Archives of Internal Medicine · doi:10.1001/2013.jamainternmed.34 · Cited by 218 · see citing works
Multiple Organ Failure Can Be Predicted as Early as 12 Hours after Injury
Sauaia, A., Moore, F. A., Moore, E. E., Norris, J. M., Lezotte, D. C., & Hamman, R. F. (1998) · doi:10.1097/00005373-199808000-00014 · Cited by 211 · see citing works
Looking Beyond Where Children Die: Determinants and Effects of Planning a Child's Location of Death
Dussel, V., Kreicbergs, U., Hilden, J. M., Watterson, J., Moore, C., Turner, B. G., Weeks, J. C., & Wolfe, J. (2009) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2007.12.017 · Cited by 210 · see citing works
Dairy Food, Calcium, and Risk of Cancer in the NIH-AARP Diet and Health Study
Park, Y., Leitzmann, M. F., Subar, A. F., Hollenbeck, A. R., & Schatzkin, A. G. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2008.578 · Cited by 214 · see citing works
“In sensitivity analyses, we analyzed the data using an indicator variable for missing responses in each covariate,(19,20) excluding participants with missing responses in any covariate.(21) We found that the results from these methods were similar to those of the main analyses.”
ResultHaukoos, J. S., & Newgard, C. D. (2007) · Academic Emergency Medicine · doi:10.1197/j.aem.2006.11.037 · Cited by 207 · see citing works
Use of Multivitamin/Mineral Prenatal Supplements: Influence on the Outcome of Pregnancy
Scholl, T. O., Hediger, M. L., Bendich, A., Schall, J. I., Smith, W., & Krueger, P. M. (1997) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009244 · Cited by 204 · see citing works
“However, because there were a number of missing samples at entry to care or 28 weeks' gestation, the effects of prenatal vitamin use on nutritional status were estimated using a multiple imputation procedure (9-11) instead of a complete-subject analysis.”
MethodRodondi, N., Peng, T., Karter, A. J., Bauer, D. C., Vittinghoff, E., Tang, S. S. K., Pettitt, D., Kerr, E. A., & Selby, J. V. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-7-200604040-00006 · Cited by 208 · see citing works
“Because of missing data on race or ethnicity (Table 1), we used complete case analysis for the multivariate models, which is less subject to bias than treating observations with missing race or ethnicity as a separate category (28).”
MethodKarahalios, A., Baglietto, L., Carlin, J. B., English, D. R., & Simpson, J. A. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-96 · Cited by 191 · see citing works
“LOCF, the missing indicator method and mean value substitution), which are generally not recommended [16,25] because they fail to account for the uncertainty in the data and may produce biased estimates [12], continue to be used.”
MethodPredicting the long-term prognosis of children with symptoms suggestive of asthma at preschool age
Caudri, D., Wijga, A. H., Schipper, C. M. A., et al. (2009) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2009.06.045 · Cited by 192 · see citing works
Growth and Fatness at Three to Six Years of Age of Children Born Small- or Large-for-Gestational Age
Hediger, M. L., Overpeck, M. D., McGlynn, A. F., Kuczmarski, R. J., Maurer, K. R., & Davis, W. W. (1999) · PEDIATRICS · doi:10.1542/peds.104.3.e33 · Cited by 191 · see citing works
Missing Data and Multiple Imputation
Cummings, P. (2013) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2013.1329 · Cited by 189 · see citing works
Knol, M. J., Janssen, K. J. M., Donders, A. R. T., Egberts, T. C. G., Heerdink, E. R., Grobbee, D. E., Moons, K. G. M., & Geerlings, M. I. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.08.028 · Cited by 190 · see citing works
Missing data in randomised controlled trials: a practical guide
Carpenter, J. R., & Kenward, M. (2007) · LSHTM Research Online (London School of Hygiene and Tropical Medicine) · Cited by 204 · see citing works
A prognostic model for ovarian cancer
Clark, T. G., Stewart, M., Altman, D. G., Gabra, H., & Smyth, J. F. (2001) · British Journal of Cancer · doi:10.1054/bjoc.2001.2030 · Cited by 192 · see citing works
“This practice can lead to unrealistic hazard ratios and underestimation of standard errors (Greenland and Finkle 1995), but may be better than excluding whole cases.”
BackgroundRelationship between atherosclerosis and late-life depression: the Rotterdam Study.
Tiemeier, H. W., van Dijck, W., Albert Hofman, D., Witteman, J. C. M., Stijnen, T., & Breteler, M. M. B. (2004) · PubMed · doi:10.1001/archpsyc.61.4.369 · Cited by 186 · see citing works
“To decrease possible bias, these missing atherosclerosis values can be filled in (imputed) by the multiple imputation method.(31) Imputation is based on the correlation between the missing variable and other variables including the atherosclerosis measures.”
Moscucci, M. M., Kline-Rogers, E. M., Share, D. A., et al. (2001) · Circulation · doi:10.1161/01.cir.104.3.263 · Cited by 186 · see citing works
DOORDUYN, Y., VAN DEN BRANDHOF, W. E., VAN DUYNHOVEN, Y. T. H. P., WANNET, W. J. B., & VAN PELT, W. (2006) · Epidemiology and Infection · doi:10.1017/s0950268805005406 · Cited by 183 · see citing works
“This method is recommended to avoid bia in analyses f epidemiological data with missing covariates [32].”
MethodGüder, G., Brenner, S., Angermann, C. E., Ertl, G., Held, M., Sachs, A. P. E., Lammers, J. J., Zanen, P., Hoes, A. W., Störk, S., et al. (2012) · Respiratory Research · doi:10.1186/1465-9921-13-13 · Cited by 181 · see citing works
“As deletion of subjects with missing values may lead to biased results we imputed missing values using a regression method with the addition of a random error term [21].”
MethodMultiple imputation: dealing with missing data
de Goeij, M. C. M., van Diepen, M., Jager, K. J., Tripepi, G. L., Zoccali, C., & Dekker, F. W. (2013) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/gft221 · Cited by 176 · see citing works
“For those details, we refer to other literature [3–6].”
MethodLarge-sample theory for parametric multiple imputation procedures
Wang, N. (1998) · Biometrika · doi:10.1093/biomet/85.4.935 · Cited by 177 · see citing works
Tomas, L. H. S., Loberiza, F. R., Klein, J. P., et al. (2005) · CHEST Journal · doi:10.1378/chest.128.1.153 · Cited by 176 · see citing works
Haukoos, J. S., & Newgard, C. D. (2007) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2007.tb01855.x · Cited by 176 · see citing works
“8,12,18 This approach has been widely used and is still an extremely common method of handling data sets with censored values, particularly because of its simplicity.”
BackgroundMethodInfluentialIntroduction to propensity scores
Williamson, E., & Forbes, A. B. (2014) · Respirology · doi:10.1111/resp.12312 · Cited by 176 · see citing works
Predicting mortality in patients with heart failure: a pragmatic approach
Bouvy, M. L., Heerdink, E. R., Leufkens, H. G. M., & Hoes, A. W. (2003) · Heart · doi:10.1136/heart.89.6.605 · Cited by 171 · see citing works
Newgard, C. D., Fildes, J., Wu, L., Hemmila, M. R., Burd, R. S., Neal, M., Mann, C. N., Shafi, S., Clark, D. E., Goble, S. M., et al. (2012) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2012.08.017 · Cited by 169 · see citing works
Sherer, M., Sander, A. M., Nick, T. G., High, W. M., Malec, J. F., & Rosenthal, M. (2002) · Archives of Physical Medicine and Rehabilitation · doi:10.1053/apmr.2002.28802 · Cited by 170 · see citing works
Peters, S. A. E., Bots, M. L., den Ruijter, H. M., Palmer, M. K., Grobbee, D. E., Crouse, J. R., O’Leary, D. H., Evans, G. W., Raichlen, J. S., Moons, K. G. M., et al. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.11.012 · Cited by 165 · see citing works
Socioeconomic status and survival after an invasive breast cancer diagnosis
Sprague, B. L., Trentham‐Dietz, A., Gangnon, R. E., Ramchandani, R., Hampton, J. M., Robert, S., Remington, P., & Newcomb, P. A. (2010) · Cancer · doi:10.1002/cncr.25589 · Cited by 162 · see citing works
Sacker, A., & Cable, N. (2006) · European Journal of Public Health · doi:10.1093/eurpub/cki189 · Cited by 158 · see citing works
A comparison of different methods to handle missing data in the context of propensity score analysis
Choi, J., Dekkers, O. M., & le Cessie, S. (2018) · European Journal of Epidemiology · doi:10.1007/s10654-018-0447-z · Cited by 155 · see citing works
Wijga, A. H. (2003) · Thorax · doi:10.1136/thorax.58.7.567 · Cited by 158 · see citing works
Heida, K. Y., Franx, A., van Rijn, B. B., Eijkemans, M. J. C., Boer, J. M. A., Verschuren, W. M. M., Oudijk, M. A., Bots, M. L., & van der Schouw, Y. T. (2015) · Hypertension · doi:10.1161/hypertensionaha.115.06005 · Cited by 159 · see citing works
- Authors
- Greenland, S., & Finkle, W. D.
- Affiliations
- University of California, Los Angeles (US)
- Corresponding author
- Greenland, Sander
- ORCID
- Greenland, Sander
- Venue
- American Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 1995-12-15
- Volume
- 142
- Issue
- 12
- Pages
- 1255-1264
- ISSN
- 0002-9262 (ISSN-L)
- 1476-6256
- OpenAlex
- W2166561686
- Cited by
- 891
- References
- 0
- Citations by year
- 2026: 13 · 2025: 18 · 2024: 13 · 2023: 24 · 2022: 26 · 2021: 21 · 2020: 33 · 2019: 29 · 2018: 34 · 2017: 32 · 2016: 27 · 2015: 40 · 2014: 57 · 2013: 53 · 2012: 67
- FWCI
- 6.57
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- American Journal of Epidemiology
- Funders
- National Institutes of Health (n01cp095680-001)
- National Cancer Institute (N01 CP 95680)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- missing covariates, missing data, logistic regression, bias, model misspecification, multiple imputation, maximum likelihood estimation, weighted estimating equations, endometrial cancer, age at menarche, case-control study
- MeSH terms
- Epidemiologic Methods, Female, Humans, Menarche, Research Design, Logistic Models, Odds Ratio, Case-Control Studies, Endometrial Neoplasms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
