The DOI matches its Crossref record.
A critical look at methods for handling missing covariates in epidemiologic regression analyses.
Abstract
Epidemiologic studies often encounter missing covariate values. While simple methods such as stratification on missing-data status, conditional-mean imputation, and complete-subject analysis are commonly employed for handling this problem, several studies have shown that these methods can be biased under reasonable circumstances. The authors review these results in the context of logistic regression and present simulation experiments showing the limitations of the methods. The method based on missing-data indicators can exhibit severe bias even when the data are missing completely at random, and regression (conditional-mean) imputation can be inordinately sensitive to model misspecification. Even complete-subject analysis can outperform these methods. More sophisticated methods, such as maximum likelihood, multiple imputation, and weighted estimating equations, have been given extensive attention in the statistics literature. While these methods are superior to simple methods, they are not commonly used in epidemiology, no doubt due to their complexity and the lack of packaged software to apply these methods. The authors contrast the results of multiple imputation to simple methods in the analysis of a case-control study of endometrial cancer, and they find a meaningful difference in results for age at menarche. In general, the authors recommend that epidemiologists avoid using the missing-indicator method and use more sophisticated methods whenever a large proportion of data are missing.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Commonly missing values are related to observed participant or disease characteristics.”
From Reporting and Methods in Clinical Prediction Research: A Systematic Review (2012) · cites this paper as [24]
In the citing paper
“Missing values in clinical study data rarely occur completely at random.”
From Reporting and Methods in Clinical Prediction Research: A Systematic Review (2012) · cites this paper as [24]
In the citing paper
“Exclusion of participants with missing values will therefore not only lead to loss of statistical power, but often also to biased results 10,12,13,23,24,57,58.”
From Reporting and Methods in Clinical Prediction Research: A Systematic Review (2012) · cites this paper as [24]
Citations
Cited by
Sorted by citations
Items 101–200 of 892
High‐risk HPV types and head and neck cancer
Michaud, D. S., Langevin, S. M., Eliot, M., Nelson, H. H., Pawlita, M., McClean, M. D., & Kelsey, K. T. (2014) · International Journal of Cancer · doi:10.1002/ijc.28811 · Cited by 158
“7% missing); using multiple imputation for missing data reduces bias.(16,17) Additional control for body mass index (BMI), income (below/above $50K per year) or marijuana use, did not change the observed associations and were left out of the final model.”
MethodVarma, J. K., Samuel, M., Marcus, R., Hoekstra, R. M., Medus, C., Segler, S. D., Anderson, B. J., Jones, T. F., Shiferaw, B., Haubert, N., et al. (2007) · Clinical Infectious Diseases · doi:10.1086/509920 · Cited by 154
“In all analyses, we controlled for the matching factors, for whether the subject was enrolled as part of a matched set, and for whether a surrogate respondent was used to report the food exposure history [30, 31].”
Dealing with Missing Predictor Values When Applying Clinical Prediction Models
Janssen, K. J. M., Vergouwe, Y., Donders, A. R. T., Harrell, F. E., Chen, Q., Grobbee, D. E., & Moons, K. G. M. (2009) · Clinical Chemistry · doi:10.1373/clinchem.2008.115345 · Cited by 152
“Malignancy present 77 (6) 19 (4) 18 (7)”
The Wells Rule Does Not Adequately Rule Out Deep Venous Thrombosis in Primary Care Patients
Oudega, R., Hoes, A. W., & Moons, K. G. M. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-143-2-200507190-00008 · Cited by 152
“We imputed the missing values of all study variables by using a linear regression method with the addition of a random error term (SPSS software, version 12.0 for Windows, SPSS, Inc., Chicago, Illinois) (17, 31, 32 ).”
MethodInfluentialHyponatremia and mortality risk: a Danish cohort study of 279 508 acutely hospitalized patients
Holland-Bill, L., Christiansen, C. F., Heide‐Jørgensen, U., Ulrichsen, S. P., Ring, T., Jørgensen, J. O. L., & Sørensen, H. T. (2015) · European Journal of Endocrinology · doi:10.1530/eje-15-0111 · Cited by 149
Avoiding pitfalls when combining multiple imputation and propensity scores
Granger, E., Sergeant, J. C., & Lunt, M. (2019) · Statistics in Medicine · doi:10.1002/sim.8355 · Cited by 148
Ruling out deep venous thrombosis in primary care
Moons, K. G. M., Hoes, A. W., & Oudega, R. (2005) · Thrombosis and Haemostasis · doi:10.1160/th04-12-0829 · Cited by 144
Preoperative Hyponatremia and Perioperative Complications
Leung, A. A. C., McAlister, F. A., Rogers, S. O., et al. (2012) · Archives of Internal Medicine · doi:10.1001/archinternmed.2012.3992 · Cited by 142
“) Data were mostly complete, but missing values were handled in the analysis by dummy coding.(23) We subsequently conducted a secondary analysis for the primary outcome according to the number of days from the preoperative sodium measurement to the time of operation.”
MethodSnijder, C. A., Kortenkamp, A., Steegers, E. A. P., Jaddoe, V. W. V., Albert Hofman, D., Hass, U., & Burdorf, A. (2012) · Human Reproduction · doi:10.1093/humrep/der474 · Cited by 140
“The threshold for imputation was 30% of missing values, all covariates were imputed (Greenland and Finkle, 1995).”
BackgroundIssues in multiple imputation of missing data for large general practice clinical databases
Marston, L., Carpenter, J. R., Walters, K. R., Morris, R. W., Nazareth, I., & Petersen, I. (2010) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.1934 · Cited by 132
“Diastolic BP mean (SD) 76 (11) 74 (11) 81 (12) 85 (11) Missing (%) 30 38 17 0”
van der Hoeven, E. J., McVerry, F., Vos, J. A., Algra, A., Puetz, V., Kappelle, L. J., Schonewille, W. J., & on behalf of the BASICS registry investigators (2016) · International Journal of Stroke · doi:10.1177/1747493016641951 · Cited by 134
Hamilton, B. H., Ko, C. Y., Richards, K., & Hall, B. L. (2010) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2009.10.021 · Cited by 134
Mortality in British hip fracture patients, 2000–2010: A population-based retrospective cohort study
Klop, C., Welsing, P. M. J., Cooper, C. C., Harvey, N. C., Elders, P. J. M., Bijlsma, J. W. J., Leufkens, H. G. M., & de Vries, F. (2014) · Bone · doi:10.1016/j.bone.2014.06.011 · Cited by 141
Faris, P., Ghali, W. A., Brant, R., Norris, C. M., Galbraith, P. D., & Knudtson, M. L. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00433-4 · Cited by 130
Dietary Fatty Acids and Pancreatic Cancer in the NIH-AARP Diet and Health Study
Thiébaut, A. C. M., Jiao, L., Silverman, D. T., Cross, A. J., Thompson, F. E., Subar, A. F., Hollenbeck, A. R., Schatzkin, A. G., & Stolzenberg‐Solomon, R. Z. (2009) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djp168 · Cited by 132
Nonmelanoma Skin Cancer and Risk for Subsequent Malignancy
Chen, J., Ruczinski, I., Jørgensen, T., Yenokyan, G., Yao, Y., Alani, R. M., Liégeois, N. J., Hoffman, S. C., Hoffman–Bolton, J., Strickland, P. T., et al. (2008) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djn260 · Cited by 130
Bärnighausen, T., Tanser, F. C., Gqwede, Z., Mbizana, C., Herbst, K., & Newell, M. (2008) · AIDS · doi:10.1097/qad.0b013e3282f2ef43 · Cited by 131
Predictors of Hypertension and Changes of Blood Pressure in HIV-Infected Patients
Thiébaut, R., El-Sadr, W. M., Friis‐Møller, N., Rickenbach, M., Reiss, P., Monforte, A. D., Morfeldt, L., Fontas, É., Kirk, O., Wit, S. D., et al. (2005) · Antiviral Therapy · doi:10.1177/135965350501000706 · Cited by 129
“Moreover, a sensitivity analysis using a multiple imputation method to take into account missing values (with n=5 samples) was performed [25] and the results were not substantially different (data not shown).”
MethodWesterhuis, M. E. M. H., Visser, G. H. A., Moons, K. G. M., van Beek, E., Benders, M. J. N. L., Bijvoet, S. M., Dessel, H. J. H. M. V., Drogtrop, A. P., van Geijn, H. P., Graziosi, G. C. M., et al. (2010) · Obstetrics and Gynecology · doi:10.1097/aog.0b013e3181dfffd6 · Cited by 126
Kim, S., Egerter, S. A., Cubbin, C., Takahashi, E. R., & Braveman, P. (2007) · Public Health Reports · doi:10.1177/003335490712200607 · Cited by 130
EuroSCORE as predictor of in-hospital mortality after percutaneous coronary intervention
Romagnoli, E., Burzotta, F., Trani, C., et al. (2008) · Heart · doi:10.1136/hrt.2007.134114 · Cited by 123
Are All Growth-restricted Newborns Created Equal(ly)?
Kramer, M. S., Platt, R. W., MSc, H. Y., McNamara, H., & Usher, R. H. (1999) · PEDIATRICS · doi:10.1542/peds.103.3.599 · Cited by 123
Self‐reported use of mental health services versus administrative records: care to recall?
Rhodes, A. E., & Fung, K. (2004) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.172 · Cited by 120
“Each missing value was substituted with each income level in successive imputations (Vach and Blettner, 1991; Greenland and Finkle, 1995).”
MethodMarroun, H. E., White, T. J. H., van der Knaap, N. J. F., Homberg, J. R., Fernández, G., Schoemaker, N. K., Jaddoe, V. W. V., Hofman, A., Verhulst, F. C., Hudziak, J. J., et al. (2014) · British Journal of Psychiatry · doi:10.1192/bjp.bp.113.127746 · Cited by 117
“46 The associations in the complete-case data-set (with no missing data imputed) were very similar to the associations found in the imputed data-sets.”
ResultFreeman, L., Bloemenkamp, K. W. M., Franssen, M. T. M., et al. (2015) · BMJ · doi:10.1136/bmj.h846 · Cited by 121
Stavseth, M. R., Clausen, T., & Røislien, J. (2019) · SAGE Open Medicine · doi:10.1177/2050312118822912 · Cited by 137
Multiple Imputation of Baseline Data in the Cardiovascular Health Study
Arnold, A. M. (2002) · American Journal of Epidemiology · doi:10.1093/aje/kwf156 · Cited by 118
“Greenland S, Finkle WD.”
BackgroundAnderson, K. K., Fuhrer, R., Schmitz, N., & MALLA, A. K. (2012) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-012-0571-0 · Cited by 119
“This was done multiple times to create several datasets with complete observations, which were analyzed in parallel and synthesized to compute summary parameter estimates that account for uncertainty introduced by the imputation process [31].”
MethodMcGue, M. K., & Christensen, K. (2002) · Experimental Aging Research · doi:10.1080/03610730290080416 · Cited by 118
“Such treatment of missing data can be expected to yield more precise estimates of parameters than analysis of summary variance-covariance matrices and mean vectors that in effect involve deleting a case from the analysis if even one observation is missing (Greenland & Finkle, 1995).”
MethodDeCourcey, D. D., Silverman, M., Oladunjoye, A. O., & Wolfe, J. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000003472 · Cited by 119
Quantifying losses to the donated blood supply due to donor deferral and miscollection
Custer, B., Johnson, E. S., Sullivan, S. D., et al. (2004) · Transfusion · doi:10.1111/j.1537-2995.2004.04160.x · Cited by 119
“Summary of predonation classification based on age, sex, and first-time versus repeat donor status * Demographic group Eligible to donate Donor Age group Deferral status (years) Sex Short-term Long-term Permanent First-time 16-24 Female 4,825 (73) 1,248 (19) 436 (7) 77 (1) 25-39 4,511 (73) 1,155 (19) 383 (6) 158 (3) 4…”
Black–White Disparities in Preterm Birth: Geographic, Social, and Health Determinants
Thoma, M. E., Drew, L. B., Hirai, A. H., Kim, T. Y., Fenelon, A. T., & Shenassa, E. D. (2019) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2019.07.007 · Cited by 118
Mortality and disability: the effect of overweight and obesity
Walter, S., Kunst, A. E., Mackenbach, J. P., Albert Hofman, D., & Tiemeier, H. W. (2009) · International Journal of Obesity · doi:10.1038/ijo.2009.176 · Cited by 120
Purdue, M. P., Hashibe, M., Berthiller, J., Vecchia, C. L., Maso, L. D., Herrero, R., Franceschi, S., Castellsagué, X., Wei, Q., Sturgis, E. M., et al. (2008) · American Journal of Epidemiology · doi:10.1093/aje/kwn306 · Cited by 116
“, Cary, North Carolina) under the assumption that the education data were missing at random (34).”
BackgroundPisters, M., Veenhof, C., van Dijk, G. V., Heymans, M., Twisk, J., & Dekker, J. (2012) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2012.02.002 · Cited by 112
Davies, R. R., Russo, M. J., Morgan, J. A., Sorabella, R. A., Naka, Y., & Chen, J. M. (2010) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2010.04.029 · Cited by 116
Rider, J. R., Wilson, K. M., Sinnott, J. A., Kelly, R. S., Mucci, L. A., & Giovannucci, E. L. (2016) · European Urology · doi:10.1016/j.eururo.2016.03.027 · Cited by 115
“The missing indicator method was used for missing data on most covariates [11].”
Lew, J., Freedman, N. D., Leitzmann, M. F., Brinton, L. A., Hoover, R. N., Hollenbeck, A. R., Schatzkin, A. G., & Park, Y. (2009) · American Journal of Epidemiology · doi:10.1093/aje/kwp120 · Cited by 113
“Those missing body mass index and height measurements were assigned the mean values in the study population (19).”
MethodLength of Hospital Stays Among Obese Individuals
Zizza, C. A., Herring, A. H., Stevens, J., & Popkin, B. M. (2004) · American Journal of Public Health · doi:10.2105/ajph.94.9.1587 · Cited by 113
The Missing Data Problem in Birth Weight Percentiles and Thresholds for "Small-for-Gestational-Age"
Hutcheon, J. A., & Platt, R. W. (2008) · American Journal of Epidemiology · doi:10.1093/aje/kwm327 · Cited by 112
“In order for the ‘‘complete case’’ approach ( 25 ) used in conventional weight-for-gestationalage charts to be valid (the use of only those cases for whom complete data are available—i.e., data on livebirths), the unobserved data must be missing completely at random (MCAR).”
BackgroundEndometriosis and pregnancy complications: a Danish cohort study
Glavind, M. T., Forman, A., Arendt, L. H., Nielsen, K. B., & Henriksen, T. B. (2016) · Fertility and Sterility · doi:10.1016/j.fertnstert.2016.09.020 · Cited by 112
Kropko, J., Goodrich, B., Gelman, A., & Hill, J. (2014) · Political Analysis · doi:10.1093/pan/mpu007 · Cited by 111
“Most researchers instead suppose that a few variables in the data are completely observed, and allow each variable’s missingness to depend only on these fully observed variables (Greenland and Finkle 1995).”
BackgroundMeta‐analysis of a binary outcome using individual participant data and aggregate data
Riley, R. D., & Steyerberg, E. W. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.4 · Cited by 108
Opstelten, W., Zuithoff, N. P. A., van Essen, G. A., van Loon, A. M., van Wijck, A. J. M., Kalkman, C. J., Verheij, T. J., & Moons, K. G. M. (2007) · Pain · doi:10.1016/j.pain.2007.02.004 · Cited by 109
“Since data omission rarely occurs at random, excluding subjects with missing values not only leads to loss of statistical power, but also to biased results (Greenland and Finkle, 1995; Harrell et al., 1996).”
BackgroundApplication of the Missing-Indicator Method in Matched Case-Control Studies with Incomplete Data
Huberman, M., & Langholz, B. M. (1999) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a009966 · Cited by 109
“In a recent review of methods for handling missing covariates in case-control studies, it was recommended that the missing-indicator method be avoided and more sophisticated methods be preferred ( 12 ).”
MethodNewgard, C. D. (2006) · Academic Emergency Medicine · doi:10.1197/j.aem.2005.09.011 · Cited by 111
Imputation of missing values of tumour stage in population-based cancer registration
Eisemann, N., Waldmann, A., & Katalinic, A. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-129 · Cited by 107
“The first approach is widely known to produce biased results [17,18].”
BackgroundModelling relative survival in the presence of incomplete data: a tutorial
Nur, U., Shack, L., Rachet, B., Carpenter, J. R., & Coleman, M. P. (2009) · International Journal of Epidemiology · doi:10.1093/ije/dyp309 · Cited by 107
Fuchs, Z., Blumstein, T., Novikov, I., Walter‐Ginzburg, A., Lyanders, M., Gindin, J., Habot, B., & Modan, B. (1998) · The Journals of Gerontology Series A · doi:10.1093/gerona/53a.6.m447 · Cited by 108
“We compared this simple indicator method for dealing with missing data with the modified indicator method (21); as the results of both methods were very close, only the simple method is presented.”
MethodSocioeconomic inequalities in frailty and frailty components among community-dwelling older citizens
Franse, C. B., van Grieken, A., Li, Q., Melis, R. J. F., Rietjens, J. A. C., & Raat, H. (2017) · PLoS ONE · doi:10.1371/journal.pone.0187946 · Cited by 108
“We computed five imputation datasets using a fully conditional specified model[35].”
MethodRidsdale, L., Godfrey, E. L., Chalder, T., Seed, P. T., King, M., Wallace, P., & Wessely, S. C. (2001) · Research Portal (King's College London) · Cited by 106
Heckbert, S. R. (1997) · Archives of Internal Medicine · doi:10.1001/archinte.1997.00440330066007 · Cited by 109
Postdiagnosis Change in Bodyweight and Survival After Breast Cancer Diagnosis
Bradshaw, P. T., Ibrahim, J. G., Stevens, J., Cleveland, R. J., Abrahamson, P. E., Satia, J. A., Teitelbaum, S. L., Neugut, A. I., & Gammon, M. D. (2012) · Epidemiology · doi:10.1097/ede.0b013e31824596a1 · Cited by 106
“improper imputation, which can perform even worse than complete case analysis.(22) Formal treatment of missing data is crucial to accurate inference, and the selection model ap-”
Fast-Track Anesthesia and Cardiac Surgery: A Retrospective Cohort Study of 7989 Patients
Svircevic, V., Nierich, A. P., Moons, K. G. M., Bruinsma, G. J. B. B., Kalkman, C. J., & van Dijk, D. (2009) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e318193c423 · Cited by 106
Li, C. I., & Daling, J. R. (2007) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-07-0546 · Cited by 106
“One study attempted to address this problem using multiple imputation (2), but there are inherent biases in using this strategy (22), particularly when the proportion of the variable with missing data is changing so rapidly over the study period, and the methodologic approach used was not described.”
BackgroundRajpathak, S. N., Xue, X., Wassertheil-Smoller, S. W., Horn, L. V., Robinson, J. G., Liu, S., Allison, M., Martin, L. W., Ho, G. Y. F., & Rohan, T. E. (2010) · American Journal of Clinical Nutrition · doi:10.3945/ajcn.2009.28579 · Cited by 104
“Missing data were handled by the complete case method by using observations without any missing data (23).”
MethodAntiretroviral Therapy Outcomes among Adolescents and Youth in Rural Zimbabwe
Bygrave, H., Mtangirwa, J., Ncube, K., et al. (2012) · PLoS ONE · doi:10.1371/journal.pone.0052856 · Cited by 104
“This is particularly true in situations where the percentage of missing data is high [14–17].”
BackgroundChaudru, V., Chompret, A., Paillerets, B. B., Spatz, A., Avril, M.-F., & Démenais, F. (2004) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djh136 · Cited by 103
Pettengell, R., Bosly, A., Szucs, T. D., et al. (2008) · British Journal of Haematology · doi:10.1111/j.1365-2141.2008.07514.x · Cited by 102
Jenkins, P. C., Flanagan, M. F., Jenkins, K. J., Sargent, J. D., Canter, C. E., Chinnock, R., Vincent, R. N., Tosteson, A. N. A., & O’Connor, G. T. (2000) · Journal of the American College of Cardiology · doi:10.1016/s0735-1097(00)00855-x · Cited by 99
“Missing data were dealt with in two ways (21,22): 1) for variables not entered into the logistic regression model, missing data were not substituted; 2) sensitivity analysis was performed on three variables missing $20% of data tested in the regression: aortic diameter, moderate-tosevere tricuspid regurgitation (TR) a…”
MethodHulsegge, G., Looman, M., Smit, H. A., Daviglus, M. L., van der Schouw, Y. T., & Verschuren, W. M. M. (2016) · Journal of the American Heart Association · doi:10.1161/jaha.115.002432 · Cited by 102
“31,32 Therefore, missing values for all determinants and covariates were multiple-imputed using the multivariate imputation by chained equations method in the statistical program R (version 3.1.0; R Foundation for Statistical Computing, Vienna, Austria).”
InfluentialAccess to High Pediatric-Readiness Emergency Care in the United States
Ray, K. N., Olson, L. M., Edgerton, E. A., et al. (2018) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2017.10.074 · Cited by 98
“Multiple imputation generates multiple simulated datasets, each containing plausible values for missing data, which are then analyzed and pooled.(13,14) To perform the imputation, we first evaluated hospital characteristics associated with nonresponse and with WPRS based on AHA linkage.”
A Comparison of Prehospital and Hospital Data in Trauma Patients
Arbabi, S., Jurkovich, G. J., Wahl, W. L., et al. (2004) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/01.ta.0000123036.20919.4b · Cited by 98
Common scientific and statistical errors in obesity research
George, B. J., Beasley, T. M., Brown, A. W., Dawson, J., Dimova, R. B., Divers, J., Goldsby, T. U., Heo, M., Kaiser, K. A., Keith, S. W., et al. (2016) · Obesity · doi:10.1002/oby.21449 · Cited by 100
“None of these is an appropriate strategy (71,72), and they are listed only so they can be recognized and avoided.”
BackgroundMultiple imputation by chained equations in praxis: Guidelines and review
Wulff, J. N., & Ejlskov, L. (2017) · The Electronic Journal of Business Research Methods · Cited by 95
Predicting Lymph Node Metastases in Early Esophageal Adenocarcinoma Using a Simple Scoring System
Lee, L., Ronellenfitsch, U., Hofstetter, W. L., et al. (2013) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2013.03.015 · Cited by 97
Wood, A. (2004) · International Journal of Epidemiology · doi:10.1093/ije/dyh297 · Cited by 96
Trends in Unintentional Drowning
Cummings, P., & Quan, L. (1999) · JAMA · doi:10.1001/jama.281.23.2198 · Cited by 97
“Because blood alcohol information was often missing, we used a multipleimputation method.(20,21) Deaths were classified by year and victim’s age (15-49 years, $50 years), and within each stratum of year and age the missing values for blood alcohol category were imputed 20 times using an approximate Bayesian bootstrap…”
BackgroundMethodRace and Productivity Outcome After Traumatic Brain Injury
Sherer, M., Nick, T. G., Sander, A. M., et al. (2003) · Journal of Head Trauma Rehabilitation · doi:10.1097/00001199-200309000-00003 · Cited by 95
Bai, G., Korfage, I. J., Groen, E. H., Jaddoe, V. W. V., Mautner, E., & Raat, H. (2016) · PLoS ONE · doi:10.1371/journal.pone.0166133 · Cited by 94
Outcomes of Antiretroviral Therapy in Children in Asia and Africa
Leroy, V., Malateste, K., Rabie, H., et al. (2012) · JAIDS Journal of Acquired Immune Deficiency Syndromes · doi:10.1097/qai.0b013e31827b70bf · Cited by 94
Daumit, G. L., Hermann, J. A., Coresh, J., et al. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-3-199902020-00002 · Cited by 94
“We then constructed regression models using only patients for whom data on these major confounders was complete (36).”
MethodCYP2D6 Inhibition and Breast Cancer Recurrence in a Population-Based Study in Denmark
Lash, T. L., Cronin-Fenton, D. P., Ahern, T. P., Rosenberg, C. L., Lunetta, K. L., Silliman, R. A., Garne, J. P., Sørensen, H. T., Hellberg, Y., Christensen, M., et al. (2011) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djr010 · Cited by 93
“Case patients and control subjects with missing data were excluded from analyses that required the variable with a missing result (45).”
BackgroundLee, Y.-C. A., Boffetta, P., Sturgis, E. M., Wei, Q., Zhang, Z., Muscat, J. E., Lazarus, P. J., Matos, E., Hayes, R. B., Winn, D. M., et al. (2008) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-08-0047 · Cited by 100
Dealing with missing data in observational health care outcome analyses
Norris, C. M., Ghali, W. A., Knudtson, M. L., Naylor, C. D., & Saunders, L. D. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00181-x · Cited by 92
Chaffee, B. W., Feldens, C. A., & Vítolo, M. R. (2014) · Annals of Epidemiology · doi:10.1016/j.annepidem.2014.01.013 · Cited by 93
“However, losses remain a limitation, as strategies to account for missing data introduce additional assumptions [47].”
Power and money: Designing statistically powerful studies while minimizing financial costs.
Allison, D. B., Allison, R. L., Faith, M. S., Paultre, F., & Pi‐Sunyer, F. X. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.1.20 · Cited by 95
Lung cancer mortality in a site producing hard metals
Wild, P. P., Perdrix, A., Romazini, S., et al. (2000) · Occupational and Environmental Medicine · doi:10.1136/oem.57.8.568 · Cited by 91
“there was some suggestive evidence of a positive interaction between smoking and exposure to hard metal dust. This is not confirmed in the present study, as a larger than expected SMR was found among non-smokers. Either this is due to some smoking misclassification, in this case any discussion on interaction is useles…”
ResultIdentifying Pulmonary Tuberculosis in Patients With Negative Sputum Smear Results
Kanaya, A. M., Glidden, D. V., & Chambers, H. F. “. (2001) · CHEST Journal · doi:10.1378/chest.120.2.349 · Cited by 91
Lifetime socioeconomic position in relation to onset of perimenopause
Wise, L. A., Krieger, N., Zierler, S., & Harlow, B. L. (2002) · Journal of Epidemiology and Community Health · doi:10.1136/jech.56.11.851 · Cited by 90
Straíf, K., Weiland, S. K., Bungers, M., Holthenrich, D., Taeger, D., Yi, S., & Keil, U. (2000) · Occupational and Environmental Medicine · doi:10.1136/oem.57.3.180 · Cited by 88
“Hazard rate ratios (RRs) and 95% confidence intervals (95% CIs) for exposure to nitrosamines were calculated with low exposure as the reference. The SAS procedure PROC PHREG. 6.12 was used to calculate time dependent and age (time marker) adjusted RRs. Trend tests were performed with exposure scores (low=0, medium=1,…”
MethodInfluentialA clinical prediction model to assess the risk of operative delivery
Schuit, E., Kwee, A., Westerhuis, M., van Dessel, H., Graziosi, G., van Lith, J., Nijhuis, J., Oei, S., Oosterbaan, H., Schuitemaker, N., et al. (2012) · BJOG: an International Journal of Obstetrics and Gynaecology · doi:10.1111/j.1471-0528.2012.03334.x · Cited by 57
Utility of inverse probability weighting in molecular pathological epidemiology
Liu, L., Nevo, D., Nishihara, R., et al. (2017) · European Journal of Epidemiology · doi:10.1007/s10654-017-0346-8 · Cited by 85
Typologies of alcohol consumption in adolescence: predictors and adult outcomes.
Cable, N., & Sacker, A. (2006) · Alcohol and Alcoholism · doi:10.1093/alcalc/agm146 · Cited by 92
“Multiple imputation is recommended for epidemiological studies with extensive missing values as long as these amount to less than 60% of the total (Barzi and Woodward, 2004; Greenland and Finkle, 1995).”
BackgroundA review on missing hydrological data processing
Gao, Y., Merz, C., Lischeid, G., & Schneider, M. (2018) · Environmental Earth Sciences · doi:10.1007/s12665-018-7228-6 · Cited by 86
“Regression-based imputation replaces missing data with predicted values from a regression estimation (Greenland and Finkle 1995).”
MethodNewgard, C. D., Zive, D. M., Jui, J., Weathers, C. C., & Daya, M. R. (2012) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2011.01275.x · Cited by 85
Correlates of Physical Activity in 2-Year-Old Toddlers: The Generation R Study
Wijtzes, A. I., Kooijman, M. N., Jong, J. C. K., de Vries, S. I., Henrichs, J., Jansen, W., Jaddoe, V. W. V., Albert Hofman, D., Moll, H. A., & Raat, H. (2013) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2013.02.029 · Cited by 84
Uijl, S. G., Leijten, F. S. S., Arends, J. B. A. M., Parra, J., van Huffelen, A. C., & Moons, K. G. M. (2007) · Epilepsia · doi:10.1111/j.1528-1167.2007.01197.x · Cited by 83
“The best way to control for this bias is to impute the “missing” values (Greenland and Finkle, 1995; Donders et al., 2006).”
BackgroundSocial inequalities in young children’s sports participation and outdoor play
Wijtzes, A. I., Jansen, W., Bouthoorn, S. H., Pot, N., Albert Hofman, D., Jaddoe, V. W. V., & Raat, H. (2014) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-014-0155-3 · Cited by 96
“To handle missing data in the data, multiple imputation was applied [29].”
BackgroundMethodHuang, C., Li, S., Caraballo, C., Masoudi, F. A., Rumsfeld, J. S., Spertus, J. A., Normand, S. T., Mortazavi, B. J., & Krumholz, H. M. (2021) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.120.007526 · Cited by 90
“52,53 The amount of missingness in clinical data might be higher for racial or ethnic minorities, which may result in the model performing worse for these subgroups.”
BackgroundIndividualized decision-making in IVF: calculating the chances of pregnancy
van Loendersloot, L. L., van Wely, M., Repping, S., Bossuyt, P. M., & van der Veen, F. (2013) · Human Reproduction · doi:10.1093/humrep/det315 · Cited by 83
“For this reason, we first performed an analysis with missingness indicators and then completed missing values by multiple imputation using the Statistical Package for the Social Sciences (version 18.0) (Greenland and Finkle, 1995).”
BackgroundMethodInfluentialMcGrother, C. W., Donaldson, M. M. K., Clayton, D. G., Abrams, K. R., & Clarke, M. (2002) · Osteoporosis International · doi:10.1007/s198-002-8343-6 · Cited by 83
“obtained [ 27 ]. Respondents who were lost to follow-up were eliminated from the analyses; however, the sensitivity of the results to this assumption was assessed using a Cox’s proportional hazard regression model to take account of person-years follow-up.”
MethodTREAD: TReatment with Exercise Augmentation for Depression: study rationale and design
Trivedi, M. H., Greer, T. L., Grannemann, B. D., Church, T. S., Galper, D. I., Sunderajan, P., Wisniewski, S. R., Chambliss, H., JORDAN, A. N., Finley, C. E., et al. (2006) · Clinical Trials · doi:10.1191/1740774506cn151oa · Cited by 81
“If missing data effects are found, mn(ore sophisticated methods will be utilized to correct for these effects (please see Efron [841, Golden [85], Greenland and Finkle [86] Rubin [871, Shafer [881, White [89,90], Zhouetal.”
MethodErmers, M. J. J., Rovers, M. M., van Woensel, J. B. M., et al. (2009) · BMJ · doi:10.1136/bmj.b897 · Cited by 80
Clinical Prediction Rule for RSV Bronchiolitis in Healthy Newborns: Prognostic Birth Cohort Study
Houben, M. L., Bont, L., Wilbrink, B., Belderbos, M. E., Kimpen, J. L. L., Visser, G. H. A., & Rovers, M. M. (2010) · PEDIATRICS · doi:10.1542/peds.2010-0581 · Cited by 80
Primary Stapes Surgery in Patients With Otosclerosis Prediction of Postoperative Outcome
Bittermann, A. J. N. (2011) · Archives of Otolaryngology - Head and Neck Surgery · doi:10.1001/archoto.2011.100 · Cited by 78
“Data are seldom missing at random; it has been shown that removal of participants with a missing value for 1 of the predictors studied (complete case analysis) commonly leads to biased results and certainly to loss of power.(18,19) To decrease bias and to increase statistical efficiency, it is better to impute missing…”
BackgroundResultSome prognostic models for traumatic brain injury were not valid
Hukkelhoven, C. W. P. M., Rampen, A. J. J., Maas, A. I. R., Farace, E., Habbema, J. D. F., Marmarou, A., Marshall, L. F., Murray, G. D., & Steyerberg, E. W. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.06.009 · Cited by 77
Vieira, V. M., Webster, T. F., Weinberg, J., et al. (2005) · Environmental Health · doi:10.1186/1476-069x-4-11 · Cited by 77
Straíf, K. (2000) · American Journal of Epidemiology · doi:10.1093/aje/152.4.297 · Cited by 77
“All analyses were restricted to cohort members with complete information on all variables included in the respective model (18).”
MethodMertens, A., Balakrishnan, K., Ramaswamy, P., Rajkumar, P., Ramaprabha, P., Natesan, D., Hubbard, A., Khush, R. S., Colford, J. M., & Arnold, B. F. (2019) · Environmental Health Perspectives · doi:10.1289/ehp3711 · Cited by 82
Metadata
- Authors
- Greenland, S., & Finkle, W. D.
- Affiliations
- University of California, Los Angeles (US)
- Corresponding author
- Greenland, Sander
- ORCID
- Greenland, Sander
- Venue
- American Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 1995-12-15
- Volume
- 142
- Issue
- 12
- Pages
- 1255-1264
- ISSN
- 0002-9262 (ISSN-L)
- 1476-6256
- OpenAlex
- W2166561686
- Cited by
- 891
- References
- 0
- Citations by year
- 2026: 13 · 2025: 18 · 2024: 13 · 2023: 24 · 2022: 26 · 2021: 21 · 2020: 33 · 2019: 29 · 2018: 34 · 2017: 32 · 2016: 27 · 2015: 40 · 2014: 57 · 2013: 53 · 2012: 67
- FWCI
- 6.57
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- American Journal of Epidemiology
- Funders
- National Institutes of Health (n01cp095680-001)
- National Cancer Institute (N01 CP 95680)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- missing covariates, missing data, logistic regression, bias, model misspecification, multiple imputation, maximum likelihood estimation, weighted estimating equations, endometrial cancer, age at menarche, case-control study
- MeSH terms
- Epidemiologic Methods, Female, Humans, Menarche, Research Design, Logistic Models, Odds Ratio, Case-Control Studies, Endometrial Neoplasms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
