The DOI matches its Crossref record.
A critical look at methods for handling missing covariates in epidemiologic regression analyses.
Abstract
Epidemiologic studies often encounter missing covariate values. While simple methods such as stratification on missing-data status, conditional-mean imputation, and complete-subject analysis are commonly employed for handling this problem, several studies have shown that these methods can be biased under reasonable circumstances. The authors review these results in the context of logistic regression and present simulation experiments showing the limitations of the methods. The method based on missing-data indicators can exhibit severe bias even when the data are missing completely at random, and regression (conditional-mean) imputation can be inordinately sensitive to model misspecification. Even complete-subject analysis can outperform these methods. More sophisticated methods, such as maximum likelihood, multiple imputation, and weighted estimating equations, have been given extensive attention in the statistics literature. While these methods are superior to simple methods, they are not commonly used in epidemiology, no doubt due to their complexity and the lack of packaged software to apply these methods. The authors contrast the results of multiple imputation to simple methods in the analysis of a case-control study of endometrial cancer, and they find a meaningful difference in results for age at menarche. In general, the authors recommend that epidemiologists avoid using the missing-indicator method and use more sophisticated methods whenever a large proportion of data are missing.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Commonly missing values are related to observed participant or disease characteristics.”
From Reporting and Methods in Clinical Prediction Research: A Systematic Review (2012) · cites this paper as [24]
In the citing paper
“Missing values in clinical study data rarely occur completely at random.”
From Reporting and Methods in Clinical Prediction Research: A Systematic Review (2012) · cites this paper as [24]
In the citing paper
“Exclusion of participants with missing values will therefore not only lead to loss of statistical power, but often also to biased results 10,12,13,23,24,57,58.”
From Reporting and Methods in Clinical Prediction Research: A Systematic Review (2012) · cites this paper as [24]
Citations
Cited by
Sorted by citations
Items 201–300 of 892
Li, T., Hutfless, S. M., Scharfstein, D. O., Daniels, M. J., Hogan, J. W., Little, R. J. A., Roy, J. A., Law, A. H., & Dickersin, K. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.08.013 · Cited by 79
Herpetic Eye Disease Study Group (2000) · Archives of Ophthalmology · doi:10.1001/archopht.118.12.1617 · Cited by 74
“Available-case and complete-case methods of analysis with missing data were used for the univariate and multivariable analyses, respectively.(31,32) No appreciable differences were observed when analysis was restricted to participants with 80% or more weekly log completion.”
MethodBradshaw, P. T., Ibrahim, J. G., Khankari, N. K., Cleveland, R. J., Abrahamson, P. E., Stevens, J., Satia, J. A., Teitelbaum, S. L., Neugut, A. I., & Gammon, M. D. (2014) · Breast Cancer Research and Treatment · doi:10.1007/s10549-014-2966-y · Cited by 75
“The first assumes a non-ignorable missing data mechanism (labeled Model 2 in results), which included a logistic regression model for the probability of PA data being observed as a function of age and previous PA, which accounts for the potential for the data to be not missing at random (NMAR)—that the probability tha…”
BackgroundMethodInfluentialAli, A. M., Dawson, S.-J., Blows, F. M., Provenzano, E., Ellis, I. O., Baglietto, L., Huntsman, D. D., Caldas, C., & Pharoah, P. D. P. (2011) · British Journal of Cancer · doi:10.1038/sj.bjc.6606078 · Cited by 75
“When missing data are not MCAR, single conditional imputation tends to result in less biased parameter estimates than CCA or MS, but the standard errors will be underestimated if the data are treated as a dataset with no missing values (Greenland and Finkle, 1995; Van der Heijden et al, 2006).”
BackgroundResultInfluentialAddressing Missing Data in Clinical Studies of Kidney Diseases
Montez‐Rath, M. E., Winkelmayer, W. C., & Desai, M. (2014) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.10141013 · Cited by 75
“Because no special software is required for its use, complete case analysis is the most common approach to handling missing data (4,5).”
MethodNewgard, C. D., Malveau, S. E., Staudenmayer, K., et al. (2012) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2012.01324.x · Cited by 74
Ridsdale, L., Darbishire, L., & Seed, P. T. (2004) · Psychological Medicine · doi:10.1017/s0033291703001247 · Cited by 73
“For the sensitivity analysis the difference in outcome scores between the two therapy groups were calculated using an additional four methods (Heckman, 1979; Little & Rubin, 1987; Greenland & Finkle, 1995).”
MethodHow can I deal with missing data in my study?
Bennett, D. (2001) · Australian and New Zealand Journal of Public Health · doi:10.1111/j.1467-842x.2001.tb00659.x · Cited by 74
The effect of California’s paid family leave policy on parent health: a quasi-experimental study.
Lee, B. C., Modrek, S., White, J. S., Batra, A., Collin, D. F., & Hamad, R. (2020) · Social Science & Medicine · doi:10.1016/j.socscimed.2020.112915 · Cited by 73
Kelder, J. C., Cowie, M., McDonagh, T. A., Hardman, S. M. C., Grobbee, D. E., Cost, B., & Hoes, A. W. (2011) · Heart · doi:10.1136/hrt.2010.220426 · Cited by 73
“Standard univariable analyses were performed. For the multivariable regression analyses we used logistic regression analyses with heart failure as the dependent variable and signs, symptoms and BNP levels (and other potential additional diagnostic tests, such as an ECG) as independent variables. When values are missin…”
MethodImputation approaches for potential outcomes in causal inference
Westreich, D., Edwards, J. K., Cole, S. R., Platt, R. W., Mumford, S. L., & Schisterman, E. F. (2015) · International Journal of Epidemiology · doi:10.1093/ije/dyv135 · Cited by 64
Screening differences and risk of cervical cancer in inflammatory bowel disease
Hutfless, S. M., FIREMAN, B. H., Kane, S. V., et al. (2008) · Alimentary Pharmacology & Therapeutics · doi:10.1111/j.1365-2036.2008.03766.x · Cited by 72
Helmhout, P. H., Staal, J. B., Heymans, M. W., Harts, C. C., Hendriks, E. J. M., & de Bie, R. A. (2010) · European Spine Journal · doi:10.1007/s00586-009-1254-8 · Cited by 71
“Several studies have shown that with multiple imputation valid results can be obtained when around 50% of the data are missing [21].”
BackgroundPredictors of Time to Death After Terminal Withdrawal of Mechanical Ventilation in the ICU
Cooke, C. R., Hotchkin, D. L., Engelberg, R. A., Rubinson, L., & Curtis, J. R. (2010) · CHEST Journal · doi:10.1378/chest.10-0289 · Cited by 70
Prediction of bacterial meningitis in children with meningeal signs: reduction of lumbar punctures
Oostenbrink, R., Moons, K. G. M., Donders, A. R. T., Grobbee, D. E., & Moll, H. (2001) · Acta Paediatrica · doi:10.1111/j.1651-2227.2001.tb02421.x · Cited by 74
“a record of a seriously sick patient may be more complete than that of a less sick patient), such that the “complete cases” would re ect a selected sample of children with meningeal signs (20).”
BackgroundMethodSocial Networks and Subjective Well-Being: A Comparison of Australia, Britain, and China
Bian, Y., Hao, M., & Li, Y. (2017) · Journal of Happiness Studies · doi:10.1007/s10902-017-9926-2 · Cited by 70
Oudega, R. (2004) · Family Practice · doi:10.1093/fampra/cmh718 · Cited by 69
O’Leary, E., Vena, J. E., Freudenheim, J. L., & Brasure, J. (2003) · Environmental Research · doi:10.1016/j.envres.2003.08.001 · Cited by 70
de Barse, L. M., Tiemeier, H. W., Leermakers, E. T. M., Voortman, T., Jaddoe, V. W. V., Edelson, L. R., Franco, O. H., & Jansen, P. W. (2015) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-015-0313-2 · Cited by 69
“multiple imputation techniques [37].”
Taylor, J. M. G. (2002) · American Journal of Epidemiology · doi:10.1093/aje/kwf110 · Cited by 69
Late response and item nonresponse in the Finbalt Health Monitor Survey
Helasoja, V. (2002) · European Journal of Public Health · doi:10.1093/eurpub/12.2.117 · Cited by 70
Bleeker, S., Derksen‐Lubsen, G., Grobbee, D. E., et al. (2006) · Acta Paediatrica · doi:10.1111/j.1651-2227.2006.00033.x · Cited by 69
“Taking into consideration uncertainties in imputed data, multiple imputations were performed by repeating the imputation five times (25).”
MethodPhysician referrals to physical therapists for the treatment of spine disorders
Freburger, J. K., Carey, T. S., & Holmes, G. M. (2005) · The Spine Journal · doi:10.1016/j.spinee.2005.03.008 · Cited by 69
Caregiving Stress, Endogenous Sex Steroid Hormone Levels, and Breast Cancer Incidence
Kroenke, C. H. (2004) · American Journal of Epidemiology · doi:10.1093/aje/kwh148 · Cited by 69
“To include women for whom information was missing and to reduce bias, we used the Greenland and Finkle (41) regression method in analyses of certain continuous caregiving variables and for continuously analyzed reproductive covariates.”
MethodCoolman, M., Timmermans, S., de Groot, C. J. M., Russcher, H., Lindemans, J. W., Albert Hofman, D., Geurts-Moespot, A. J., Sweep, F. C. G. J., Jaddoe, V. V. W., & Steegers, E. A. P. (2012) · Obstetrics and Gynecology · doi:10.1097/aog.0b013e318256187f · Cited by 71
Maslova, E., Hansen, S., Strøm, M., Halldorsson, T. I., & Olsen, S. F. (2013) · British Journal Of Nutrition · doi:10.1017/s0007114513003395 · Cited by 67
“1 % (socioeconomic status, parity, pre-pregnancy BMI, parental history of asthma and allergies, maternal physical activity and breast-feeding); if #1 %, then the missing observations were deleted((26)).”
BackgroundRovers, M. M., Glasziou, P., Appelman, C. L., Burke, P., McCormick, D. P., Damoiseaux, R. A. M. J., Little, P. S., Saux, N. L., & Hoes, A. W. (2007) · PEDIATRICS · doi:10.1542/peds.2006-2092 · Cited by 64
Kappen, T. H., Moons, K. G. M., van Wolfswinkel, L., Kalkman, C. J., Vergouwe, Y., & van Klei, W. A. (2013) · Anesthesiology · doi:10.1097/aln.0000000000000009 · Cited by 66
Predicting serious bacterial infection in young children with fever without apparent source
Moons, K. G. M. (2001) · Acta Paediatrica · doi:10.1080/080352501317130236 · Cited by 67
Advanced Statistics: Missing Data in Clinical Research—Part 2: Multiple Imputation
Newgard, C. D., & Haukoos, J. S. (2007) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2007.tb01856.x · Cited by 67
Gobardhan, P. D., Elias, S. G., Madsen, E. V. E., et al. (2008) · Annals of Oncology · doi:10.1093/annonc/mdn535 · Cited by 66
Schwenkglenks, M. M., Pettengell, R., Jackisch, C., Paridaens, R., Constenla, M., Bosly, A., Szucs, T. D., & Leonard, R. C. (2010) · Supportive Care in Cancer · doi:10.1007/s00520-010-0840-y · Cited by 65
“ity analyses were performed to address concerns that this approach can lead to biased estimation results [ 13 , 14].”
MethodPrognosis after temporal lobe epilepsy surgery: The value of combining predictors
Uijl, S. G., Leijten, F. S. S., Arends, J. B. A. M., et al. (2008) · Epilepsia · doi:10.1111/j.1528-1167.2008.01695.x · Cited by 66
“As some values were missing, and missing values usually do not occur at random, we imputed the missing values to prevent bias, using single imputation by linear regression with the addition of a random error term (Greenland & Finkle, 1995; Donders et al., 2006).”
BackgroundMethodNewgard, C. D., Mann, N. C., Hsia, R. Y.-J., et al. (2013) · Academic Emergency Medicine · doi:10.1111/acem.12213 · Cited by 66
Toh, S., Gagne, J. J., Rassen, J. A., FIREMAN, B. H., Kulldorff, M., & Brown, J. S. (2013) · Medical Care · doi:10.1097/mlr.0b013e31829b1bb1 · Cited by 65
“Some, like the complete-case analysis and the missing indicator approach, are easy to implement but are valid only under very strong assumptions.(70,71) Investigators should consider methods that require weaker assumptions, such as multiple imputation,(72,73) inverse probability weighting,(74) and PS calibration.”
MethodDynamic Effects of Depressive Symptoms on Osteoarthritis Knee Pain
Rathbun, A. M., Stuart, E. A., Shardell, M. D., Yau, M. S., Baumgarten, M., & Hochberg, M. C. (2017) · Arthritis Care & Research · doi:10.1002/acr.23239 · Cited by 66
A broader perspective on education and mortality: Are we influenced by other people's education?
Kravdal, Ø. (2007) · Social Science & Medicine · doi:10.1016/j.socscimed.2007.10.009 · Cited by 64
Regular dental visits are associated with earlier stage at diagnosis for oral and pharyngeal cancer
Langevin, S. M., Michaud, D. S., Eliot, M., Peters, E. S., McClean, M. D., & Kelsey, K. T. (2012) · Cancer Causes & Control · doi:10.1007/s10552-012-0061-4 · Cited by 64
“To compensate for the missing race and household income values, multiple imputation was employed using multivariate normal regression, based on age, sex, and education data; multiple imputation results in less biased findings when dealing with missing covariate data [27].”
MethodGroen, E. H., Lingsma, H. F., Caudri, D., Levie, D., Wijga, A. H., Koppelman, G. H., Duijts, L., Jaddoe, V. W. V., Smit, H. A., Kerkhof, M., et al. (2013) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2013.07.007 · Cited by 65
Prenatal maternal psychopathology and stress and offspring HPA axis function at 6 years
Molenaar, N. M., Tiemeier, H. W., van Rossum, E. F. C., Hillegers, M. H. J., Bockting, C., Hoogendijk, W. J. G., van den Akker, E. L., den Berg, M. P. L., & Marroun, H. E. (2018) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2018.09.003 · Cited by 68
“As recommended, imputations were based on the relations between all variables in the study (Greenland and Finkle, 1995).”
Cremer, O. L., Moons, K. G. M., van Dijk, G. W., van Balen, P., & Kalkman, C. J. (2006) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/01.ta.0000195981.63776.ba · Cited by 65
Lapane, K. L., Yang, S., Driban, J. B., Liu, S., Dube, C. E., McAlindon, T. E., & Eaton, C. B. (2014) · Arthritis & Rheumatology · doi:10.1002/art.38933 · Cited by 64
“2%), we carried the last observation forward (28).”
Prediction of vesico‐ureteric reflux in childhood urinary tract infection: a multivariate approach
Oostenbrink, R., van der Heijden, A., Moons, K., et al. (2000) · Acta Paediatrica · doi:10.1111/j.1651-2227.2000.tb00380.x · Cited by 64
“These data may not be missing at random, since precision of documenting information may depend on the seriousness of the clinical presentation of the patient (24).”
BackgroundPreschool diet and adult risk of breast cancer
Michels, K. B., ROSNER, B. A., Chumlea, W. C., Colditz, G. A., & Willett, W. C. (2005) · International Journal of Cancer · doi:10.1002/ijc.21407 · Cited by 62
“The multiple imputation method used in the present analysis does not involve sampling of the parameter values in the imputation model but assumes that the parameter estimates are known without error, and are therefore not changed at each imputation by adding error to them.(44)”
MethodWijtzes, A. I., Jansen, W., Jansen, P. W., Jaddoe, V. W. V., Albert Hofman, D., & Raat, H. (2013) · Preventive Medicine · doi:10.1016/j.ypmed.2013.08.014 · Cited by 63
Development of socioeconomic inequalities in obesity among Dutch pre-school and school-aged children
Bouthoorn, S. H., Wijtzes, A. I., Jaddoe, V. W. V., Albert Hofman, D., Raat, H., & van Lenthe, F. J. (2014) · Obesity · doi:10.1002/oby.20843 · Cited by 62
“Five imputed datasets were created and analyzed together (25).”
MethodKoenen, K. C., Vivo, I. D., Rich-Edwards, J. W., Smoller, J. W., Wright, R. J., & Purcell, S. (2009) · BMC Psychiatry · doi:10.1186/1471-244x-9-29 · Cited by 64
“[116-118]”
The Toronto score for in-hospital mortality after percutaneous coronary interventions
Chowdhary, S., Ivanov, J., Mackie, K., et al. (2009) · American Heart Journal · doi:10.1016/j.ahj.2008.08.026 · Cited by 62
Mortality and Predictors of Mortality in a Cohort of Brazilian Type 2 Diabetic Patients
Salles, G. F., Bloch, K. V., & Cardoso, C. R. L. (2004) · Diabetes Care · doi:10.2337/diacare.27.6.1299 · Cited by 62
Angermann, C. E., Störk, S., Costard‐Jäckle, A., et al. (2004) · European Heart Journal · doi:10.1016/j.ehj.2004.06.032 · Cited by 62
Zhu, X. (2014) · Open Journal of Statistics · doi:10.4236/ojs.2014.411088 · Cited by 59
Chaudru, V., Laud, K., Avril, M., Minière, A., Chompret, A., Paillerets, B. B., Démenais, F., & The French Familial Melanoma Study Group (2005) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-04-0777 · Cited by 61
“To deal with missing covariates, we used the complete case method, which implies coding subjects with missing covariates as unknown for disease status (29).”
Methodvan den Beukel, T. O., de Goeij, M. C. M., Dekker, F. W., Siegert, C. E. H., & Halbesma, N. (2013) · Clinical Journal of the American Society of Nephrology · doi:10.2215/cjn.10761012 · Cited by 60
Dijk, J. M., van der Graaf, Y., Grobbee, D. E., et al. (2004) · Stroke · doi:10.1161/01.str.0000130513.77186.26 · Cited by 59
Predicting serious bacterial infection in young children with fever without apparent source
Bleeker, S., Moons, K., Derksen‐Lubsen, G., et al. (2001) · Acta Paediatrica · doi:10.1111/j.1651-2227.2001.tb01566.x · Cited by 59
“Taking into consideration uncertainties in imputed data, multiple imputations were performed by repeating the imputation ve times (28).”
MethodDiagnostic research on routine care data
Oostenbrink, R., Moons, K. G. M., Bleeker, S. E., Moll, H. A., & Grobbee, D. E. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00080-5 · Cited by 59
“Whether missingness depends on unobserved variables, however, can obviously not be assessed from the observed data but can only be judged [56–58].”
BackgroundMethodResultInfluentialIncidence and predictors of hepatocellular carcinoma in patients with autoimmune hepatitis
Colapietro, F., Maisonneuve, P., Lytvyak, E., Beuers, U. H., Verdonk, R. C., van der Meer, A. J., van Hoek, B., Kuiken, S. D., Brouwer, J. T., Muratori, P., et al. (2023) · Journal of Hepatology · doi:10.1016/j.jhep.2023.09.010 · Cited by 60
Obesity and outcomes in patients hospitalized with pneumonia
Kahlon, S., Eurich, D. T., Padwal, R. S., et al. (2012) · Clinical Microbiology and Infection · doi:10.1111/j.1469-0691.2012.04003.x · Cited by 58
“We did not attempt to impute missing values for BMI [22].”
Kanwar, N., Scott, H. M., Norby, B., Loneragan, G., Vinasco, J., Cottell, J., Chalmers, G., Chengappa, M., Bai, J., & Boerlin, P. (2014) · Scientific Reports · doi:10.1038/srep05100 · Cited by 49
Berthiller, J., Straíf, K., Agudo, A., Ahrens, W., dos Santos, A. B., Boccia, S., Cadoni, G., Canova, C., Castellsagué, X., Chen, C., et al. (2015) · International Journal of Epidemiology · doi:10.1093/ije/dyv146 · Cited by 57
Terada, T., Johnson, J. A., Norris, C. M., et al. (2016) · Journal of the American Heart Association · doi:10.1161/jaha.116.003282 · Cited by 58
“In addition, inclusion of “missing” and “not completed” as separate categories for analysis may have influenced our estimates.(51) Nonetheless, although some patients had missing data in a few covariate categories, the main predictor (ie, BMI category), outcomes (ie, mortality, complications, and LOS), and the other c…”
ResultAn, U., Pazokitoroudi, A., Alvarez, M., Huang, L., Bacanu, S., Schork, A. J., Kendler, K. S., Pajukanta, P., Flint, J., Zaitlen, N., et al. (2023) · Nature Genetics · doi:10.1038/s41588-023-01558-w · Cited by 60
“We considered K-Nearest Neighbors (KNN), missForest 19 and MICE 3 , missing data 1–7 (see Supplementary Note Section S1 for additional related work).”
MethodA Framework for Investigating Change over Time
Singer, J. D., & Willett, J. B. (2003) · doi:10.1093/acprof:oso/9780195152968.003.0001 · Cited by 59
Christensen, K., Frederiksen, H., Vaupel, J. W., & McGue, M. K. (2003) · Behavior Genetics · doi:10.1023/a:1022501817781 · Cited by 56
“This treatment of missing data has been shown to produce more precise estimates of parameters than analysis of summary variance-covariance matrices and mean vectors where cases were deleted when even one observation was missing (Greenland and Finkle, 1995).”
Thygesen, L. C., Johansen, C., Keiding, N., Giovannucci, E. L., & Grønbæk, M. (2008) · Addiction · doi:10.1111/j.1360-0443.2008.02241.x · Cited by 58
“Thirdly, we used multiple imputation by generating 10 copies of the original data set, in each of which missing values for all covariates except alcohol intake were replaced by values generated randomly according to a linear regression model with self-reported covariates and register-based covariates the year before p…”
MethodMoore, E. E., Romaniuk, H., Olsson, C. A., Jayasinghe, Y., Carlin, J. B., & Patton, G. C. (2010) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2010.01.004 · Cited by 56
Using decision trees to understand structure in missing data
Tierney, N. J., Harden, F. A., Harden, M. J., & Mengersen, K. L. (2015) · BMJ Open · doi:10.1136/bmjopen-2014-007450 · Cited by 58
Steele, F., Kallis, C., & Joshi, H. E. (2006) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2006.00420.x · Cited by 56
“This approach may introduce bias into estimates of their effects on partnership transitions (Greenland and Finkle, 1995).”
BackgroundUnsafe from the Start: Serious Misuse of Car Safety Seats at Newborn Discharge
Hoffman, B. D., Gallardo, A. R., & Carlson, K. F. (2015) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2015.11.047 · Cited by 56
“Results based on imputed data are presented herein; for comparison, we also conducted complete case analyses, and found no meaningful differences in overall outcomes.(20) All analyses were performed using SAS version 9.”
MethodHensor, E. M. A., McKeigue, P., Ling, S. F., Colombo, M., Barrett, J. H., Nam, J. L., Freeston, J. E., Buch, M. H., Spiliopoulou, A., Agakov, F. V., et al. (2019) · Lara D. Veeken · doi:10.1093/rheumatology/kez049 · Cited by 56
Prenatal Exposures and Glucose Metabolism in Adulthood
Thomas, C., Hyppönen, E., & Power, C. (2007) · Diabetes Care · doi:10.2337/dc06-1881 · Cited by 55
“As recommended, 10 copies of the original dataset imputed for missing data were created, and a regression analysis combining the results from each dataset was undertaken (24).”
MethodGrundmeier, R. W., Song, L., Ramos, M. J., Fiks, A. G., Elliott, M. N., Fremont, A., Pace, W., Wasserman, R. C., & Localio, R. (2015) · Health Services Research · doi:10.1111/1475-6773.12295 · Cited by 55
“The missing indicator method— using a missing category indicator for missing variables such as race/ethnicity —also produces biased estimates and (Greenland and Finkle 1995; Knol et al. 2010).”
MethodBrown, C. R. L., Dodds, L. A., Attenborough, R., Bryanton, J., Rose, A. E., Flowerdew, G. J., Langille, D. B., Lauzon, L., & Semenic, S. E. (2013) · CMAJ Open · doi:10.9778/cmajo.20120011 · Cited by 55
“Fifth, the creation of a dummy “missing” category is not an ideal way to treat missing covariates, because it tends to underestimate uncertainty in some analyses where sample size is inflated, while overestimating uncertainty in other analyses because the degrees of freedom is increased.(29) In our study, however, a c…”
BackgroundInverse-Probability-Weighted Estimation for Monotone and Nonmonotone Missing Data
Sun, B., Perkins, N. J., Cole, S. R., et al. (2017) · American Journal of Epidemiology · doi:10.1093/aje/kwx350 · Cited by 55
“This is also referred to as the “stratifiedMAR assumption” as discussed by Greenland and Finkle (14), and it is strictly stronger than MAR assumption 1.”
BackgroundPetersen, I., Welch, C. A., Nazareth, I., Walters, K. R., Marston, L., Morris, R. W., Carpenter, J. R., Morris, T. P., & Pham, T. M. (2019) · Clinical Epidemiology · doi:10.2147/clep.s191437 · Cited by 55
“,36–38 Using weight measurements recorded for individuals who were registered with general practices contributing data to THIN in 2010, we found that both the observed weight measurements and missingness in weight were associated with sex, age, social deprivation, and disease status.”
BackgroundGuided Multiple Imputation of Missing Data
Fraser, G. E., & Yan, R. (2007) · Epidemiology · doi:10.1097/01.ede.0000254708.40228.8b · Cited by 55
“Unfortunately, in multivariate work this approach typically produces biased results.(2,3) Others have assumed that subjects who, for instance, skip a dietary question do so because they do not eat that food; a zero is then imputed.”
BackgroundQuinteros, M. E., Lu, S., Blazquez, C. A., Cárdenas-Ramírez, J. P., Ximena Ossa, G., Delgado-Saborit, J. M., Harrison, R. M., & Ruiz-Rudolph, P. (2018) · Atmospheric Environment · doi:10.1016/j.atmosenv.2018.11.053 · Cited by 55
Racial and Ethnic Disparities in Cancer Care During the COVID-19 Pandemic
Patel, M. I., Ferguson, J. M., Castro, E., Pereira-Estremera, C. D., Armaiz-Peña, G. N., Duron, Y., Hlubocky, F., Infantado, A., Nuqui, B., Julian, D., et al. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.22009 · Cited by 53
Bruintjes, T. D., Companjen, J., van der Zaag–Loonen, H. J., & van Benthem, P. P. G. (2014) · Clinical Otolaryngology · doi:10.1111/coa.12217 · Cited by 52
Metformin and cancer in type 2 diabetes: a systematic review and comprehensive bias evaluation
Farmer, R. E., Ford, D., Forbes, H. J., Chaturvedi, N., Kaplan, R. S., Smeeth, L. L., & Bhaskaran, K. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw275 · Cited by 53
“65 With these three studies having > 20% missing data, the effect of residual confounding could be large.”
BackgroundDavison, K., Lung, Y., Lin, S., Tong, H., Kobayashi, K. M., & Fuller‐Thomson, E. (2019) · BMC Psychiatry · doi:10.1186/s12888-019-2309-y · Cited by 55
Moscucci, M. M., O’Connor, G. T., Ellis, S. G., et al. (1999) · Journal of the American College of Cardiology · doi:10.1016/s0735-1097(99)00266-1 · Cited by 53
LightGBM: an efficient and accurate method for predicting pregnancy diseases
Liao, H., Zhang, X., Zhao, C., Chen, Y., Zeng, X., & Li, H. (2021) · Journal of Obstetrics and Gynaecology · doi:10.1080/01443615.2021.1945006 · Cited by 57
“For example, covariate data are often missing in epidemiologic studies (Greenland and Finkle 1995) and developing prognostic models (Clark and Altman 2003) because the patients transferred from one hospital to another, and did not accept relevant examination checks of lost records and other reasons.”
BackgroundPreoperative Hypernatremia Predicts Increased Perioperative Morbidity and Mortality
Leung, A. A. C., McAlister, F. A., Finlayson, S. R. G., & Bates, D. W. (2013) · The American Journal of Medicine · doi:10.1016/j.amjmed.2013.02.039 · Cited by 51
Propensity scores using missingness pattern information: a practical guide
Blake, H. A., Leyrat, C., Mansfield, K. E., Seaman, S. R., Tomlinson, L. A., Carpenter, J. R., & Williamson, E. (2020) · Statistics in Medicine · doi:10.1002/sim.8503 · Cited by 49
The determinants of undergraduate degree performance: How important is gender?
Barrow, M., Reilly, B., & Woodfield, R. (2009) · British Educational Research Journal · doi:10.1080/01411920802642322 · Cited by 51
“The approach is not entirely free of criticism as highlighted by the work of Greenland and Finkle (1995) Gender and undergraduate degree performance 583”
MethodKuo, W.-H., Yen, A. M., Lee, P.-H., et al. (2009) · British Journal of Cancer · doi:10.1038/sj.bjc.6604898 · Cited by 51
“We used the indicator method to deal with covariates with missing values in the Cox regression model (Greenland and Finkle, 1995).”
MethodMoons, K. G. M., Bots, M. L., Salonen, J. T., Elwood, P. C., de Concalves, A. F., Nikitin, Y., Sivenius, J., Inzitari, D., Benetou, V., Tuomilehto, J., et al. (2002) · Journal of Epidemiology & Community Health · doi:10.1136/jech.56.suppl_1.i30 · Cited by 51
Diabetes and Risk of Tuberculosis Relapse: Nationwide Nested Case-Control Study
Lee, P.-H., Lin, H., Huang, A., Wei, S.-H., Lai, M., & Lin, H.-H. (2014) · PLoS ONE · doi:10.1371/journal.pone.0092623 · Cited by 52
“Because of the small proportion of case-control pairs with missing covariates (16% of all pairs), we only included the pairs with complete information in the main multivariable analysis [22].”
BackgroundInpatient Hemodialysis Initiation: Reasons, Risk Factors and Outcomes
Crews, D. C., Jaar, B. G., Plantinga, L., et al. (2009) · Nephron Clinical Practice · doi:10.1159/000245066 · Cited by 50
“Statistical Analysis Baseline characteristics of the sample stratified by location of dialysis initiation were compared using χ 2 tests for categorical variables and t tests for continuous variables. Cox-proportional hazards regression was used to test the association between location of dialysis initiation and mortal…”
MethodContraceptive Use and Associated Factors among Women Enrolling into HIV Care in Southwestern Uganda
Muyindike, W. R., Fatch, R., Steinfield, R., et al. (2012) · Infectious Diseases in Obstetrics and Gynecology · doi:10.1155/2012/340782 · Cited by 50
“As this approach may introduce bias [33], we also imputed the missing values using multiple (five) imputations by chained equations in Stata and conducted multivariable logistic regression using this imputed dataset.”
MethodKrieger, N., Chen, J. T., Ware, J. H., & Kaddour, A. (2008) · Cancer Causes & Control · doi:10.1007/s10552-008-9202-1 · Cited by 52
“, sociodemographic and tumor characteristics), it is reasonable to posit that our use of multiple imputation was justified, given the key Missing At Random (MAR) criterion, whereby the probability of missingness depends only on variables that are observed [30, 31].”
QTc Interval Prolongation Is a Predictor of Future Strokes in Patients With Type 2 Diabetes Mellitus
Cardoso, C. R. L., Salles, G. F., & Deccache, W. (2003) · Stroke · doi:10.1161/01.str.0000085084.15144.66 · Cited by 50
Imputation methods to improve inference in SNP association studies
Dai, J. Y., Ruczinski, I., LeBlanc, M. L., & Kooperberg, C. L. (2006) · Genetic Epidemiology · doi:10.1002/gepi.20180 · Cited by 50
The Impact of Heart Failure on the Classification of COPD Severity
Güder, G., Rutten, F. H., Brenner, S., et al. (2012) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2012.05.008 · Cited by 49
“The imputation was based on the correlations between each variable with missing values and all other variables.(25) The count of missing values per variable was !20, except for diffusion capacity with carbon monoxide (n 5 125).”
The cusp catastrophe model as cross-sectional and longitudinal mixture structural equation models.
Chow, S., Witkiewitz, K. A., Grasman, R. P. P. P., & Maisto, S. A. (2015) · Psychological Methods · doi:10.1037/a0038962 · Cited by 50
Köse, T., Özgür, S., Coşgun, E., Keskınoğlu, A., & Keskinoğlu, P. (2020) · BioMed Research International · doi:10.1155/2020/1895076 · Cited by 49
“In contrast, if the observed data are highly predictive of the missing values, the imputations will be more consistent across imputations, resulting in smaller, but still accurate, standard errors (Figure 1) [33].”
ResultFarragher, T., Lunt, M., Plant, D., Bunn, D., Barton, A., & Symmons, D. (2010) · Arthritis care & research · doi:10.1002/acr.20207 · Cited by 53
Heritability of P300 amplitude development from adolescence to adulthood
Carlson, S. R., & Iacono, W. G. (2006) · Psychophysiology · doi:10.1111/j.1469-8986.2006.00450.x · Cited by 48
Surgical Site Infection in Orthopaedic Oncology
Gradl, G., de Witte, P. B., Evans, B. T., et al. (2014) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.l.01514 · Cited by 48
Metadata
- Authors
- Greenland, S., & Finkle, W. D.
- Affiliations
- University of California, Los Angeles (US)
- Corresponding author
- Greenland, Sander
- ORCID
- Greenland, Sander
- Venue
- American Journal of Epidemiology
- Publisher
- Oxford University Press (OUP)
- Published
- 1995-12-15
- Volume
- 142
- Issue
- 12
- Pages
- 1255-1264
- ISSN
- 0002-9262 (ISSN-L)
- 1476-6256
- OpenAlex
- W2166561686
- Cited by
- 891
- References
- 0
- Citations by year
- 2026: 13 · 2025: 18 · 2024: 13 · 2023: 24 · 2022: 26 · 2021: 21 · 2020: 33 · 2019: 29 · 2018: 34 · 2017: 32 · 2016: 27 · 2015: 40 · 2014: 57 · 2013: 53 · 2012: 67
- FWCI
- 6.57
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- American Journal of Epidemiology
- Funders
- National Institutes of Health (n01cp095680-001)
- National Cancer Institute (N01 CP 95680)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- missing covariates, missing data, logistic regression, bias, model misspecification, multiple imputation, maximum likelihood estimation, weighted estimating equations, endometrial cancer, age at menarche, case-control study
- MeSH terms
- Epidemiologic Methods, Female, Humans, Menarche, Research Design, Logistic Models, Odds Ratio, Case-Control Studies, Endometrial Neoplasms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
