The Analysis of Social Science Data with Missing Values
Abstract
Methods for handling missing data in social science data sets are reviewed. Limitations of common practical approaches, including complete-case analysis, available-case analysis and imputation, are illustrated on a simple missing-data problem with one complete and one incomplete variable. Two more principled approaches, namely maximum likelihood under a model for the data and missing-data mechanism and multiple imputation, are applied to the bivariate problem. General properties of these methods are outlined, and applications to more complex missing-data problems are discussed. The EM algorithm, a convenient method for computing maximum likelihood estimates in missing-data problems, is described and applied to two common models, the multivariate normal model for continuous data and the multinomial model for discrete data. Multiple imputation under explicit or implicit models is recommended as a method that retains the advantages of imputation and overcomes its limitations.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Linear mixed models permit valid inferences when the missing data mechanism is missing at random or missing completely at random.
“LMM allows for valid inferences under the assumption that the missing mechanism is missing at random or missing completely at random (Little & Rubin, 1989).”
From Academic Achievement Trajectories of Homeless and Highly Mobile Students: Resilience in the Context of Chronic and Acute Risk (2013) · cites this paper as Little & Rubin, 1989
Compared with traditional missing data methods, maximum likelihood estimation yields more efficient and less biased parameters even when the assumption of data being missing at random is not strictly satisfied.
“Compared with traditional missing data methods, ML yields more efficient and less biased parameters even when the assumption of data being missing at random is not strictly satisfied (Little and Rubin, 1989).”
From Efficacy and mechanisms of mobile application-delivered Acceptance and Commitment Therapy for posttraumatic stress disorder in China: Study protocol for a randomized controlled trial (2022) · cites this paper as Little and Rubin, 1989
Cites
Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444 · see citing works
Self-citationOn structural equation modeling with data that are not missing completely at random
Muthén, B., Kaplan, D., & Hollis, M. (1987) · Psychometrika · doi:10.1007/bf02294365 · Cited by 883 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013) · Educational and Psychological Measurement · doi:10.1177/0013164413495237 · Cited by 3,931 · see citing works
Multiple imputation after 18+ years
Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476908 · Cited by 4,072 · see citing works
Self-citationLongitudinal associations between teen dating violence victimization and adverse health outcomes
Exner-Cortens, D., Eckenrode, J., & Rothman, E. (2013) · Pediatrics · doi:10.1542/peds.2012-1029 · Cited by 837 · see citing works
Reporting Analyses of Covariance Structures
Boomsma, A. (2000) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0703_6 · Cited by 655 · see citing works
Comparison of performance of data imputation methods for numeric dataset
Jadhav, A., Pramod, D., & Ramanathan, K. (2019) · Applied Artificial Intelligence · doi:10.1080/08839514.2019.1637138 · Cited by 471 · see citing works
Cole, D. A., Maxwell, S. E., Martin, J. M., Peeke, L. G., Seroczynski, A. D., Tram, J. M., Hoffman, K. B., Ruiz, M. D., Jacquez, F., & Maschman, T. (2001) · Child Development · doi:10.1111/1467-8624.00375 · Cited by 368 · see citing works
Schleider, J., & Weisz, J. (2018) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12811 · Cited by 320 · see citing works
Missing Data in Multiple Item Scales: A Monte Carlo Analysis of Missing Data Techniques
Roth, P. L., Switzer, F. S., & Switzer, D. M. (1999) · Organizational Research Methods · doi:10.1177/109442819923001 · Cited by 344 · see citing works
Cutuli, J. J., Desjardins, C. D., Herbers, J. E., Long, J. D., Heistad, D., Chan, C.-K., Hinz, E., & Masten, A. S. (2013) · Child Development · doi:10.1111/cdev.12013 · Cited by 235 · see citing works
“LMM allows for valid inferences under the assumption that the missing mechanism is missing at random or missing completely at random (Little & Rubin, 1989).”
Eiden, R. D., Edwards, E. P., & Leonard, K. E. (2007) · Developmental Psychology · doi:10.1037/0012-1649.43.5.1187 · Cited by 229 · see citing works
Bernaards, C. A., & Sijtsma, K. (2000) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3503_03 · Cited by 177 · see citing works
Intention-to-treat approach to data from randomized controlled trials: a sensitivity analysis
Wright, C. C., & Sim, J. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00155-0 · Cited by 171 · see citing works
Acosta, J., Chinman, M., Ebener, P., Malone, P. S., Phillips, A., & Wilks, A. (2019) · Journal of Youth and Adolescence · doi:10.1007/s10964-019-01013-2 · Cited by 166 · see citing works
Frequencies and patterns of adverse childhood events in LGBTQ+ youth
Craig, S. L., Austin, A., Levenson, J., Leung, V. W. Y., Eaton, A. D., & D’Souza, S. A. (2020) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2020.104623 · Cited by 164 · see citing works
Craig, S. L., & Austin, A. (2016) · Children and Youth Services Review · doi:10.1016/j.childyouth.2016.02.022 · Cited by 158 · see citing works
A multimethod examination of the stability of depressive symptoms in childhood and adolescence
Tram, J. M., & Cole, D. A. (2006) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.115.4.674 · Cited by 115 · see citing works
Andrews, P. W., Kornstein, S. G., Halberstadt, L. J., Gardner, C. O., & Neale, M. C. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00159 · Cited by 107 · see citing works
Pretreatment Attrition in a Comparative Treatment Outcome Study on Panic Disorder
Hofmann, S. G., Barlow, D. H., Papp, L. A., Detweiler, M. F., Ray, S. E., Shear, M. K., Woods, S. W., & Gorman, J. M. (1998) · American Journal of Psychiatry · doi:10.1176/ajp.155.1.43 · Cited by 93 · see citing works
Cole, D. A., & Martin, N. C. (2005) · Psychological Assessment · doi:10.1037/1040-3590.17.2.144 · Cited by 79 · see citing works
CopyMean: A new method to predict monotone missing values in longitudinal studies
Genolini, C., Lacombe, A., Écochard, R., & Subtil, F. (2016) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2016.04.010 · Cited by 52 · see citing works
Can family affectedness inform infant sibling outcomes of autism spectrum disorders?
Schwichtenberg, A. J., Young, G. S., Sigman, M., Hutman, T., & Ozonoff, S. (2010) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2010.02267.x · Cited by 67 · see citing works
Rhode, M. P., & Arriaza, B. T. (2006) · American Journal of Physical Anthropology · doi:10.1002/ajpa.20333 · Cited by 61 · see citing works
Missing data techniques for multilevel data: implications of model misspecification
Black, A. C., Harel, O., & Betsy McCoach, D. (2011) · Journal of Applied Statistics · doi:10.1080/02664763.2010.529882 · Cited by 55 · see citing works
Aderka, I. M., McLean, C. P., Huppert, J. D., Davidson, J. R. T., & Foa, E. B. (2013) · Behaviour Research and Therapy · doi:10.1016/j.brat.2013.03.007 · Cited by 49 · see citing works
Garner, B. R., Gotham, H. J., Tueller, S. J., Ball, E. L., Kaiser, D., Stilen, P., Speck, K., Vandersloot, D., Rieckmann, T. R., Chaple, M., et al. (2017) · Addiction Science & Clinical Practice · doi:10.1186/s13722-017-0095-8 · Cited by 31 · see citing works
Craig, S. L., & Furman, E. (2018) · Journal of Social Service Research · doi:10.1080/01488376.2018.1436631 · Cited by 25 · see citing works
Mechanisms of Mindfulness: A Longitudinal Study of a Mindfulness-Based Stress Reduction Program
Davis, K. M., Wojcik, C. M., Baillie, A. J., Foley, E., Goddard, T., Lau, M. A., & Haigh, E. A. P. (2024) · Mindfulness · doi:10.1007/s12671-024-02359-w · Cited by 25 · see citing works
Social Workers in Israel: Daily Stressors, Work Benefits, Burnout and Well-Being
Savaya, R., Levin, L., & Roziner, I. (2021) · The British Journal of Social Work · doi:10.1093/bjsw/bcaa087 · Cited by 20 · see citing works
Colaianne, B. A., Lavelle, B. D., Small, M. L., & Roeser, R. W. (2023) · Journal of Research on Adolescence · doi:10.1111/jora.12795 · Cited by 14 · see citing works
Davis, H. A., Ortiz, A. M. L., & Smith, G. T. (2018) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsx113 · Cited by 14 · see citing works
Linking psychological need experiences to daily and recurring dreams
Weinstein, N., Campbell, R., & Vansteenkiste, M. (2018) · Motivation and Emotion · doi:10.1007/s11031-017-9656-0 · Cited by 14 · see citing works
Kenter, R. M., van Straten, A., Hobbel, S., Smit, F., Bosmans, J., Beekman, A., & Cuijpers, P. (2013) · Trials · doi:10.1186/1745-6215-14-412 · Cited by 13 · see citing works
Graham, S., Irving, J., Cano, I., & Edwards, M. (2018) · Alcoholism Treatment Quarterly · doi:10.1080/07347324.2018.1500873 · Cited by 13 · see citing works
Zhao, Z., Zhao, C., Ren, Z., Shi, C., & Lai, L. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100585 · Cited by 10 · see citing works
“Compared with traditional missing data methods, ML yields more efficient and less biased parameters even when the assumption of data being missing at random is not strictly satisfied (Little and Rubin, 1989).”
Method
- Authors
- LITTLE, R. J. A., & RUBIN, D. B.
- Affiliations
- University of California, Los Angeles (US)
- Harvard University (US)
- Venue
- Sociological Methods & Research
- Publisher
- SAGE Publications
- Published
- 1989-11
- Volume
- 18
- Issue
- 2-3
- Pages
- 292-326
- ISSN
- 0049-1241 (ISSN-L)
- 1552-8294
- OpenAlex
- W2155725368
- Cited by
- 1,016
- References
- 2
- Citations by year
- 2026: 14 · 2025: 32 · 2024: 42 · 2023: 42 · 2022: 63 · 2021: 73 · 2020: 53 · 2019: 50 · 2018: 39 · 2017: 44 · 2016: 53 · 2015: 59 · 2014: 42 · 2013: 34 · 2012: 33
- FWCI
- 6.12
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Sociological Methods & Research (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Applications
- Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
- Keywords
- missing data, missing data mechanisms, multiple imputation, maximum likelihood estimation, expectation-maximization algorithm, complete case analysis, available case analysis, imputation, multivariate normal model, multinomial model, social science data analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
