Dealing with missing data in a multi-question depression scale: a comparison of imputation methods
Open accessAbstract
Background: Missing data present a challenge to many research projects. The problem is often pronounced in studies utilizing self-report scales, and literature addressing different strategies for dealing with missing data in such circumstances is scarce. The objective of this study was to compare six different imputation techniques for dealing with missing data in the Zung Self-reported Depression scale (SDS). Methods: 1580 participants from a surgical outcomes study completed the SDS. The SDS is a 20 question scale that respondents complete by circling a value of 1 to 4 for each question. The sum of the responses is calculated and respondents are classified as exhibiting depressive symptoms when their total score is over 40. Missing values were simulated by randomly selecting questions whose values were then deleted (a missing completely at random simulation). Additionally, a missing at random and missing not at random simulation were completed. Six imputation methods were then considered; 1) multiple imputation, 2) single regression, 3) individual mean, 4) overall mean, 5) participant's preceding response, and 6) random selection of a value from 1 to 4. For each method, the imputed mean SDS score and standard deviation were compared to the population statistics. The Spearman correlation coefficient, percent misclassified and the Kappa statistic were also calculated. Results: When 10% of values are missing, all the imputation methods except random selection produce Kappa statistics greater than 0.80 indicating 'near perfect' agreement. MI produces the most valid imputed values with a high Kappa statistic (0.89), although both single regression and individual mean imputation also produced favorable results. As the percent of missing information increased to 30%, or when unbalanced missing data were introduced, MI maintained a high Kappa statistic. The individual mean and single regression method produced Kappas in the 'substantial agreement' range (0.76 and 0.74 respectively). Conclusion: Multiple imputation is the most accurate method for dealing with missing data in most of the missind data scenarios we assessed for the SDS. Imputing the individual's mean is also an appropriate and simple method for dealing with missing data that may be more interpretable to the majority of medical readers. Researchers should consider conducting methodological assessments such as this one when confronted with missing data. The optimal method should balance validity, ease of interpretability for readers, and analysis expertise of the research team.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148 · see citing works
RUBIN, D. B. (1976) · Biometrika · doi:10.1093/biomet/63.3.581 · Cited by 9,925 · see citing works
A self-rating depression scale
ZUNG, W. W. K. (1965) · Archives of General Psychiatry · doi:10.1001/archpsyc.1965.01720310065008 · Cited by 9,813 · see citing works
Analysis of Incomplete Multivariate Data
Schafer, J. L. (1997) · doi:10.1201/9781439821862 · Cited by 6,012 · see citing works
Analysis of Incomplete Multivariate Data
Booth, D. E., & Schafer, J. L. (2000) · Technometrics · doi:10.2307/1271466 · Cited by 5,627 · see citing works
Heberlein, T. A., & Baumgartner, R. (1978) · American Sociological Review · doi:10.2307/2094771 · Cited by 999 · see citing works
Rubin, D. B. (1976) · Biometrika · doi:10.2307/2335739 · Cited by 946 · see citing works
Missing Data in Likert Ratings: A Comparison of Replacement Methods
Downey, R. G., & King, C. V. (1998) · The Journal of General Psychology · doi:10.1080/00221309809595542 · Cited by 657 · see citing works
Imputing Cross-Sectional Missing Data: Comparison of Common Techniques
Hawthorne, G., Hawthorne, G., & Elliott, P. (2005) · Australian & New Zealand Journal of Psychiatry · doi:10.1080/j.1440-1614.2005.01630.x · Cited by 382 · see citing works
On the Treatment of Grouped Observations in Life Studies
Thompson, W. A. (1977) · Biometrics · doi:10.2307/2529360 · Cited by 226 · see citing works
Zung Self-Rating Depression Scale and Depression Status Inventory
Zung, W. W. K. (1986) · Springer eBooks · doi:10.1007/978-3-642-70486-4_21 · Cited by 217 · see citing works
Functional Assessment of Cancer Therapy (FACT-G): Non-response to individual questions
Fairclough, D. L., & Cella, D. F. (1996) · Quality of Life Research · doi:10.1007/bf00433916 · Cited by 210 · see citing works
Faris, P., Ghali, W. A., Brant, R., Norris, C. M., Galbraith, P. D., & Knudtson, M. L. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00433-4 · Cited by 130 · see citing works
Self-citationMethods for Addressing Missing Data in Psychiatric and Developmental Research
Croy, C. D., & Novins, D. K. (2005) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/01.chi.0000181044.06337.6f · Cited by 109 · see citing works
Dealing with missing data in observational health care outcome analyses
Norris, C. M., Ghali, W. A., Knudtson, M. L., Naylor, C. D., & Saunders, L. D. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00181-x · Cited by 92 · see citing works
Self-citationImputation of missing values in the case of a multiple item instrument measuring alcohol consumption
Gmel, G. E. (2001) · Statistics in Medicine · doi:10.1002/sim.837 · Cited by 87 · see citing works
Mosconi, P., Torri, V., Cifani, S., Ruggiata, R., Meyerowitz, B. E., Apolone, G., Liberati, A., & Interdisciplinary Group for Cancer Care Evaluation (GIVIO) (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980315/15)17:5/7 · Cited by 17 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Gillis, C., Li, C., Lee, L., Awasthi, R., Augustin, B., Gamsa, A., Liberman, A. S., Stein, B., Charlebois, P., Feldman, L. S., et al. (2014) · Anesthesiology · doi:10.1097/aln.0000000000000393 · Cited by 876 · see citing works
Modular cognitive-behavioral therapy for body dysmorphic disorder: A randomized controlled trial
Wilhelm, S., Phillips, K. A., Didie, E., Buhlmann, U., Greenberg, J. L., Fama, J. M., Keshaviah, A., & Steketee, G. (2014) · Behavior Therapy · doi:10.1016/j.beth.2013.12.007 · Cited by 238 · see citing works
Peters, S. A. E., Bots, M. L., den Ruijter, H. M., Palmer, M. K., Grobbee, D. E., Crouse, J. R., O’Leary, D. H., Evans, G. W., Raichlen, J. S., Moons, K. G. M., et al. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.11.012 · Cited by 165 · see citing works
Alvares, G. A., Quintana, D. S., Kemp, A. H., Van Zwieten, A., Balleine, B. W., Hickie, I. B., & Guastella, A. J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0070468 · Cited by 154 · see citing works
Whittingham, K., Sanders, M. R., McKinlay, L., & Boyd, R. N. (2016) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsv118 · Cited by 113 · see citing works
Lakkaraju, H., Kleinberg, J., Leskovec, J., Ludwig, J., & Mullainathan, S. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098066 · Cited by 110 · see citing works
Social anxiety and self-compassion in adolescents
Gill, C., Watson, L., Williams, C., & Chan, S. W. Y. (2018) · Journal of Adolescence · doi:10.1016/j.adolescence.2018.10.004 · Cited by 88 · see citing works
Internalized stigma, anticipated discrimination and perceived public stigma in adults with ADHD
Masuch, T. V., Bea, M., Alm, B., Deibler, P., & Sobanski, E. (2019) · ADHD Attention Deficit and Hyperactivity Disorders · doi:10.1007/s12402-018-0274-9 · Cited by 85 · see citing works
Depressive symptoms in persons with acute coronary syndrome: specific symptom scales and prognosis
Doyle, F., Conroy, R., McGee, H., & Delaney, M. (2010) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2009.07.013 · Cited by 74 · see citing works
Alvares, G. A., Balleine, B. W., & Guastella, A. J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0094778 · Cited by 63 · see citing works
Crooks, C. V., Exner-Cortens, D., Burm, S., Lapointe, A., & Chiodo, D. (2017) · The Journal of Primary Prevention · doi:10.1007/s10935-016-0457-0 · Cited by 64 · see citing works
Examining the association among fear of COVID‐19, psychological distress, and delays in cancer care
Caston, N. E., Lawhon, V. M., Smith, K. L., Gallagher, K., Angove, R., Anderson, E., Balch, A., Azuero, A., Huang, C. S., & Rocque, G. B. (2021) · Cancer Medicine · doi:10.1002/cam4.4391 · Cited by 56 · see citing works
Kindness reduces avoidance goals in socially anxious individuals
Trew, J. L., & Alden, L. E. (2015) · Motivation and Emotion · doi:10.1007/s11031-015-9499-5 · Cited by 55 · see citing works
Shapiro, A. F., Nahm, E. Y., Gottman, J. M., & Content, K. (2011) · American Journal of Orthopsychiatry · doi:10.1111/j.1939-0025.2011.01102.x · Cited by 55 · see citing works
Development and Testing of a 3-Item Screening Tool for Problematic Internet Use
Moreno, M. A., Arseniev-Koehler, A., & Selkie, E. (2016) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2016.05.067 · Cited by 45 · see citing works
Thorpe, J. M., Thorpe, C. T., Kennelty, K. A., & Chewning, B. A. (2012) · Medical Care · doi:10.1097/mlr.0b013e31821a933f · Cited by 32 · see citing works
Horvath, J., Steinert, T., & Jaeger, S. (2018) · International Journal of Law and Psychiatry · doi:10.1016/j.ijlp.2018.02.004 · Cited by 22 · see citing works
Multidimensional variability in ecological assessments predicts two clusters of suicidal patients.
Bonilla-Escribano, P., Ramírez, D., Baca-García, E., Courtet, P., Artés-Rodríguez, A., & López-Castromán, J. (2023) · Scientific Reports · doi:10.1038/s41598-023-30085-1 · Cited by 17 · see citing works
Hirsch, S., Thilo, N., Steinert, T., & Flammer, E. (2021) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-021-02083-z · Cited by 9 · see citing works
- Authors
- Shrive, F. M., Stuart, H., Quan, H., & Ghali, W. A.
- Affiliations
- University of Calgary (CA)
- Queen's University (CA)
- Corresponding author
- Ghali, William A
- Venue
- BMC Medical Research Methodology
- Published
- 2006-12
- Volume
- 6
- Issue
- 1
- Pages
- 57-57
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC1716168
- OpenAlex
- W1603573412
- Cited by
- 650
- References
- 17
- Citations by year
- 2026: 16 · 2025: 30 · 2024: 34 · 2023: 33 · 2022: 49 · 2021: 60 · 2020: 55 · 2019: 39 · 2018: 41 · 2017: 50 · 2016: 34 · 2015: 38 · 2014: 30 · 2013: 35 · 2012: 35
- FWCI
- 7.86
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Heart and Stroke Foundation of Canada
- Government of Canada
- Fondation pour la Recherche Médicale
- Canadian Institutes of Health Research
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Keywords
- missing data, multiple imputation, mean imputation, Zung Self-Rating Depression Scale, depression measurement, surgical outcomes, missing completely at random, missing at random, missing not at random, imputation method comparison, self-report scale
- MeSH terms
- Alberta, Attitude to Health, Data Interpretation, Statistical, Depression, Female, Hospitals, Teaching, Humans, Male, Preoperative Care, Psychiatric Status Rating Scales, Psychometrics, Surveys and Questionnaires, Regression Analysis, Research Design, Self-Assessment, Surgical Procedures, Operative, Reproducibility of Results, Outcome Assessment, Health Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
