Standards should be applied in the prevention and handling of missing data for patient-centered outcomes research: a systematic review and expert consensus
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Objectives To recommend methodology standards in the prevention and handling of missing data for primary patient-centered outcomes research (PCOR). Study Design and Setting We searched National Library of Medicine Bookshelf and Catalog, regulatory agencies’ and organizations’ websites in January 2012 for guidance documents that had formal recommendations regarding missing data. We extracted the characteristics of included guidance documents and recommendations. Using a two-round modified Delphi survey, a multidisciplinary panel proposed mandatory standards on the prevention and handling of missing data for PCOR. Results We identified 1,790 records and assessed 30 as having relevant recommendations. We proposed 10 standards as mandatory, covering three domains. First, the single best approach is to prospectively prevent missing data occurrence. Second, use of valid statistical methods that properly reflect multiple sources of uncertainty is critical when analyzing missing data. Third, transparent and thorough reporting of missing data allows readers to judge the validity of the findings. Conclusions We urge researchers to adopt rigorous methodology and promote good science by applying best practices to the prevention and handling of missing data. Developing guidance on the prevention and handling of missing data for observational studies and studies that utilize existing records is a priority for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group* (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-152-11-201006010-00232 · Cited by 3,824
U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research, U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research, & U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health (2006) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-4-79 · Cited by 2,856
Review: a gentle introduction to imputation of missing values.
Donders, A. R. T., van der Heijden, G. J. M. G., Stijnen, T., & Moons, K. G. M. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.014 · Cited by 2,582
The prevention and treatment of missing data in clinical trials
Little, R. J., D'Agostino, R., Cohen, M. L., Dickersin, K., Emerson, S. S., Farrar, J. T., Frangakis, C., Hogan, J. W., Molenberghs, G., Murphy, S. A., et al. (2012) · New England Journal of Medicine · doi:10.1056/nejmsr1203730 · Cited by 1,884
Self-citationA critical look at methods for handling missing covariates in epidemiologic regression analyses.
Greenland, S., & Finkle, W. D. (1995) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117592 · Cited by 891
Imputation of missing longitudinal data: a comparison of methods
Engels, J. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00170-7 · Cited by 478
Handling drop‐out in longitudinal studies
Hogan, J. W., Roy, J., & Korkontzelou, C. (2004) · Statistics in Medicine · doi:10.1002/sim.1728 · Cited by 404
Self-citationQuality of reporting of observational longitudinal research.
Tooth, L. (2005) · American Journal of Epidemiology · doi:10.1093/aje/kwi042 · Cited by 363
Cox, E., Martin, B. C., Van Staa, T., Garbe, E., Siebert, U., & Johnson, M. L. (2009) · Value in Health · doi:10.1111/j.1524-4733.2009.00601.x · Cited by 307
Cited by
O. B. O. T. C.-S. G., Grant, S., Mayo-Wilson, E., Montgomery, P., Macdonald, G., Michie, S., Hopewell, S., & Moher, D. (2018) · Trials · doi:10.1186/s13063-018-2735-z · Cited by 348
Patient-Reported Outcome Measures (PROMs) for Acute Stroke: Rationale, Methods and Future Directions
Reeves, M., Lisabeth, L., Williams, L., Katzan, I., Kapral, M., Deutsch, A., & Prvu-Bettger, J. (2018) · Stroke · doi:10.1161/strokeaha.117.018912 · Cited by 147
“Statistical techniques to minimize selection bias include accounting for missing PROM data using statistical imputation methods, such as multiple imputation and inverse probability weighting.(34,35) It is important to recognize that these statistical approaches typically assume data are missing at random—meaning that…”
MethodMayo-Wilson, E., Fusco, N., Li, T., Hong, H., Canner, J. K., & Dickersin, K. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.05.007 · Cited by 131
“On one hand, it is efficient to collect multiple outcomes, and it is good practice for trialists to conduct sensitivity analyses to evaluate whether their findings are robust under different assumptions [37, 38].”
BackgroundSelf-citationCherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy
Mayo-Wilson, E., Li, T., Fusco, N., Bertizzolo, L., Canner, J. K., Cowley, T., Doshi, P., Ehmsen, J., Gresham, G., Guo, N., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.07.014 · Cited by 108
Self-citation1000 Norms Project: protocol of a cross-sectional study cataloging human variation
McKay, M. J., Cartwright, J., Ferreira, P., Simic, M., Vanicek, N., Hiller, C. E., Nightingale, E. J., Moloney, N. A., Quinlan, K. G., Pourkazemi, F., et al. (2016) · Physiotherapy · doi:10.1016/j.physio.2014.12.002 · Cited by 53
Metadata
- Authors
- Li, T., Hutfless, S., Scharfstein, D. O., Daniels, M. J., Hogan, J. W., Little, R. J. A., Roy, J. A., Law, A. H., & Dickersin, K.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2014-01
- Volume
- 67
- Issue
- 1
- Pages
- 15-32
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2047671141
- Cited by
- 76
- References
- 14
- FWCI
- 1.45
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Standards should be applied in the prevention and handling of missing data for patient-centered outcomes research: a systematic review and expert consensus. https://app.cochat.ai/papers/10.1016/j.jclinepi.2013.08.013
- The paper
- Li, T., Hutfless, S., Scharfstein, D. O., Daniels, M. J., Hogan, J. W., Little, R. J. A., Roy, J. A., Law, A. H., & Dickersin, K. (2014). Standards should be applied in the prevention and handling of missing data for patient-centered outcomes research: a systematic review and expert consensus. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2013.08.013
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.