The impact of missing data and how it is handled on the rate of false‐positive results in drug development
The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
In drug development, a common choice for the primary analysis is to assess mean changes via analysis of (co)variance with missing data imputed by carrying the last or baseline observations forward (LOCF, BOCF). These approaches assume that data are missing completely at random (MCAR). Multiple imputation (MI) and likelihood‐based repeated measures (MMRM) are less restrictive as they assume data are missing at random (MAR). Nevertheless, LOCF and BOCF remain popular, perhaps because it is thought that the bias in these methods lead to protection against falsely concluding that a drug is more effective than the control. We conducted a simulation study that compared the rate of false positive results or regulatory risk error (RRE) from BOCF, LOCF, MI, and MMRM in 32 scenarios that were generated from a 2 5 full factorial arrangement with data missing due to a missing not at random (MNAR) mechanism. Both BOCF and LOCF inflated RRE were compared to MI and MMRM. In 12 of the 32 scenarios, BOCF yielded inflated RRE compared with eight scenarios for LOCF, three scenarios for MI and four scenarios for MMRM. In no situation did BOCF or LOCF provide adequate control of RRE when MI and MMRM did not. Both MI and MMRM are better choices than either BOCF or LOCF for the primary analysis. Copyright © 2007 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Small sample inference for fixed effects from restricted maximum likelihood
Kenward, M. G., & Roger, J. H. (1997) · Biometrics · doi:10.2307/2533558 · Cited by 4,552 · cited 1×
“Denominator degrees of freedom were estimated using the Kenward–Roger approximation [24].”
MethodA comparison of inclusive and restrictive strategies in modern missing data procedures.
Collins, L. M., Schafer, J. L., & Kam, C.-M. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.4.330 · Cited by 2,389
ACCOUNTING FOR DROPOUT BIAS USING MIXED-EFFECTS MODELS
Mallinckrodt, C. H., Clark, W. S., & David, S. R. (2001) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-100104194 · Cited by 398 · cited 2×
“Research [6,13,15] and analytic proof [15,22] have shown that the behavior of LOCF is in fact uncertain, with frequent inflation of Type I error when treatments are not different and overestimation of the magnitude of treatment differences when the treatments are different.”
BackgroundMethodSelf-citation
Cited by
MMRM vs. LOCF: A Comprehensive Comparison Based on Simulation Study and 25 NDA Datasets
Siddiqui, O., Hung, H. M. J., & O'Neill, R. (2009) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543400802609797 · Cited by 301
Strategies for dealing with missing data in clinical trials: from design to analysis.
(2013) · PubMed Central · Cited by 288
Whiteley, P., Haracopos, D., Knivsberg, A.-M., Reichelt, K. L., Parlar, S., Jacobsen, J., Seim, A., Pedersen, L., Schondel, M., & Shattock, P. (2010) · Nutritional Neuroscience · doi:10.1179/147683010x12611460763922 · Cited by 266
Lumley, M. A., Schubiner, H., Lockhart, N. A., Kidwell, K. M., Harte, S. E., Clauw, D. J., & Williams, D. A. (2017) · Pain · doi:10.1097/j.pain.0000000000001036 · Cited by 242
Kane, J. M., Cohen, M., Zhao, J., Alphs, L., & Panagides, J. (2010) · Journal of Clinical Psychopharmacology · doi:10.1097/jcp.0b013e3181d35d6b · Cited by 126
Seng, E. K., & Holroyd, K. A. (2010) · Annals of Behavioral Medicine · doi:10.1007/s12160-010-9223-3 · Cited by 76
MMRM versus MI in Dealing with Missing Data—A Comparison Based on 25 NDA Data Sets
Siddiqui, O. (2011) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543401003777995 · Cited by 63
“However, Barnes et al. (2008) find a larger average standard error from MI compared with MMRM in a simulation study. Under a null hypothesis (i.e., no treatment effect), the larger average standard error from MI results in a wider confidence interval (CI), and hence it controls type I error rate conservatively. Conseq…”
BackgroundMethodResultInfluentialThe impact of analytic method on interpretation of outcomes in longitudinal clinical trials
Prakash, A., Risser, R. C., & Mallinckrodt, C. H. (2008) · International Journal of Clinical Practice · doi:10.1111/j.1742-1241.2008.01808.x · Cited by 59
Self-citationImpact of missing data on type 1 error rates in non-inferiority trials
Yoo, B. (2010) · Pharmaceutical Statistics · doi:10.1002/pst.378 · Cited by 12
Metadata
- Authors
- Barnes, S. A., Mallinckrodt, C. H., Lindborg, S. R., & Carter, M. K.
- Venue
- Pharmaceutical Statistics
- Publisher
- Wiley
- Published
- 2007-09-12
- Volume
- 7
- Issue
- 3
- Pages
- 215-225
- ISSN
- 1539-1604 (ISSN-L)
- 1539-1612
- OpenAlex
- W2119304424
- Cited by
- 44
- References
- 3
- FWCI
- 5.11
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.