Methods to analyze treatment effects in the presence of missing data for a continuous heavy drinking outcome measure when participants drop out from treatment in alcohol clinical trials
Open accessAbstract
Background: Attrition is common in alcohol clinical trials and the resultant loss of data represents an important methodological problem. In the absence of a simulation study, the drinking outcomes among those who are lost to follow‐up are not known. Individuals who drop out of treatment and continue to provide drinking data, however, may be a reasonable proxy group for making inferences about the drinking outcomes of those lost to follow‐up. Methods: We used data from the COMBINE study, a multisite, randomized clinical trial, to examine drinking during the 4 months of treatment among individuals who dropped out of treatment but continued to provide drinking data (i.e., “treatment dropouts;” n = 185). First, we estimated the observed treatment effect size for naltrexone versus placebo in a sample that included both treatment completers ( n = 961) and treatment dropouts ( n = 185; total N = 1,146), as well as the observed treatment effect size among just those who dropped out of treatment ( n = 185). In both the total sample ( N = 1,146) and the dropout sample ( n = 185), we then deleted the drinking data after treatment dropout from those 185 individuals to simulate missing data. Using the deleted data sets, we then estimated the effect of naltrexone on the continuous outcome percent heavy drinking days using 6 methods to handle missing data (last observation carried forward, baseline observation carried forward, placebo mean imputation, missing = heavy drinking days, multiple imputation ( MI ), and full information maximum likelihood [ FIML ]). Results: MI and FIML produced effect size estimates that were most similar to the true effects observed in the full data set in all analyses, while missing = heavy drinking days performed the worst. Conclusions: Although missing drinking data should be avoided whenever possible, MI and FIML yield the best estimates of the treatment effect for a continuous outcome measure of heavy drinking when there is dropout in an alcohol clinical trial.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
FIML has been shown to produce estimates that are the best estimates of true effects under missingness, outperforming other approaches.
“this approach has been shown to produce estimates that are the best estimates of true effects (observed in complete data sets) under conditions of missingness (e.g., Witkiewitz et al., 2014), outperforming other approaches to handling missing data.”
From ACTing towards better living during COVID-19: The effects of Acceptance and Commitment therapy for individuals affected by COVID-19 (2022) · cites this paper as Witkiewitz et al., 2014
The Abstract (Conclusions) says FIML and multiple imputation (MI) yielded the best estimates of treatment effects for a continuous heavy-drinking outcome when participants dropped out of an alcohol trial. The Discussion says they performed equally well, so the source does not support FIML outperforming MI or establish the claim for missing data generally. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,384 · see citing works
Missing data: our view of the state of the art
Schafer, J. L., & Graham, J. W. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.147 · Cited by 11,638 · see citing works
RUBIN, D. B. (1976) · Biometrika · doi:10.1093/biomet/63.3.581 · Cited by 9,925 · see citing works
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473 · see citing works
Anton, R. F., O’Malley, S. S., Ciraulo, D. A., Cisler, R. A., Couper, D., Donovan, D. M., Gastfriend, D. R., Hosking, J. D., Johnson, B. A., LoCastro, J. S., et al. (2006) · JAMA · doi:10.1001/jama.295.17.2003 · Cited by 1,930 · see citing works
Self-citationThe prevention and treatment of missing data in clinical trials
Little, R. J., D'Agostino, R., Cohen, M. L., Dickersin, K., Emerson, S. S., Farrar, J. T., Frangakis, C., Hogan, J. W., Molenberghs, G., Murphy, S. A., et al. (2012) · New England Journal of Medicine · doi:10.1056/nejmsr1203730 · Cited by 1,884 · see citing works
Measurement of drinking behavior using the Form 90 family of instruments.
Miller, W. R., & Boca, F. K. D. (1994) · Journal of Studies on Alcohol Supplement · doi:10.15288/jsas.1994.s12.112 · Cited by 450 · see citing works
Recommendations for the Primary Analysis of Continuous Endpoints in Longitudinal Clinical Trials
Mallinckrodt, C. H., Lane, P. W., Schnell, D. J., Peng, Y., & Mancuso, J. P. (2008) · Drug Information Journal · doi:10.1177/009286150804200402 · Cited by 358 · see citing works
Litten, R. Z., Ryan, M. L., Fertig, J. B., Falk, D. E., Johnson, B., Dunn, K. E., Green, A. I., Pettinati, H. M., Ciraulo, D. A., Sarid-Segal, O., et al. (2013) · Journal of Addiction Medicine · doi:10.1097/adm.0b013e31829623f4 · Cited by 287 · see citing works
Self-citationFalk, D. E., Wang, X. Q., Liu, L., Fertig, J. B., Mattson, M. E., Ryan, M., Johnson, B. A., Stout, R. L., & Litten, R. Z. (2010) · Alcoholism Clinical and Experimental Research · doi:10.1111/j.1530-0277.2010.01290.x · Cited by 189 · see citing works
Self-citationMissing Data in Alcohol Clinical Trials: A Comparison of Methods
Hallgren, K. A., & Witkiewitz, K. (2013) · Alcoholism: Clinical and Experimental Research · doi:10.1111/acer.12205 · Cited by 176 · see citing works
Self-citationThe COMBINE Study Research Group (2003) · Alcoholism: Clinical and Experimental Research · doi:10.1111/j.1530-0277.2003.tb02873.x · Cited by 174 · see citing works
Hedeker, D., Mermelstein, R. J., & Demirtaş, H. (2007) · Addiction · doi:10.1111/j.1360-0443.2007.01946.x · Cited by 154 · see citing works
Demirtaş, H., Freels, S. A., & Yucel, R. M. (2008) · Journal of Statistical Computation and Simulation · doi:10.1080/10629360600903866 · Cited by 153 · see citing works
Substance User Treatment Dropout from Client and Clinician Perspectives: A Pilot Study
Palmer, R. S., Murphy, M. K., Piselli, A., & Ball, S. A. (2009) · Substance Use & Misuse · doi:10.1080/10826080802495237 · Cited by 137 · see citing works
McKellar, J., Kelly, J., Harris, A., & Moos, R. (2006) · Addictive Behaviors · doi:10.1016/j.addbeh.2005.05.024 · Cited by 125 · see citing works
Nich, C., & Carroll, K. M. (2002) · Drug and Alcohol Dependence · doi:10.1016/s0376-8716(02)00111-4 · Cited by 109 · see citing works
ML Versus MI for Missing Data With Violation of Distribution Conditions
Yuan, K., Yang‐Wallentin, F., & Bentler, P. M. (2012) · Sociological Methods & Research · doi:10.1177/0049124112460373 · Cited by 106 · see citing works
Sample Size Determination for Hierarchical Longitudinal Designs with Differential Attrition Rates
Roy, A., Bhaumik, D. K., Aryal, S., & Gibbons, R. D. (2007) · Biometrics · doi:10.1111/j.1541-0420.2007.00769.x · Cited by 87 · see citing works
Del Re, A. C., Maisel, N. C., Blodgett, J. C., & Finney, J. W. (2013) · BMJ Open · doi:10.1136/bmjopen-2013-003464 · Cited by 62 · see citing works
Fertig, J. B., Ryan, M. L., Falk, D. E., Litten, R. Z., Mattson, M. E., Ransom, J. H., Rickman, W. J., Scott, C., Ciraulo, D. A., Green, A. I., et al. (2012) · Alcoholism Clinical and Experimental Research · doi:10.1111/j.1530-0277.2011.01716.x · Cited by 53 · see citing works
Self-citationTull, M. T., & Gratz, K. L. (2011) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2011.08.014 · Cited by 53 · see citing works
Missing data assumptions and methods in a smoking cessation study
Barnes, S. A., Larsen, M. D., Schroeder, D. R., Hanson, A. C., & Decker, P. A. (2010) · Addiction · doi:10.1111/j.1360-0443.2009.02809.x · Cited by 51 · see citing works
Elbreder, M. F., Silva, R. D. S. E., Pillon, S. C., & Laranjeira, R. (2010) · Alcohol and Alcoholism · doi:10.1093/alcalc/agq078 · Cited by 32 · see citing works
Hedden, S. L., Woolson, R. F., Carter, R. E., Palesch, Y. Y., Upadhyaya, H. P., & Malcolm, R. J. (2008) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2008.09.011 · Cited by 31 · see citing works
Measurement and choice of drinking outcome variables in the COMBINE Study.
Anton, R., & Randall, C. (2005) · Journal of Studies on Alcohol, Supplement · doi:10.15288/jsas.2005.s15.104 · Cited by 23 · see citing works
Self-citationGueorguieva, R. V., Wu, R., Krystal, J. H., Donovan, D. M., & O’Malley, S. S. (2013) · Addictive Behaviors · doi:10.1016/j.addbeh.2013.01.024 · Cited by 23 · see citing works
Self-citationStereotyping and the treatment of missing data for drug and alcohol clinical trials
Arndt, S. V. (2009) · Substance Abuse Treatment Prevention and Policy · doi:10.1186/1747-597x-4-2 · Cited by 19 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- FDA 2006 Medical Review of Vivitrol 21 897 US Government Rockville, Maryland
- National Research Council The prevention and treatment of missing data in clinical trials Washington, DC National Academies Press 2010 http://www.nap.edu/catalog.php?recordid-12955 24983040
Cited by
Sorted by citations
Roos, C. R., Bowen, S., & Witkiewitz, K. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000249 · Cited by 92 · see citing works
Self-citationMurphy, J. L., Cordova, M. J., & Dedert, E. A. (2022) · Psychological Services · doi:10.1037/ser0000506 · Cited by 57 · see citing works
Shepherd, K., Golijani-Moghaddam, N., & Dawson, D. L. (2022) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2021.12.003 · Cited by 31 · see citing works
- Authors
- Witkiewitz, K., Falk, D. E., Kranzler, H. R., Litten, R. Z., Hallgren, K. A., O'Malley, S. S., Anton, R. F., & Workgroup, I. C. W. T. A. C. T. I. (.
- Affiliations
- University of New Mexico (US)
- National Institute on Alcohol Abuse and Alcoholism (US)
- Philadelphia VA Medical Center (US)
- Department of Psychiatry Yale School of Medicine New Haven Connecticut
- Medical University of South Carolina (US)
- Corresponding author
- Witkiewitz, Katie
- ORCID
- Venue
- Alcoholism: Clinical and Experimental Research
- Publisher
- Wiley
- Published
- 2014-11
- Volume
- 38
- Issue
- 11
- Pages
- 2826-2834
- ISSN
- 0145-6008 (ISSN-L)
- 1530-0277
- PMC
- PMC4244651
- OpenAlex
- W2096009052
- Cited by
- 60
- References
- 28
- Citations by year
- 2026: 3 · 2025: 2 · 2024: 4 · 2023: 3 · 2022: 3 · 2021: 5 · 2020: 4 · 2019: 7 · 2018: 9 · 2017: 6 · 2016: 10 · 2015: 4
- FWCI
- 7.05
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Alcoholism Clinical and Experimental Research (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute on Alcohol Abuse and Alcoholism (1R01 AA022328-01)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Medicine › Epidemiology › Substance Abuse Treatment and Outcomes
- Keywords
- missing data, attrition, treatment discontinuation, heavy drinking, naltrexone, alcoholism treatment, multiple imputation, full information maximum likelihood, last observation carried forward, COMBINE study, treatment effect estimation
- MeSH terms
- Adult, Alcohol Drinking, Alcoholism, Female, Follow-Up Studies, Humans, Male, Middle Aged, Patient Dropouts, Research Design, Statistics as Topic, Treatment Outcome
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
