Handling drop‐out in longitudinal clinical trials: a comparison of the LOCF and MMRM approaches
Open accessAbstract
This study compares two methods for handling missing data in longitudinal trials: one using the last‐observation‐carried‐forward (LOCF) method and one based on a multivariate or mixed model for repeated measurements (MMRM). Using data sets simulated to match six actual trials, I imposed several drop‐out mechanisms, and compared the methods in terms of bias in the treatment difference and power of the treatment comparison. With equal drop‐out in Active and Placebo arms, LOCF generally underestimated the treatment effect; but with unequal drop‐out, bias could be much larger and in either direction. In contrast, bias with the MMRM method was much smaller; and whereas MMRM rarely caused a difference in power of greater than 20%, LOCF caused a difference in power of greater than 20% in nearly half the simulations. Use of the LOCF method is therefore likely to misrepresent the results of a trial seriously, and so is not a good choice for primary analysis. In contrast, the MMRM method is unlikely to result in serious misinterpretation, unless the drop‐out mechanism is missing not at random (MNAR) and there is substantially unequal drop‐out. Moreover, MMRM is clearly more reliable and better grounded statistically. Neither method is capable of dealing on its own with trials involving MNAR drop‐out mechanisms, for which sensitivity analysis is needed using more complex methods. Copyright © 2007 John Wiley & Sons, Ltd.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Linear mixed models analysis uses all available data and produces unbiased parameter estimates under the assumption of data being missing at random, making it suitable for intention-to-treat analyses in longitudinal studies with data missing at random.
“Building on a likelihood-based approach, the linear mixed models analysis uses all available data and produces unbiased parameter estimates under the assumption of data being missing at random, making it suitable for intention-to-treat analyses in longitudinal studies with data missing at random [56-58].”
From Comparing internet-based cognitive behavioral therapy with standard care for women with fear of birth: Randomized controlled trial (2018) · cites this paper as [57]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,384 · see citing works
RUBIN, D. B. (1976) · Biometrika · doi:10.1093/biomet/63.3.581 · Cited by 9,925 · see citing works
Linear Mixed Models for Longitudinal Data
Molenberghs, G., & Verbeke, G. N. (2000) · Springer series in statistics · doi:10.1007/978-1-4419-0300-6 · Cited by 2,681 · see citing works
Roderick J.A. Little, Donald B. Rubin — Statistical Analysis with Missing Data
Bo., N., Little, R. J. A., & Rubin, D. B. (1988) · Population · doi:10.2307/1533221 · Cited by 2,155 · see citing works
Gueorguieva, R., & Krystal, J. H. (2004) · Archives of General Psychiatry · doi:10.1001/archpsyc.61.3.310 · Cited by 1,529 · see citing works
Analyzing incomplete longitudinal clinical trial data
Molenberghs, G. (2004) · Biostatistics · doi:10.1093/biostatistics/5.3.445 · Cited by 463 · see citing works
ACCOUNTING FOR DROPOUT BIAS USING MIXED-EFFECTS MODELS
Mallinckrodt, C. H., Clark, W. S., & David, S. R. (2001) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-100104194 · Cited by 398 · see citing works
Analyzing incomplete longitudinal clinical trial data
Molenberghs, G. (2004) · Biostatistics · doi:10.1093/biostatistics/kxh001 · Cited by 378 · see citing works
A multiple imputation strategy for clinical trials with truncation of patient data
Lavori, P. W., Dawson, R., & Shera, D. M. (1995) · Statistics in Medicine · doi:10.1002/sim.4780141707 · Cited by 291 · see citing works
Assessing and interpreting treatment effects in longitudinal clinical trials with missing data
Mallinckrodt, C. H., Sanger, T. M., Dubé, S., DeBrota, D. J., Molenberghs, G., Carroll, R. J., Potter, W. Z., & Tollefson, G. D. (2003) · Biological Psychiatry · doi:10.1016/s0006-3223(02)01867-x · Cited by 246 · see citing works
Mallinckrodt, C. H., Clark, W. S., & David, S. R. (2001) · Drug Information Journal · doi:10.1177/009286150103500418 · Cited by 160 · see citing works
A review on linear mixed models for longitudinal data, possibly subject to dropout
Molenberghs, G., & Verbeke, G. N. (2001) · Statistical Modelling · doi:10.1177/1471082x0100100402 · Cited by 125 · see citing works
Choice of the primary analysis in longitudinal clinical trials
Mallinckrodt, C. H., Watkin, J. G., Molenberghs, G., & Carroll, R. J. (2004) · Pharmaceutical Statistics · doi:10.1002/pst.124 · Cited by 121 · see citing works
Mallinckrodt, C. H., Kaiser, C. J., Watkin, J. G., Molenberghs, G., & Carroll, R. J. (2004) · Clinical Trials · doi:10.1191/1740774504cn049oa · Cited by 95 · see citing works
Marginal analysis of incomplete longitudinal binary data: a cautionary note on LOCF imputation
Cook, R. J., Zeng, L., & Yi, G. Y. (2004) · Biometrics · doi:10.1111/j.0006-341x.2004.00234.x · Cited by 88 · see citing works
Analysis of Smoking Trends with Incomplete Longitudinal Binary Responses
Preisser, J. S., Gałecki, A. T., Lohman, K. K., & Wagenknecht, L. E. (2000) · Journal of the American Statistical Association · doi:10.1080/01621459.2000.10474299 · Cited by 38 · see citing works
Type I error rates from likelihood‐based repeated measures analyses of incomplete longitudinal data
Mallinckrodt, C. H., Kaiser, C. J., Watkin, J. G., Detke, M. J., Molenberghs, G., & Carroll, R. J. (2004) · Pharmaceutical Statistics · doi:10.1002/pst.131 · Cited by 36 · see citing works
Brown, D. J. (2002) · Statistics in Medicine · doi:10.1002/sim.1329 · Cited by 11 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Mallinckrodt CH, Clark WS, David SR. Type I error rates from mixed effects model repeated measures compared with fixed effects ANOVA with missing values imputed via LOCF. Drug Informa- tion Journal 2001; 35:1214-1225.
Cited by
Sorted by citations
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883 · see citing works
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2012) · International Journal of Surgery · doi:10.1016/j.ijsu.2011.10.001 · Cited by 2,345 · see citing works
“Many authors have severely criticised last observation carried forward [229 e 231].”
BackgroundMoher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.004 · Cited by 1,858 · see citing works
Lock, J., Le Grange, D., Agras, W. S., Moye, A., Bryson, S. W., & Jo, B. (2010) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2010.128 · Cited by 866 · see citing works
Douillard, J. Y., Siena, S., Cassidy, J., Tabernero, J., Burkes, R., Barugel, M., Humblet, Y., Bodoky, G., Cunningham, D., Jassem, J., et al. (2014) · Annals of Oncology · doi:10.1093/annonc/mdu141 · Cited by 606 · see citing works
Uher, R., Tansey, K. E., Dew, T., Maier, W., Mors, O., Hauser, J., Dernovsek, M. Z., Henigsberg, N., Souery, D., Farmer, A., et al. (2014) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2014.14010094 · Cited by 418 · see citing works
de Vibe, M., Solhaug, I., Tyssen, R., Friborg, O., Rosenvinge, J. H., Sørlie, T., & Bjørndal, A. (2013) · BMC Medical Education · doi:10.1186/1472-6920-13-107 · Cited by 398 · see citing works
“The last-observation-carried-for-ward method of imputation was chosen as this is a conservative method used in instances in which there is an equal drop-out rate in the intervention and the control group [39].”
MethodGold, M., Alderton, C., Zvartau-Hind, M., Egginton, S., Saunders, A. M., Irizarry, M., Craft, S., Landreth, G., Linnamägi, Ü., & Sawchak, S. (2010) · Dementia and Geriatric Cognitive Disorders · doi:10.1159/000318845 · Cited by 361 · see citing works
“Donepezil, the most widely used of the ChEI in the treatment of AD, was used as an active control for assay sensitivity in this study. There was no expectation that the effect of donepezil would depend on the APOE e 4 allele status; thus, a smaller number of subjects were enrolled in this group.The full safety populat…”
Recommendations for the Primary Analysis of Continuous Endpoints in Longitudinal Clinical Trials
Mallinckrodt, C. H., Lane, P. W., Schnell, D. J., Peng, Y., & Mancuso, J. P. (2008) · Drug Information Journal · doi:10.1177/009286150804200402 · Cited by 358 · see citing works
Self-citationGenome-Wide Pharmacogenetics of Antidepressant Response in the GENDEP Project
Uher, R., Perroud, N. A., Ng, M., Hauser, J., Henigsberg, N., Maier, W., Mors, O. N., Placentino, A., Rietschel, M., Souery, D., et al. (2010) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2009.09070932 · Cited by 350 · see citing works
Including all individuals is not enough: lessons for intention-to-treat analysis
White, I., Carpenter, J., & Horton, N. (2012) · Clinical Trials · doi:10.1177/1740774512450098 · Cited by 299 · see citing works
“LOCF does not require the data to be MCAR, although some authors claim that it does [17,18]: MCAR would instead require the missing data to be equal to the observed data in expectation at the final time point [5].”
BackgroundMMRM vs. LOCF: A Comprehensive Comparison Based on Simulation Study and 25 NDA Datasets
Siddiqui, O., Hung, H. M. J., & O'Neill, R. (2009) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543400802609797 · Cited by 301 · see citing works
“Several researchers, including Mallinckrodt et al. (2001), David and Mallinckrodt (2001), Liu and Gould (2002), Lane (2008), and Barnes et al. (2008), have published simulation study results to compare the performance of LOCF analysis with likelihood-based analysis in evaluating treatment efficacy…”
BackgroundResultCooper, A. A., & Conklin, L. R. (2015) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.05.001 · Cited by 264 · see citing works
Agomelatine in the Treatment of Major Depressive Disorder
Stahl, S. M., D, M. F. M., Trivedi, M. H., Caputo, A., Shah, A., & Post, A. (2010) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.09m05471blu · Cited by 262 · see citing works
de Vibe, M., Bjørndal, A., Tipton, E., Hammerstrøm, K., & Kowalski, K. (2012) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.4073/csr.2012.3 · Cited by 253 · see citing works
Assessing treatments used to reduce rumination and/or worry: A systematic review
Querstret, D., & Cropley, M. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2013.08.004 · Cited by 253 · see citing works
“30 given repeated warnings about the dangers of using the LOCF method it seems clear that its use as the sole form of analysis should be discontinued (Lane, 2008; Shapiro, 2001; Barnes, Mallinckrodt, Lindborg, & Carter, 2008; Streiner, 2002).”
MethodGradisar, M., Dohnt, H., Gardner, G., Paine, S., Starkey, K., Menne, A., Slater, A., Wright, H., Hudson, J. L., Weaver, E., et al. (2011) · SLEEP · doi:10.5665/sleep.1432 · Cited by 241 · see citing works
Brinkborg, H., Michanek, J., Hesser, H., & Berglund, G. (2011) · Behaviour Research and Therapy · doi:10.1016/j.brat.2011.03.009 · Cited by 235 · see citing works
“This method of accounting for missing data may not accurately estimate treatment effects in certain scenarios (Lane, 2008).”
MethodLast observation carried forward versus mixed models in the analysis of psychiatric clinical trials.
Hamer, R. M., & Simpson, P. M. (2009) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2009.09040458 · Cited by 233 · see citing works
Comparisons of approaches to pelvic floor muscle training for urinary incontinence in women
Hay-Smith, J. E., Herderschee, R., Dumoulin, C. L., & Herbison, P. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009508 · Cited by 232 · see citing works
Genetic predictors of response to antidepressants in the GENDEP project
Uher, R., Huezo-Diaz, P., Perroud, N., Smith, R., Rietschel, M., Mors, O., Hauser, J., Maier, W., Kozel, D., Henigsberg, N., et al. (2009) · The Pharmacogenomics Journal · doi:10.1038/tpj.2009.12 · Cited by 226 · see citing works
Holroyd, K. A., Cottrell, C., O'donnell, F., Cordingley, G. E., Drew, J., Carlson, B. W., & Himawan, L. K. (2010) · BMJ · doi:10.1136/bmj.c4871 · Cited by 221 · see citing works
Ljótsson, B., Hedman, E., Andersson, E., Hesser, H., Lindfors, P., Hursti, T., Rydh, S., Rück, C., Lindefors, N., & Andersson, G. (2011) · American Journal of Gastroenterology · doi:10.1038/ajg.2011.139 · Cited by 218 · see citing works
“, last observation carried forward) in most missing data situations (”
BackgroundWinters‐Stone, K. M., Dobek, J., Nail, L. M., Bennett, J. A., Leo, M. C., Naik, A. M., & Schwartz, A. L. (2011) · Breast Cancer Research and Treatment · doi:10.1007/s10549-011-1444-z · Cited by 214 · see citing works
“HLM is superior to other approaches to handling missing data, such as mean imputation or last observation carried forward [42, 43].”
Winters-Stone, K. M., Dobek, J., Bennett, J. A., Nail, L. M., Leo, M. C., & Schwartz, A. (2012) · Journal of Cancer Survivorship · doi:10.1007/s11764-011-0210-x · Cited by 211 · see citing works
“HLM uses maximum likelihood estimation for handling missing data, which is superior to many other approaches in handling missing data, such as mean imputation or last observation carried forward [56, 57].”
MethodA Brief Intervention Reduces Hazardous and Harmful Drinking in Emergency Department Patients
D'Onofrio, G., Fiellin, D. A., Pantalon, M. V., Chawarski, M. C., Owens, P. H., Degutis, L. C., Busch, S. H., Bernstein, S. L., & O'Connor, P. G. (2012) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2012.02.006 · Cited by 207 · see citing works
Feedback or biofeedback to augment pelvic floor muscle training for urinary incontinence in women
Herderschee, R., Hay-Smith, J. E., Herbison, P., Roovers, J., & Heineman, M. J. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009252 · Cited by 202 · see citing works
Hesser, H. (2015) · Internet Interventions · doi:10.1016/j.invent.2015.02.003 · Cited by 192 · see citing works
McIntyre, R. S., Cohen, M., Zhao, J., Alphs, L. D., Macek, T. A., & Panagides, J. (2009) · Bipolar Disorders · doi:10.1111/j.1399-5618.2009.00748.x · Cited by 189 · see citing works
“MMRM analysis is included as it is becoming the preferred method for evaluating missing data in clinical trials (37, 38).”
MethodDifferential efficacy of escitalopram and nortriptyline on dimensional measures of depression
Uher, R., Maier, W., Hauser, J., Marušič†, A., Schmael, C., Mors, O. N., Henigsberg, N., Souery, D., Placentino, A., Rietschel, M., et al. (2009) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.108.057554 · Cited by 183 · see citing works
Missing Data in Alcohol Clinical Trials: A Comparison of Methods
Hallgren, K. A., & Witkiewitz, K. (2013) · Alcoholism: Clinical and Experimental Research · doi:10.1111/acer.12205 · Cited by 176 · see citing works
“The current findings are consistent with prior simulation studies of missing data in clinical trials (e.g., Lane, 2008; Mallinckrodt et al., 2001) and we echo the recommendations of many others to use MI, FIML, or other recommended modern missing data analytic tools (e.g., weighted estimating…”
BackgroundMethodResultInfluentialWestin, V. Z., Schulin, M., Hesser, H., Karlsson, M., Noe, R. Z., Olofsson, U., Stalby, M., Wisung, G., & Andersson, G. (2011) · Behaviour Research and Therapy · doi:10.1016/j.brat.2011.08.001 · Cited by 168 · see citing works
McIntyre, R. S., Cohen, M., Zhao, J., Alphs, L. D., Macek, T. A., & Panagides, J. (2010) · Journal of Affective Disorders · doi:10.1016/j.jad.2009.12.028 · Cited by 162 · see citing works
Uher, R., Mors, O., Rietschel, M., Rajewska-Rager, A., Petrovic, A., Zobel, A., Henigsberg, N., Mendlewicz, J., Aitchison, K., Farmer, A., et al. (2011) · Journal of Clinical Psychiatry · doi:10.4088/jcp.10m06419 · Cited by 139 · see citing works
Does Personality Moderate the Effects of Mindfulness Training for Medical and Psychology Students?
de Vibe, M., Solhaug, I., Tyssen, R., Friborg, O., Rosenvinge, J. H., Sørlie, T., Halland, E., & Bjørndal, A. (2015) · Mindfulness · doi:10.1007/s12671-013-0258-y · Cited by 144 · see citing works
“%) were imputed using the last observation carried forward—a conservative statistical method when drop-out rates in the intervention and the control group are equal (Lane 2008).”
MethodMissing outcomes in randomized trials: addressing the dilemma.
Altman, D. G. (2009) · PubMed Central · Cited by 149 · see citing works
Dinov, I. D., Heavner, B., Tang, M., Glusman, G., Chard, K., Darcy, M., Madduri, R., Pa, J., Spino, C., Kesselman, C., et al. (2016) · PLOS ONE · doi:10.1371/journal.pone.0157077 · Cited by 145 · see citing works
“The model-based approaches included generalized linear models (GLM) [47], mixed effect modeling with repeated measurements (MMRM) [48, 49], change-based models [50], and generalized estimating equations (GEE) [51].”
MethodThomas, S., Thomas, P., Kersten, P., Jones, R., Green, C., Nock, A., Slingsby, V., Smith, A. D., Baker, R. D., Galvin, K., et al. (2013) · Journal of Neurology Neurosurgery & Psychiatry · doi:10.1136/jnnp-2012-303816 · Cited by 147 · see citing works
Longitudinal relations between sleep quality, time in bed and adolescent problem behaviour
Meijer, A. M., Reitz, E., Deković, M., van den Wittenboer, G. L. H., & Stoel, R. D. (2010) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2010.02261.x · Cited by 143 · see citing works
“It appears to be a reliable method, not resulting in serious misinterpretation (Lane, 2008).”
MethodLjótsson, B., Hesser, H., Andersson, E., Lackner, J. M., El Alaoui, S., Falk, L., Aspvall, K., Fransson, J., Hammarlund, K., Löfström, A., et al. (2014) · Behaviour Research and Therapy · doi:10.1016/j.brat.2014.01.007 · Cited by 141 · see citing works
Kaldo, V., Jernelöv, S., Blom, K., Ljótsson, B., Brodin, M., Jörgensen, M., Kraepelien, M., Rück, C., & Lindefors, N. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.06.001 · Cited by 132 · see citing works
“, last observation carried forward) in most missing data situations (Lane, 2008; Mallinckrodt, Clark, & David, 2001).”
BackgroundMethodRandomized study of adjunctive belimumab in participants with generalized myasthenia gravis
Hewett, K., Sanders, D. B., Grove, R. A., Broderick, C. L., Rudo, T. J., Bassiri, A. E., Zvartau-Hind, M. E., Bril, V., On behalf of the BEL115123 Study Group, BEL115123 Study Group, et al. (2018) · Neurology · doi:10.1212/wnl.0000000000005323 · Cited by 131 · see citing works
“The MMRM models fitted participants as a random effect and used an unstructured covariance matrix.(25) Given the exploratory nature of the study, Bayesian interpretation for the MMRM analysis of QMG and MGC assuming noninformative priors was also generated, and posterior probabilities that the treatment difference was…”
MethodTrajectories of change in depression severity during treatment with antidepressants
Uher, R., Muthén, B., Souery, D., Mors, O. N., Jaracz, J., Placentino, A., Petrovic, A., Zobel, A. W., Henigsberg, N., Rietschel, M., et al. (2009) · Psychological Medicine · doi:10.1017/s0033291709991528 · Cited by 128 · see citing works
Winters-Stone, K. M., Dobek, J. C., Bennett, J. A., Dieckmann, N. F., Maddalozzo, G. F., Ryan, C. W., & Beer, T. M. (2015) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2014.08.010 · Cited by 126 · see citing works
“Therefore, it is not necessary to delete subjects who have missing data (eg, listwise deletion) or to use methods (eg, last-observation-carried-forward) that have been shown to produce biased parameter estimates.(43) Using maximum likelihood estimation in this way has been shown to perform on par with multiple imputat…”
MethodImproving Adherence to Medication in Stroke Survivors: A Pilot Randomised Controlled Trial
O’Carroll, R. E., Chambers, J. A., Dennis, M. S., Sudlow, C. L. M., & Johnston, M. (2013) · Annals of Behavioral Medicine · doi:10.1007/s12160-013-9515-5 · Cited by 122 · see citing works
Karner, M., Kocjan, A., Stein, J. M., Schreiber, S., von Boyen, G. B. T., Uebel, P., Schmidt, C., Kupčinskas, L., Dina, I., Zuelch, F., et al. (2014) · The American Journal of Gastroenterology · doi:10.1038/ajg.2014.104 · Cited by 117 · see citing works
Analysis of longitudinal data from animals with missing values using SPSS
Duricki, D. A., Soleman, S., & Moon, L. D. F. (2016) · Nature Protocols · doi:10.1038/nprot.2016.048 · Cited by 113 · see citing works
Vallance, J. K., Courneya, K. S., Plotnikoff, R. C., & Mackey, J. R. (2008) · Annals of Behavioral Medicine · doi:10.1007/s12160-008-9019-x · Cited by 107 · see citing works
“LOCF has been shown to give a valid analysis when the data are missing completely at random, loss-to-follow-up rate is under 10%, and there is equal lossto-follow-up across the groups [25].”
MethodBody weight as a predictor of antidepressant efficacy in the GENDEP project
Uher, R., Mors, O. N., Hauser, J., Rietschel, M., Maier, W., Kozel, D., Henigsberg, N., Souery, D., Placentino, A., Perroud, N. A., et al. (2009) · Journal of Affective Disorders · doi:10.1016/j.jad.2009.02.013 · Cited by 96 · see citing works
“The mixed models account for the clustering of individuals within centres, allow use of all repeated measurements, and provide unbiased estimates in the presence of dropout without imputation (Gueorguieva and Krystal, 2004; Lane, 2008).”
BackgroundMethodPharmacogenetics of antidepressant response: A polygenic approach
García‐González, J., Tansey, K. E., Hauser, J., Henigsberg, N., Maier, W., Mors, O. N., Placentino, A., Rietschel, M., Souery, D., Žagar, T., et al. (2017) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2017.01.011 · Cited by 91 · see citing works
“Percent-age change was preferred to absolute change because it is less correlat ed with initial severity, relatively independent of the scale, and closely reflects clinician's impression of improvement (Uher et al., 2009; Lane, 2008; Mallinckrodt, Clark and David, 2001).”
BackgroundAndersson, E., Steneby, S., Karlsson, K., Ljótsson, B., Hedman‐Lagerlöf, E., Enander, J., Kaldo, V., Andersson, G., Lindefors, N., & Rück, C. (2014) · Psychological Medicine · doi:10.1017/s0033291714000543 · Cited by 88 · see citing works
“Linear mixed models have the advantage of incorporating the non-independence of intra-individual change and have been shown to be superior to traditional intention-to-treat methods for handling missing data (Mallinckrodt et al. 2001; Lane, 2008).”
MethodCohen, S. P., Hanling, S. R., Bicket, M. C., White, R. L., Veizi, E., Kurihara, C., Zhao, Z., Hayek, S. M., Guthmiller, K. B., Griffith, S., et al. (2015) · BMJ · doi:10.1136/bmj.h1748 · Cited by 87 · see citing works
Lim, S. L., Ong, K. W., Johal, J., Han, C. Y., Yap, Q. V., Chan, Y. H., Chooi, Y. C., Zhang, Z. P., Chandra, C. C., Thiagarajah, A. G., et al. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.12417 · Cited by 87 · see citing works
Oveisi, S., Ardabili, H. E., Dadds, M. R., Majdzadeh, R., Mohammadkhani, P., Alaghband‐Rad, J., & Shahrivar, Z. (2010) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2009.05.008 · Cited by 83 · see citing works
Peters, E., Landau, S., McCrone, P., Cooke, M. A., Fisher, P., Steel, C., Evans, R., Carswell, K., Dawson, K., Williams, S., et al. (2010) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2010.01572.x · Cited by 91 · see citing works
“Linear mixed modelling is a likelihood approach and therefore valid under the less restrictive assumption that the data are missing at random, unlike last observation carried forward analysis, which has been discredited in the statistical literature for dealing with missing data (43).”
Lu, K., Luo, X., & Chen, P. (2008) · The International Journal of Biostatistics · doi:10.2202/1557-4679.1098 · Cited by 82 · see citing works
Boswood, A., Gordon, S. G., Häggström, J. U., Wess, G., Stepien, R. L., Oyama, M. A., Keene, B. W., Bonagura, J. D., MacDonald, K. A., Patteson, M. W., et al. (2017) · Journal of Veterinary Internal Medicine · doi:10.1111/jvim.14885 · Cited by 81 · see citing works
“Comparing groups in longitudinal studies is challenging, particularly when the rate at which individuals leave those groups is unequal, not at random and potentially related to what is being measured.(25) For example, dogs with bigger hearts are more likely to be missing from the PP population because they have met th…”
BackgroundA closer look at the baseline-observation-carried-forward (BOCF)
D’Souza, D. N. (2010) · Patient Preference and Adherence · doi:10.2147/ppa.s8135 · Cited by 70 · see citing works
“In addition, if there is equal drop-out in both the active and placebo arms, LOCF is liable to underestimate the treatment effect, and in cases of unequal drop-outs, the bias can be much larger in either direction.(2) In addition, with the LOCF method, one has to assume that subjects’ measurement remains at the same l…”
Jarskog, L. F., Lowy, M. T., Grove, R. A., Keefe, R. S. E., Horrigan, J. P., Ball, M. P., Breier, A. F., Buchanan, R. W., Carter, C. S., Csernansky, J. G., et al. (2015) · Schizophrenia Research · doi:10.1016/j.schres.2015.01.041 · Cited by 67 · see citing works
“, 2014), with the exception that covariancewasmodeled using a first order anti-dependence covariance structure, allowing more closely spaced visits to be more highly correlated than those that are further apart, rather than an unstructured correlation matrix (Lane, 2008).”
Missing data treatments in intervention studies: What was, what is, and what should be
Rioux, C., & Little, T. D. (2019) · International Journal of Behavioral Development · doi:10.1177/0165025419880609 · Cited by 65 · see citing works
“…this depends on the study because LOCF can also lead to a liberal bias that exaggerates treatment effects (Cook, Zeng, & Yi, 2004; Lachin, 2016; Lane, 2008; Mallinckrodt, Clark, & David, 2001; Saha & Jones, 2009; Siddiqui, Hung, & O’Neill, 2009; Tang et al., 2005; Wood, White, Hillsdon, &…”
BackgroundMethodInfluentialOpdenacker, J., Delecluse, C., & Boen, F. (2009) · Journal of Sport and Exercise Psychology · doi:10.1123/jsep.31.6.743 · Cited by 66 · see citing works
“A recent study showed that mixed model analyses are more reliable and better grounded statistically for handling missing data in longitudinal trials compared with imputation methods such as the last-observation-carried-forward method (Lane, 2008).”
MethodPharmacogenetics of antidepressant response
Keers, R., & Aitchison, K. J. (2010) · Expert Review of Neurotherapeutics · doi:10.1586/ern.10.186 · Cited by 66 · see citing works
Thase, M. E., Kornstein, S. G., Germain, J., Jiang, Q. L., Guico‐Pabia, C. J., & Ninan, P. T. (2009) · CNS Spectrums · doi:10.1017/s1092852900020125 · Cited by 66 · see citing works
“8 (5, 20) 7(5,12) 6(5,11) 6 (4,10) 9(6,18) 7 (6,10)”
MMRM versus MI in Dealing with Missing Data—A Comparison Based on 25 NDA Data Sets
Siddiqui, O. (2011) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543401003777995 · Cited by 63 · see citing works
“( 2001 ), David and Mallinckrodt ( 2001 ), Liu and Gould ( 2002 ), Lane ( 2008 ), Barnes et al.”
Friberg, L. E., de Greef, R., Kerbusch, T., & Karlsson, M. (2009) · Clinical Pharmacology & Therapeutics · doi:10.1038/clpt.2009.44 · Cited by 63 · see citing works
Kemani, M. K., Olsson, G. L., Lekander, M., Hesser, H., Andersson, E., & Wicksell, R. K. (2015) · The Clinical Journal of Pain · doi:10.1097/ajp.0000000000000203 · Cited by 61 · see citing works
Wang, B., Susukida, R., Mojtabai, R., Amin‐Esmaeili, M., & Rosenblum, M. (2021) · Journal of the American Statistical Association · doi:10.1080/01621459.2021.1981338 · Cited by 60 · see citing works
de Groot, J. F., Takken, T., van Brussel, M., Gooskens, R. H. J. M., Schoenmakers, M. A. G. C., Versteeg, C., Vanhees, L., & Helders, P. J. M. (2011) · Neurorehabilitation and neural repair · doi:10.1177/1545968311400094 · Cited by 60 · see citing works
“Results showed no changes between T1 and T2, making the feed-forward technique appropriate to use because it assumes no changes compared with the last measurement.(52) We also experienced many missing values because of software problems.”
BackgroundLitten, R. Z., Castle, I. P., Falk, D. E., Ryan, M., Fertig, J. B., Chen, C. M., & Yi, H. (2013) · Alcoholism Clinical and Experimental Research · doi:10.1111/acer.12197 · Cited by 59 · see citing works
“This might be better addressed by analyses of simulated incomplete data for individual patients pooled from different studies (Lane, 2008; Siddiqui, 2011; Siddiqui et al., 2009).”
BackgroundHäggström, J. U., Boswood, A., O'Grady, M., Jöns, O., Smith, S., Swift, S., Borgarelli, M., Gavaghan, B. J., Kresken, J., Patteson, M. W., et al. (2013) · Journal of Veterinary Internal Medicine · doi:10.1111/jvim.12181 · Cited by 59 · see citing works
“Finally, it does not require the carrying forward of any data for dogs that have already reached the primary endpoint, a technique which may bias the outcome when there is an unequal rate of removal from the study.(22) In studies in which determination of the effect of a drug on QoL is a primary aim, it is necessary t…”
BackgroundThe impact of analytic method on interpretation of outcomes in longitudinal clinical trials
Prakash, A., Risser, R. C., & Mallinckrodt, C. H. (2008) · International Journal of Clinical Practice · doi:10.1111/j.1742-1241.2008.01808.x · Cited by 59 · see citing works
Missing Data in Longitudinal Trials – Part B, Analytic Issues
Siddique, J., Brown, C. H., Hedeker, D., Duan, N., Gibbons, R. D., Miranda, J. M., & Lavori, P. W. (2008) · Psychiatric Annals · doi:10.3928/00485713-20081201-09 · Cited by 58 · see citing works
Cimino, S., Cerniglia, L., & Paciello, M. (2014) · Child Psychiatry & Human Development · doi:10.1007/s10578-014-0462-6 · Cited by 54 · see citing works
“Consistent with several perspective studies [35], the drop-off rate in the sample at the second time point (T2, 3 years after) was *45 % of the original number of subjects.”
Woolley, S. B., Cardoni, A. A., & Goethe, J. W. (2009) · Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy · doi:10.1592/phco.29.12.1408 · Cited by 52 · see citing works
The Anti-Amyloid Monoclonal Antibody Lecanemab: 16 Cautionary Notes
Kepp, K. P., Sensi, S. L., Johnsen, K. B., Barrio, J. R., Høilund‐Carlsen, P. F., Neve, R. L., Alavi, A. A., Herrup, K., Perry, G. L., Robakis, N. K. R. K., et al. (2023) · Journal of Alzheimer s Disease · doi:10.3233/jad-230099 · Cited by 56 · see citing works
Lerner, D., Adler, D. A., Rogers, W. H., Chang, H., Greenhill, A., Cymerman, E., & Azocar, F. (2015) · Psychiatric Services · doi:10.1176/appi.ps.201400350 · Cited by 53 · see citing works
“Mixed-effects modeling was used to assess the change from baseline to follow-up in each outcome, assuming a person-level random effect (37).”
MethodRawal, R. B., Deal, A. M., Ebert, C. S., Dhandha, V. H., Mitchell, C. A., Hang, A. X., Gore, M. R., Senior, B. A., & Zanation, A. M. (2015) · Rhinology Journal · doi:10.4193/rhin14.196 · Cited by 53 · see citing works
“However, in accordance to the multiple criticisms of this method, last observation carried forward would have been inappropriate for quality of life and olfaction data and therefore was not used (37-40).”
Mayer, J. S., Hees, K., Medda, J., Grimm, O., Asherson, P., Bellina, M., Colla, M., Ibáñez, P., Koch, E., Martinez-Nicolas, A., et al. (2018) · Trials · doi:10.1186/s13063-017-2426-1 · Cited by 48 · see citing works
Specification of covariance structure in longitudinal data analysis for randomized clinical trials
Lu, K., & Mehrotra, D. V. (2009) · Statistics in Medicine · doi:10.1002/sim.3820 · Cited by 52 · see citing works
Well-Being From the Knife? Psychological Effects of Aesthetic Surgery
Margraf, J., Meyer, A. H., & Lavallee, K. L. (2013) · Clinical Psychological Science · doi:10.1177/2167702612471660 · Cited by 51 · see citing works
“Note that mixed models have been shown to provide more efficient and less biased results compared with complete case analyses or analyses in which missing values are imputed using the “last observation carried forward” method (Lane, 2008).”
ResultHobfoll, S. E., Blais, R. K., Stevens, N. R., Walt, L., & Gengler, R. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000041 · Cited by 50 · see citing works
“Moreover, mixed models with full information maximum likelihood is a preferred method of handling missing data due to incomplete outcome data at the 6-week and 12-week follow-up assessments (Lane, 2008).”
MethodCimino, S., Cerniglia, L., Paciello, M., & Sinesi, S. (2012) · European Eating Disorders Review · doi:10.1002/erv.2218 · Cited by 50 · see citing works
“Consistent with prior prospective studies (Lane, 2008), the drop-off rate at the second assessment (T2) 3 years after T1, and the third assessment (T3), 3 years after T2, were ~53% and ~62%, respectively.”
MethodChanges in body weight during pharmacological treatment of depression
Uher, R., Mors, O. N., Hauser, J., Rietschel, M., Maier, W., Kozel, D., Henigsberg, N., Souery, D., Placentino, A., Keers, R., et al. (2010) · The International Journal of Neuropsychopharmacology · doi:10.1017/s1461145710000933 · Cited by 49 · see citing works
“The mixed models account for the clustering of individuals within centres, allow use of all repeated measurements, and provide unbiased estimates in the presence of missing data under less restrictive assumptions than methods used for single time-point analyses (Gueorguieva & Krystal, 2004 ; Lane, 2008).”
MethodSelf-help for stress and burnout without therapist contact: An online randomised controlled trial
Hofer, P. D., Waadt, M., Aschwanden, R., Milidou, M., Acker, J., Meyer, A. H., Lieb, R., & Gloster, A. T. (2018) · Work & Stress · doi:10.1080/02678373.2017.1402389 · Cited by 48 · see citing works
“These analyses lead to more efficient and less biased results compared with complete case analyses or analyses in which missing values have been imputed prior to the analysis using the last observation carried forward method (Lane, 2008).”
ResultRondung, E., Ternström, E., Hildingsson, I., Haines, H. M., Sundin, Ö., Ekdahl, J., Karlström, A., Larsson, B., Segeblad, B., Baylis, R., et al. (2018) · JMIR Mental Health · doi:10.2196/10420 · Cited by 47 · see citing works
“Building on a likelihood-based approach, the linear mixed models analysis uses all available data and produces unbiased parameter estimates under the assumption of data being missing at random, making it suitable for intention-to-treat analyses in longitudinal studies with data missing at random [56-58].”
MethodNon-steroidal anti-inflammatory drugs and efficacy of antidepressants in major depressive disorder
Uher, R., Carver, S., Power, R. A., Mors, O., Maier, W., Rietschel, M., Hauser, J., Dernovsek, M. Z., Henigsberg, N., Souery, D., et al. (2012) · Psychological Medicine · doi:10.1017/s0033291712000190 · Cited by 45 · see citing works
“These models allow the use of all available information across repeated measurements, separate effects that generalize across individuals and recruitment centres from those that are individual- or centre-specific and efficiently handle missing data (Gueorguieva & Krystal, 2004 ; Lane, 2008).”
Joseph, R., Sim, J., Ogollah, R., & Lewis, M. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.09.002 · Cited by 50 · see citing works
“This was the most frequently used imputation method in the review (in more than a third of the trials with late dropouts) despite recommendations against its use [22,23,43].”
MethodKim, J., Seo, B.-K., Choi, J.-B., Kim, H.-J., Kim, T., Lee, M.-H., Kang, K., Kim, J., Shin, K.-M., Lee, S., et al. (2015) · Trials · doi:10.1186/s13063-015-0857-0 · Cited by 47 · see citing works
“Therefore, care should be taken when using the LOCF method for analysis of clinical trial data [39, 40].”
MethodTwo randomized controlled trials of SB742457 in mild‐to‐moderate Alzheimer's disease
Maher‐Edwards, G., Watson, C. B., Ascher, J. A., Barnett, C., Boswell, D., Davies, J. T., Fernández, M., Kurz, A., Zanetti, O. O., Safirstein, B., et al. (2015) · Alzheimer s & Dementia Translational Research & Clinical Interventions · doi:10.1016/j.trci.2015.04.001 · Cited by 47 · see citing works
“Primary inference was conducted using a mixed model for repeated measures analysis using restricted maximum likelihood estimation [23].”
MethodPower and Sample Size for Longitudinal Models in R – The longpower Package and Shiny App
Iddi, S., & Donohue, M. (2022) · The R Journal · doi:10.32614/rj-2022-022 · Cited by 38 · see citing works
“This approach, common in trials of treatments of Alzheimer’s and other therapeutic areas, is often referred to as the MMRM (Mallinckrodt et al., 2001, 2003; Lane, 2008).”
MethodSpineli, L. M., Pandis, N., & Salanti, G. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1131 · Cited by 46 · see citing works
“Nevertheless, despite a well-designed study protocol, some of the randomized participants may leave the trial early without providing any information or after providing partial information, or they can be excluded after randomization (Higgins et al., 2008; Lane, 2008; Spineli et al., 2013).”
BackgroundMissing Data: Turning Guidance Into Action
Mallinckrodt, C. H., Roger, J., Chuang‐Stein, C., Molenberghs, G., Lane, P. W., O’Kelly, M., Ratitch, B., Xu, L., Gilbert, S., Mehrotra, D. V., et al. (2013) · Statistics in Biopharmaceutical Research · doi:10.1080/19466315.2013.848822 · Cited by 46 · see citing works
“For example, Mallinckrodt et al. (2008), Lane (2008), and Siddiqui et al. (2009) summarized comparisons between LOCF and newer analytic methods in regards to bias from missing data.”
BackgroundSelf-citationKivitz, A., Fairfax, M., Sheldon, E. A., Xiang, Q., Jones, B. A., Gammaitoni, A. R., & Gould, E. M. (2008) · Clinical Therapeutics · doi:10.1016/j.clinthera.2008.12.015 · Cited by 44 · see citing works
Hoecke, A.-S. V., Delecluse, C., Bogaerts, A., & Boen, F. (2014) · Journal of Aging and Physical Activity · doi:10.1123/japa.2012-0261 · Cited by 43 · see citing works
Arrogi, A., Schotte, A., Bogaerts, A., Boen, F., & Seghers, J. (2017) · BMC Public Health · doi:10.1186/s12889-016-3965-1 · Cited by 43 · see citing works
“Given the main advantage of mixed model analysis in handling with missing values, our longitudinal design allowed us to reliably assess the effects of the intervention with mixed model analysis [59].”
MethodWyssen, A., Meyer, A. H., Messerli‐Bürgy, N., Forrer, F., Vanhulst, P., Lalanne, D., & Munsch, S. (2021) · European Eating Disorders Review · doi:10.1002/erv.2856 · Cited by 40 · see citing works
“Multilevel models have been shown to provide more efficient and less biased results compared with complete case analyses or analyses in which missing values are imputed using the last observation carried forward method (Lane, 2008).”
ResultBradley, A. J., & Lenox‐Smith, A. (2013) · Journal of Psychopharmacology · doi:10.1177/0269881113494937 · Cited by 41 · see citing works
“Studies have demonstrated that differences in drop outs between treatments introduce significant bias in estimations of treatment efficacy using the LOCF methodology (Lane, 2008; Mallinckrodt et al., 2001; Siddiqui et al., 2009).”
MethodKötter, T., & Niebuhr, F. (2016) · Advances in Medical Education and Practice · doi:10.2147/amep.s110424 · Cited by 40 · see citing works
Gewandter, J. S., McDermott, M., McKeown, A., Smith, S. M., Williams, M. R., Hunsinger, M., Farrar, J. T., Turk, D. C., & Dworkin, R. H. (2014) · Pain · doi:10.1016/j.pain.2014.06.018 · Cited by 39 · see citing works
Reddy, V. P., Kozielska, M., Johnson, M. G., Vermeulen, A., de Greef, R., Liu, J., Groothuis, G. M. M., Danhof, M., & Proost, J. H. (2011) · Clinical Pharmacokinetics · doi:10.2165/11590590-000000000-00000 · Cited by 38 · see citing works
“independent of or dependent on any observed or unobserved variables.[48-51] The main objective of confirmatory clinical trials is typically to delineate the differences between drug and placebo treatment and not the effectiveness of treatment.”
Background
- Authors
- Lane, P.
- Affiliations
- GlaxoSmithKline (United Kingdom) (GB)
- Corresponding author
- Lane, Peter
- ORCID
- Lane, Peter
- Venue
- Pharmaceutical Statistics
- Publisher
- Wiley
- Published
- 2007-03-09
- Volume
- 7
- Issue
- 2
- Pages
- 93-106
- ISSN
- 1539-1604 (ISSN-L)
- 1539-1612
- OpenAlex
- W2068456795
- Cited by
- 262
- References
- 17
- Citations by year
- 2026: 5 · 2025: 12 · 2024: 9 · 2023: 4 · 2022: 12 · 2021: 17 · 2020: 9 · 2019: 8 · 2018: 5 · 2017: 14 · 2016: 10 · 2015: 22 · 2014: 24 · 2013: 20 · 2012: 12
- FWCI
- 17.61
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Pharmaceutical Statistics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- dropout, missing data, last observation carried forward, bias in treatment effect estimation, statistical power, missing not at random, sensitivity analysis
- MeSH terms
- Clinical Trials as Topic, Computer Simulation, Data Interpretation, Statistical, Humans, Longitudinal Studies, Patient Dropouts, Bias, Multivariate Analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
