ACCOUNTING FOR DROPOUT BIAS USING MIXED-EFFECTS MODELS
Abstract
Treatment effects are often evaluated by comparing change over time in outcome measures. However, valid analyses of longitudinal data can be problematic when subjects discontinue (dropout) prior to completing the study. This study assessed the merits of likelihood-based repeated measures analyses (MMRM) compared with fixed-effects analysis of variance where missing values were imputed using the last observation carried forward approach (LOCF) in accounting for dropout bias. Comparisons were made in simulated data and in data from a randomized clinical trial. Subject dropout was introduced in the simulated data to generate ignorable and nonignorable missingness. Estimates of treatment group differences in mean change from baseline to endpoint from MMRM were, on average, markedly closer to the true value than estimates from LOCF in every scenario simulated. Standard errors and confidence intervals from MMRM accurately reflected the uncertainty of the estimates, whereas standard errors and confidence intervals from LOCF underestimated uncertainty.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Missing data were managed using LMM, which is based on all observed data and accounts for data missing at random.
“Missing data were managed using LMM, which is based on all observed data and accounts for data missing at random [108-110].”
Cites
Littell, R. C. (1996) · Medical Entomology and Zoology · Cited by 10,345 · see citing works
Linear Mixed Models for Longitudinal Data
Verbeke, G. (1997) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-2294-1_3 · Cited by 3,506 · see citing works
Linear Mixed Models for Longitudinal Data
Molenberghs, G., & Verbeke, G. N. (2000) · Springer series in statistics · doi:10.1007/978-1-4419-0300-6 · Cited by 2,681 · see citing works
Multiple imputation in health‐are databases: An overview and some applications
Rubin, D. B., & Nathaniel Schenker, A. (1991) · Statistics in Medicine · doi:10.1002/sim.4780100410 · Cited by 1,553 · see citing works
Using the general linear mixed model to analyse unbalanced repeated measures and longitudinal data
Cnaan, A., Laird, N. M., & Slasor, P. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19971030)16:20 · Cited by 1,227 · see citing works
Tollefson, G. D., Beasley, C. M., Tran, P. V., Street, J. S., Krueger, J. A., Tamura, R., Graffeo, K. A., & Thieme, M. E. (1997) · American Journal of Psychiatry · doi:10.1176/ajp.154.4.457 · Cited by 891 · see citing works
Some Conceptual and Statistical Issues in Analysis of Longitudinal Psychiatric Data
Gibbons, R. D. (1993) · Archives of General Psychiatry · doi:10.1001/archpsyc.1993.01820210073009 · Cited by 719 · see citing works
Missing data in longitudinal studies
Laird, N. M. (1988) · Statistics in Medicine · doi:10.1002/sim.4780070131 · Cited by 713 · see citing works
A multiple imputation strategy for clinical trials with truncation of patient data
Lavori, P. W., Dawson, R., & Shera, D. M. (1995) · Statistics in Medicine · doi:10.1002/sim.4780141707 · Cited by 291 · see citing works
Statistical handling of drop‐outs in longitudinal clinical trials
Heyting, A., Tolboom, J. T. B. M., & Essers, J. G. A. (1992) · Statistics in Medicine · doi:10.1002/sim.4780111603 · Cited by 236 · see citing works
Clinical trials in psychiatry: should protocol deviation censor patient data?
Lavori, P. W. (1992) · PubMed · Cited by 162 · see citing works
Siddiqui, O. I., & Ali, M. W. (1998) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543409808835259 · Cited by 135 · see citing works
Vleck, L. D. V., & GREGORY, K. E. (1992) · Journal of Animal Science · doi:10.2527/1992.70157x · Cited by 20 · see citing works
SOLAS for Missing Data Analysis by Statistical Solutions Ltd.: Software Review
Bernaards, C. A. (1999) · Structural Equation Modeling A Multidisciplinary Journal · Cited by 20 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Gueorguieva, R., & Krystal, J. H. (2004) · Archives of General Psychiatry · doi:10.1001/archpsyc.61.3.310 · Cited by 1,529 · see citing works
“Tradit ional and mixed-effects approaches produce biased results in the presence of informative dropout, but the bias may be smaller in mixed-effects models.(6,7) Mixed models can also be used for sensitivity analysis.”
BackgroundMethodNeurostimulation for Parkinson's disease with early motor complications.
Schuepbach, W., Rau, J., Knudsen, K., Volkmann, J., Krack, P., Timmermann, L., Hälbig, T., Hesekamp, H., Navarro, S., Meier, N., et al. (2013) · New England Journal of Medicine · doi:10.1056/nejmoa1205158 · Cited by 1,255 · see citing works
Tohen, M., Vieta, E., Calabrese, J., Ketter, T. A., Sachs, G., Bowden, C., Mitchell, P. B., Centorrino, F., Risser, R., Baker, R. W., et al. (2003) · Archives of General Psychiatry · doi:10.1001/archpsyc.60.11.1079 · Cited by 908 · see citing works
Wardlaw, D., Cummings, S. R., Meirhaeghe, J. V., Bastian, L., Tillman, J. B., Ranstam, J., Eastell, R., Shabe, P., Talmadge, K., & Boonen, S. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60010-6 · Cited by 843 · see citing works
McGlashan, T. H., Zipursky, R. B., Perkins, D., Addington, J., Miller, T., Woods, S. W., A. Hawkins, K., E. Hoffman, R., Preda, A., Epstein, I., et al. (2006) · American Journal of Psychiatry · doi:10.1176/ajp.2006.163.5.790 · Cited by 556 · see citing works
“This Cox proportional hazards model included treatment and baseline scores for positive symptoms on the Scale of Prodromal Symptoms as the terms in the model.”
MethodInfluentialBerenson, J. R., Pflugmacher, R., Jarzem, P. F., Zonder, J. A., Schechtman, K. B., Tillman, J. B., Bastian, L., Ashraf, T., & Vrionis, F. D. (2011) · The Lancet Oncology · doi:10.1016/s1470-2045(11)70008-0 · Cited by 521 · see citing works
Analyzing incomplete longitudinal clinical trial data
Molenberghs, G. (2004) · Biostatistics · doi:10.1093/biostatistics/5.3.445 · Cited by 463 · see citing works
Fitzgerald, P. B., Hoy, K. E., McQueen, S., Maller, J. J., Herring, S. E., Segrave, R. A., Bailey, M., Been, G., Kulkarni, J., & Daskalakis, Z. J. (2009) · Neuropsychopharmacology · doi:10.1038/npp.2008.233 · Cited by 420 · see citing works
Detke, M. J., Lu, Y., Goldstein, D. J., Hayes, J. R., & Demitrack, M. A. (2002) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v63n0407 · Cited by 384 · see citing works
Recommendations for the Primary Analysis of Continuous Endpoints in Longitudinal Clinical Trials
Mallinckrodt, C. H., Lane, P. W., Schnell, D. J., Peng, Y., & Mancuso, J. P. (2008) · Drug Information Journal · doi:10.1177/009286150804200402 · Cited by 358 · see citing works
“Mallinckrodt and colleagues (7,8,11) compared MMRM with LOCF ANOVA in a series of simulation studies with MNAR missingness.”
Self-citationMultiple imputation as a flexible tool for missing data handling in clinical research
Enders, C. K. (2016) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.11.008 · Cited by 356 · see citing works
Reininghaus, U., Kempton, M. J., Valmaggia, L., Craig, T. K. J., Garety, P., Onyejiaka, A., Gayer-Anderson, C., So, S. H., Hubbard, K., Beards, S., et al. (2016) · Schizophrenia Bulletin · doi:10.1093/schbul/sbv190 · Cited by 304 · see citing works
Collaborative care for bipolar disorder, Part II: impact on clinical outcome, function, and costs.
Bauer, M. S., McBride, L., Williford, W. O., Glick, H., Kinosian, B., Altshuler, L., Beresford, T., Kilbourne, A. M., & Sajatovic, M. (2006) · Psychiatric Services · doi:10.1176/ps.2006.57.7.937 · Cited by 312 · see citing works
MMRM vs. LOCF: A Comprehensive Comparison Based on Simulation Study and 25 NDA Datasets
Siddiqui, O., Hung, H. M. J., & O'Neill, R. (2009) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543400802609797 · Cited by 301 · see citing works
“Several researchers, including Mallinckrodt et al. (2001), David and Mallinckrodt (2001), Liu and Gould (2002), Lane (2008), and Barnes et al. (2008), have published simulation study results to compare the performance of LOCF analysis with likelihood-based analysis in evaluating treatment efficacy…”
BackgroundMethodResultInfluentialDetke, M. J., Wiltse, C. G., Mallinckrodt, C. H., McNamara, R. K., Demitrack, M. A., & Bitter, I. (2004) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2004.01.002 · Cited by 282 · see citing works
Self-citationDuloxetine 60 mg once daily dosing versus placebo in the acute treatment of major depression
Detke, M. J., Lu, Y., GOLDSTEIN, D. J., McNamara, R. K., & Demitrack, M. A. (2002) · Journal of Psychiatric Research · doi:10.1016/s0022-3956(02)00060-2 · Cited by 278 · see citing works
Hesser, H., Gustafsson, T., Lundén, C., Henrikson, O., Fattahi, K., Johnsson, E., Westin, V. Z., Carlbring, P., Mäki-Torkko, E., Kaldo, V., et al. (2012) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0027021 · Cited by 257 · see citing works
Handling drop‐out in longitudinal clinical trials: a comparison of the LOCF and MMRM approaches
Lane, P. (2008) · Pharmaceutical Statistics · doi:10.1002/pst.267 · Cited by 262 · see citing works
Assessing and interpreting treatment effects in longitudinal clinical trials with missing data
Mallinckrodt, C. H., Sanger, T. M., Dubé, S., DeBrota, D. J., Molenberghs, G., Carroll, R. J., Potter, W. Z., & Tollefson, G. D. (2003) · Biological Psychiatry · doi:10.1016/s0006-3223(02)01867-x · Cited by 246 · see citing works
“Carrying observations forward may therefore bias estimates of treatment effects and the associated standard errors (Gibbons et al 1993; Heyting et al 1992; Lavori et al 1995; Mallinckrodt et al 2001a, 2001b; Siddiqui and Ali 1998; Verbeke and Molenberghs 2000).”
BackgroundMethodResultInfluentialSelf-citationGoldstein, D. J., Lu, Y., Detke, M. J., Wiltse, C., Mallinckrodt, C., & Demitrack, M. A. (2004) · Journal of Clinical Psychopharmacology · doi:10.1097/01.jcp.0000132448.65972.d9 · Cited by 314 · see citing works
“This approach has been shown to be more robust in accounting for the potential bias resulting from missing data points than the last-observation-carriedforward (LOCF) approach.(29,30) The model included the fixed categorical effects of treatment, investigator, visit, and treatment-by-visit interaction, and continuous,…”
Self-citationHauser-Ulrich, S., Künzli, H., Meier-Peterhans, D., & Kowatsch, T. (2020) · JMIR mHealth and uHealth · doi:10.2196/15806 · Cited by 210 · see citing works
“We used an LMM approach because of its ability to handle missing data and to control for type 1 error in the case of incomplete or missing data [88,89].”
MethodBoonen, S., Meirhaeghe, J. V., Bastian, L., Cummings, S. R., Ranstam, J., Tillman, J. B., Eastell, R., Talmadge, K., & Wardlaw, D. (2011) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.364 · Cited by 231 · see citing works
“Results were analyzed by intent to treat, including all data available at baseline and follow-up from all (300) randomized patients (the full analysis set); we used repeated-measures analysis of variance using mixed models that assumed a compound symmetry covariance structure to perform an analysis using unbalanced da…”
MethodChildhood Bereavement: Psychopathology in the 2 Years Postparental Death
Cerel, J., Fristad, M. A., Verducci, J. S., Weller, R. A., & Weller, E. B. (2006) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/01.chi.0000215327.58799.05 · Cited by 229 · see citing works
“Mallinckrodt et al. (2001) conducted a simulation study to compare imputation versus maximum likelihood estimators when some data are missing at random and not ignorable.”
MethodGenetic predictors of response to antidepressants in the GENDEP project
Uher, R., Huezo-Diaz, P., Perroud, N., Smith, R., Rietschel, M., Mors, O., Hauser, J., Maier, W., Kozel, D., Henigsberg, N., et al. (2009) · The Pharmacogenomics Journal · doi:10.1038/tpj.2009.12 · Cited by 226 · see citing works
Haslam, C., Cruwys, T., Chang, M. X. L., Bentley, S. V., Haslam, S. A., Dingle, G. A., & Jetten, J. (2019) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000427 · Cited by 207 · see citing works
Woods, S. W., Breier, A. F., Zipursky, R. B., Perkins, D. O., Addington, J., Miller, T. J., Hawkins, K. A., Márquez, E. M. A., Lindborg, S. R., Tohen, M. F., et al. (2003) · Biological Psychiatry · doi:10.1016/s0006-3223(03)00321-4 · Cited by 220 · see citing works
Holroyd, K. A., Cottrell, C., O'donnell, F., Cordingley, G. E., Drew, J., Carlson, B. W., & Himawan, L. K. (2010) · BMJ · doi:10.1136/bmj.c4871 · Cited by 221 · see citing works
“tests to compare attrition and side effects across the relevant treatment groups. Efficacy analyses were intention to treat analyses that included all randomised participants (n=232). We used a mixed model with fixed effects for treatment (four treatment conditions), time (defined as the natural log of months), and th…”
MethodLjótsson, B., Hedman, E., Andersson, E., Hesser, H., Lindfors, P., Hursti, T., Rydh, S., Rück, C., Lindefors, N., & Andersson, G. (2011) · American Journal of Gastroenterology · doi:10.1038/ajg.2011.139 · Cited by 218 · see citing works
Safarinejad, M. R. (2010) · Andrologia · doi:10.1111/j.1439-0272.2009.01013.x · Cited by 212 · see citing works
“All analyses were intention to treat (ITT) (Lee et al., 1991) using a stratified Fisher’s exact test with the last observation carried forward (LOCF) (Mallinckrodt et al., 2001a,b) for subjects who did not complete the full study schedule.”
MethodHieronymus, F., Emilsson, J. F., Nilsson, S., & Eriksson, E. (2015) · Molecular Psychiatry · doi:10.1038/mp.2015.53 · Cited by 204 · see citing works
Salim, A., Mackinnon, A., Christensen, H., & Griffiths, K. (2008) · Psychiatry Research · doi:10.1016/j.psychres.2007.08.005 · Cited by 194 · see citing works
“The advantages of MI and ML over LOCF have been clearly demonstrated when multiple observations are taken over time and thus when some information about response is available (Mallinckrodt et al., 2001).”
BackgroundMethodAmariglio, R. E., Donohue, M., Marshall, G. A., Rentz, D. M., Salmon, D. P., Ferris, S. H., Karantzoulis, S., Aisen, P. S. S., & Sperling, R. A. (2015) · JAMA Neurology · doi:10.1001/jamaneurol.2014.3375 · Cited by 201 · see citing works
Hesser, H. (2015) · Internet Interventions · doi:10.1016/j.invent.2015.02.003 · Cited by 192 · see citing works
Differential efficacy of escitalopram and nortriptyline on dimensional measures of depression
Uher, R., Maier, W., Hauser, J., Marušič†, A., Schmael, C., Mors, O. N., Henigsberg, N., Souery, D., Placentino, A., Rietschel, M., et al. (2009) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.108.057554 · Cited by 183 · see citing works
Gadbury, G. L., Coffey, C. S., & Allison, D. B. (2003) · Obesity Reviews · doi:10.1046/j.1467-789x.2003.00109.x · Cited by 181 · see citing works
“LOCF and the method of replacing missing data with mean values requires the missing values to be MCAR (47) (or MCAR2 if X is used in the analysis) to yield unbiased estimates of a treatment effect.”
MethodBrannan, S. K., Mallinckrodt, C. H., Brown, E. B., Wohlreich, M. M., Watkin, J. G., & Schatzberg, A. F. (2004) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2004.04.011 · Cited by 179 · see citing works
Self-citationThe development of a single-item Food Choice Questionnaire
Onwezen, M. C., Reinders, M. J., Verain, M. C. D., & Snoek, H. M. (2018) · Food Quality and Preference · doi:10.1016/j.foodqual.2018.05.005 · Cited by 185 · see citing works
Missing Data in Alcohol Clinical Trials: A Comparison of Methods
Hallgren, K. A., & Witkiewitz, K. (2013) · Alcoholism: Clinical and Experimental Research · doi:10.1111/acer.12205 · Cited by 176 · see citing works
“The current findings are consistent with prior simulation studies of missing data in clinical trials (e.g., Lane, 2008; Mallinckrodt et al., 2001) and we echo the recommendations of many others to use MI, FIML, or other recommended modern missing data analytic tools (e.g., weighted estimating…”
BackgroundMethodResultInfluentialCorya, S. A., Williamson, D., Sanger, T. M., Briggs, S. D., Case, M., & Tollefson, G. (2006) · Depression and Anxiety · doi:10.1002/da.20130 · Cited by 170 · see citing works
“…(LOCF) methodology for handling missing data, the current analyses utilized a mixed–effects model repeated measures regression (MMRM), which has been shown to provide highly accurate modeling of treatment outcome, while accounting for subject dropout [Mallinckrodt et al., 2001].”
MethodCorya, S. A. (2005) · doi:10.4088/jcp.v66n1012 · Cited by 162 · see citing works
Mallinckrodt, C. H., Clark, W. S., & David, S. R. (2001) · Drug Information Journal · doi:10.1177/009286150103500418 · Cited by 160 · see citing works
Self-citationMcGlashan, T. H. (2006) · American Journal of Psychiatry · doi:10.1176/appi.ajp.163.5.790 · Cited by 160 · see citing works
Uher, R., Mors, O., Rietschel, M., Rajewska-Rager, A., Petrovic, A., Zobel, A., Henigsberg, N., Mendlewicz, J., Aitchison, K., Farmer, A., et al. (2011) · Journal of Clinical Psychiatry · doi:10.4088/jcp.10m06419 · Cited by 139 · see citing works
Lavedan, C. N., Licamele, L., Volpi, S., Hamilton, J., Heaton, C., Mack, K., Lannan, R., Thompson, A. D., Wolfgang, C. D., & Polymeropoulos, M. H. (2008) · Molecular Psychiatry · doi:10.1038/mp.2008.56 · Cited by 153 · see citing works
Beunckens, C., Molenberghs, G., & Kenward, M. G. (2005) · Clinical Trials · doi:10.1191/1740774505cn119oa · Cited by 152 · see citing works
Meirhaeghe, J. V., Bastian, L., Boonen, S., Ranstam, J., Tillman, J. B., & Wardlaw, D. (2013) · Spine · doi:10.1097/brs.0b013e31828e8e22 · Cited by 151 · see citing works
Krogh, J., Saltin, B., Gluud, C. N., & Nordentoft, M. (2009) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.08m04241 · Cited by 147 · see citing works
Randomized trial of buprenorphine for treatment of concurrent opiate and cocaine dependence
Montoya, I. D. (2004) · Clinical Pharmacology & Therapeutics · doi:10.1016/j.clpt.2003.09.004 · Cited by 147 · see citing works
“Although mixed regression models do not eliminate dropout bias, they reduce the bias in comparison with more conventional alternatives.(32) Piecewise linear regression models were fit by use of a linear spline technique so that a separate slope was estimated for each study phase (maintenance and withdrawal) with both…”
MethodMallinckrodt, C. H., Clark, W. S., Carroll, R. J., & Molenberghs, G. (2003) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-120019265 · Cited by 138 · see citing works
“The simulation studies cited herein (Mallinckrodt et al., 2001a,b) illustrated, and other authors have noted (Lavori, 1992; Little and Yau, 1996; Molenberghs et al., 2002; Verbeke and Molenberghs, 2000), that conservative behavior of LOCF is not guaranteed.”
BackgroundMethodResultInfluentialSelf-citationDoes analysis using "last observation carried forward" introduce bias in dementia research?
Molnar, F., Hutton, B., & Fergusson, D. A. (2008) · Canadian Medical Association Journal · doi:10.1503/cmaj.080820 · Cited by 137 · see citing works
A Randomized Trial of a Web‐Based Intervention to Improve Migraine Self‐Management and Coping
Bromberg, J., Wood, M. E., Black, R. A., Surette, D. A., Zacharoff, K. L., & Chiauzzi, E. J. (2012) · Headache: The Journal of Head and Face Pain · doi:10.1111/j.1526-4610.2011.02031.x · Cited by 136 · see citing works
Kaldo, V., Jernelöv, S., Blom, K., Ljótsson, B., Brodin, M., Jörgensen, M., Kraepelien, M., Rück, C., & Lindefors, N. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.06.001 · Cited by 132 · see citing works
“c) Insomnia at a clinical level defined as more than 10 points on ISI according to Morin (1993), d) Ability to read and write in Swedish and no foreseeable practical problems to participate in the study, e) No comorbid sleep disorders primarily requiring other treatment (e.”
BackgroundTrajectories of change in depression severity during treatment with antidepressants
Uher, R., Muthén, B., Souery, D., Mors, O. N., Jaracz, J., Placentino, A., Petrovic, A., Zobel, A. W., Henigsberg, N., Rietschel, M., et al. (2009) · Psychological Medicine · doi:10.1017/s0033291709991528 · Cited by 128 · see citing works
Duloxetine in the treatment of major depressive disorder: a placebo- and paroxetine-controlled trial
David, P., Wang, F., Mallinckrodt, C. H., Walker, D. J., & Detke, M. J. (2006) · European Psychiatry · doi:10.1016/j.eurpsy.2006.03.004 · Cited by 128 · see citing works
Self-citationKrogh, J., Videbech, P., Thomsen, C. E., Gluud, C. N., & Nordentoft, M. (2012) · PLoS ONE · doi:10.1371/journal.pone.0048316 · Cited by 126 · see citing works
“The mixed-effects function is able to handle missing continuous data using a likelihood estimation of missing data [32].”
MethodDeuschl, G., Schüpbach, M. W. M., Knudsen, K., Pinsker, M. O., Cornu, P., Rau, J., Agid, Y., & Schade‐Brittinger, C. (2012) · Parkinsonism & Related Disorders · doi:10.1016/j.parkreldis.2012.07.004 · Cited by 125 · see citing works
Zonisamide improves wearing‐off in Parkinson's disease: A randomized, double‐blind study
Murata, M., Hasegawa, K., Kanazawa, I., Fukasaka, J., Kochi, K., Shimazu, R., & The Japan Zonisamide on PD Study Group (2015) · Movement Disorders · doi:10.1002/mds.26286 · Cited by 121 · see citing works
Choice of the primary analysis in longitudinal clinical trials
Mallinckrodt, C. H., Watkin, J. G., Molenberghs, G., & Carroll, R. J. (2004) · Pharmaceutical Statistics · doi:10.1002/pst.124 · Cited by 121 · see citing works
“A series of studies [12,13,21,22] compared the MMRM approach to an LOCF ANOVA approach in simulated data with varying amounts of MCAR, MAR, and MNAR missingness.”
Self-citationThe Delayed-Start Study Design
D’Agostino, R. B. (2009) · New England Journal of Medicine · doi:10.1056/nejmsm0904209 · Cited by 119 · see citing works
Igarashi, M., Nakagawa-Nagahama, Y., Miura, M., Kashiwabara, K., Yaku, K., Sawada, M., Sekine, R., Fukamizu, Y., Sato, T., Sakurai, T., et al. (2022) · npj Aging · doi:10.1038/s41514-022-00084-z · Cited by 125 · see citing works
The Cost-Effectiveness of Continuous Glucose Monitoring in Type 1 Diabetes
Huang, E. S., O’Grady, M. J., Basu, A., Winn, A. N., John, P. M., Lee, J. M., Meltzer, D. O., Kollman, C., Laffel, L. M., Tamborlane, W. V., et al. (2010) · Diabetes Care · doi:10.2337/dc09-2042 · Cited by 117 · see citing works
“Experienced utilities at baseline, week 13, and week 26 were modeled using a random-effects linear model (9,10).”
Polanczyk, G. V., Zeni, C., Genro, J. P., Guimarães, A. P., Roman, T., Hutz, M. H., & Rohde, L. A. (2007) · Archives of General Psychiatry · doi:10.1001/archpsyc.64.2.218 · Cited by 115 · see citing works
Clinical trial on tonal tinnitus with tailor-made notched music training
Stein, A., Wunderlich, R., Lau, P., Engell, A., Wollbrink, A., Shaykevich, A., Kuhn, J., Holling, H., Rudack, C., & Pantev, C. (2016) · BMC Neurology · doi:10.1186/s12883-016-0558-7 · Cited by 113 · see citing works
“In anticipation of dropouts and missing data, we chose a likelihood-based mixed-effects model for repeated measures analysis as the primary analysis [46, 47].”
MethodUncertainty Analysis With High Dimensional Dependence Modelling
Carbno, C. C. (2007) · Technometrics · doi:10.1198/tech.2007.s464 · Cited by 110 · see citing works
Carrión, R. E., Auther, A. M., McLaughlin, D., Olsen, R. H., Addington, J., Bearden, C. E., Cadenhead, K. S., Cannon, T. D., Mathalon, D. H., McGlashan, T. H., et al. (2018) · Schizophrenia Bulletin · doi:10.1093/schbul/sby126 · Cited by 107 · see citing works
Lieberman, J. A., Greenhouse, J. B., Hamer, R. M., Krishnan, K. R. R., Nemeroff, C. B., Sheehan, D. V., Thase, M. E., & Keller, M. B. (2005) · Neuropsychopharmacology · doi:10.1038/sj.npp.1300571 · Cited by 105 · see citing works
“Neuropsychopharmacology Contemporary approaches to handling missing data (Mallinckrodt et al, 2003, 2001; Entsuah, 1996) are highly preferable to the aforementioned conventional methods commonly used in studies designed for purposes of regulatory approval.”
MethodJohansson, R., Björklund, M., Hornborg, C., Karlsson, S., Hesser, H., Ljótsson, B., Rousseau, A., Frederick, R. J., & Andersson, G. (2013) · PeerJ · doi:10.7717/peerj.102 · Cited by 102 · see citing works
“%)”
Reininghaus, U., Klippel, A., Steinhart, H., Vaessen, T., van Nierop, M. M., Viechtbauer, W., Batink, T., Kasanova, Z., van Aubel, E., van Winkel, R., et al. (2019) · Trials · doi:10.1186/s13063-019-3912-4 · Cited by 100 · see citing works
COMPARISON OF ALTERNATIVE STRATEGIES FOR ANALYSIS OF LONGITUDINAL TRIALS WITH DROPOUTS
Liu, G. F., & Gould, A. L. (2002) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-120015744 · Cited by 101 · see citing works
Body weight as a predictor of antidepressant efficacy in the GENDEP project
Uher, R., Mors, O. N., Hauser, J., Rietschel, M., Maier, W., Kozel, D., Henigsberg, N., Souery, D., Placentino, A., Perroud, N. A., et al. (2009) · Journal of Affective Disorders · doi:10.1016/j.jad.2009.02.013 · Cited by 96 · see citing works
“It has been repeatedly demonstrated that mixed models are preferable to end-point analyses with last-observation-carried forward procedure (Lane, 2008; Mallinckrodt et al., 2001).”
MethodEdwards, K. M., Banyard, V. L., Sessarego, S. N., Waterman, E. A., Mitchell, K. J., & Chang, H. (2019) · Prevention Science · doi:10.1007/s11121-019-01000-w · Cited by 95 · see citing works
“One strength of multilevel modeling in which the longitudinal time trend at both the cluster level and individual level are preserved through intraclass correlation, and variance structure is that this technique minimizes bias due to attrition (Gad and Youssif 2006; Mallinckrodt et al. 2001).”
Sadeh‐Sharvit, S., Camp, T. D., Horton, S. E., Hefner, J. D., Berry, J. M., Grossman, E., & Hollon, S. D. (2023) · Journal of Medical Internet Research · doi:10.2196/46781 · Cited by 115 · see citing works
“Missing data points were treated as missing at random [24].”
Mallinckrodt, C. H., Kaiser, C. J., Watkin, J. G., Molenberghs, G., & Carroll, R. J. (2004) · Clinical Trials · doi:10.1191/1740774504cn049oa · Cited by 95 · see citing works
“However, the present results and previous empirical studies [4,12-14] have clearly shown that conservative behavior of LOCF is not guaranteed.”
BackgroundMethodResultInfluentialSelf-citationFitzgerald, P. B., Hoy, K. E., Gunewardene, R., Slack, C., Ibrahim, S., Bailey, M., & Daskalakis, Z. J. (2010) · Psychological Medicine · doi:10.1017/s0033291710001923 · Cited by 91 · see citing works
“Empirical studies have confirmed the advantages of mixed models over last observation carry-forward (LOCF) analysis, with mixed effects (Mallinckrodt et al. 2001a, b).”
BackgroundPharmacogenetics of antidepressant response: A polygenic approach
García‐González, J., Tansey, K. E., Hauser, J., Henigsberg, N., Maier, W., Mors, O. N., Placentino, A., Rietschel, M., Souery, D., Žagar, T., et al. (2017) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2017.01.011 · Cited by 91 · see citing works
“Remission was defined as a score below a consensus cut -off that corresponds to absence of depression for each scale (Hamilton, 1967; Montgomery and Asberg, 1979; Beck et al., 1961).”
BackgroundFonseca, A., Alves, S., Monteiro, F., Gorayeb, R., & Canavarro, M. C. (2020) · Behavior Therapy · doi:10.1016/j.beth.2019.09.007 · Cited by 82 · see citing works
Efficacy of Iloperidone in the Treatment of Schizophrenia
Potkin, S. G., Litman, R. E., Torres, R., & Wolfgang, C. D. (2008) · Journal of Clinical Psychopharmacology · doi:10.1097/jcp.0b013e3181692787 · Cited by 89 · see citing works
“That study also used a more contemporary approach to imputation of missing data, a mixedeffects model repeated measures method that has been shown to be less vulnerable to the bias that may be introduced by missing data compared with the LOCF approach, as it takes into account the actual duration of treatment.(27,28)…”
ResultBrecht, S., & Courtecuisse, C. (2007) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v68n1110 · Cited by 89 · see citing works
Andersson, E., Steneby, S., Karlsson, K., Ljótsson, B., Hedman‐Lagerlöf, E., Enander, J., Kaldo, V., Andersson, G., Lindefors, N., & Rück, C. (2014) · Psychological Medicine · doi:10.1017/s0033291714000543 · Cited by 88 · see citing works
“Linear mixed models have the advantage of incorporating the non-independence of intra-individual change and have been shown to be superior to traditional intention-to-treat methods for handling missing data (Mallinckrodt et al. 2001; Lane, 2008).”
MethodPregabalin for the treatment of generalized anxiety disorder: a novel pharmacologic intervention
Bandelow, B., Wedekind, D., & León, T. (2007) · Expert Review of Neurotherapeutics · doi:10.1586/14737175.7.7.769 · Cited by 85 · see citing works
Suh, S. A., Kim, H., Yang, H.-C., Cho, E. R., Lee, S. K., & Shin, C. (2013) · SLEEP · doi:10.5665/sleep.2452 · Cited by 85 · see citing works
Debray, T. P. A., Schuit, E., Efthimiou, O., Reitsma, J. B., Ioannidis, J. P. A., Salanti, G., Moons, K. G. M., & Workpackage, O. B. O. G. (2016) · Statistical Methods in Medical Research · doi:10.1177/0962280216660741 · Cited by 82 · see citing works
Lu, K., Luo, X., & Chen, P. (2008) · The International Journal of Biostatistics · doi:10.2202/1557-4679.1098 · Cited by 82 · see citing works
Outcome of Dialectical Behaviour Therapy for Concurrent Eating and Substance Use Disorders
Courbasson, C. M. A., Nishikawa, Y., & Dixon, L. (2011) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.748 · Cited by 82 · see citing works
Beard, J. I. L., & Delgadillo, J. (2019) · Depression and Anxiety · doi:10.1002/da.22931 · Cited by 81 · see citing works
Profiling multiple provider prescribing of opioids, benzodiazepines, stimulants, and anorectics
Wilsey, B. L., Fishman, S. M., Gilson, A. M., Casamalhuapa, C., Baxi, H., Zhang, H., & Li, C. (2010) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2010.05.007 · Cited by 79 · see citing works
Courbasson, C. M. A., Nishikawa, Y., & Shapira, L. B. (2010) · Eating Disorders · doi:10.1080/10640266.2011.533603 · Cited by 79 · see citing works
“Even while accounting for nonrandom missing data, MMRM analyses have been demonstrated to be very precise at modeling treatment outcome (eg, Mallinckrodt, Clark, & David, 2001 ).”
MethodOgura, T., Azuma, A., Inoue, Y., Taniguchi, H., Chida, K., Bando, M., Niimi, Y., Kakutani, S., Suga, M., Sugiyama, Y., et al. (2015) · Respiratory Investigation · doi:10.1016/j.resinv.2015.06.001 · Cited by 78 · see citing works
“[15] Mallinckrodt CH, Clark WS, David SR.”
MethodParsons, S. K., Shih, M., DuHamel, K. N., Ostroff, J. S., Mayer, D. K., Austin, J., Martini, D. R., Williams, S. E., Mee, L. L., Sexson, S., et al. (2005) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsj078 · Cited by 78 · see citing works
Labelle, L. E., Lawlor-Savage, L., Campbell, T. S., Faris, P., & Carlson, L. E. (2015) · The Journal of Positive Psychology · doi:10.1080/17439760.2014.927902 · Cited by 77 · see citing works
Seng, E. K., & Holroyd, K. A. (2010) · Annals of Behavioral Medicine · doi:10.1007/s12160-010-9223-3 · Cited by 76 · see citing works
“The Behavioral Migraine Management program [ 34 ] is a psychological treatment program based on Social Cognitive Theory.”
BackgroundMethodInfluentialSavitz, J. B., Preskorn, S., Teague, T. K., Drevets, D. A., Yates, W. R., & Drevets, W. (2012) · BMJ Open · doi:10.1136/bmjopen-2011-000643 · Cited by 72 · see citing works
Bårdsen, T., Røksund, O. D., Benestad, M. R., Hufthammer, K. O., Clemm, H. H., Mikalsen, I. B., Øymar, K. A. A., Markestad, T., Halvorsen, T., & Vollsæter, M. (2022) · Thorax · doi:10.1136/thoraxjnl-2021-218400 · Cited by 73 · see citing works
Thase, M. E., Larsen, K. G., & Kennedy, S. H. (2011) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.111.093336 · Cited by 70 · see citing works
Hide, M., Park, H., Igarashi, A., Ye, Y., Kim, T., Yagami, A., Roh, J. Y., Lee, J., Chinuki, Y., Youn, S. W., et al. (2017) · Journal of Dermatological Science · doi:10.1016/j.jdermsci.2017.03.009 · Cited by 70 · see citing works
“Overall AE incidence was similar across treatment groups (54.8%, 57.7%, and 55.4% in omalizumab 300 mg, 150 mg, and placebo groups, Abbreviations: AE, adverse event; ANCOVA, analysis of covariance; BMI, body mass index; BOCF, baseline-observation-carried-forward; CIU, chronic idiopathic urticaria; CSU, chronic spontan…”
MethodInfluentialFitzgerald, P. B., Hoy, K. E., Reynolds, J., Singh, A., Gunewardene, R., Slack, C., Ibrahim, S., & Daskalakis, Z. J. (2019) · Brain stimulation · doi:10.1016/j.brs.2019.09.001 · Cited by 69 · see citing works
On Efficiency of Constrained Longitudinal Data Analysis Versus Longitudinal Analysis of Covariance
Lu, K. (2010) · Biometrics · doi:10.1111/j.1541-0420.2009.01332.x · Cited by 69 · see citing works
“This is reminiscent of the MMRM (mixed-effects model repeated measures) model proposed by Mallinckrodt, Clark, and David (2001), which is particularly suitable for analyzing clinical trial data since patients in clinical trials are often evaluated at a fixed and relative small number of time points…”
MethodMolnar, F., Man‐Son‐Hing, M., Hutton, B., & Fergusson, D. A. (2009) · PubMed Central · Cited by 65 · see citing works
Thase, M. E., Kornstein, S. G., Germain, J., Jiang, Q. L., Guico‐Pabia, C. J., & Ninan, P. T. (2009) · CNS Spectrums · doi:10.1017/s1092852900020125 · Cited by 66 · see citing works
Proof-of-concept randomized controlled trial of pregnenolone in schizophrenia
Marx, C. E., Lee, J., Subramaniam, M., Rapisarda, A., Bautista, D., Chan, E. S., Kilts, J. D., Buchanan, R. W., Eu, P. W., Verma, S. K., et al. (2014) · Psychopharmacology · doi:10.1007/s00213-014-3673-4 · Cited by 64 · see citing works
- Authors
- Mallinckrodt, C. H., Clark, W. S., & David, S. R.
- Affiliations
- Eli Lilly (United States) (US)
- Venue
- Journal of Biopharmaceutical Statistics
- Publisher
- Informa UK Limited
- Published
- 2001-01-01
- Volume
- 11
- Issue
- 1-2
- Pages
- 9-21
- ISSN
- 1054-3406 (ISSN-L)
- 1520-5711
- OpenAlex
- W2073155558
- Cited by
- 398
- References
- 14
- Citations by year
- 2026: 12 · 2025: 13 · 2024: 21 · 2023: 12 · 2022: 27 · 2021: 19 · 2020: 20 · 2019: 13 · 2018: 14 · 2017: 14 · 2016: 18 · 2015: 21 · 2014: 16 · 2013: 23 · 2012: 19
- FWCI
- 11.96
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Biopharmaceutical Statistics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- mixed models, last observation carried forward, missing data, non-ignorable missingness, treatment effect estimation, longitudinal data, randomized controlled trials, standard error estimation
- MeSH terms
- Analysis of Variance, Humans, Longitudinal Studies, Patient Dropouts, Models, Statistical, Bias, Confidence Intervals, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
