The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Analyzing incomplete longitudinal clinical trial data
Open accessAbstract
Using standard missing data taxonomy, due to Rubin and co-workers, and simple algebraic derivations, it is argued that some simple but commonly used methods to handle incomplete longitudinal clinical trial data, such as complete case analyses and methods based on last observation carried forward, require restrictive assumptions and stand on a weaker theoretical foundation than likelihood-based methods developed under the missing at random (MAR) framework. Given the availability of flexible software for analyzing longitudinal sequences of unequal length, implementation of likelihood-based MAR analyses is not limited by computational considerations. While such analyses are valid under the comparatively weak assumption of MAR, the possibility of data missing not at random (MNAR) is difficult to rule out. It is argued, however, that MNAR analyses are, themselves, surrounded with problems and therefore, rather than ignoring MNAR analyses altogether or blindly shifting to them, their optimal place is within sensitivity analysis. The concepts developed here are illustrated using data from three clinical trials, where it is shown that the analysis method may have an impact on the conclusions of the study.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,265
RUBIN, D. B. (1976) · Biometrika · doi:10.1093/biomet/63.3.581 · Cited by 9,925
Analysis of Incomplete Multivariate Data
Schafer, J. L. (1997) · doi:10.1201/9781439821862 · Cited by 6,012 · cited 1×
“Multiple imputation acknowledges the uncertainty stemming from filling in missing values rather than observing them (Rubin 1987, Schafer 1997).”
BackgroundAnalysis of Incomplete Multivariate Data
Booth, D. E. (2000) · Technometrics · doi:10.1080/00401706.2000.10486013 · Cited by 5,627
Statistical Analysis With Missing Data
Ghosh, S. (1988) · Technometrics · doi:10.1080/00401706.1988.10488446 · Cited by 4,576
Linear Mixed Models for Longitudinal Data
Verbeke, G. (1997) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-2294-1_3 · Cited by 3,506
Linear Mixed Models for Longitudinal Data
Molenberghs, G., & Verbeke, G. (2000) · Springer Series in Statistics · doi:10.1007/978-1-4419-0300-6 · Cited by 2,646
Self-citationAnalysis of Semiparametric Regression Models for Repeated Outcomes in the Presence of Missing Data
Robins, J. M., Rotnitzky, A., & Zhao, L. P. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476493 · Cited by 1,595
Cited by
Horton, N. J., & Kleinman, K. P. (2007) · The American Statistician · doi:10.1198/000313007x172556 · Cited by 786
Faria, R., Gomes, M., Epstein, D., & White, I. R. (2014) · PharmacoEconomics · doi:10.1007/s40273-014-0193-3 · Cited by 673
“CD-MCAR covariate-dependent missing completely at random, CEA cost-effectiveness analysis, HR-QOL health-related quality of life, IPW inverse probability weighting, MAR missing at random, MCAR missing completely at random, MI multiple imputation, MI-JM MI: joint modelling, MI-MICE MI: chained equations, MNAR missing n…”
BackgroundInfluentialSchool-based programmes for preventing smoking
Thomas, R. E., McLellan, J., & Perera, R. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001293.pub3 · Cited by 614
Twisk, J., de Boer, M., de Vente, W., & Heymans, M. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.03.017 · Cited by 568
Wall, M., McDermott, M. P., Kieburtz, K. D., Corbett, J. J., Feldon, S. E., Friedman, D. I., Katz, D. M., Keltner, J. L., Schron, E. B., & Kupersmith, M. J. (2014) · JAMA · doi:10.1001/jama.2014.3312 · Cited by 550
Differential dropout and bias in randomised controlled trials: when it matters and when it may not
Bell, M. L., Kenward, M. G., Fairclough, D. L., & Horton, N. J. (2013) · BMJ · doi:10.1136/bmj.e8668 · Cited by 460
Multiple imputation: current perspectives
Kenward, M. G., & Carpenter, J. (2007) · Statistical Methods in Medical Research · doi:10.1177/0962280206075304 · Cited by 456
Advances in Analysis of Longitudinal Data
Gibbons, R. D., Hedeker, D., & DuToit, S. (2010) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.032408.153550 · Cited by 452
Handling missing data in RCTs; a review of the top medical journals
Bell, M. L., Fiero, M., Horton, N. J., & Hsu, C.-H. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-118 · Cited by 395
“One trial performed worst-case imputation, LOCF, and multiple imputation.”
BackgroundMethodInfluentialDealing With Missing Outcome Data in Randomized Trials and Observational Studies
Groenwold, R. H. H., Donders, A. R. T., Roes, K. C. B., Harrell, F. E., & Moons, K. G. M. (2012) · American Journal of Epidemiology · doi:10.1093/aje/kwr302 · Cited by 398
New-onset atrial fibrillation predicts long-term mortality after coronary artery bypass graft
El-Chami, M. F., Kilgo, P., Thourani, V., Lattouf, O. M., Delurgio, D. B., Guyton, R. A., Leon, A. R., & Puskas, J. D. (2010) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2009.10.058 · Cited by 359
Multiple imputation as a flexible tool for missing data handling in clinical research
Enders, C. K. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.11.008 · Cited by 355
Recommendations for the Primary Analysis of Continuous Endpoints in Longitudinal Clinical Trials
Mallinckrodt, C. H., Lane, P. W., Schnell, D., Peng, Y., & Mancuso, J. P. (2008) · Drug Information Journal · doi:10.1177/009286150804200402 · Cited by 354
Burt, L. A., Billington, E. O., Rose, M. S., Raymond, D. A., Hanley, D. A., & Boyd, S. K. (2019) · JAMA · doi:10.1001/jama.2019.11889 · Cited by 330
Analyzing longitudinal data with missing values
Enders, C. K. (2011) · Rehabilitation Psychology · doi:10.1037/a0025579 · Cited by 321
kml and kml3d:R Packages to Cluster Longitudinal Data
Genolini, C., Alacoque, X., Sentenac, M., & Arnaud, C. (2015) · Journal of Statistical Software · doi:10.18637/jss.v065.i04 · Cited by 312
Should multiple imputation be the method of choice for handling missing data in randomized trials?
Sullivan, T. R., White, I. R., Salter, A. B., Ryan, P., & Lee, K. J. (2018) · Statistical Methods in Medical Research · doi:10.1177/0962280216683570 · Cited by 308
Including all individuals is not enough: Lessons for intention-to-treat analysis
White, I. R., Carpenter, J., & Horton, N. J. (2012) · Clinical Trials · doi:10.1177/1740774512450098 · Cited by 305
Attrition in randomized controlled clinical trials: methodological issues in psychopharmacology
Leon, A. C., Mallinckrodt, C. H., Chuang-Stein, C., Archibald, D. G., Archer, G. E., & Chartier, K. (2006) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.10.020 · Cited by 303
MMRM vs. LOCF: A Comprehensive Comparison Based on Simulation Study and 25 NDA Datasets
Siddiqui, O., Hung, H. M. J., & O'Neill, R. (2009) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543400802609797 · Cited by 301
Farren, J., Jalmbrant, M., Falconieri, N., Mitchell-Jones, N., Bobdiwala, S., Al-Memar, M., Tapp, S., Van Calster, B., Wynants, L., Timmerman, D., et al. (2020) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2019.10.102 · Cited by 299
Practical and statistical issues in missing data for longitudinal patient-reported outcomes
Bell, M. L., & Fairclough, D. L. (2014) · Statistical Methods in Medical Research · doi:10.1177/0962280213476378 · Cited by 285
A Primer on the Use of Modern Missing-Data Methods in Psychosomatic Medicine Research
Enders, C. K. (2006) · Psychosomatic Medicine · doi:10.1097/01.psy.0000221275.75056.d8 · Cited by 274
Magyar-Russell, G., Thombs, B. D., Cai, J. X., Baveja, T., Kuhl, E. A., Singh, P. P., Montenegro Braga Barroso, M., Arthurs, E., Roseman, M., Amin, N., et al. (2011) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2011.02.014 · Cited by 267
Daclizumab HYP versus Interferon Beta-1a in Relapsing Multiple Sclerosis
Kappos, L., Wiendl, H., Selmaj, K., Arnold, D. L., Havrdova, E., Boyko, A., Kaufman, M., Rose, J., Greenberg, S., Sweetser, M., et al. (2015) · New England Journal of Medicine · doi:10.1056/nejmoa1501481 · Cited by 265
Handling drop‐out in longitudinal clinical trials: a comparison of the LOCF and MMRM approaches
Lane, P. (2008) · Pharmaceutical Statistics · doi:10.1002/pst.267 · Cited by 262
Missing data imputation versus full information maximum likelihood with second-level dependencies
Larsen, R. (2011) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2011.607721 · Cited by 255
Generalized pairwise comparisons of prioritized outcomes in the two‐sample problem
Buyse, M. (2010) · Statistics in Medicine · doi:10.1002/sim.3923 · Cited by 258
Modified intention to treat reporting in randomised controlled trials: systematic review
Abraha, I., & Montedori, A. (2010) · BMJ · doi:10.1136/bmj.c2697 · Cited by 255
Hedman, E., Ljótsson, B., Kaldo, V., Hesser, H., El Alaoui, S., Kraepelien, M., Andersson, E., Rück, C., Svanborg, C., Andersson, G., et al. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.10.023 · Cited by 224
Haslam, C., Cruwys, T., Chang, M. X. L., Bentley, S. V., Haslam, S. A., Dingle, G. A., & Jetten, J. (2019) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000427 · Cited by 207
Chan, A.-W., Hrobjartsson, A., Jorgensen, K. J., Gotzsche, P. C., & Altman, D. G. (2008) · BMJ · doi:10.1136/bmj.a2299 · Cited by 202
“Missing data from losses to follow-up can be handled with various statistical methods that can each produce different results.”
BackgroundExtended-Release Guanfacine for Hyperactivity in Children With Autism Spectrum Disorder
Scahill, L., McCracken, J. T., King, B. H., Rockhill, C., Shah, B., Politte, L., Sanders, R., Minjarez, M., Cowen, J., Mullett, J., et al. (2015) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2015.15010055 · Cited by 201
Salim, A., Mackinnon, A., Christensen, H., & Griffiths, K. (2008) · Psychiatry Research · doi:10.1016/j.psychres.2007.08.005 · Cited by 194
Blankers, M., Koeter, M. W. J., & Schippers, G. M. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1448 · Cited by 194
“Despite its wide application in clinical trials, however, recent empirical studies have cautioned against the use of this technique [21] and have demonstrated its bias [22].”
Rollman, B. L., Belnap, B. H., Mazumdar, S., Houck, P. R., Zhu, F., Gardner, W., Reynolds, C. F., Schulberg, H. C., & Shear, M. K. (2005) · Archives of General Psychiatry · doi:10.1001/archpsyc.62.12.1332 · Cited by 197
Hesser, H. (2015) · Internet Interventions · doi:10.1016/j.invent.2015.02.003 · Cited by 192
Discordance between reported intention-to-treat and per protocol analyses
Porta, N., Bonet, C., & Cobo, E. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.09.013 · Cited by 189
Missing Data in Alcohol Clinical Trials: A Comparison of Methods
Hallgren, K. A., & Witkiewitz, K. (2013) · Alcoholism: Clinical and Experimental Research · doi:10.1111/acer.12205 · Cited by 176
“P A A uthor M anuscript when data are MNAR, in which case sensitivity analyses (e.g., pattern mixture modeling and/or selection models; Enders, 2011) are recommended (Little et al., 2012; Mallinckrodt et al 2008; Molenberghs et al., 2004).”
BackgroundMethodAbraha, I., Cherubini, A., Cozzolino, F., De Florio, R., Luchetta, M. L., Rimland, J. M., Folletti, I., Marchesi, M., Germani, A., Orso, M., et al. (2015) · BMJ · doi:10.1136/bmj.h2445 · Cited by 165
Lintvedt, O. K., Griffiths, K. M., Sørensen, K., Østvik, A. R., Wang, C. E. A., Eisemann, M., & Waterloo, K. (2013) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.770 · Cited by 157
Thayakaran, R., Adderley, N. J., Sainsbury, C., Torlinska, B., Boelaert, K., Šumilo, D., Price, M., Thomas, G. N., Toulis, K. A., & Nirantharakumar, K. (2019) · BMJ · doi:10.1136/bmj.l4892 · Cited by 138
School‐based programmes for preventing smoking
Thomas, R. E., McLellan, J., & Perera, R. (2013) · Evidence-Based Child Health: A Cochrane Review Journal · doi:10.1002/ebch.1937 · Cited by 120
Copy Mean: A New Method to Impute Intermittent Missing Values in Longitudinal Studies
Genolini, C., Écochard, R., & Jacqmin-Gadda, H. (2013) · Open Journal of Statistics · doi:10.4236/ojs.2013.34a004 · Cited by 83
“[11] G. Molenberghs, H. Thijs, I. Jansen, C. Beunckens, M. Kenward, C. Mallinckrodt and R. Carroll, “Analyzing Incomplete Longitudinal Clinical Trial Data,” Biostatistics, Vol. 5, No. 3, 2004, pp. 445-464. doi:10.1093/biostatistics/kxh001 [12] J. Graham, S. Hofer and A. Piccinin, “Analysis with Missing Data in Drug Pr…”
BackgroundInfluentialEl Alaoui, S., Hedman, E., Kaldo, V., Hesser, H., Kraepelien, M., Andersson, E., Rück, C., Andersson, G., Ljótsson, B., & Lindefors, N. (2015) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0039198 · Cited by 99
Seng, E. K., Singer, A. B., Metts, C., Grinberg, A. S., Patel, Z. S., Marzouk, M., Rosenberg, L., Day, M., Minen, M. T., Lipton, R. B., et al. (2019) · Headache: The Journal of Head and Face Pain · doi:10.1111/head.13657 · Cited by 103
Cognitive behavioral therapy for misophonia: A randomized clinical trial
Jager, I. J., Vulink, N. C. C., Bergfeld, I. O., Loon, A. J. J. M., & Denys, D. A. J. P. (2021) · Depression and Anxiety · doi:10.1002/da.23127 · Cited by 97
Dahlin, M., Andersson, G., Magnusson, K., Johansson, T., Sjögren, J., Håkansson, A., Pettersson, M., Kadowaki, Å., Cuijpers, P., & Carlbring, P. (2016) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.12.007 · Cited by 86
CopyMean: A new method to predict monotone missing values in longitudinal studies
Genolini, C., Lacombe, A., Écochard, R., & Subtil, F. (2016) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2016.04.010 · Cited by 52
Seng, E. K., & Holroyd, K. A. (2010) · Annals of Behavioral Medicine · doi:10.1007/s12160-010-9223-3 · Cited by 76
Online Cognitive–Behavioural Treatment of Bulimic Symptoms: A Randomized Controlled Trial
Ruwaard, J., Lange, A., Broeksteeg, J., Renteria‐Agirre, A., Schrieken, B., Dolan, C. V., & Emmelkamp, P. (2013) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.1767 · Cited by 75
Sex Differences in the Contributions of Visceral and Total Body Fat to Blood Pressure in Adolescence
Pausova, Z., Mahboubi, A., Abrahamowicz, M., Leonard, G. T., Perron, M., Richer, L., Veillette, S., Gaudet, D., & Paus, T. (2012) · Hypertension · doi:10.1161/hypertensionaha.111.180372 · Cited by 70
Cruwys, T., Haslam, C., Rathbone, J. A., Williams, E., Haslam, S. A., & Walter, Z. C. (2022) · The British Journal of Psychiatry · doi:10.1192/bjp.2021.128 · Cited by 64
Holroyd, K. A., Labus, J. S., & Carlson, B. (2009) · Pain · doi:10.1016/j.pain.2009.02.019 · Cited by 61
The impact of analytic method on interpretation of outcomes in longitudinal clinical trials
Prakash, A., Risser, R. C., & Mallinckrodt, C. H. (2008) · International Journal of Clinical Practice · doi:10.1111/j.1742-1241.2008.01808.x · Cited by 59
Woolley, S. B., Cardoni, A. A., & Goethe, J. W. (2009) · Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy · doi:10.1592/phco.29.12.1408 · Cited by 52
Bupropion for the Treatment of Apathy in Alzheimer Disease: A Randomized Clinical Trial
Maier, F., Spottke, A., Bach, J.-P., Bartels, C., Buerger, K., Dodel, R., Fellgiebel, A., Fliessbach, K., Frölich, L., Hausner, L., et al. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2020.6027 · Cited by 46
Behavioral Migraine Management Modifies Behavioral and Cognitive Coping in People With Migraine
Seng, E. K., & Holroyd, K. A. (2014) · Headache: The Journal of Head and Face Pain · doi:10.1111/head.12426 · Cited by 44
Parati, G., Agabiti-Rosei, E., Bakris, G. L., Bilo, G., Branzi, G., Cecchi, F., Chrostowska, M., De la Sierra, A., Domenech, M., Dorobantu, M., et al. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-021038 · Cited by 39
Aarsland, D., Khalifa, K., Bergland, A. K., Soennesyn, H., Oppedal, K., Holteng, L. B. A., Oesterhus, R., Nakling, A., Jarholm, J. A., de Lucia, C., et al. (2023) · The American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2022.10.002 · Cited by 36
Rogers, M. L., Bafna, A., & Galynker, I. (2022) · Suicide and Life-Threatening Behavior · doi:10.1111/sltb.12870 · Cited by 26
Enhancing clinical trial design of interventions for posttraumatic stress disorder
Leon, A. C., & Davis, L. L. (2009) · Journal of Traumatic Stress · doi:10.1002/jts.20466 · Cited by 26
Altalib, H. H., Galluzzo, D., Argraves, S., Goulet, J., Bornovski, Y., Cheung, K.-H., Jackson-Shaheed, E., Tolchin, B., Fenton, B. T., & Pugh, M. J. (2019) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s234852 · Cited by 9
Metadata
- Authors
- Molenberghs, G.
- Venue
- Biostatistics
- Publisher
- Oxford University Press (OUP)
- Published
- 2004-07-01
- Volume
- 5
- Issue
- 3
- Pages
- 445-464
- ISSN
- 1465-4644 (ISSN-L)
- 1468-4357
- OpenAlex
- W2123275168
- Cited by
- 463
- References
- 43
- FWCI
- 18.70
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.