MMRM vs. LOCF: A Comprehensive Comparison Based on Simulation Study and 25 NDA Datasets
Abstract
In recent years, the use of the last observation carried forward (LOCF) approach in imputing missing data in clinical trials has been greatly criticized, and several likelihood-based modeling approaches are proposed to analyze such incomplete data. One of the proposed likelihood-based methods is the Mixed-Effect Model Repeated Measure (MMRM) model. To compare the performance of LOCF and MMRM approaches in analyzing incomplete data, two extensive simulation studies are conducted, and the empirical bias and Type I error rates associated with estimators and tests of treatment effects under three missing data paradigms are evaluated. The simulation studies demonstrate that LOCF analysis can lead to substantial biases in estimators of treatment effects and can greatly inflate Type I error rates of the statistical tests, whereas MMRM analysis on the available data leads to estimators with comparatively small bias, and controls Type I error rates at a nominal level in the presence of missing completely at random (MCAR) or missing at random (MAR) and some possibility of missing not at random (MNAR) data. In a sensitivity analysis of 48 clinical trial datasets obtained from 25 New Drug Applications (NDA) submissions of neurological and psychiatric drug products, MMRM analysis appears to be a superior approach in controlling Type I error rates and minimizing biases, as compared to LOCF ANCOVA analysis. In the exploratory analyses of the datasets, no clear evidence of the presence of MNAR missingness is found.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Missing data were managed using LMM, which is based on all observed data and accounts for data missing at random.
“Missing data were managed using LMM, which is based on all observed data and accounts for data missing at random [108-110].”
Cites
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,384 · see citing works
RUBIN, D. B. (1976) · Biometrika · doi:10.1093/biomet/63.3.581 · Cited by 9,925 · see citing works
Random-Effects Models for Longitudinal Data
Laird, N. M., & Ware, J. H. (1982) · Biometrics · doi:10.2307/2529876 · Cited by 8,923 · see citing works
Statistical Analysis with Missing Data
Little, R. J. A., & Rubin, D. B. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/9781119013563 · Cited by 8,473 · see citing works
Fitzmaurice, G. M., Laird, N. M., & Ware, J. H. (2011) · Wiley Series in Probability and Statistics · doi:10.1002/9781119513469 · Cited by 4,639 · see citing works
Linear Mixed Models for Longitudinal Data
Verbeke, G. (1997) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-2294-1_3 · Cited by 3,506 · see citing works
Linear Mixed Models for Longitudinal Data
Molenberghs, G., & Verbeke, G. N. (2000) · Springer series in statistics · doi:10.1007/978-1-4419-0300-6 · Cited by 2,681 · see citing works
Modeling the Drop-Out Mechanism in Repeated-Measures Studies
Little, R. J. A. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476615 · Cited by 1,522 · see citing works
Informative Drop-Out in Longitudinal Data Analysis
Diggle, P. J., & Kenward, M. G. (1994) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2986113 · Cited by 1,472 · see citing works
(2004) · Choice Reviews Online · doi:10.5860/choice.42-2265 · Cited by 1,385 · see citing works
Pattern-Mixture Models for Multivariate Incomplete Data
Little, R. J. A. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10594302 · Cited by 1,026 · see citing works
Missing Data in Clinical Studies
Molenberghs, G., & Kenward, M. G. (2007) · Wiley eBooks · doi:10.1002/9780470510445 · Cited by 969 · see citing works
Missing data in longitudinal studies
Laird, N. M. (1988) · Statistics in Medicine · doi:10.1002/sim.4780070131 · Cited by 713 · see citing works
Analyzing incomplete longitudinal clinical trial data
Molenberghs, G. (2004) · Biostatistics · doi:10.1093/biostatistics/5.3.445 · Cited by 463 · see citing works
Semiparametric Regression for Repeated Outcomes with Nonignorable Nonresponse
Rotnitzky, A. G., Robins, J. M., & Scharfstein, D. O. (1998) · Journal of the American Statistical Association · doi:10.1080/01621459.1998.10473795 · Cited by 431 · see citing works
A Class of Pattern-Mixture Models for Normal Incomplete Data
Little, R. J. A. (1994) · Biometrika · doi:10.1093/biomet/81.3.471 · Cited by 403 · see citing works
ACCOUNTING FOR DROPOUT BIAS USING MIXED-EFFECTS MODELS
Mallinckrodt, C. H., Clark, W. S., & David, S. R. (2001) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-100104194 · Cited by 398 · cited 4× · see citing works
“Several researchers, including Mallinckrodt et al. (2001), David and Mallinckrodt (2001), Liu and Gould (2002), Lane (2008), and Barnes et al. (2008), have published simulation study results to compare the performance of LOCF analysis with likelihood-based analysis in evaluating treatment efficacy…”
BackgroundMethodResultInfluentialAnalyzing incomplete longitudinal clinical trial data
Molenberghs, G. (2004) · Biostatistics · doi:10.1093/biostatistics/kxh001 · Cited by 378 · see citing works
Last observation carry‐forward and last observation analysis
Shao, J., & Zhong, B. (2003) · Statistics in Medicine · doi:10.1002/sim.1519 · Cited by 279 · see citing works
Handling drop‐out in longitudinal clinical trials: a comparison of the LOCF and MMRM approaches
Lane, P. (2008) · Pharmaceutical Statistics · doi:10.1002/pst.267 · Cited by 262 · cited 2× · see citing works
“Several researchers, including Mallinckrodt et al. (2001), David and Mallinckrodt (2001), Liu and Gould (2002), Lane (2008), and Barnes et al. (2008), have published simulation study results to compare the performance of LOCF analysis with likelihood-based analysis in evaluating treatment efficacy…”
BackgroundResultAssessing and interpreting treatment effects in longitudinal clinical trials with missing data
Mallinckrodt, C. H., Sanger, T. M., Dubé, S., DeBrota, D. J., Molenberghs, G., Carroll, R. J., Potter, W. Z., & Tollefson, G. D. (2003) · Biological Psychiatry · doi:10.1016/s0006-3223(02)01867-x · Cited by 246 · see citing works
Statistical handling of drop‐outs in longitudinal clinical trials
Heyting, A., Tolboom, J. T. B. M., & Essers, J. G. A. (1992) · Statistics in Medicine · doi:10.1002/sim.4780111603 · Cited by 236 · see citing works
Mallinckrodt, C. H., Clark, W. S., & David, S. R. (2001) · Drug Information Journal · doi:10.1177/009286150103500418 · Cited by 160 · see citing works
Siddiqui, O. I., & Ali, M. W. (1998) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543409808835259 · Cited by 135 · see citing works
Self-citationMissing Data in Longitudinal Studies
Donald, H., & Gibbons Robert, D. (2006) · Wiley series in probability and statistics · doi:10.1002/0470036486.ch14 · Cited by 102 · see citing works
COMPARISON OF ALTERNATIVE STRATEGIES FOR ANALYSIS OF LONGITUDINAL TRIALS WITH DROPOUTS
Liu, G. F., & Gould, A. L. (2002) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-120015744 · Cited by 101 · see citing works
Mallinckrodt, C. H., Kaiser, C. J., Watkin, J. G., Molenberghs, G., & Carroll, R. J. (2004) · Clinical Trials · doi:10.1191/1740774504cn049oa · Cited by 95 · see citing works
Marginal analysis of incomplete longitudinal binary data: a cautionary note on LOCF imputation
Cook, R. J., Zeng, L., & Yi, G. Y. (2004) · Biometrics · doi:10.1111/j.0006-341x.2004.00234.x · Cited by 88 · see citing works
EFFECTS OF COVARIATE OMISSION AND CATEGORIZATION WHEN ANALYSING RANDOMIZED TRIALS WITH THE COX MODEL
Schmoor, C., & Schumacher, M. C. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970215)16:3 · Cited by 73 · see citing works
Michiels, B., Molenberghs, G., Bijnens, L., Vangeneugden, T., & Thijs, H. (2002) · Statistics in Medicine · doi:10.1002/sim.1064 · Cited by 67 · see citing works
Barnes, S. A., Mallinckrodt, C. H., Lindborg, S. R., & Carter, M. K. (2007) · Pharmaceutical Statistics · doi:10.1002/pst.310 · Cited by 44 · see citing works
Last observation carry‐forward and last observation analysis
Carpenter, J. R., Kenward, M., Evans, S. J. W., & White, I. R. (2004) · Statistics in Medicine · doi:10.1002/sim.1891 · Cited by 35 · see citing works
A Local Influence Sensitivity Analysis for Incomplete Longitudinal Depression Data
Shen, S., Beunckens, C., Mallinckrodt, C. H., & Molenberghs, G. (2006) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543400600609510 · Cited by 13 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Strategy for intention to treat analysis in randomised trials with missing outcome data
White, I. R., Horton, N. J., Carpenter, J. R., statistics, R. I. M. A. S., & Pocock, S. (2011) · BMJ · doi:10.1136/bmj.d40 · Cited by 849 · see citing works
Douillard, J. Y., Siena, S., Cassidy, J., Tabernero, J., Burkes, R., Barugel, M., Humblet, Y., Bodoky, G., Cunningham, D., Jassem, J., et al. (2014) · Annals of Oncology · doi:10.1093/annonc/mdu141 · Cited by 606 · see citing works
Moriarty, P. M., Thompson, P. D., Cannon, C. P., Guyton, J. R., Bergeron, J., Zieve, F. J., Bruckert, E., Jacobson, T. A., Kopecky, S. L., Baccara-Dinet, M. T., et al. (2015) · Journal of Clinical Lipidology · doi:10.1016/j.jacl.2015.08.006 · Cited by 470 · see citing works
Randomized controlled trial of atorvastatin in mild to moderate Alzheimer disease: LEADe
Feldman, H. H., Doody, R. S., Kivipelto, M., Sparks, D. L., Waters, D. D., Jones, R. W., Schwam, E., Schindler, R., Hey-Hadavi, J., DeMicco, D. A., et al. (2010) · Neurology · doi:10.1212/wnl.0b013e3181d6476a · Cited by 438 · see citing works
Uher, R., Tansey, K. E., Dew, T., Maier, W., Mors, O., Hauser, J., Dernovsek, M. Z., Henigsberg, N., Souery, D., Farmer, A., et al. (2014) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2014.14010094 · Cited by 418 · see citing works
Community-Implemented Trauma Therapy for Former Child Soldiers in Northern Uganda
Ertl, V., Pfeiffer, A., Schauer, E., Elbert, T. R., & Neuner, F. (2011) · JAMA · doi:10.1001/jama.2011.1060 · Cited by 321 · see citing works
Including all individuals is not enough: lessons for intention-to-treat analysis
White, I., Carpenter, J., & Horton, N. (2012) · Clinical Trials · doi:10.1177/1740774512450098 · Cited by 299 · see citing works
GAD65 antigen therapy in recently diagnosed type 1 diabetes mellitus
Ludvigsson, J., Krisky, D., Casas, R., Battelino, T., Castaño, L., Greening, J., Kordonouri, O., Otonkoski, T., Pozzilli, P., Robert, J.-J., et al. (2012) · New England Journal of Medicine · doi:10.1056/nejmoa1107096 · Cited by 340 · see citing works
Monticone, M., Ambrosini, E., Cazzaniga, D., Rocca, B., & Ferrante, S. (2014) · European Spine Journal · doi:10.1007/s00586-014-3241-y · Cited by 280 · see citing works
Gibbons, R. D., Hur, K., Brown, C. H., Davis, J. M., & Mann, J. J. (2012) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.2044 · Cited by 271 · see citing works
Feldman, T. E., Mauri, L., Kahwash, R., Litwin, S. E., Ricciardi, M. J., van der Harst, P., Pěnička, M., Fail, P. S., Kaye, D. M., Petrie, M. C., et al. (2017) · Circulation · doi:10.1161/circulationaha.117.032094 · Cited by 269 · see citing works
Foa, E. B., Yusko, D. A., McLean, C. P., Suvak, M. K., Bux, D. A., Oslin, D., O’Brien, C. P., Imms, P., Riggs, D. S., & Volpicelli, J. (2013) · JAMA · doi:10.1001/jama.2013.8268 · Cited by 267 · see citing works
Whiteley, P., Haracopos, D., Knivsberg, A.-M., Reichelt, K. L., Parlar, S., Jacobsen, J., Seim, A., Pedersen, L., Schondel, M., & Shattock, P. (2010) · Nutritional Neuroscience · doi:10.1179/147683010x12611460763922 · Cited by 266 · see citing works
Hauser-Ulrich, S., Künzli, H., Meier-Peterhans, D., & Kowatsch, T. (2020) · JMIR mHealth and uHealth · doi:10.2196/15806 · Cited by 210 · see citing works
Last observation carried forward versus mixed models in the analysis of psychiatric clinical trials.
Hamer, R. M., & Simpson, P. M. (2009) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2009.09040458 · Cited by 233 · see citing works
Decramer, M., Anzueto, A., Kerwin, E., Kaelin, T., Richard, N., Crater, G., Tabberer, M. M., Harris, S. S., & Church, A. (2014) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(14)70065-7 · Cited by 233 · see citing works
Deodhar, A. A., van der Heijde, D. M. F. M., Gensler, L. S., Kim, T., Maksymowych, W. P., Østergaard, M., Poddubnyy, D., Marzo‐Ortega, H., Bessette, L., Tomita, T., et al. (2019) · The Lancet · doi:10.1016/s0140-6736(19)32971-x · Cited by 218 · see citing works
Growth hormone plus childhood low-dose estrogen in Turner's syndrome
Ross, J. L., Quigley, C. A., Cao, D., Feuillan, P., Kowal, K., Chipman, J. J., & Cutler, G. B. (2011) · New England Journal of Medicine · doi:10.1056/nejmoa1005669 · Cited by 212 · see citing works
Ginsberg, H. N., Rader, D. J., Raal, F. J., Guyton, J. R., Baccara‐Dinet, M. T., Lorenzato, C., Pordy, R. C., & Stroes, E. S. G. (2016) · Cardiovascular Drugs and Therapy · doi:10.1007/s10557-016-6685-y · Cited by 212 · see citing works
A Brief Intervention Reduces Hazardous and Harmful Drinking in Emergency Department Patients
D'Onofrio, G., Fiellin, D. A., Pantalon, M. V., Chawarski, M. C., Owens, P. H., Degutis, L. C., Busch, S. H., Bernstein, S. L., & O'Connor, P. G. (2012) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2012.02.006 · Cited by 207 · see citing works
Evidence for benefit of statins to modify cognitive decline and risk in Alzheimer’s disease
Geifman, N., Brinton, R. D., Kennedy, R. E., Schneider, L. S., & Butte, A. J. (2017) · Alzheimer's Research & Therapy · doi:10.1186/s13195-017-0237-y · Cited by 191 · see citing works
Farnier, M., Jones, P. J. H., Severance, R., Averna, M. R., Steinhagen‐Thiessen, E., Colhoun, H. M., Du, Y. E., Hanotin, C., & Donahue, S. (2015) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2015.11.010 · Cited by 188 · see citing works
Missing Data in Alcohol Clinical Trials: A Comparison of Methods
Hallgren, K. A., & Witkiewitz, K. (2013) · Alcoholism: Clinical and Experimental Research · doi:10.1111/acer.12205 · Cited by 176 · see citing works
Two-year outcome of vagus nerve stimulation in treatment-resistant depression
Bajbouj, M., Merkl, A., Schlaepfer, T. E., Frick, C., Zobel, A., Maier, W., O'Keane, V., Corcoran, C., Adolfsson, R., Trimble, M., et al. (2010) · Journal of Clinical Psychopharmacology · doi:10.1097/jcp.0b013e3181db8831 · Cited by 178 · see citing works
Wilkinson, D. G., Windfeld, K., & Colding‐Jørgensen, E. (2014) · The Lancet Neurology · doi:10.1016/s1474-4422(14)70198-x · Cited by 164 · see citing works
Thase, M. E., Mahableshwarkar, A. R., Dragheim, M., Loft, H., & Vieta, E. (2016) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2016.03.007 · Cited by 163 · see citing works
Baldwin, D. S., Loft, H., & Dragheim, M. (2011) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2011.11.008 · Cited by 158 · see citing works
Shah, S. J., Feldman, T. E., Ricciardi, M. J., Kahwash, R., Lilly, S. M., Litwin, S. E., Nielsen, C., van der Harst, P., Hoendermis, E. S., Pěnička, M., et al. (2018) · JAMA Cardiology · doi:10.1001/jamacardio.2018.2936 · Cited by 157 · see citing works
Moriarty, P. M., Jacobson, T. A., Bruckert, E., Thompson, P. D., Guyton, J. R., Baccara-Dinet, M. T., & Gipe, D. (2014) · Journal of Clinical Lipidology · doi:10.1016/j.jacl.2014.09.007 · Cited by 154 · see citing works
Block, G. A., Fishbane, S. N., Rodríguez, M., Smits, G., Shemesh, S., Pérgola, P. E., Wolf, M. S., & Chertow, G. M. (2014) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2014.10.014 · Cited by 152 · see citing works
Inhaled aztreonam lysine vs. inhaled tobramycin in cystic fibrosis: A comparative efficacy trial
Assael, B. M., Pressler, T., Bilton, D., Fayon, M. J., Fischer, R., Chiron, R., Larosa, M., Knoop, C., McElvaney, N. G., Lewis, S. A., et al. (2012) · Journal of Cystic Fibrosis · doi:10.1016/j.jcf.2012.07.006 · Cited by 139 · see citing works
Leiter, L. A., Cariou, B., Müller‐Wieland, D., Colhoun, H. M., Prato, S. D., Tinahones, F. J., Ray, K. K., Bujas‐Bobanovic, M., Domenger, C., Mandel, J., et al. (2017) · Diabetes Obesity and Metabolism · doi:10.1111/dom.13114 · Cited by 138 · see citing works
Self-management intervention for chronic pain in older adults: A randomised controlled trial
Nicholas, M. K., Asghari, A., Blyth, F. M., Wood, B. M., Murray, R., McCabe, R., Brnabic, A., Beeston, L., Corbett, M., Sherrington, C., et al. (2013) · Pain · doi:10.1016/j.pain.2013.02.009 · Cited by 137 · see citing works
Mahableshwarkar, A. R., Jacobsen, P. L., & Chen, Y. (2012) · Current Medical Research and Opinion · doi:10.1185/03007995.2012.761600 · Cited by 135 · see citing works
A Randomized Trial of a Web‐Based Intervention to Improve Migraine Self‐Management and Coping
Bromberg, J., Wood, M. E., Black, R. A., Surette, D. A., Zacharoff, K. L., & Chiauzzi, E. J. (2012) · Headache: The Journal of Head and Face Pain · doi:10.1111/j.1526-4610.2011.02031.x · Cited by 136 · see citing works
Lintzeris, N., Dunlop, A. J., Haber, P., Lubman, D. I., Graham, R., Hutchinson, S., Arunogiri, S., Hayes, V., Hjelmström, P., Svedberg, A., et al. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.9041 · Cited by 129 · see citing works
How to prove that your therapy is effective, even when it is not: a guideline
Cuijpers, P., & Cristea, I. A. (2016) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796015000864 · Cited by 122 · see citing works
Welslau, M., Diéras, V., Sohn, J., Hurvitz, S. A., Lalla, D., Fang, L., Althaus, B., Guardino, E., & Miles, D. (2014) · Cancer · doi:10.1002/cncr.28465 · Cited by 126 · see citing works
Deuschl, G., Schüpbach, M. W. M., Knudsen, K., Pinsker, M. O., Cornu, P., Rau, J., Agid, Y., & Schade‐Brittinger, C. (2012) · Parkinsonism & Related Disorders · doi:10.1016/j.parkreldis.2012.07.004 · Cited by 125 · see citing works
Maleki-Yazdi, M. R., Kaelin, T., Richard, N., Zvarich, M. T., & Church, A. (2014) · Respiratory Medicine · doi:10.1016/j.rmed.2014.10.002 · Cited by 122 · see citing works
Zonisamide improves wearing‐off in Parkinson's disease: A randomized, double‐blind study
Murata, M., Hasegawa, K., Kanazawa, I., Fukasaka, J., Kochi, K., Shimazu, R., & The Japan Zonisamide on PD Study Group (2015) · Movement Disorders · doi:10.1002/mds.26286 · Cited by 121 · see citing works
Monticone, M., Ferrante, S., Teli, M., Rocca, B., Foti, C., Lovi, A., & Brayda Bruno, M. (2014) · European Spine Journal · doi:10.1007/s00586-013-2889-z · Cited by 119 · see citing works
Bode, B. W., McGill, J. B., Lorber, D. L., Gross, J. L., Chang, P.-C., Bregman, D. B., & for the Affinity 1 Study Group (2015) · Diabetes Care · doi:10.2337/dc15-0075 · Cited by 114 · see citing works
Harun, S. N., Wainwright, C. E., Klein, K., & Hennig, S. (2016) · Paediatric Respiratory Reviews · doi:10.1016/j.prrv.2016.03.002 · Cited by 112 · see citing works
TOMBAL, B. F., Fred Saad, C., Penson, D. F., Hussain, M., Sternberg, C. N., Morlock, R., Ramaswamy, K., Ivanescu, C., & Attard, G. (2019) · The Lancet Oncology · doi:10.1016/s1470-2045(18)30898-2 · Cited by 107 · see citing works
Monticone, M., Ambrosini, E., Rocca, B., Magni, S., Brivio, F., & Ferrante, S. (2014) · European Spine Journal · doi:10.1007/s00586-014-3478-5 · Cited by 107 · see citing works
Kastelein, J. J. P., Robinson, J. G., Farnier, M., Krempf, M., Langslet, G., Lorenzato, C., Gipe, D. A., & Baccara‐Dinet, M. T. (2014) · Cardiovascular Drugs and Therapy · doi:10.1007/s10557-014-6523-z · Cited by 102 · see citing works
Mathur, V. S., Kumar, J., Crawford, P. W., Hait, H. I., Sciascia, T. R., & for the TR02 Study Investigators (2017) · American Journal of Nephrology · doi:10.1159/000484573 · Cited by 99 · see citing works
A randomized placebo-controlled trial of an omega-3 fatty acid and vitamins E+C in schizophrenia
Bentsen, H., Osnes, K., Refsum, H., Solberg, D. K., & Bøhmer, T. (2013) · Translational Psychiatry · doi:10.1038/tp.2013.110 · Cited by 97 · see citing works
Koch, A., Pizzichini, E., Hamilton, A. L., Hart, L., Korducki, L., Salvo, M. D., & Paggiaro, P. (2014) · International Journal of COPD · doi:10.2147/copd.s62502 · Cited by 95 · see citing works
Baldwin, D. S., Florea, I., Jacobsen, P. L., Zhong, W., & Nomikos, G. G. (2016) · Journal of Affective Disorders · doi:10.1016/j.jad.2016.07.015 · Cited by 94 · see citing works
Maheu, E., Cadet, C., Marty, M., Moyse, D., Kerloch, I., Coste, P., Dougados, M., Mazières, B., Spector, T. D., Halhol, H., et al. (2013) · Annals of the Rheumatic Diseases · doi:10.1136/annrheumdis-2012-202485 · Cited by 86 · see citing works
Chen, Y., Yang, Y., Hung, H., & Wang, S. (2011) · Contemporary Clinical Trials · doi:10.1016/j.cct.2011.04.006 · Cited by 84 · see citing works
Self-citationTran, P., Skolnick, P., Czobor, P., Huang, N. Y., Bradshaw, M., McKinney, A., & Fava, M. (2012) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2011.09.003 · Cited by 81 · see citing works
Nangaku, M., Kondo, K., Kokado, Y., Ueta, K., Kaneko, G., Tandai, T., Kawaguchi, Y., & Komatsu, Y. (2021) · Journal of the American Society of Nephrology · doi:10.1681/asn.2020091311 · Cited by 83 · see citing works
Cella, D., Ivanescu, C., Holmstrom, S., Bui, C. N., Spalding, J., & Fizazi, K. (2015) · Annals of Oncology · doi:10.1093/annonc/mdu510 · Cited by 79 · see citing works
Vokes, T., Mannstadt, M., Levine, M. A., Clarke, B. L., Lakatos, P., Chen, K. S., Piccolo, R. S., Krasner, A. S., Shoback, D., & Bilezikian, J. P. (2017) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2017-01471 · Cited by 79 · see citing works
Ogura, T., Azuma, A., Inoue, Y., Taniguchi, H., Chida, K., Bando, M., Niimi, Y., Kakutani, S., Suga, M., Sugiyama, Y., et al. (2015) · Respiratory Investigation · doi:10.1016/j.resinv.2015.06.001 · Cited by 78 · see citing works
Odenstedt, J., Linderoth, B., Bergfeldt, L., Ekre, O., Grip, L., Mannheimer, C., & Andréll, P. (2011) · Heart Rhythm · doi:10.1016/j.hrthm.2011.01.029 · Cited by 76 · see citing works
Savitz, J. B., Preskorn, S., Teague, T. K., Drevets, D. A., Yates, W. R., & Drevets, W. (2012) · BMJ Open · doi:10.1136/bmjopen-2011-000643 · Cited by 72 · see citing works
Challenges in detecting disease modification in Parkinson's disease clinical trials
Athauda, D. S., & Foltynie, T. (2016) · Parkinsonism & Related Disorders · doi:10.1016/j.parkreldis.2016.07.019 · Cited by 73 · see citing works
Wild, B., Hünnemeyer, K., Sauer, H., Hain, B., Mack, I., Schellberg, D., Müller-Stich, B. P., Weiner, R., Meile, T., Rudofsky, G., et al. (2015) · Surgery for Obesity and Related Diseases · doi:10.1016/j.soard.2015.05.018 · Cited by 70 · see citing works
On Efficiency of Constrained Longitudinal Data Analysis Versus Longitudinal Analysis of Covariance
Lu, K. (2010) · Biometrics · doi:10.1111/j.1541-0420.2009.01332.x · Cited by 69 · see citing works
Cognitive exposure versus avoidance in patients with chronic pain: Adherence matters
Nicholas, M. K., Asghari̇, A., Sharpe, L., Brnabic, A. J. M., Wood, B. M., Overton, S., Tonkin, L., Sousa, M. D., Finniss, D. G., Beeston, L., et al. (2013) · European Journal of Pain · doi:10.1002/j.1532-2149.2013.00383.x · Cited by 68 · see citing works
Missing data treatments in intervention studies: What was, what is, and what should be
Rioux, C., & Little, T. D. (2019) · International Journal of Behavioral Development · doi:10.1177/0165025419880609 · Cited by 65 · see citing works
Publication Bias, with a Focus on Psychiatry: Causes and Solutions
Turner, E. H. (2013) · CNS Drugs · doi:10.1007/s40263-013-0067-9 · Cited by 64 · see citing works
Samtani, M. N., Raghavan, N., Novak, G., Nandy, P., & Narayan, V. A. (2014) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s62323 · Cited by 65 · see citing works
Nangaku, M., Kondo, K., Ueta, K., Kokado, Y., Kaneko, G., Matsuda, H., Kawaguchi, Y., & Komatsu, Y. (2021) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/gfab055 · Cited by 65 · see citing works
Fiorillo, D., McLean, C., Pistorello, J., Hayes, S. C., & Follette, V. M. (2017) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2016.11.003 · Cited by 61 · see citing works
Siler, T. M., Kerwin, E., Singletary, K., Brooks, J., & Church, A. (2015) · COPD Journal of Chronic Obstructive Pulmonary Disease · doi:10.3109/15412555.2015.1034256 · Cited by 64 · see citing works
MMRM versus MI in Dealing with Missing Data—A Comparison Based on 25 NDA Data Sets
Siddiqui, O. (2011) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543401003777995 · Cited by 63 · see citing works
Self-citationBalasubramanyam, A., Coraza, I., Smith, E. O., Scott, L. W., Patel, P., Iyer, D., Taylor, A. A., Giordano, T. P., Sekhar, R. V., Clark, P., et al. (2011) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2010-3067 · Cited by 63 · see citing works
Fountoulakis, K. Ν., Vieta, E., Young, A. H., Yatham, L. N., Grunze, H., Blier, P., Moeller, H. J., & Kasper, S. F. (2016) · The International Journal of Neuropsychopharmacology · doi:10.1093/ijnp/pyw072 · Cited by 63 · see citing works
Wilens, T. E., Robertson, B., Sikirica, V., Harper, L., Young, J. L., Bloomfield, R., Lyne, A., Rynkowski, G., & Cutler, A. J. (2015) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2015.08.016 · Cited by 63 · see citing works
Parkinsonism and Related Disorders
Slow, E., & Lang, A. E. (2019) · DeckerMed Transitional Year Weekly Curriculum™ · doi:10.2310/tywc.1178 · Cited by 62 · see citing works
Colhoun, H. M., Robinson, J. G., Farnier, M., Cariou, B., Blom, D. J., Kereiakes, D. J., Lorenzato, C., Pordy, R. C., & Chaudhari, U. (2014) · BMC Cardiovascular Disorders · doi:10.1186/1471-2261-14-121 · Cited by 61 · see citing works
Corticosteroid-free Kidney Transplantation Improves Growth
Webb, N. J. A., Douglas, S. E., Rajai, A., Roberts, S. A., Grenda, R., Marks, S. D., Watson, A. R., Fitzpatrick, M. M., Vondrák, K., Maxwell, H., et al. (2015) · Transplantation · doi:10.1097/tp.0000000000000498 · Cited by 61 · see citing works
Monteiro, F., Pereira, M., Canavarro, M. C., & Fonseca, A. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17134679 · Cited by 58 · see citing works
Monticone, M., Ambrosini, E., Rocca, B., Cazzaniga, D., Liquori, V., & Foti, C. (2016) · European Journal of Pain · doi:10.1002/ejp.756 · Cited by 80 · see citing works
Bennett, L., Zhao, Z., Barber, B. R., Zhou, X., Peeters, M., Zhang, J., Xu, F., Wiezorek, J. S., & Douillard, J.-Y. (2011) · British Journal of Cancer · doi:10.1038/bjc.2011.409 · Cited by 59 · see citing works
Litten, R. Z., Castle, I. P., Falk, D. E., Ryan, M., Fertig, J. B., Chen, C. M., & Yi, H. (2013) · Alcoholism Clinical and Experimental Research · doi:10.1111/acer.12197 · Cited by 59 · see citing works
Berlin, I., Chen, H., & Covey, L. S. (2010) · Addiction · doi:10.1111/j.1360-0443.2010.03109.x · Cited by 56 · see citing works
Raz, I., Ziegler, A. G., Linn, T., Schernthaner, G., Bonńici, F., Distiller, L. A., Giordano, C. S. D. S., Giorgino, F., de Vries, L., Mauricio, D., et al. (2014) · Diabetes Care · doi:10.2337/dc13-1391 · Cited by 57 · see citing works
RetractedHylan G-F 20 Improves Hip Osteoarthritis: A Prospective, Randomized Study
Spitzer, A. I., Bockow, B. I., Brander, V. A., Yates, J. W., MacCarter, D. K., Gudger, G. K., Haller, S., Lake, S. L., & Magilavy, D. B. (2010) · The Physician and Sportsmedicine · doi:10.3810/psm.2010.06.1781 · Cited by 57 · see citing works
Soares, C. N., Thase, M. E., Clayton, A. H., Guico‐Pabia, C. J., Focht, K., Jiang, Q. L., Kornstein, S. G., Ninan, P. T., Kane, C. P., & Cohen, L. S. (2010) · Menopause The Journal of The North American Menopause Society · doi:10.1097/gme.0b013e3181d88962 · Cited by 57 · see citing works
Comparisons of Analysis Methods for Proof‐of‐Concept Trials
Karlsson, K. E., Vong, C., Bergstrand, M., Jonsson, E. N., & Karlsson, M. (2013) · CPT Pharmacometrics & Systems Pharmacology · doi:10.1038/psp.2012.24 · Cited by 55 · see citing works
Nicholas, M. K., Asghari, A., Blyth, F., Wood, B. M., Murray, R., McCabe, R., Brnabic, A. J. M., Beeston, L., Corbett, M., Sherrington, C., et al. (2016) · Pain · doi:10.1097/j.pain.0000000000000729 · Cited by 56 · see citing works
Mayer, J. S., Hees, K., Medda, J., Grimm, O., Asherson, P., Bellina, M., Colla, M., Ibáñez, P., Koch, E., Martinez-Nicolas, A., et al. (2018) · Trials · doi:10.1186/s13063-017-2426-1 · Cited by 48 · see citing works
Genome-wide pharmacogenomic study of citalopram-induced side effects in STAR*D
Adkins, D. E., Clark, S. L., Åberg, K. A., Hettema, J. M., Bukszár, J., McClay, J. L., de Souza, R. P., & van den Oord, E. J. C. G. (2012) · Translational Psychiatry · doi:10.1038/tp.2012.57 · Cited by 52 · see citing works
Ogawa, H., Isshiki, T., Kimura, T., Yokoi, H., Nanto, S., Takayama, M., Kitagawa, K., Nishikawa, M., Miyazaki, S., Ikeda, Y., et al. (2015) · Journal of Cardiology · doi:10.1016/j.jjcc.2015.07.019 · Cited by 51 · see citing works
Boettcher, J., Magnusson, K., Marklund, A., Berglund, E., Blomdahl, R., Braun, U., Delin, L., Lundén, C., Sjöblom, K., Sommer, D., et al. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.04.052 · Cited by 47 · see citing works
Schminkey, D. L., von Oertzen, T., & Bullock, L. (2016) · Research in Nursing & Health · doi:10.1002/nur.21724 · Cited by 48 · see citing works
Mewton, L. R., & Andrews, G. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.08.038 · Cited by 49 · see citing works
Monticone, M., Ambrosini, E., Cazzaniga, D., Rocca, B., Motta, L., Cerri, C., Brayda‐Bruno, M., & Lovi, A. (2016) · European Spine Journal · doi:10.1007/s00586-016-4528-y · Cited by 48 · see citing works
Hengartner, M. P. (2019) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2019-111161 · Cited by 48 · see citing works
Shaibani, A. I., Pope, L. E., Thisted, R. A., & Hepner, A. (2012) · Pain Medicine · doi:10.1111/j.1526-4637.2011.01316.x · Cited by 48 · see citing works
Kim, J., Seo, B.-K., Choi, J.-B., Kim, H.-J., Kim, T., Lee, M.-H., Kang, K., Kim, J., Shin, K.-M., Lee, S., et al. (2015) · Trials · doi:10.1186/s13063-015-0857-0 · Cited by 47 · see citing works
Missing Data: Turning Guidance Into Action
Mallinckrodt, C. H., Roger, J., Chuang‐Stein, C., Molenberghs, G., Lane, P. W., O’Kelly, M., Ratitch, B., Xu, L., Gilbert, S., Mehrotra, D. V., et al. (2013) · Statistics in Biopharmaceutical Research · doi:10.1080/19466315.2013.848822 · Cited by 46 · see citing works
Missing Data Handling in Chronic Pain Trials
Kim, Y. (2011) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543406.2011.550112 · Cited by 46 · see citing works
Robinson, J. G., Colhoun, H. M., Bays, H. E., Jones, P. H., Du, Y. E., Hanotin, C., & Donahue, S. (2014) · Clinical Cardiology · doi:10.1002/clc.22327 · Cited by 45 · see citing works
- Authors
- Siddiqui, O., Hung, H. M. J., & O'Neill, R.
- Affiliations
- United States Food and Drug Administration (US)
- a Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research, Food and Drug Administration , Silver Spring, Maryland, USA
- Corresponding author
- Siddiqui, Ohidul
- ORCID
- Hung, H. M. James
- Venue
- Journal of Biopharmaceutical Statistics
- Publisher
- Informa UK Limited
- Published
- 2009-02-12
- Volume
- 19
- Issue
- 2
- Pages
- 227-246
- ISSN
- 1054-3406 (ISSN-L)
- 1520-5711
- OpenAlex
- W2129152730
- Cited by
- 301
- References
- 32
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 7 · 2023: 15 · 2022: 10 · 2021: 12 · 2020: 18 · 2019: 14 · 2018: 16 · 2017: 23 · 2016: 22 · 2015: 25 · 2014: 32 · 2013: 31 · 2012: 20
- FWCI
- 17.15
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Biopharmaceutical Statistics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Medicine › Pediatrics, Perinatology and Child Health › Pharmaceutical studies and practices
- Keywords
- last observation carried forward, missing data imputation, clinical trials, type I error rate, estimation bias, missing completely at random, missing at random, missing not at random, new drug applications, neurological drugs, psychiatric medication, covariance analysis
- MeSH terms
- Algorithms, Clinical Trials as Topic, Computer Simulation, Data Interpretation, Statistical, Humans, Longitudinal Studies, Psychotropic Drugs, Regression Analysis, United States, United States Food and Drug Administration, Reproducibility of Results, Models, Statistical, Bias, Likelihood Functions, Least-Squares Analysis, Databases, Factual
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review