Clinical trials with multiple outcomes: a statistical perspective on their design, analysis, and interpretation

Pocock, Stuart J.

doi:10.1016/s0197-2456(97)00008-1

Cited by 184Statistical Methods in Clinical TrialsMeta-analysis and systematic reviewsStatistical Methods and Bayesian Inference

Clinical trials with multiple outcomes: a statistical perspective on their design, analysis, and interpretation

Pocock, Stuart J.·1997·Controlled Clinical Trials

DOI 10.1016/s0197-2456(97)00008-1

Open this paper in CoChat — free account

Abstract

This article tackles both practical and statistical issues in the handling of multiple outcomes in clinical trials, with relevance to trial design, analysis, and reporting. Specific topics illustrated by examples include: the advantage of prespecifying priorities amongst outcomes and analyses, corrections for multiple significance testing and their limited value, problems with adverse event data, the use of a single global test of significance for clinically related outcomes, the use of a combined outcome for clinical event data, and the value of exploring interrelationships amongst outcomes. The problems in handling multiple outcomes are enhanced by trials being too small, dichotomous attitudes (is the trial "positive" or not?), obsession with p-values, and the manipulative instincts of human nature. While predeclarations of priorities in analysis and reporting of multiple outcomes are important in suppressing distortive claims, it would be unfortunate if too inflexible an approach suppressed unpredictable findings from being seriously considered.

Integrity signals

Registry

Crossref · 10.1016/s0197-2456(97)00008-1

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Clinical trial design parameters strongly influence data outcome, interpretation, and conclusions.
“It is very important to emphasize that clinical trial design parameters strongly influence data outcome, interpretation, and conclusions [267, 268].”
From Inorganic nitrite therapy: historical perspective and future directions (2011) · cites this paper as [267]

Randomised trial of cholesterol lowering in 4444 patients with coronary heart disease: the Scandinavian Simvastatin Survival Study (4S)
OLSSON, A. G., McMurray, J. J. V., Thorgeirsson, G., Spaulding, C. M., Slattery, J., Pyörälà, K., Færgeman, O., Wright, R. S., Berg, K. M., Wilhelmsen, L., et al. (1994) · The Lancet · Cited by 8,467 · see citing works
A sharper Bonferroni procedure for multiple tests of significance
HOCHBERG, Y. (1988) · Biometrika · doi:10.1093/biomet/75.4.800 · Cited by 4,829 · see citing works
An improved Bonferroni procedure for multiple tests of significance
SIMES, R. J. (1986) · Biometrika · doi:10.1093/biomet/73.3.751 · Cited by 2,396 · see citing works
Final Report on the Aspirin Component of the Ongoing Physicians' Health Study
Steering Committee of the Physicians' Health Study Research Group (1989) · New England Journal of Medicine · doi:10.1056/nejm198907203210301 · Cited by 2,269 · see citing works
Procedures for Comparing Samples with Multiple Endpoints
O’Brien, P. C. (1984) · Biometrics · doi:10.2307/2531158 · Cited by 1,163 · see citing works
Effect of simvastatin on coronary atheroma: the Multicentre Anti-Atheroma Study (MAAS)
M-heart investigators (1994) · The Lancet · doi:10.1016/s0140-6736(94)92082-6 · Cited by 674 · see citing works
Statistical Problems in the Reporting of Clinical Trials
Pocock, S., Hughes, M. D., & Lee, R. J. (1987) · New England Journal of Medicine · doi:10.1056/nejm198708133170706 · Cited by 660 · see citing works
Coronary angioplasty versus coronary artery bypass surgery: the Randomised Intervention Treatment of Angina (RITA) trial
participants, R. T. (1993) · The Lancet · doi:10.1016/0140-6736(93)90348-k · Cited by 600 · see citing works
Meta-analysis of randomised trials comparing coronary angioplasty with bypass surgery
Pocock, S., Henderson, R. A., Rickards, A. F., Hampton, J. R., King, S. B., Hamm, C. W., Puel, J., Hueb, W. A., Goy, J. J., & Rodriguez, A. (1995) · The Lancet · doi:10.1016/s0140-6736(95)92897-9 · Cited by 527 · see citing works
The Analysis of Multiple Endpoints in Clinical Trials
Pocock, S. J., Geller, N. L., & Tsiatis, A. A. (1987) · Biometrics · doi:10.2307/2531989 · Cited by 524 · see citing works
Sulfinpyrazone in the Prevention of Sudden Death after Myocardial Infarction
The Anturane Reinfarction Trial Research Group (1980) · New England Journal of Medicine · doi:10.1056/nejm198001313020502 · Cited by 346 · see citing works
A Comparison of Bronchodilator Therapy with or without Inhaled Corticosteroid Therapy for Obstructive Airways Disease
Kerstjens, H. A. M., Brand, P. L. P., Hughes, M. D., Robinson, N. J., Postma, D. S., SLUITER, H. J., Bleecker, E. R., Dekhuijzen, P. N. R., de Jong, P. M., Mengelers, H. J. J., et al. (1992) · New England Journal of Medicine · doi:10.1056/nejm199211123272003 · Cited by 339 · see citing works
Quality of Life, Employment Status, and Anginal Symptoms After Coronary Angioplasty or Bypass Surgery
Pocock, S., Henderson, R. A., Seed, P. T., Treasure, T., & Hampton, J. (1996) · Circulation · doi:10.1161/01.cir.94.2.135 · Cited by 280 · see citing works
Multiplicity Considerations in the Design and Analysis of Clinical Trials
Cook, R. J., & Farewell, V. T. (1996) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2983471 · Cited by 257 · see citing works
Randomized Trial of Cholesterol Lowering in 4,444 Patients with Coronary Heart Disease
&NA; (1995) · Journal of Cardiopulmonary Rehabilitation · doi:10.1097/00008483-199505000-00010 · Cited by 166 · see citing works
An approximate likelihood ratio test for a normal mean vector with nonnegative components with application to clinical trials
Tang, D., Gnecco, C., & Geller, N. L. (1989) · Biometrika · doi:10.1093/biomet/76.3.577 · Cited by 122 · see citing works

Flagged first, then by citations

Sorted by citations

Adjusting for multiple testing—when and how?
Bender, R., & Lange, S. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00314-0 · Cited by 3,060 · see citing works
. Cognitive behavior therapy for schizophrenia: effect sizes, clinical models, and methodological rigor.
Wykes, T., Steel, C., Everitt, B., & Tarrier, N. (2007) · Schizophrenia Bulletin · doi:10.1093/schbul/sbm114 · Cited by 1,029 · see citing works
Follow-up of glycemic control and cardiovascular outcomes in type 2 diabetes
Hayward, R. A., Reaven, P. D., Wiitala, W. L., Bahn, G. D., Reda, D. J., Ge, L., McCarren, M., Duckworth, W. C., & Emanuele, N. V. (2015) · New England Journal of Medicine · doi:10.1056/nejmoa1414266 · Cited by 668 · see citing works
Trastuzumab containing regimens for early breast cancer
Moja, L., Tagliabue, L., Balduzzi, S., Parmelli, E., Pistotti, V., Guarneri, V., & D’Amico, R. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006243.pub2 · Cited by 610 · see citing works
The efficacy of single‐session motivational interviewing in reducing drug consumption and perceptions of drug‐related risk and harm among young people: results from a multi‐site cluster randomized trial
McCambridge, J., & Strang, J. (2003) · Addiction · doi:10.1111/j.1360-0443.2004.00564.x · Cited by 435 · see citing works
An introduction to multiplicity issues in clinical trials: the what, why, when and how
Li, G., Taljaard, M., Van den Heuvel, E. R., Levine, M. A., Cook, D. J., Wells, G. A., Devereaux, P. J., & Thabane, L. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw320 · Cited by 381 · see citing works
Multiplicity in randomised trials I: endpoints and treatments
Schulz, K. F., & Grimes, D. A. (2005) · The Lancet · doi:10.1016/s0140-6736(05)66461-6 · Cited by 366 · see citing works
A review of modern multiple hypothesis testing, with particular attention to the false discovery proportion
Farcomeni, A. (2007) · Statistical Methods in Medical Research · doi:10.1177/0962280206079046 · Cited by 222 · see citing works
Methodologic discussions for using and interpreting composite endpoints are limited, but still identify major concerns
González, I. F., Permanyer-Miralda, G., Busse, J. W., Bryant, D., Montori, V. M., Alonso‐Coello, P., Walter, S. D., & Guyatt, G. H. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.10.020 · Cited by 220 · see citing works
Analyzing multiple endpoints in clinical trials of pain treatments: IMMPACT recommendations
Turk, D. C., Dworkin, R. H., McDermott, M., Bellamy, N., Burke, L. B., Chandler, J. M., Cleeland, C. S., Cowan, P., Dimitrova, R., Farrar, J. T., et al. (2008) · Pain · doi:10.1016/j.pain.2008.06.025 · Cited by 218 · see citing works
Trastuzumab-containing regimens for metastatic breast cancer
Balduzzi, S., Mantarro, S., Guarneri, V., Tagliabue, L., Pistotti, V., Moja, L., & D’Amico, R. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006242.pub2 · Cited by 217 · see citing works
Comparisons of methods for multiple hypothesis testing in neuropsychological research
Blakesley, R. E., Mazumdar, S., Dew, M. A., Houck, P. R., Tang, G., Reynolds, C. F., & Butters, M. A. (2009) · Neuropsychology · doi:10.1037/a0012850 · Cited by 206 · see citing works
Factors associated with aggressive responses in pet dogs
Hsu, Y., & Sun, L. (2010) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2010.01.013 · Cited by 200 · see citing works
Analysis of Randomized Controlled Trials
Peduzzi, P. N. (2002) · Epidemiologic Reviews · doi:10.1093/epirev/24.1.26 · Cited by 204 · see citing works
Effect of Citicoline on Functional and Cognitive Status Among Patients With Traumatic Brain Injury
ZAFONTE, R. D., Bagiella, E., Ansel, B. M., Novack, T. A., Friedewald, W. T., Hesdorffer, D. C., Timmons, S. D., Jallo, J., Eisenberg, H. M., Hart, T., et al. (2012) · JAMA · doi:10.1001/jama.2012.13256 · Cited by 187 · see citing works
Electrical stimulation enhances sensory recovery: A randomized controlled trial
Wong, J. N., Olson, J. L., Morhart, M. J., & Chan, K. M. (2015) · Annals of Neurology · doi:10.1002/ana.24397 · Cited by 177 · see citing works
Selective serotonin re-uptake inhibitors (SSRIs) for preventing migraine and tension-type headaches
Moja, L., Cusi, C., Sterzi, R., & Canepari, C. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002919.pub2 · Cited by 136 · see citing works
A trial of clarithromycin for the treatment of suboptimally controlled asthma
Sutherland, E. R., King, T. S., Icitovic, N., Ameredes, B. T., Bleecker, E. R., Boushey, H. A., Calhoun, W. J., Castro, M., Cherniack, R. M., Chinchilli, V. M., et al. (2010) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2010.07.024 · Cited by 136 · see citing works
Large sample inference for a win ratio analysis of a composite outcome based on prioritized components
Bebu, I., & Lachin, J. M. (2015) · Biostatistics · doi:10.1093/biostatistics/kxv032 · Cited by 134 · see citing works
Key Issues in End Point Selection for Heart Failure Trials: Composite End Points
Neaton, J. D., Gray, G. W., Zuckerman, B. D., & Konstam, M. A. (2005) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2005.08.350 · Cited by 127 · see citing works
Efficacy of psychosocial interventions in cancer care: Evidence is weaker than it first looks
Coyne, J. C., Lepore, S. J., & Palmer, S. C. (2006) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm3202_5 · Cited by 122 · see citing works
Inorganic nitrite therapy: historical perspective and future directions
Kevil, C. G., Kolluru, G. K., Pattillo, C. B., & Giordano, T. (2011) · Free Radical Biology and Medicine · doi:10.1016/j.freeradbiomed.2011.04.042 · Cited by 113 · see citing works
Deterioration over time in effect of Motivational Interviewing in reducing drug consumption and related risk among young people
McCambridge, J., & Strang, J. (2005) · Addiction · doi:10.1111/j.1360-0443.2005.01013.x · Cited by 112 · see citing works
A Randomized Controlled Trial of a Web-Based Early Intervention for Children and their Parents Following Unintentional Injury
Cox, C. M., Kenardy, J. A., & Hendrikz, J. K. (2009) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsp095 · Cited by 103 · see citing works
A Randomized Trial Testing the Superiority of a Postdischarge Care Management Model for Stroke Survivors
Allen, K. R., Hazelett, S. E., Jarjoura, D. G., Hua, K., Wright, K., Weinhardt, J., & Kropp, D. J. (2009) · Journal of Stroke and Cerebrovascular Diseases · doi:10.1016/j.jstrokecerebrovasdis.2009.02.002 · Cited by 92 · see citing works
Design and Analysis of Studies with Binary- Event Composite Endpoints
Mascha, E. J., & Sessler, D. I. (2011) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e31821796d3 · Cited by 90 · see citing works
Effectiveness of a postdischarge care management model for stroke and transient ischemic attack: A randomized trial
Allen, K. R., Hazelett, S. E., Jarjoura, D. G., Wickstrom, G. C., Hua, K., Weinhardt, J., & Wright, K. (2002) · Journal of Stroke and Cerebrovascular Diseases · doi:10.1053/jscd.2002.127106 · Cited by 86 · see citing works
Sealant and Fluoride Varnish in Caries: a Randomized Trial
Bravo, M., Montero, J., Bravo, J. J., Baca, P., & Llodrá, J. C. (2005) · Journal of Dental Research · doi:10.1177/154405910508401209 · Cited by 85 · see citing works
The effect of subantimicrobial-dose–doxycycline periodontal therapy on serum biomarkers of systemic inflammation
Payne, J. B., Golub, L. M., Stoner, J. A., Lee, H., Reinhardt, R. A., Sorsa, T. A., & Slepian, M. J. (2011) · The Journal of the American Dental Association · doi:10.14219/jada.archive.2011.0165 · Cited by 83 · see citing works
Some issues with composite endpoints in clinical trials
Chi, G. Y. H. (2005) · Fundamental and Clinical Pharmacology · doi:10.1111/j.1472-8206.2005.00370.x · Cited by 79 · see citing works
The effectiveness and cost-effectiveness of a mindfulness training programme in schools compared with normal school provision (MYRIAD): study protocol for a randomised controlled trial
the MYRIAD team, Kuyken, W., Nuthall, E., Byford, S., Crane, C., Dalgleish, T., Ford, T., Greenberg, M. T., Ukoumunne, O. C., Viner, R. M., et al. (2017) · Trials · doi:10.1186/s13063-017-1917-4 · Cited by 77 · see citing works
Single versus multiple drug focus in substance abuse clinical trials research.
Rounsaville, B., Petry, N., & Carroll, K. (2003) · Drug and Alcohol Dependence · doi:10.1016/s0376-8716(03)00033-4 · Cited by 81 · see citing works
Biological Importance and Statistical Significance
Lovell, D. P. (2013) · Journal of Agricultural and Food Chemistry · doi:10.1021/jf401124y · Cited by 75 · see citing works
Sample size determination in clinical trials with multiple co‐primary binary endpoints
Sozu, T., Sugimoto, T., & Hamasaki, T. (2010) · Statistics in Medicine · doi:10.1002/sim.3972 · Cited by 68 · see citing works
Effects of 2- vs 4-Week Attending Physician Inpatient Rotations on Unplanned Patient Revisits, Evaluations by Trainees, and Attending Physician Burnout
Lucas, B., Trick, W. E., Evans, A. T., Mba, B., Smith, J. S., Das, K., Clarke, P., Varkey, A. B., Mathew, S. C., & Weinstein, R. A. (2012) · JAMA · doi:10.1001/jama.2012.36522 · Cited by 68 · see citing works
Evidence-based medicine: assessment of knowledge of basic epidemiological and research methods among medical doctors
Novack, L., Jotkowitz, A., Knyazer, B., & Novack, V. (2006) · Postgraduate Medical Journal · doi:10.1136/pgmj.2006.049262 · Cited by 68 · see citing works
Selective serotonin reuptake inhibitors (SSRIs) and serotonin-norepinephrine reuptake inhibitors (SNRIs) for the prevention of migraine in adults
Banzi, R., Cusi, C., Randazzo, C., Sterzi, R., Tedesco, D., & Moja, L. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002919.pub3 · Cited by 69 · see citing works
Trastuzumab containing regimens for early breast cancer
Moja, L., Brambilla, C., Compagnoni, A., & Pistotti, V. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006243 · Cited by 72 · see citing works
LOPSIDED REASONING ON LOPSIDED TESTS AND MULTIPLE COMPARISONS
Hurlbert, S. H., & Lombardi, C. M. (2012) · Australian & New Zealand Journal of Statistics · doi:10.1111/j.1467-842x.2012.00652.x · Cited by 65 · see citing works
Selective serotonin reuptake inhibitors (SSRIs) and serotonin-norepinephrine reuptake inhibitors (SNRIs) for the prevention of tension-type headache in adults
Banzi, R., Cusi, C., Randazzo, C., Sterzi, R., Tedesco, D., & Moja, L. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011681 · Cited by 66 · see citing works
Surrogates, substudies, and real clinical end points in trials of drug-eluting stents**Editorials published in the Journal of the American College of Cardiologyreflect the views of the authors and do not necessarily represent the views of JACCor the American College of Cardiology.
Kereiakes, D. J., Kuntz, R. E., Mauri, L., & Krucoff, M. W. (2005) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2005.01.005 · Cited by 65 · see citing works
Are multiple primary outcomes analysed appropriately in randomised controlled trials? A review
Vickerstaff, V., Ambler, G., King, M., Nazareth, I., & Omar, R. (2015) · Contemporary Clinical Trials · doi:10.1016/j.cct.2015.07.016 · Cited by 64 · see citing works
PROactive: time for a critical appraisal
Betteridge, D. J., DeFronzo, R. A., & CHILTON, R. J. (2008) · European Heart Journal · doi:10.1093/eurheartj/ehn114 · Cited by 53 · see citing works
Less-Affected Hand Function in Children With Hemiparetic Unilateral Cerebral Palsy: A Comparison Study With Typically Developing Peers
Rich, T. L., Menk, J. S., Rudser, K. D., Feyma, T. J., & Gillick, B. T. (2017) · Neurorehabilitation and neural repair · doi:10.1177/1545968317739997 · Cited by 53 · see citing works
Subantimicrobial dose doxycycline effects on alveolar bone loss in post‐menopausal women
Payne, J. B., Stoner, J. A., Nummikoski, P. V., Reinhardt, R. A., Goren, A. D., Wolff, M. S., Lee, H., Lynch, J. C., Valente, R. M., & Golub, L. M. (2007) · Journal Of Clinical Periodontology · doi:10.1111/j.1600-051x.2007.01115.x · Cited by 51 · see citing works
Clinical attrition due to biased preclinical assessments of potential efficacy.
Lindner, M. (2007) · Pharmacology and Therapeutics · doi:10.1016/j.pharmthera.2007.05.002 · Cited by 52 · see citing works
Factors affecting power of tests for multiple binary outcomes
Mascha, E. J., & Imrey, P. B. (2010) · Statistics in Medicine · doi:10.1002/sim.4066 · Cited by 46 · see citing works
Power and sample size for clinical trials when efficacy is required in multiple endpoints: application to an Alzheimer's treatment trial
Xiong, C., Yu, K. F., Gao, F., Yan, Y., & Zhang, Z. (2005) · Clinical Trials · doi:10.1191/1740774505cn112oa · Cited by 44 · see citing works
Intercentre reproducibility of cardiac apparent diffusion coefficient and fractional anisotropy in healthy volunteers
Tunnicliffe, E. M., Scott, A. D., Ferreira, P., Ariga, R., McGill, L., Nielles‐Vallespin, S., Neubauer, S., Pennell, D. J., Robson, M. D., & Firmin, D. N. (2014) · Journal of Cardiovascular Magnetic Resonance · doi:10.1186/1532-429x-16-31 · Cited by 43 · see citing works
Serial treatments of primed low-frequency rTMS in stroke: Characteristics of responders vs. nonresponders
Carey, J. R., Deng, H., Gillick, B. T., Cassidy, J. M., Anderson, D. C., Zhang, L., & Thomas, W. (2014) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-130358 · Cited by 42 · see citing works
Interhemispheric Pathways Are Important for Motor Outcome in Individuals with Chronic and Severe Upper Limb Impairment Post Stroke
Hayward, K. S., Neva, J. L., Mang, C. S., Peters, S., Wadden, K. P., Ferris, J. K., & Boyd, L. A. (2017) · Neural Plasticity · doi:10.1155/2017/4281532 · Cited by 40 · see citing works
Cardiovascular Risk Associated With Acarbose Versus Metformin as the First-Line Treatment in Patients With Type 2 Diabetes: A Nationwide Cohort Study
Chang, C., Chang, Y., Lin, J., Chen, S., Chuang, L., & Lai, M. (2015) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2014-2443 · Cited by 40 · see citing works
Assessing Change in Intervention Research: The Benefits of Composite Outcomes
Moreau, D., & Wiebels, K. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245920931930 · Cited by 39 · see citing works
Composite Endpoints for Clinical Trials
Kleist, P. (2007) · International Journal of Pharmaceutical Medicine · doi:10.2165/00124363-200721030-00001 · Cited by 40 · see citing works
Research Fundamentals: Selection and Development of Clinical Outcome Measures
Singer, A. J., Thode, H. C., & Hollander, J. E. (2000) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2000.tb02249.x · Cited by 39 · see citing works
Efficacy of sub‐antimicrobial dose doxycycline in post‐menopausal women: clinical outcomes
Reinhardt, R. A., Stoner, J. A., Golub, L. M., Wolff, M. S., Lee, H., Meinberg, T. A., Lynch, J. C., RYAN, M. E., Sorsa, T. A., & Payne, J. B. (2007) · Journal Of Clinical Periodontology · doi:10.1111/j.1600-051x.2007.01114.x · Cited by 37 · see citing works
The effects of long-acting bronchodilators on total mortality in patients with stable chronic obstructive pulmonary disease
Kliber, A., Lynd, L. D., & Sin, D. D. (2010) · Respiratory Research · doi:10.1186/1465-9921-11-56 · Cited by 36 · see citing works
Design and Analysis of Group Sequential Clinical Trials with Multiple Primary Endpoints
Kosorok, M. R., Yuan-jun, S., & DeMets, D. L. (2004) · Biometrics · doi:10.1111/j.0006-341x.2004.00146.x · Cited by 36 · see citing works
Prominent medical journals often provide insufficient information to assess the validity of studies with negative results
Hebert, R. S., Wright, S. M., Dittus, R. S., & Elasy, T. A. (2002) · Journal of Negative Results in BioMedicine · doi:10.1186/1477-5751-1-1 · Cited by 36 · see citing works
A Comparison of Primed Low-frequency Repetitive Transcranial Magnetic Stimulation Treatments in Chronic Stroke
Cassidy, J. M., Chu, H., Anderson, D. C., Krach, L. E., Snow, L. M., Kimberley, T. J., & Carey, J. R. (2015) · Brain stimulation · doi:10.1016/j.brs.2015.06.007 · Cited by 35 · see citing works
Bicycling to school improves the cardiometabolic risk factor profile: a randomised controlled trial
Østergaard, L. J., Børrestad, L. A. B., Tarp, J., & Andersen, L. B. (2012) · BMJ Open · doi:10.1136/bmjopen-2012-001307 · Cited by 35 · see citing works
Clinical Trials in Rehabilitation: Single or Multiple Outcomes?
Bagiella, E. (2009) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2009.08.133 · Cited by 35 · see citing works
The International Conference on Harmonization Guideline “Statistical Principles for Clinical Trials”: Issues in Applying the Guideline in Practice
Phillips, A., Ebbutt, A., France, L., & Morgan, D. D. (2000) · Drug Information Journal · doi:10.1177/009286150003400202 · Cited by 34 · see citing works
Reporting of primary analyses and multiplicity adjustment in recent analgesic clinical trials: ACTTION systematic review and recommendations
Gewandter, J. S., Smith, S. M., McKeown, A., Burke, L. B., Hertz, S. H., Hunsinger, M., Katz, N. P., Lin, A. H., McDermott, M., Rappaport, B. A., et al. (2013) · Pain · doi:10.1016/j.pain.2013.11.009 · Cited by 33 · see citing works
Design and Rationale for a Randomized, Controlled Trial of Interpersonal Psychotherapy and Citalopram for Depression in Coronary Artery Disease (CREATE)
Frasure‐Smith, N., Koszycki, D., Swenson, J. R., Baker, B., van Zyl, L. T., Laliberté, M.-A., Abramson, B. L., Lambert, J., Gravel, G., & Lespérance, F. (2006) · Psychosomatic Medicine · doi:10.1097/01.psy.0000195833.68482.27 · Cited by 33 · see citing works
Polycystic ovary syndrome and chronic autoimmune thyroiditis
Mayrink, J., Benetti‐Pinto, C. L., Garmes, H. M., Jales, R. M., & Juliato, C. R. T. (2014) · Gynecological Endocrinology · doi:10.3109/09513590.2014.958990 · Cited by 32 · see citing works
Immunological Response to Highly Active Antiretroviral Therapy in Children with Clinically Stable HIV‐1 Infection
Rosenblatt, H. M., Stanley, K., Song, L. Y., Johnson, G. M., WIZNIA, A. A., Nachman, S., & Krogstad, P. A. (2005) · The Journal of Infectious Diseases · doi:10.1086/431597 · Cited by 32 · see citing works
Immune responses to stress after stress management training in patients with rheumatoid arthritis
de Brouwer, S. J. M., van Middendorp, H., Kraaimaat, F. W., Radstake, T. R. D. J., Joosten, I., Donders, A. R. T., Eijsbouts, A. M. M., Koulil, S. S., van Riel, P. L. C. M., & Evers, A. W. M. (2013) · Arthritis Research & Therapy · doi:10.1186/ar4390 · Cited by 31 · see citing works
(262) Reporting of primary analyses and multiplicity adjustment in recent analgesic clinical trials: ACTTION systematic review and recommendations
Gewandter, J. S., Smith, S. M., McKeown, A., Burke, L. B., Hertz, S. H., Hunsinger, M., Katz, N. P., Lin, A. H., McDermott, M., Rappaport, B. A., et al. (2014) · Journal of Pain · doi:10.1016/j.jpain.2014.01.170 · Cited by 29 · see citing works
Some design issues in a community intervention trial
Feng, Z., & Thompson, B. (2002) · Controlled Clinical Trials · doi:10.1016/s0197-2456(02)00206-4 · Cited by 29 · see citing works
Multiple analyses in clinical trials: sound science or data dredging?
Lord, S. J., Gebski, V., & KEECH, A. C. (2004) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2004.tb06376.x · Cited by 29 · see citing works
Multidimensional individualised Physical ACTivity (Mi-PACT) – a technology-enabled intervention to promote physical activity in primary care: study protocol for a randomised controlled trial
Peacock, O. J., Western, M. J., Batterham, A. M., Stathi, A., Standage, M., Tapp, A., Bennett, P., & Thompson, D. (2015) · Trials · doi:10.1186/s13063-015-0892-x · Cited by 28 · see citing works
Transparency of Outcome Reporting and Trial Registration of Randomized Controlled Trials Published in the Journal of Consulting and Clinical Psychology
Azar, M., Riehm, K. E., McKay, D., & Thombs, B. D. (2015) · PLoS ONE · doi:10.1371/journal.pone.0142894 · Cited by 28 · see citing works
Crossed Random Effect Models for Multiple Outcomes in a Study of Teratogenesis
Coull, B. A., Hobert, J. P., RYAN, L. M., & Holmes, L. B. (2001) · Journal of the American Statistical Association · doi:10.1198/016214501753381841 · Cited by 28 · see citing works
Utility‐based designs for randomized comparative trials with categorical outcomes
Murray, T. A., Thall, P. F., & Yuan, Y. (2016) · Statistics in Medicine · doi:10.1002/sim.6989 · Cited by 27 · see citing works
Randomized Controlled Trials 5: Determining the Sample Size and Power for Clinical Trials and Cohort Studies
Greene, T. (2015) · Methods in molecular biology · doi:10.1007/978-1-4939-2428-8_13 · Cited by 26 · see citing works
A comparison of methods for analysing multiple outcome measures in randomised controlled trials using a simulation study
Vickerstaff, V., Ambler, G., & Omar, R. Z. (2021) · Biometrical Journal · doi:10.1002/bimj.201900040 · Cited by 26 · see citing works
Examining the predictive validity of NIH peer review scores
Lindner, M. D., & Nakamura, R. K. (2015) · PLOS ONE · doi:10.1371/journal.pone.0126938 · Cited by 24 · see citing works
Computer-enhanced interventions for drug use and HIV risk in the emergency room: Preliminary results on psychological precursors of behavior change
Bonar, E. E., Walton, M. A., Cunningham, R. M., Chermack, S. T., Bohnert, A. M., Barry, K. L., Booth, B. M., & Blow, F. C. (2013) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2013.08.005 · Cited by 24 · see citing works
A robust method for comparing two treatments in a confirmatory clinical trial via multivariate time‐to‐event methods that jointly incorporate information from longitudinal and time‐to‐event data
Saville, B. R., Herring, A. H., & Koch, G. G. (2009) · Statistics in Medicine · doi:10.1002/sim.3740 · Cited by 24 · see citing works
Practical usage of O'Brien's OLS and GLS statistics in clinical trials
Dallow, N. S., Leonov, S. L., & Roger, J. (2007) · Pharmaceutical Statistics · doi:10.1002/pst.268 · Cited by 24 · see citing works
Immune responses to stress in rheumatoid arthritis and psoriasis
de Brouwer, S. J. M., van Middendorp, H., Stormink, C., Kraaimaat, F., Joosten, I., Radstake, T. R. D. J., de Jong, E. M. G. J., Schalkwijk, J., Donders, A. R. T., Eijsbouts, A. M. M., et al. (2014) · Rheumatology · doi:10.1093/rheumatology/keu221 · Cited by 23 · see citing works
Mindfulness meditation training to reduce symptom distress in transplant patients: rationale, design, and experience with a recycled waitlist
Gross, C. R., Kreitzer, M. J., Reilly-Spong, M., Winbush, N. Y., Schomaker, E. K., & Thomas, W. T. B. (2009) · Clinical Trials · doi:10.1177/1740774508100982 · Cited by 23 · see citing works
Characteristics of clinical trials assessing antimicrobial treatment of bovine respiratory disease, 1970-2005
O'Connor, A. M. C., Wellman, N. G., Rice, M., & Funk, L. (2010) · Journal of the American Veterinary Medical Association · doi:10.2460/javma.237.6.701 · Cited by 20 · see citing works
Evidence‐Based Review Series on Endoscopic Thyroidectomy: Real Progress and Future Trends
Dionigi, G. (2008) · World Journal of Surgery · doi:10.1007/s00268-008-9834-z · Cited by 21 · see citing works
High dimensional predictions of suicide risk in 4.2 million US Veterans using ensemble transfer learning
Dhaubhadel, S., Ganguly, K., Ribeiro, R. M., Cohn, J. D., Hyman, J. M., Hengartner, N., Kolade, B., Singley, A., Tanmoy Bhattacharya, A., Finley, P. D., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-51762-9 · Cited by 20 · see citing works
Riluzole for Degenerative Cervical Myelopathy
Fehlings, M. G., Pedro, K. M., Alvi, M. A., Badhiwala, J. H., Ahn, H., Farhadi, H. F., Shaffrey, C. I., Nassr, A. N., Mummaneni, P. V., Arnold, P. M., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.15643 · Cited by 19 · see citing works
Bayesian multiple imputation for missing multivariate longitudinal data from a Parkinson's disease clinical trial
Luo, S., Lawson, A. B., He, B., Elm, J. J., & Tilley, B. C. (2012) · Statistical Methods in Medical Research · doi:10.1177/0962280212469358 · Cited by 19 · see citing works
Measuring outcomes in the longer term after a stroke
Murray, J., Young, J. B., & Förster, A. (2009) · Clinical Rehabilitation · doi:10.1177/0269215509341525 · Cited by 19 · see citing works
Effectiveness of a Theory‐Driven Nutritional Education Program in Improving Calcium Intake among Older Mauritian Adults
Bhurosy, T., & Jeewon, R. (2013) · The Scientific World JOURNAL · doi:10.1155/2013/750128 · Cited by 18 · see citing works
Fetal Hemodynamic Parameters in Low Risk Pregnancies: Doppler Velocimetry of Uterine, Umbilical, and Middle Cerebral Artery
Figueira, C. O., Surita, F. G., Dertkigil, M. S. J., Pereira, S. L., Bennini, J. R., Morais, S. S., Mayrink, J., & Cecatti, J. G. (2016) · The Scientific World JOURNAL · doi:10.1155/2016/1693704 · Cited by 18 · see citing works
Cardiovascular Risk Associated with Methotrexate versus Retinoids in Patients with Psoriasis: A Nationwide Taiwanese Cohort Study
Tsai, M.-H., Chan, T. C.-C., Lee, M., & Lai, M. (2021) · Clinical Epidemiology · doi:10.2147/clep.s305126 · Cited by 16 · see citing works
Electoral Turnovers
Marx, B., Pons, V., & Rollet, V. (2022) · National Bureau of Economic Research · doi:10.3386/w29766 · Cited by 16 · see citing works
Recommendations for Improving the Design, Conduct, and Analysis of Clinical Trials in Hospital‐Acquired Pneumonia and Ventilator‐Associated Pneumonia
Powers, J. H. (2010) · Clinical Infectious Diseases · doi:10.1086/653036 · Cited by 14 · see citing works
Electoral Turnovers
Marx, B., Pons, V., & Rollet, V. (2024) · The Review of Economic Studies · doi:10.1093/restud/rdae108 · Cited by 13 · see citing works
Evidence-based approaches to reducing in-patient stress – Part 2: Synthetic pheromone preparations
Hewson, C. J. (2014) · Veterinary Nursing Journal · doi:10.1111/vnj.12140 · Cited by 13 · see citing works
Self-modeling regression for multivariate curve data
Coull, B. A., & STAUDENMAYER, J. W. (2004) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 13 · see citing works
Modelling Multivariate Outcomes in Hierarchical Data, with Application to Cluster Randomised Trials
Turner, R., Omar, R., & Thompson, S. G. (2005) · Biometrical Journal · doi:10.1002/bimj.200310147 · Cited by 13 · see citing works
Type-2 diabetes mellitus reduces cortical thickness and decreases oxidative metabolism in sensorimotor regions after stroke
Ferris, J. K., Peters, S., Brown, K. E., Tourigny, K., & Boyd, L. A. (2017) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1177/0271678x17703887 · Cited by 12 · see citing works
Effect of dietary fibre type on physical activity and behaviour in kennelled dogs
Bosch, G., Beerda, B., van de Hoek, E., Hesta, M., van der Poel, A. F. B., Janssens, G. P. J., & Hendriks, W. H. (2009) · Applied Animal Behaviour Science · doi:10.1016/j.applanim.2009.08.001 · Cited by 12 · see citing works

ELSEVIER Clinical Trials with Multiple Outcomes: A Statistical Perspective on their Design, Analysis, and Interpretation
Pocock, S. (2015)
Statistical Problems in the Reporting of Clinical Trials
Pocock, S., Hughes, M. D., & Lee, R. J. (1987) · New England Journal of Medicine · doi:10.1056/nejm198708133170706 · Cited by 660 · see citing works
Editorial
Dawkins, J. V. (1993) · European Polymer Journal · doi:10.1016/0014-3057(93)90069-r
Clinical trials must cope better with multiplicity
Wittes, J. T. (2012) · Nature Medicine · doi:10.1038/nm1112-1607 · Cited by 6 · see citing works
The use and abuse of multiple outcomes in randomized controlled depression trials
Tyler, K. M. B., Normand, S. T., & Horton, N. J. (2010) · Contemporary Clinical Trials · doi:10.1016/j.cct.2010.12.007 · Cited by 49 · see citing works

Authors: Pocock, S. J.
Affiliations: London School of Hygiene & Tropical Medicine (GB)
Corresponding author: Pocock, Stuart J.
ORCID: Pocock, Stuart J.
Venue: Controlled Clinical Trials
Publisher: Elsevier BV
Published: 1997-12
Volume: 18
Issue: 6
Pages: 530-545
ISSN: 0197-2456 (ISSN-L)
DOI: 10.1016/s0197-2456(97)00008-1
OpenAlex: W2021056870
Cited by: 184
References: 16
Citations by year: 2026: 4 · 2025: 4 · 2024: 14 · 2022: 3 · 2021: 5 · 2020: 4 · 2019: 1 · 2018: 4 · 2017: 11 · 2016: 8 · 2015: 19 · 2014: 9 · 2013: 14 · 2012: 8
FWCI: 2.33
Type: journal article
Language: en
Open access: closed
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations: Controlled Clinical Trials (publishedVersion)
PubMed (publishedVersion)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords: multiple outcomes, clinical trial design, outcome prioritization, trial reporting, p-values, trial size
MeSH terms: Clinical Trials as Topic, Coronary Disease, Humans, Research Design, Cluster Analysis, Outcome Assessment, Health Care

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Clinical trials with multiple outcomes: a statistical perspective on their design, analysis, and interpretation

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks