The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Clinical trials with multiple outcomes: A statistical perspective on their design, analysis, and interpretation
Abstract
This article tackles both practical and statistical issues in the handling of multiple outcomes in clinical trials, with relevance to trial design, analysis, and reporting. Specific topics illustrated by examples include: the advantage of prespecifying priorities amongst outcomes and analyses, corrections for multiple significance testing and their limited value, problems with adverse event data, the use of a single global test of significance for clinically related outcomes, the use of a combined outcome for clinical event data, and the value of exploring interrelationships amongst outcomes. The problems in handling multiple outcomes are enhanced by trials being too small, dichotomous attitudes (is the trial "positive" or not?), obsession with p-values, and the manipulative instincts of human nature. While predeclarations of priorities in analysis and reporting of multiple outcomes are important in suppressing distortive claims, it would be unfortunate if too inflexible an approach suppressed unpredictable findings from being seriously considered.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It is very important to emphasize that clinical trial design parameters strongly influence data outcome, interpretation, and conclusions [267, 268].”
From Inorganic nitrite therapy: historical perspective and future directions (2011) · cites this paper as [267]
Citations
Cites
OLSSON, A. G., McMurray, J. J. V., Thorgeirsson, G., Spaulding, C. M., Slattery, J., Pyörälà, K., Færgeman, O., Wright, R. S., Berg, K. M., Wilhelmsen, L., et al. (1994) · The Lancet · Cited by 8,467
A sharper Bonferroni procedure for multiple tests of significance
HOCHBERG, Y. (1988) · Biometrika · doi:10.1093/biomet/75.4.800 · Cited by 4,829
An improved Bonferroni procedure for multiple tests of significance
SIMES, R. J. (1986) · Biometrika · doi:10.1093/biomet/73.3.751 · Cited by 2,396
Final Report on the Aspirin Component of the Ongoing Physicians' Health Study
Steering Committee of the Physicians' Health Study Research Group (1989) · New England Journal of Medicine · doi:10.1056/nejm198907203210301 · Cited by 2,267
Procedures for Comparing Samples with Multiple Endpoints
O’Brien, P. C. (1984) · Biometrics · doi:10.2307/2531158 · Cited by 1,163
Statistical Problems in the Reporting of Clinical Trials
Pocock, S., Hughes, M. D., & Lee, R. J. (1987) · New England Journal of Medicine · doi:10.1056/nejm198708133170706 · Cited by 663
Effect of simvastatin on coronary atheroma: the Multicentre Anti-Atheroma Study (MAAS)
MAAS Investigators (1994) · The Lancet · doi:10.1016/s0140-6736(94)92082-6 · Cited by 644
participants, R. T. (1993) · The Lancet · doi:10.1016/0140-6736(93)90348-k · Cited by 600
Meta-analysis of randomised trials comparing coronary angioplasty with bypass surgery
Pocock, S., Henderson, R. A., Rickards, A. F., Hampton, J. R., King, S. B., Hamm, C. W., Puel, J., Hueb, W. A., Goy, J. J., & Rodriguez, A. (1995) · The Lancet · doi:10.1016/s0140-6736(95)92897-9 · Cited by 527
The Analysis of Multiple Endpoints in Clinical Trials
Pocock, S. J., Geller, N. L., & Tsiatis, A. A. (1987) · Biometrics · doi:10.2307/2531989 · Cited by 524
Cited by
Sorted by citations
Items 101–183 of 183
Gewandter, J. S., Smith, S. M., McKeown, A., Edwards, K., Narula, A., Pawlowski, J. R., Rothstein, D., Desjardins, P. J., Dworkin, S. F., Gross, R. A., et al. (2015) · The Journal of the American Dental Association · doi:10.1016/j.adaj.2014.12.023 · Cited by 12
Assessing Sustained Effects of Communities That Care on Youth Protective Factors
Kim, B. E., Oesterle, S., Hawkins, J. D., & Shapiro, V. B. (2015) · Journal of the Society for Social Work and Research · doi:10.1086/684163 · Cited by 11
Risk factors for human‐directed aggression in a referral level clinical population
Lord, M., Loftus, B., Blackwell, E. J., & Casey, R. A. (2017) · Veterinary Record · doi:10.1136/vr.103638 · Cited by 10
Neva, J. L., Brown, K. E., Wadden, K. P., Mang, C. S., Borich, M. R., Meehan, S. K., & Boyd, L. A. (2019) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-190916 · Cited by 10
Trastuzumab containing regimens for metastatic breast cancer
Moja, L., Compagnoni, A., Brambilla, C., McGowan, J., Nurbhai, M., & Pistotti, V. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006242 · Cited by 10
Guerguerian, A., FARRELL, C. A., Gauthier, M., & Lacroix, J. (2004) · Pediatric Critical Care Medicine · doi:10.1097/01.pcc.0000130991.23217.88 · Cited by 9
Ji, Q., Zhang, F., Su, Q., He, T., Wu, Z., Zhu, K., Chen, X., Wang, Z., Hou, S., & Gui, L. (2024) · BMC Genomics · doi:10.1186/s12864-024-10480-2 · Cited by 8
The effect of statins on muscle symptoms in primary care: the StatinWISE series of 200 N-of-1 RCTs
Herrett, E. L., Williamson, E., Brack, K. E., Perkins, A., Thayne, A., Shakur‐Still, H., Roberts, I., Prowse, D., Beaumont, D., Jamal, Z., et al. (2021) · Health Technology Assessment · doi:10.3310/hta25160 · Cited by 9
Friendship Quality and Social Information Processing in Clinically Anxious Children
Baker, J. R., & Hudson, J. L. (2013) · Child Psychiatry & Human Development · doi:10.1007/s10578-013-0374-x · Cited by 10
A Statistical Perspective on the Design of Drug-Court Studies
Merrall, E. L. C., & Bird, S. M. (2009) · Evaluation Review · doi:10.1177/0193841x08330819 · Cited by 8
Roos, M., Schatz, C., & Stawarczyk, B. (2016) · Materials · doi:10.3390/ma9070512 · Cited by 8
Hu, J., Liu, S., Liu, W., Zhang, H., Chen, J., & Shang, H. (2017) · Annals of Translational Medicine · doi:10.21037/atm.2017.06.06 · Cited by 8
Group psychoeducation for forensic long-term patients with schizophrenia
Aho‐Mustonen, K. (2011) · UEF eRepo (University of Eastern Finland) · Cited by 8
Cassidy, J. M., Carey, J. R., Lu, C., Krach, L. E., Feyma, T. J., Durfee, W. K., & Gillick, B. T. (2015) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2015.09.010 · Cited by 7
Deave, T., Hawkins, A., Kumar, A., Hayes, M., Cooper, N. J., Watson, M. C., Ablewhite, J., Coupland, C. A. C., Sutton, A. J., Majsak‐Newman, G., et al. (2017) · PLoS ONE · doi:10.1371/journal.pone.0172584 · Cited by 7
Symmetry of cortical planning for initiating stepping in sub-acute stroke.
Peters, S., Ivanova, T., Lakhani, B., Boyd, L., Staines, W. R., Handy, T., & Garland, S. (2018) · Clinical Neurophysiology · doi:10.1016/j.clinph.2018.01.018 · Cited by 6
Pratt, C. M. (1999) · European Heart Journal · doi:10.1053/euhj.1999.1850 · Cited by 7
Motor Neurone Disease: Basic Designs, Sample Sizes and Pitfalls
Brooks, B., Sanjak, M., Belden, D., & Waclawik, A. (2001) · Springer eBooks · doi:10.1007/978-1-4471-3787-0_34 · Cited by 8
Eckstein, F., Stamm, T., & Collins, J. E. (2024) · Osteoarthritis Imaging · doi:10.1016/j.ostima.2024.100256 · Cited by 6
Use and Interpretation of Composite End Points in Orthopaedic Trials
Busse, J. W., Bhandari, M., González, I. F., Montori, V. M., & Guyatt, G. H. (2012) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.k.01287 · Cited by 6
Live birth potential of good morphology and vitrified blastocysts presenting abnormal cell divisions
Azzarello, A., Hoest, T., Hay‐Schmidt, A., & Mikkelsen, A. L. (2017) · Reproductive Biology · doi:10.1016/j.repbio.2017.03.004 · Cited by 6
Peters, S., Brown, K. E., Garland, S. J., Staines, R. W., Handy, T. C., & Boyd, L. A. (2018) · European Journal of Neuroscience · doi:10.1111/ejn.14136 · Cited by 6
Do simple screening statistical tools help to detect reporting bias?
Pirracchio, R., Resche‐Rigon, M., Chevret, S., & Journois, D. (2013) · Annals of Intensive Care · doi:10.1186/2110-5820-3-29 · Cited by 6
Evaluation of Antiarrhythmic Drug Efficacy in Patients with an ICD:
Pratt, C. M., Camm, A. J., Bigger, J. T., Breithardt, G., Campbell, R. W. F., Epstein, A. E., KAPPENBERGER, L. J., Kück, K., Pocock, S., Saksena, S. V., et al. (1999) · Journal of Cardiovascular Electrophysiology · doi:10.1111/j.1540-8167.1999.tb00212.x · Cited by 6
Mournet, A. M., Gunin, G. B., Shinall, J. A., Brennan, E., Jadav, N., Istvan, E. M., Kleiman, E. M., & Bal, V. H. (2024) · Autism Research · doi:10.1002/aur.3095 · Cited by 5
Peters, S., Brown, K. E., Garland, S. J., Staines, R. W., Handy, T. C., Francisco, B. A., & Boyd, L. A. (2018) · Brain Research · doi:10.1016/j.brainres.2018.11.027 · Cited by 5
When to Adjust for Multiple Testing: A Unifying Guiding Principle
Hoffmann, S., Lemster, S., Collins, G., Hapfelmeier, A., Heinze, G., Mayr, A. J., Schmid, M. C., Wilcke, J. C., & Boulesteix, A. (2026) · Biometrical Journal · doi:10.1002/bimj.70148 · Cited by 5
50 Tips for Clinical Trialists
Harvey, L. A., Glinsky, J. V., & Herbert, R. (2017) · Brain Impairment · doi:10.1017/brimp.2017.30 · Cited by 4
Greene, T. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1138-8_16 · Cited by 4
Assessing Change in Intervention Research: The Benefits of Composite OutcomesPreprint
Moreau, D., & Wiebels, K. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/t9hw3 · Cited by 7
Collin, C., Eyraud, C., Martin, P., Michel, M., Roux, É. L., & Alberti, C. (2025) · BMC Medicine · doi:10.1186/s12916-025-04245-1 · Cited by 5
Strengthening Interactions between Statisticians and Collaborators: Objectives and Sample Sizes
Dressler, E. V. M. (2013) · Journal of Biometrics & Biostatistics · doi:10.4172/2155-6180.1000e127 · Cited by 3
Hussein, H., Taylor, R. S., Manyara, A. M., Purvis, A., Emsley, R., Duarte, R. V., Wells, V., Jiang, Y., & Dibben, G. O. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111724 · Cited by 3
The promise and pitfalls of composite endpoints in sepsis and COVID ‐19 clinical trials
Brown, P. M., Rogne, T., & Solligård, E. (2020) · Pharmaceutical Statistics · doi:10.1002/pst.2070 · Cited by 3
Goldie, F. C., Fulton, R. L., Dawson, J., Bluhmki, E., & Lees, K. R. (2013) · International Journal of Stroke · doi:10.1111/ijs.12131 · Cited by 3
Epidemiology 4 Multiplicity in randomised trials I: endpoints and treatments
Schulz, K. F., & Grimes, D. A. (2005) · Cited by 3
Statistical Issues for Combining Multiple Caries Diagnostics for Demonstrating Caries Efficacy
Katz, B. P., & Huntington, E. (2004) · Journal of Dental Research · doi:10.1177/154405910408301s22 · Cited by 3
Ang, D. C., Davuluri, S., Kaplan, S. G., Keefe, F. J., Rini, C. K., Miles, C. M., & Chen, H. (2024) · Trials · doi:10.1186/s13063-024-08158-x · Cited by 2
PARAMETRIC CONTROL OF FAMILYWISE ERROR RATES WITH DEPENDENT P-VALUES
Blakesley, R. E. (2008) · D-Scholarship@Pitt (University of Pittsburgh) · Cited by 2
Modeling and Inferential Thoughts for Consecutive Gap Times Observed with Death and Censoring
Geffray, S. (2013) · Journal of the Iranian Statistical Society · Cited by 2
Simultaneous sequential monitoring of efficacy and safety led to masking of effects
van EEKELEN, R., Hoop, E. O., & van der Tweel, I. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.02.024 · Cited by 2
A Bayesian Group Sequential Approach for Multiple Endpoints
Mor, M. K., & Anderson, S. (2005) · Sequential Analysis · doi:10.1081/sqa-200046829 · Cited by 2
Prolonged cardiac activation, stressful events and worry in daily life
Pieper, S. (2000) · Leiden Repository (Leiden University) · Cited by 2
Fiscus, S. A., Kovács, A., Petch, L., Hu, C., WIZNIA, A. A., MOFENSON, L. M., Yogev, R., McIntosh, K., Pelton, S. I., Napravnik, S., et al. (2007) · AIDS Research and Therapy · doi:10.1186/1742-6405-4-2 · Cited by 2
Fulton, R. L. (2013) · Enlighten: Theses (The University of Glasgow) · Cited by 2
Statistical Issues in Clinical Trials with Multiple Primary Endpoints
Sozu, T., Sugimoto, T., & Hamasaki, T. (2013) · Japanese Journal of Biometrics · doi:10.5691/jjb.34.35 · Cited by 1
Scales to climb borderline personalities: when science goes nowhere
Moja, L., Virgili, G., Liberati, A., Gensini, G. F., Gusinu, R., & Conti, A. (2007) · Internal and Emergency Medicine · doi:10.1007/s11739-007-0086-y · Cited by 1
Lucas, B., Trick, W. E., Evans, A. T., Benjamin, ., Clarke, P., Varkey, A. B., Mathew, S. C., & Weinstein, R. A. (2017) · Cited by 1
Leach, J. M., Aban, I., & Cutter, G. (2020) · Springer eBooks · doi:10.1007/978-3-319-52677-5_70-1 · Cited by 1
Williams, L. R., Voysey, M., Pollard, A. J., & Grassly, N. C. (2025) · AJE Advances Research in Epidemiology · doi:10.1093/ajeadv/uuaf016 · Cited by 1
Leach, J. M., Aban, I., & Cutter, G. (2022) · Springer eBooks · doi:10.1007/978-3-319-52636-2_70 · Cited by 1
Kavelaars, X., Mulder, J., & Kaptein, M. C. (2024) · Multivariate Behavioral Research · doi:10.1080/00273171.2024.2337340 · Cited by 1
Western, M. J. (2016) · The University of Bath Online Publications Store (The University of Bath) · Cited by 1
Conclusions: The Two-Front War
(2006) · Springer eBooks · doi:10.1007/0-387-21813-0_15 · Cited by 1
Botelho, R. V. (2010) · Journal of Neurosurgery Spine · doi:10.3171/2009.10.spine09836 · Cited by 1
An exploration of evidence synthesis methods for adverse events
Warren, F. C. (2010) · Figshare · Cited by 1
Collective Evidence in Drug Evaluation
Li, Q. H. (2015) · ICSA book series in statistics · doi:10.1007/978-3-319-12694-4_10
Sequential sample size re-estimation in clinical trials with multiple co-primary endpoints
Ntambwe, L. I. (2026) · Warwick Research Archive Portal (University of Warwick) · doi:10.82444/warw.33685981
18 Definitive Phase III and Phase IV Clinical Trials
Davis, B. R., & Baraniuk, S. (2007) · Handbook of statistics · doi:10.1016/s0169-7161(07)27018-x
Multivariate global testing and adaptive designs
Minas, G. (2026) · Warwick Research Archive Portal (University of Warwick) · doi:10.82444/warw.33634321
Kim, B. (2014) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York)
Analyzing and Interpreting Clinical Trials
Spino, C. (2016) · Gastroenterology · doi:10.1053/j.gastro.2016.02.065
Studien und statistische Ergebnisse in der Forensik
Ramsthaler, F., Birngruber, C. G., Kettner, M., Verhoff, M. A., & Burkholder, I. (2016) · Rechtsmedizin · doi:10.1007/s00194-015-0063-0
Kannan, S., & Gowri, S. (2015) · Journal of Scientometric Research · doi:10.4103/2320-0057.167245
Optimization of Repetitive Transcranial Magnetic Stimulation With Priming In Chronic Stroke
Cassidy, J. M. (2014) · University of Minnesota Digital Conservancy (University of Minnesota)
Gokhale, S. (2015) · Journal of Health Policy & Outcomes Research · doi:10.7365/jhpor.2015.2.2
Etudes cliniques avec paramètres composites (composite endpoints)
Kleist, P. (2008) · Forum Médical Suisse ‒ Swiss Medical Forum · doi:10.4414/fms.2008.06659
Multiple Outcomes in Heart Failure Research: Composite Endpoints and Multivariate Modelling
Brown, P. M. (2017) · ERA: Education and Research Archive (University of Alberta) · doi:10.7939/r3901zt68
BAYESIAN BASED SEMIPARAMETRIC MULTIDIMENSIONAL APPROACHES TO ANALYSIS PARKINSON'S DISEASE
Zhang, J. (2018) · Digital Commons-TMC (Texas Medical Center)
Pocock, S. (2006) · Encyclopedia of Environmetrics · doi:10.1002/9780470057339.vam043
(2013) · Wiley series in probability and statistics · doi:10.1002/9781118458167.refs
Electoral TurnoversPreprint
Marx, B., Pons, V., & Rollet, V. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4039485
Addington, J. (2012) · Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature · doi:10.3410/f.13853958.15292058
Sozu, T., Sugimoto, T., Hamasaki, T., & Evans, S. R. (2015) · SpringerBriefs in statistics · doi:10.1007/978-3-319-22005-5_6
RETRACTED ARTICLE: A Skew-Normal Bayesian Semi-parametric Latent Trait Linear Mixed Effect Model
He, W., Zgibor, J. C., & Kim, J. (2024) · Journal of Statistical Theory and Practice · doi:10.1007/s42519-024-00373-8
RetractedAng, D. C., Davuluri, S., Kaplan, S. G., Keefe, F. J., Rini, C. K., Miles, C. M., & Chen, H. (2024) · Research Square · doi:10.21203/rs.3.rs-3924330/v1
Registration and multiple outcome testing in the HEALing communities study
Gorman, D. M., & Fitzpatrick, B. G. (2025) · International Journal of Drug Policy · doi:10.1016/j.drugpo.2025.105028
Ang, D. C., Davuluri, S., Kaplan, S., Keefe, F., Rini, C., Miles, C., & Chen, H. (2024) · Figshare · doi:10.6084/m9.figshare.26722578
Ang, D. C., Davuluri, S., Kaplan, S., Keefe, F., Rini, C., Miles, C., & Chen, H. (2024) · Figshare · doi:10.6084/m9.figshare.26722575
Ang, D. C., Davuluri, S., Kaplan, S., Keefe, F., Rini, C., Miles, C., & Chen, H. (2024) · Figshare · doi:10.6084/m9.figshare.25854683
Ang, D. C., Davuluri, S., Kaplan, S. G., Keefe, F., Rini, C., Miles, C., & Chen, H. (2024) · Figshare · doi:10.6084/m9.figshare.25854652
Pedersen, T. B., Poulsen, C. A., Lund, M., Albitar, M. X., Østergaard, L. D., Feddersen, S., Lund, L., & Poulsen, M. H. (2026) · Scandinavian Journal of Urology · doi:10.2340/sju.v61.45464
Metadata
- Authors
- Pocock, S. J.
- Affiliations
- London School of Hygiene & Tropical Medicine (GB)
- Corresponding author
- Pocock, Stuart J.
- ORCID
- Pocock, Stuart J.
- Venue
- Controlled Clinical Trials
- Publisher
- Elsevier BV
- Published
- 1997-12
- Volume
- 18
- Issue
- 6
- Pages
- 530-545
- ISSN
- 0197-2456 (ISSN-L)
- OpenAlex
- W2021056870
- Cited by
- 184
- References
- 16
- Citations by year
- 2026: 4 · 2025: 4 · 2024: 14 · 2022: 3 · 2021: 5 · 2020: 4 · 2019: 1 · 2018: 4 · 2017: 11 · 2016: 8 · 2015: 19 · 2014: 9 · 2013: 14 · 2012: 8
- FWCI
- 2.33
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Controlled Clinical Trials (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- multiple outcomes, clinical trial design, outcome prioritization, trial reporting, p-values, trial size
- MeSH terms
- Clinical Trials as Topic, Coronary Disease, Humans, Research Design, Cluster Analysis, Outcome Assessment, Health Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review