Challenges and Solutions to Pre- and Post-Randomization Subgroup Analyses
Open accessAbstract
Subgroup analyses are commonly performed in the clinical trial setting with the purpose of illustrating that the treatment effect was consistent across different patient characteristics or identifying characteristics that should be targeted for treatment. There are statistical issues involved in performing subgroup analyses, however. These have been given considerable attention in the literature for analyses where subgroups are defined by a pre-randomization feature. Although subgroup analyses are often performed with subgroups defined by a post-randomization feature--including analyses that estimate the treatment effect among compliers--discussion of these analyses has been neglected in the clinical literature. Such analyses pose a high risk of presenting biased descriptions of treatment effects. We summarize the challenges of doing all types of subgroup analyses described in the literature. In particular, we emphasize issues with post-randomization subgroup analyses. Finally, we provide guidelines on how to proceed across the spectrum of subgroup analyses.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Post hoc subgroup analyses have a high risk of spurious (false positive) findings.
“Post hoc subgroup analyses (ie, performed after looking at the data) should be avoided owing to the high risk of spurious (false positive) findings.441”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [441]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · see citing works
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241 · see citing works
Statistical Analysis With Missing Data
Lahiff, M., Little, R. J. A., & Rubin, D. B. (1989) · Journal of the American Statistical Association · doi:10.2307/2289883 · Cited by 17,384 · see citing works
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225 · see citing works
Austin, P. C. (2011) · Multivariate Behavioral Research · doi:10.1080/00273171.2011.568786 · Cited by 12,242 · see citing works
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799 · see citing works
An Introduction to Categorical Data Analysis
Agresti, A. (2006) · Wiley series in probability and statistics · doi:10.1002/0470114754 · Cited by 6,699 · see citing works
Marginal Structural Models and Causal Inference in Epidemiology
Robins, J. M., Hernán, M. Á., & Brumback, B. (2000) · Epidemiology · doi:10.1097/00001648-200009000-00011 · Cited by 5,581 · see citing works
Statistical Analysis With Missing Data
Ghosh, S., Little, R. J. A., & Rubin, D. B. (1988) · Technometrics · doi:10.2307/1269814 · Cited by 4,730 · see citing works
Identification of Causal Effects Using Instrumental Variables
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476902 · Cited by 4,230 · see citing works
Multiple Comparison Procedures
Hochberg, Y., & Tamhane, A. C. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316672 · Cited by 2,389 · see citing works
Multiple Comparison Procedures
Hahs-Vaughn, D., & Lomax, R. (2013) · Routledge eBooks · doi:10.4324/9780203137819-17 · Cited by 2,132 · see citing works
Principal Stratification in Causal Inference
Frangakis, C., & Rubin, D. B. (2002) · Biometrics · doi:10.1111/j.0006-341x.2002.00021.x · Cited by 1,496 · see citing works
Statistics in Medicine — Reporting of Subgroup Analyses in Clinical Trials
Wang, R., Lagakos, S. W., Ware, J. H., Hunter, D. J., & Drazen, J. M. (2007) · New England Journal of Medicine · doi:10.1056/nejmsr077003 · Cited by 1,445 · see citing works
Subgroup analysis and other (mis)uses of baseline data in clinical trials
Assmann, S. F., Pocock, S. J., Enos, L. E., & Kasten, L. E. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02039-0 · Cited by 1,167 · see citing works
Yusuf, S. (1991) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1991.03470010097038 · Cited by 1,014 · see citing works
Statistical Problems in the Reporting of Clinical Trials
Pocock, S., Hughes, M. D., & Lee, R. J. (1987) · New England Journal of Medicine · doi:10.1056/nejm198708133170706 · Cited by 660 · see citing works
Estimating Outcome Distributions for Compliers in Instrumental Variables Models
Imbens, G. W., & Rubin, D. B. (1997) · The Review of Economic Studies · doi:10.2307/2971731 · Cited by 468 · see citing works
Clinical Trials: A Methodologic Perspective
Piantadosi, S. (2005) · Wiley Series in Probability and Statistics · doi:10.1002/0471740136 · Cited by 366 · see citing works
Debate: Subgroup analyses in clinical trials: fun to look at - but don't believe them!
Sleight, P. (2000) · Trials · doi:10.1186/cvm-1-1-025 · Cited by 135 · see citing works
The Value of Statistical Analysis Plans in Observational Research
Thomas, L. E., & Peterson, E. D. (2012) · JAMA · doi:10.1001/jama.2012.9502 · Cited by 104 · see citing works
The Problem of Cogent Subgroups: A Clinicostatistical Tragedy
Feinstein, A. R. (1998) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(98)00004-3 · Cited by 85 · see citing works
Analysis of clinical trial outcomes: Some comments on subgroup analyses
Buyse, M. E. (1989) · Controlled Clinical Trials · doi:10.1016/0197-2456(89)90057-3 · Cited by 62 · see citing works
Bulpitt, C. (1988) · The Lancet · doi:10.1016/s0140-6736(88)92956-x · Cited by 59 · see citing works
Subgroups, treatment effects, and baseline risks: Some lessons from major cardiovascular trials
Parker, A. B., & Naylor, C. D. (2000) · American Heart Journal · doi:10.1067/mhj.2000.106610 · Cited by 57 · see citing works
The use and abuse of subgroup analysis in epidemiological research
Stallones, R. A. (1987) · Preventive Medicine · doi:10.1016/0091-7435(87)90082-x · Cited by 55 · see citing works
Cd, F., & Rp, B. (1983) · PubMed · Cited by 44 · see citing works
Altstein, L. L., & Li, G. (2013) · Biometrics · doi:10.1111/j.1541-0420.2012.01818.x · Cited by 42 · see citing works
Outcome based subgroup analysis: a neglected concern
Hirji, K. F., & Fagerland, M. W. (2009) · Trials · doi:10.1186/1745-6215-10-33 · Cited by 41 · see citing works
Pieper, K. S., Tsiatis, A. A., Davidian, M., Hasselblad, V., Kleiman, N. S., Boersma, E. H., Chang, W.-C., Griffin, J., Armstrong, P. W., Califf, R. M., et al. (2004) · Circulation · doi:10.1161/01.cir.0000112570.97220.89 · Cited by 36 · see citing works
Self-citationPrimer: the fallacy of subgroup analysis
Guillemin, F. (2007) · Nature Clinical Practice Rheumatology · doi:10.1038/ncprheum0528 · Cited by 34 · see citing works
Figures in clinical trial reports: current practice & scope for improvement
Pocock, S., Travison, T. G., & Wruck, L. M. (2007) · Trials · doi:10.1186/1745-6215-8-36 · Cited by 34 · see citing works
Post hoc subgroup analysis and the truth of a clinical trial
Adams, K. F. (1998) · American Heart Journal · doi:10.1016/s0002-8703(98)70116-4 · Cited by 31 · see citing works
Within Trial Variation—A False Trail?
Altman, D. G. (1998) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(98)00005-5 · Cited by 30 · see citing works
Analysis of clinical trial outcomes: Alternative approaches to subgroup analysis
Schneider, B. (1989) · Controlled Clinical Trials · doi:10.1016/0197-2456(89)90056-1 · Cited by 25 · see citing works
Multiplicity considerations for subgroup analysis subject to consistency constraint
Alosh, M., & Huque, M. F. (2013) · Biometrical Journal · doi:10.1002/bimj.201200065 · Cited by 18 · see citing works
An Introduction to Categorical Data Analysis, 2nd Edition by Alan Agresti
Liski, E. P. (2007) · International Statistical Review · doi:10.1111/j.1751-5823.2007.00030_6.x · Cited by 17 · see citing works
Clinical Trials: A Methodological Perspective, 2nd Edn by S. Piantadosi
Davis, J. W. (2006) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2006.00455_13.x · Cited by 7 · see citing works
Debate: A subversive view of subsets - a dissident clinician's opinion
Julian, D. G. (2000) · Trials · doi:10.1186/cvm-1-1-028 · Cited by 6 · see citing works
Hagisawa, K., Nishioka, T., Umemura, S., et al. (2003) · Circulation · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [32]Allison P. Missing data. Series: quantitative applications in the social sciences. A SAGE University Paper; 2001.
Cited by
Sorted by citations
SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials
Hróbjartsson, A., Boutron, I., Hopewell, S., Moher, D., Schulz, K. F., Collins, G. S., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081660 · Cited by 171 · see citing works
- Authors
- Desai, M., Pieper, K. S., & Mahaffey, K.
- Affiliations
- Stanford Medicine (US)
- Duke Clinical Research Institute (US)
- Corresponding author
- Desai, Manisha
- Venue
- Current Cardiology Reports
- Published
- 2014-08-19
- Volume
- 16
- Issue
- 10
- Pages
- 531-531
- ISSN
- 1523-3782 (ISSN-L)
- 1534-3170
- PMC
- PMC4200313
- OpenAlex
- W2087519912
- Cited by
- 26
- References
- 40
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 2 · 2023: 3 · 2022: 2 · 2021: 3 · 2020: 5 · 2019: 1 · 2018: 1 · 2017: 2
- FWCI
- 0.08
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Current Cardiology Reports (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- subgroup analysis, complier average causal effect, treatment effect heterogeneity, bias in treatment effect estimation, clinical trial interpretation
- MeSH terms
- Data Interpretation, Statistical, Humans, Research Design, Bias, Randomized Controlled Trials as Topic, Guidelines as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
