The Challenge of Subgroup Analyses — Reporting without Distorting
The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Subgroup analyses are an important part of the analysis of a comparative clinical trial. However, they are commonly overinterpreted14 and can lead to further research that is misguided or, worse, to suboptimal patient care.Consider a randomized, clinical trial designed to determine whether a new treatment is more effective than an established treatment and assessed with a test, based on all randomized patients, of the null hypothesis that the treatments have equal efficacy, as measured in terms of the primary end point. Then, subgroup analyses are conducted to assess whether different types of patients respond differently to the new . . .
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In addition, analyses of HRQoL, IBS symptoms and stool characteristics data were performed in the subgroup of those definitely constipated subjects (n = 19), as defined on the only basis of <3 stools per week; a lower significance level (P < 0.001) was applied for judging results in this subgroup, as recommended.36”
Citations
Cites
Clopidogrel and Aspirin versus Aspirin Alone for the Prevention of Atherothrombotic Events
Bhatt, D. L., Fox, K. A. A., Hacke, W., Berger, P. B., Black, H. R., Boden, W. E., Cacoub, P., Cohen, E. A., Creager, M. A., Easton, J. D., et al. (2006) · New England Journal of Medicine · doi:10.1056/nejmoa060989 · Cited by 2,837
Subgroup analysis and other (mis)uses of baseline data in clinical trials
Assmann, S. F., Pocock, S. J., Enos, L. E., & Kasten, L. E. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02039-0 · Cited by 1,167
Yusuf, S. (1991) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1991.03470010097038 · Cited by 1,014
Enoxaparin versus Unfractionated Heparin with Fibrinolysis for ST-Elevation Myocardial Infarction
Antman, E. M., Morrow, D. A., McCabe, C. H., Murphy, S. A., Ruda, M., Sadowski, Z., Budaj, A., López-Sendón, J. L., Guneri, S., Jiang, F., et al. (2006) · New England Journal of Medicine · doi:10.1056/nejmoa060898 · Cited by 609
Cornell, R. G., Wolf, F. M., Bailar, J. C., & Mosteller, F. (1989) · Journal of the American Statistical Association · doi:10.2307/2289981 · Cited by 295
Yusuf, S. (1991) · JAMA · doi:10.1001/jama.266.1.93 · Cited by 129
Subgroups, treatment effects, and baseline risks: Some lessons from major cardiovascular trials
Parker, A. B., & Naylor, C. D. (2000) · American Heart Journal · doi:10.1067/mhj.2000.106610 · Cited by 57
Bailar, J. C. (1986) · Cited by 28
Cited by
Husereau, D., Drummond, M., Petrou, S., Carswell, C., Moher, D., Greenberg, D., Augustovski, F., Briggs, A. H., Mauskopf, J., & Loder, E. (2013) · Value in Health · doi:10.1016/j.jval.2013.02.002 · Cited by 1,998
“[48] Lagakos SW.”
BackgroundAntman, E. M., Hand, M., Armstrong, P. W., Bates, E. R., Green, L. A., Halasyamani, L. K., Hochman, J. S., Krumholz, H. M., Lamas, G. A., Mullany, C. J., et al. (2008) · Circulation · doi:10.1161/circulationaha.107.188209 · Cited by 1,562
“to provide the point estimate and 95% CI of the treatment effect in various subgroups to allow readers to see the range of possible treatment effects.(82) The writing group encourages randomization of additional patients in future trials to clarify a number of questions, such as 1) the benefits of reviparin compared w…”
BackgroundStatistics in Medicine — Reporting of Subgroup Analyses in Clinical Trials
Wang, R., Lagakos, S. W., Ware, J. H., Hunter, D. J., & Drazen, J. M. (2007) · New England Journal of Medicine · doi:10.1056/nejmsr077003 · Cited by 1,445
Self-citationPfeffer, M. A., Claggett, B., Assmann, S. F., Boineau, R., Anand, I. S., Clausell, N., Desai, A. S., Diaz, R., Fleg, J. L., Gordeev, I., et al. (2015) · Circulation · doi:10.1161/circulationaha.114.013255 · Cited by 1,065
Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK): Explanation and Elaboration
Altman, D. G., McShane, L. M., Sauerbrei, W., & Taube, S. E. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001216 · Cited by 1,013
“However, a significant effect in one group and a nonsignificant effect in the other is not sound evidence that the effect of the marker differs by subgroup [93,94].”
BackgroundAntman, E. M., Armstrong, P. W., Green, L. A., Halasyamani, L. K., Hochman, J. S., Krumholz, H. M., Lamas, G. A., Smith, S. C., Hand, M., Bates, E. R., et al. (2008) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2007.10.001 · Cited by 892
Bolland, M. J., Grey, A., Avenell, A., Gamble, G. D., & Reid, I. R. (2011) · BMJ · doi:10.1136/bmj.d2040 · Cited by 851
“The current analysis has some limitations. We used the publicly accessible, limited access dataset of the WHI clinical trials for these analyses, so the analysis is limited to the information available in this dataset. Several vascular end points are potentially evaluable; we pre-specified those which allowed the most…”
MethodHusereau, D., Drummond, M., Augustovski, F., de Bekker-Grob, E., Briggs, A. H., Carswell, C., Caulley, L., Chaiyakunapruk, N., Greenberg, D., Loder, E., et al. (2022) · Value in Health · doi:10.1016/j.jval.2021.10.008 · Cited by 847
Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine
(2021) · New England Journal of Medicine · doi:10.1056/nejmc2036242 · Cited by 831
Vitamin D supplementation and prevention of type 2 diabetes
Pittas, A. G., Dawson-Hughes, B., Sheehan, P., Ware, J. H., Knowler, W. C., Aroda, V. R., Brodsky, I., Ceglia, L., Chadha, C., Chatterjee, R., et al. (2019) · New England Journal of Medicine · doi:10.1056/nejmoa1900906 · Cited by 673
Subgroup identification from randomized clinical trial data
Foster, J. C., Taylor, J. M. G., & Ruberg, S. J. (2011) · Statistics in Medicine · doi:10.1002/sim.4322 · Cited by 555
Sample Size Planning for Statistical Power and Accuracy in Parameter Estimation
Maxwell, S. E., Kelley, K., & Rausch, J. R. (2008) · Annual Review of Psychology · doi:10.1146/annurev.psych.59.103006.093735 · Cited by 542
“…on interim analyses (e.g., Jennison & Turnbull 2006, Mehta & Patel 2006), complications in planning subgroup analyses (e.g., Brookes et al. 2004, Lagakos 2006), the impact of noncompliance on power in randomized studies (e.g., Jo 2002), and Bayesian approaches to sample size planning (e.g.,…”
BackgroundPersonalized evidence based medicine: predictive approaches to heterogeneous treatment effects
Kent, D. M., Steyerberg, E., & van Klaveren, D. (2018) · BMJ · doi:10.1136/bmj.k4245 · Cited by 479
Estimating treatment effect heterogeneity in randomized program evaluation
Imai, K., & Ratkovic, M. (2013) · The Annals of Applied Statistics · doi:10.1214/12-aoas593 · Cited by 480
“However, the most commonly used method, subgroup analysis, is often inappropriate and remains one of the most debated practices in the medical research community [e.g., Rothwell (2005), Lagakos (2006)].”
MethodThe Predictive Approaches to Treatment effect Heterogeneity (PATH) statement
Kent, D. M., Paulus, J. K., van Klaveren, D., D'Agostino, R., Goodman, S., Hayward, R., Ioannidis, J. P. A., Patrick-Lake, B., Morton, S., Pencina, M., et al. (2020) · Annals of Internal Medicine · doi:10.7326/m18-3667 · Cited by 469
Reporting recommendations for tumor marker prognostic studies (REMARK): explanation and elaboration
Altman, D. G., McShane, L. M., Sauerbrei, W., & Taube, S. E. (2012) · BMC Medicine · doi:10.1186/1741-7015-10-51 · Cited by 372
“However, a significant effect in one group and a non-significant effect in the other is not sound evidence that the effect of the marker differs by subgroup [186,187].”
BackgroundPisters, K. M. W., Evans, W. K., Azzoli, C. G., Kris, M. G., Smith, C. A., Desch, C. E., Somerfield, M. R., Brouwers, M. C., Darling, G., Ellis, P. M., et al. (2007) · Journal of Clinical Oncology · doi:10.1200/jco.2007.14.1226 · Cited by 363
GUYONNET, D., CHASSANY, O., DUCROTTE, P., PICARD, C., MOURET, M., MERCIER, C. ‐., & MATUCHANSKY, C. (2007) · Alimentary Pharmacology & Therapeutics · doi:10.1111/j.1365-2036.2007.03362.x · Cited by 365
Decreased response to inhaled steroids in overweight and obese asthmatic children
Forno, E., Lescher, R., Strunk, R., Weiss, S., Fuhlbrigge, A., & Celedón, J. C. (2011) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2010.12.010 · Cited by 341
“The probability of a type I error in subgroup analysis increases significantly with the number of subgroups tested.(51) Second, CAMP excluded children with severe asthma, and thus we had limited ability to evaluate modification of the effect of inhaled budesonide by BMI in these children.”
BackgroundSubgroup Analysis via Recursive Partitioning
Su, X., Tsai, C., Wang, H., Nickerson, D. M., Li, B., & Rosset, S. (2009) · Journal of International Crisis and Risk Communication Research · Cited by 324
Associations of Insulin Resistance and Adiponectin With Mortality in Women With Breast Cancer
Duggan, C., Irwin, M. L., Xiao, L., Henderson, K. D., Smith, A. W., Baumgartner, R. N., Baumgartner, K. B., Bernstein, L., Ballard-Barbash, R., & McTiernan, A. (2011) · Journal of Clinical Oncology · doi:10.1200/jco.2009.26.4473 · Cited by 294
“006 to assess statistical significance.(24) All P values are two-sided.”
BackgroundKaul, S., & Diamond, G. A. (2010) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2009.06.065 · Cited by 269
Bolland, M. J., Grey, A., Gamble, G. D., & Reid, I. R. (2011) · The American Journal of Clinical Nutrition · doi:10.3945/ajcn.111.015032 · Cited by 260
“To minimize these risks, we prespecified the variable of interest (personal calcium or vitamin D supplement use) before carrying out any analyses, assessed its effect by using interaction tests, and followed recommended approaches for subgroup analysis and interpretation (24, 25).”
BackgroundMethodPrecision medicine in cardiology
Antman, E. M., & Loscalzo, J. (2016) · Nature Reviews Cardiology · doi:10.1038/nrcardio.2016.101 · Cited by 250
A basic study design for expedited safety signal evaluation based on electronic healthcare data
Schneeweiss, S. (2010) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.1926 · Cited by 255
“19,20 Patient subgroups that should be considered include duration of drug use and dose categories of the study drug.”
BackgroundCaffeine for Apnea of Prematurity Trial: Benefits May Vary in Subgroups
Davis, P. G., Schmidt, B., Roberts, R. S., Doyle, L. W., Asztalos, E., Haslam, R., Sinha, S., & Tin, W. (2010) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2009.09.069 · Cited by 228
Wykrzykowska, J. J., Garg, S., Girasis, C., de Vries, T., Morel, M.-A., van Es, G.-A., Buszman, P., Linke, A., Ischinger, T., Klauss, V., et al. (2010) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2010.03.044 · Cited by 220
Colchicine in Patients with Chronic Coronary Disease
(2021) · New England Journal of Medicine · doi:10.1056/nejmc2034992 · Cited by 217
Gender medicine and oncology: report and consensus of an ESMO workshop
Wagner, A. D., Oertelt-Prigione, S., Adjei, A., Buclin, T., Cristina, V., Csajka, C., Coukos, G., Dafni, U., Dotto, G.-P., Ducreux, M., et al. (2019) · Annals of Oncology · doi:10.1093/annonc/mdz414 · Cited by 214
Chemotherapy for isolated locoregional recurrence of breast cancer (CALOR): a randomised trial
Aebi, S., Gelber, S., Anderson, S. J., Láng, I., Robidoux, A., Martín, M., Nortier, J. W. R., Paterson, A. H. G., Rimawi, M. F., Cañada, J. M. B., et al. (2014) · The Lancet Oncology · doi:10.1016/s1470-2045(13)70589-8 · Cited by 209
Chan, A.-W., Hrobjartsson, A., Jorgensen, K. J., Gotzsche, P. C., & Altman, D. G. (2008) · BMJ · doi:10.1136/bmj.a2299 · Cited by 202
“Selective reporting of a favourable subset of analyses done can lead to inaccurate interpretation of results, similar to selective publication or selective outcome reporting.”
ResultProne position for acute respiratory failure in adults
Bloomfield, R., Noble, D. W., & Sudlow, A. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008095.pub2 · Cited by 193
Estimating the magnitude of trastuzumab effects within patient subgroups in the HERA trial
Untch, M., Gelber, R. D., Jackisch, C., Procter, M., Baselga, J., Bell, R., Cameron, D., Bari, M., Smith, I., Leyland-Jones, B., et al. (2008) · Annals of Oncology · doi:10.1093/annonc/mdn005 · Cited by 197
“Subgroup analyses must be interpreted with caution due to the increased likelihood of false-positive and false-negative results arising by the play of chance [10].”
BackgroundKent, D. M., van Klaveren, D., Paulus, J. K., D'Agostino, R., Goodman, S., Hayward, R., Ioannidis, J. P. A., Patrick-Lake, B., Morton, S., Pencina, M., et al. (2020) · Annals of Internal Medicine · doi:10.7326/m18-3668 · Cited by 179
Cornelissen, J. J., van der Holt, B., Verhoef, G. E. G., van 't Veer, M. B., van Oers, M. H. J., Schouten, H. C., Ossenkoppele, G., Sonneveld, P., Maertens, J., van Marwijk Kooy, M., et al. (2009) · Blood · doi:10.1182/blood-2008-07-168625 · Cited by 173
“Median age, y (range) 31 (16-55) 26 (15-54)”
Model-Based Recursive Partitioning for Subgroup Analyses
Seibold, H., Zeileis, A., & Hothorn, T. (2016) · The International Journal of Biostatistics · doi:10.1515/ijb-2015-0032 · Cited by 167
“To be conservative one can see the confidence intervals for parameters in the subgroup-specific models as shown in our examples as a range of possible values and hence as a measure of variability rather than significance [42].”
BackgroundWakelin, K. E., Perman, G., & Simonds, L. M. (2022) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2586 · Cited by 170
“N J = F i v e F a c e t M i nd f u l n e ss Q u e s t i o nn a i r e : N o n - j ud g m e n t s ub s c a l e ( B a e r e t a l. , 2006 ) .”
BackgroundInfluentialPerinatal Systemic Inflammatory Response Syndrome and Retinopathy of Prematurity
Sood, B. G., Madan, A., Saha, S., Schendel, D., Thorsen, P., Skogstrand, K., Hougaard, D., Shankaran, S., & Carlo, W. (2010) · Pediatric Research · doi:10.1203/pdr.0b013e3181d01a36 · Cited by 171
“Therefore, reliance should be placed on the observed effect size rather than on statistical significance testing as a basis for decision making (36).”
Hancock, M., Herbert, R. D., & Maher, C. G. (2009) · Physical Therapy · doi:10.2522/ptj.20080351 · Cited by 164
Detecting moderator effects using subgroup analyses
Wang, R., & Ware, J. H. (2013) · Prevention Science · doi:10.1007/s11121-011-0221-x · Cited by 165
“Figure 1 in Lagakos (2006) depicts the probability that multiple subgroup analyses will yield at least one, two, or three false positive findings as a function of the number of subgroup analyses conducted.”
Chlorthalidone vs. Hydrochlorothiazide for Hypertension–Cardiovascular Events
Ishani, A., Cushman, W. C., Leatherman, S. M., Lew, R., Woods, P. A., Glassman, P. A., Taylor, A. A., Hau, C., Klint, A., Huang, G. D., et al. (2022) · New England Journal of Medicine · doi:10.1056/nejmoa2212270 · Cited by 172
Overinterpretation and Misreporting of Diagnostic Accuracy Studies: Evidence of “Spin”
Ochodo, E. A., de Haan, M. C., Reitsma, J. B., Hooft, L., Bossuyt, P. M., & Leeflang, M. M. G. (2013) · Radiology · doi:10.1148/radiol.12120527 · Cited by 158
“Selective reporting of results in abstract 7 (6) [2–10] 4 (8) [3–20]”
BackgroundInfluentialManchikanti, L., Hirsch, J. A., & Smith, H. S. (2008) · Pain Physician · doi:10.36076/ppj.2008/11/717 · Cited by 161
VanderWeele, T. J., & Knol, M. J. (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-154-10-201105170-00008 · Cited by 167
Subgroup Analysis via Recursive PartitioningPreprint
Su, X., Tsai, C., Wang, H., Nickerson, D. M., & Li, B. (2009) · SSRN Electronic Journal · doi:10.2139/ssrn.1341380 · Cited by 164
Hodgkins, P., Shaw, M., Coghill, D., & Hechtman, L. (2012) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-012-0286-5 · Cited by 147
Lehtinen, P., Ruohola, A., Vanto, T., Vuorinen, T., Ruuskanen, O., & Jartti, T. T. (2006) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2006.11.003 · Cited by 140
“To avoid multiple testing and consequent false-positive results, we performed interaction tests instead of testing subgroups, for which we provided only estimates with CIs.41 Although our children had not had previous wheezing requiring outpatient visit or hospitalization, we cannot assure that previous viral infectio…”
MethodStatistical Considerations for Subgroup Analyses
Xiaofei, W., Piantadosi, S., Le‐Rademacher, J. G., & Mandrekar, S. J. (2020) · Journal of Thoracic Oncology · doi:10.1016/j.jtho.2020.12.008 · Cited by 144
Ensrud, K. E., Stock, J. L., Barrett-Connor, E. L., Grady, D. G., Mosca, L. J., Khaw, K., Zhao, Q., Agnusdei, D., & Cauley, J. A. (2007) · Journal of Bone and Mineral Research · doi:10.1359/jbmr.070904 · Cited by 146
“The risk subgroup analysis was prespecified, but our results are subject to the limitations of subgroup analyses.((42)) Although”
ResultPharmacologic Management of Chronic Low Back Pain: Synthesis of the Evidence
White, A. P., Arnold, P. M., Norvell, D. C., Ecker, E. D., & Fehlings, M. G. (2011) · Spine · doi:10.1097/brs.0b013e31822f178f · Cited by 147
Janjigian, Y. Y., Park, B. J., Zakowski, M. F., Ladanyi, M., Pao, W. J., D’Angelo, S. P., Kris, M. G., Shen, R., Zheng, J., & Azzoli, C. G. (2010) · Journal of Thoracic Oncology · doi:10.1097/jto.0b013e318202bffe · Cited by 134
Understanding of interaction (subgroup) analysis in clinical trials
Brankovic, M., Kardys, I., Steyerberg, E. W., Lemeshow, S., Markovic, M., Rizopoulos, D., & Boersma, E. (2019) · European Journal of Clinical Investigation · doi:10.1111/eci.13145 · Cited by 130
“RERI=OR T ×OR B ×OR T×B −ORT −ORB +1 (7)”
Is There an “Abortion Trauma Syndrome”? Critiquing the Evidence
Robinson, G. E., Stotland, N. L., Russo, N. F., Lang, J. A., & Occhiogrosso, M. (2009) · Harvard Review of Psychiatry · doi:10.1080/10673220903149119 · Cited by 140
Szilagyi, P. G., Albertin, C. S., Humiston, S. G., Rand, C. M., Schaffer, S. J., Brill, H., Stankaitis, J. A., Yoo, B., Blumkin, A. K., & Stokley, S. K. (2013) · Academic Pediatrics · doi:10.1016/j.acap.2013.01.002 · Cited by 120
Manchikanti, L., Singh, V., Smith, H. S., & Hirsch, J. A. (2009) · Pain Physician · doi:10.36076/ppj.2009/12/73 · Cited by 120
Bernhard, M., Becker, T. K., Nowe, T., Mohorovicic, M., Sikinger, M., Brenner, T., Richter, G. M., Radeleff, B., Meeder, P., Büchler, M. W., et al. (2007) · Resuscitation · doi:10.1016/j.resuscitation.2006.09.014 · Cited by 113
Garg, S., Serruys, P. W. J. C., Silber, S., Wykrzykowska, J. J., van Geuns, R., Richardt, G., Buszman, P. E., Kelbæk, H., van Boven, A. J., Hofma, S. H., et al. (2011) · JACC: Cardiovascular Interventions · doi:10.1016/j.jcin.2011.01.008 · Cited by 112
Goveia, M. G., Rodriguez, Z. M., Dallas, M. J., Itzler, R. F., Boslego, J. W., Heaton, P. M., & DiNubile, M. J. (2007) · The Pediatric Infectious Disease Journal · doi:10.1097/inf.0b013e31814521cb · Cited by 112
“Definitive conclusions about safety and efficacy can not be confidently drawn from a small post hoc subgroup analysis.(9,10) Nonetheless, our findings suggest that the pentavalent rotavirus vaccine can be safely and effectively administered to healthy premature infants.”
BackgroundMichalek, J. E., & Pavúk, M. (2008) · Journal of Occupational and Environmental Medicine · doi:10.1097/jom.0b013e31815f889b · Cited by 114
“A P value driven approach will increase the number of false positive results.(15) We attempted to avoid this pitfall by using external information rather than P values to hypothesize strata of interest.”
BackgroundJordan, S. E., Watkins, A. J., Storey, M., Allen, S. J., Brooks, C. J., Garaiová, I., Heaven, M. L., Jones, R. H., Plummer, S. F., Russell, I. T., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0067912 · Cited by 129
“We acknowledge the limitations of post hoc subgroup analyses [66,67], and present these solely to illustrate how outcome distribution affects data weighting, not to guide clinical practice.”
Interaction Trees with Censored Survival Data
Su, X., Zhou, T., Yan, X., Fan, J., & Song, Y. (2008) · The International Journal of Biostatistics · doi:10.2202/1557-4679.1071 · Cited by 128
Lokhorst, H. M., van der Holt, B., Cornelissen, J. J., Kersten, M. J., van Oers, M. H. J., Raymakers, R. A. P., Minnema, M. C., Zweegman, S., Janssen, J. J. W. M., Zijlmans, M., et al. (2012) · Blood · doi:10.1182/blood-2011-11-393801 · Cited by 104
Heterogeneity of Treatment Effects: Implications for Guidelines, Payment, and Quality Assessment
Greenfield, S., Kravitz, R. L., Duan, N., & Kaplan, S. H. (2007) · The American Journal of Medicine · doi:10.1016/j.amjmed.2007.02.002 · Cited by 118
Pheochromocytoma penetrance varies by RET mutation in MEN 2A
Quayle, F. J., Fialkowski, E., Benveniste, R. J., & Moley, J. F. (2007) · Surgery · doi:10.1016/j.surg.2007.09.013 · Cited by 103
Lazar, A., Cole, B. F., Bonetti, M., & Gelber, R. D. (2010) · Journal of Clinical Oncology · doi:10.1200/jco.2009.27.9182 · Cited by 103
A Bayesian Approach to Subgroup Identification
Berger, J. O., Wang, X., & Shen, L. (2014) · Journal of Biopharmaceutical Statistics · doi:10.1080/10543406.2013.856026 · Cited by 101
The Effect of Crohn's Disease on Outcomes After Restorative Proctocolectomy
Reese, G., Lovegrove, R. E., Tilney, H. S., Yamamoto, T., Heriot, A. G., Fazio, V. W., & Tekkis, P. P. (2006) · Diseases of the Colon & Rectum · doi:10.1007/s10350-006-0777-x · Cited by 101
Post-hoc data analysis: benefits and limitations
Curran‐Everett, D., & Milgrom, H. (2013) · Current Opinion in Allergy and Clinical Immunology · doi:10.1097/aci.0b013e3283609831 · Cited by 103
Sargeant, J. M., O'Connor, A. M. C., Gardner, I. A., Dickson, J. S., Torrence, M. E., participants, C. M., Dohoo, I. R., Lefebvre, S. L., Morley, P. S., Ramirez, A., et al. (2010) · Zoonoses and Public Health · doi:10.1111/j.1863-2378.2009.01312.x · Cited by 108
“Experience from human health care suggests that authors should resist the temptation to perform post hoc subgroup analyses (7, 12, 56, 80, 114).”
BackgroundSargeant, J. M., O’connor, A. M., Gardner, I. A., Dickson, J. S., Torrence, M. E., Dohoo, C. M. P. I. R., Lefebvre, S. L., Morley, P. S., Ramirez, A., & Snedeker, K. (2010) · Journal of Food Protection · doi:10.4315/0362-028x-73.3.579 · Cited by 97
Porst, H., McVary, K. T., Montorsi, F., Sutherland, P. D., Elion‐Mboussa, A., Wolka, A. M., & Viktrup, L. (2009) · European Urology · doi:10.1016/j.eururo.2009.04.033 · Cited by 103
Selecting Optimal Subgroups for Treatment Using Many Covariates
VanderWeele, T. J., Luedtke, A. R., van der Laan, M. J., & Kessler, R. C. (2019) · Epidemiology · doi:10.1097/ede.0000000000000991 · Cited by 104
Szilagyi, P. G., Humiston, S. G., Gallivan, S., Albertin, C. S., Sandler, M., & Blumkin, A. K. (2011) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/archpediatrics.2011.73 · Cited by 94
Nonselective and Cyclooxygenase-2-Selective NSAIDs and Acute Kidney Injury
Winkelmayer, W. C., Waikar, S. S., Mogun, H., & Solomon, D. H. (2008) · The American Journal of Medicine · doi:10.1016/j.amjmed.2008.06.035 · Cited by 93
Holland, G. N., Vaudaux, J. D., Shiramizu, K. M., Yu, F., Goldenberg, D. T., Gupta, A., Carlson, M., Read, R. W., Novack, R. D., & Kuppermann, B. D. (2008) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2007.09.040 · Cited by 91
Wallach, J. D., Sullivan, P. G., Trepanowski, J. F., Steyerberg, E. W., & Ioannidis, J. P. A. (2016) · BMJ · doi:10.1136/bmj.i5826 · Cited by 89
Kawakami, K., Ohkusa, Y., Kuroki, R., Tanaka, T., Koyama, K., Harada, Y., Iwanaga, K., Yamaryo, T., & Oishi, K. (2010) · Vaccine · doi:10.1016/j.vaccine.2010.08.010 · Cited by 86
Saam, D. E., Liptak, J. M., Stalker, M. J., & Chun, R. (2011) · Journal of the American Veterinary Medical Association · doi:10.2460/javma.238.2.195 · Cited by 86
Lee, H., Myung, W., Jeong, B., Choi, H. Y., Jhun, B. W., & Kim, H. (2018) · Environment International · doi:10.1016/j.envint.2018.06.041 · Cited by 88
Cleveland, M. J., Lanza, S. T., Ray, A. E., Turrisi, R., & Mallett, K. A. (2011) · Psychology of Addictive Behaviors · doi:10.1037/a0026130 · Cited by 89
“First, as the number of potential moderators increase, higher Type I error rates are likely (Lagakos, 2006).”
BackgroundStudy Designs and Statistical Analyses for Biomarker Research
Gosho, M., Nagashima, K., & Sato, Y. (2012) · Sensors · doi:10.3390/s120708966 · Cited by 93
“When multiple subgroup analyses are performed, the results are commonly over-interpreted and can lead to further research that is misguided, or worse, lead to suboptimal patient care due to substantial inflation of the probability of a false positive result [51].”
BackgroundCharlton, V. (2019) · Health Care Analysis · doi:10.1007/s10728-019-00381-x · Cited by 80
“…criteria for the use of subgroup analysis because of the loss of statistical power that comes from dividing a trial population into smaller subsets, and the potential for bias to be introduced through ‘data dredging’ (i.e. searching for patterns in data that have arisen from chance) [29, 70, 79].”
Peters, S., Dafni, U. G., Boyer, M. J., Ruysscher, D. D., Finn, C. F., Felip, E., Garrido, P., Girard, N., Gückenberger, M., Haanen, J. B. A. G., et al. (2019) · Annals of Oncology · doi:10.1093/annonc/mdy553 · Cited by 82
“• The conclusion is instead based on an unplanned, post hoc exploratory analysis requested by the regulator with an additional cutoff, different from the one pre-specified in the protocol (1% instead of planned 25%), contrary to basic statistical principles [9].”
ResultMitterberger, M. J., Aigner, F., Horninger, W., Ulmer, H., Cavuto, S., Halpern, E. J., & Frauscher, F. (2010) · European Radiology · doi:10.1007/s00330-010-1860-1 · Cited by 78
“The subgroup analysis could theoretically increase the possibility of finding a falsely positive significant result [ 27 ].”
BackgroundVariable selection for qualitative interactions
Gunter, L., Zhu, J., & Murphy, S. A. (2009) · Statistical Methodology · doi:10.1016/j.stamet.2009.05.003 · Cited by 96
“Much of the statistical literature suggests that the search for qualitative interactions should be severely limited and qualitative interactions that are found should be initiallymistrusted [9,14,11,16].”
BackgroundEconomic evaluation of sevelamer in patients with end-stage renal disease
Manns, B., Klarenbach, S. W., Lee, H., Culleton, B. F., Shrive, F. M., & Tonelli, M. A. (2007) · Nephrology Dialysis Transplantation · doi:10.1093/ndt/gfm367 · Cited by 79
“In addition, since this test for interaction was non-significant after correcting for multiple comparisons [33], it could be argued that the benefit of sevelamer in older populations requires confirmation in specifically designed trials.”
ResultPost hoc analysis of the Exenatide‐ PD trial—Factors that predict response
Athauda, D. S., Maclagan, K., Budnik, N., Zampedri, L., Hibbert, S., Avilés-Olmos, I., Chowdhury, K., Skene, S. S., Limousin, P. D., & Foltynie, T. (2018) · European Journal of Neuroscience · doi:10.1111/ejn.14096 · Cited by 76
“05), there is a high probability of finding at least one significant subgroup- by- interaction treatment effect (Lagakos, 2006), thus examining the general direction of effects and plausibility of any potential trends is important in generating future hypotheses.”
BackgroundMethodInfluentialRoehrborn, C. G., Kaminetsky, J. C., Auerbach, S. M., Montelongo, R. M., Elion‐Mboussa, A., & Viktrup, L. (2009) · British Journal of Urology · doi:10.1111/j.1464-410x.2009.08822.x · Cited by 79
Study Design and “Evidence” in Patient-oriented Research
Concato, J. (2013) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201303-0521oe · Cited by 76
Rizvi, S., Rienties, B., Rogaten, J., & Kizilcec, R. F. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106973 · Cited by 74
“This twice-over analysis approach helped us avoid overinterpretation of Jo urn al Pr e-p roo f 8 potentially noisy subgroup results (Lagakos, 2006).”
MethodExploration of Heterogeneous Treatment Effects via Concave Fusion
Ma, S., Huang, J., Zhang, Z., & Liu, M. (2019) · The International Journal of Biostatistics · doi:10.1515/ijb-2018-0026 · Cited by 80
Regularized outcome weighted subgroup identification for differential treatment effects
Xu, Y., Yu, M., Zhao, Y., Li, Q., Wang, S., & Shao, J. (2015) · Biometrics · doi:10.1111/biom.12322 · Cited by 88
“The controversy and challenges of subgroup identification have been well documented (Brookes et al., 2004; Lagakos, 2006; Ruberg et al., 2010).”
Identification of predicted individual treatment effects in randomized clinical trials
Lamont, A. E., Lyons, M. D., Jaki, T., Stuart, E. A., Feaster, D. J., Tharmaratnam, K., Oberski, D. L., Ishwaran, H., Wilson, D. K., & Horn, M. L. V. (2016) · Statistical Methods in Medical Research · doi:10.1177/0962280215623981 · Cited by 78
Risk assessment in pulmonary arterial hypertension: Insights from the GRIPHON study
Sitbon, O., Chin, K., Channick, R. N., Benza, R. L., Scala, L. D., Gaine, S., Ghofrani, H. A., Lang, I. M., McLaughlin, V. V., Preiss, R., et al. (2020) · The Journal of Heart and Lung Transplantation · doi:10.1016/j.healun.2019.12.013 · Cited by 75
“01 to account for the increased risk of a type 1 error associated with multiple subgroup analyses.(16)”
BackgroundKhullar, V., Cambronero, J., Angulo, J. C., Wooning, M., Blauwet, M. B., Dorrepaal, C. A., & Martin, N. (2013) · BMC Urology · doi:10.1186/1471-2490-13-45 · Cited by 70
“these results, we did not report p values for mean treatment differences versus placebo [24].”
ResultMcCabe, C. J., Bergmann, L., Bosanquet, N., Ellis, M., Enzmann, H. G., von Euler, M., Jönsson, B., Kallen, K., Newling, D. W. W., Nüssler, V., et al. (2008) · Annals of Oncology · doi:10.1093/annonc/mdn603 · Cited by 71
“When faced with a dossier based on subgroup analysis, regulatory agencies must consider several important factors that could affect how the results are interpreted and the findings used in patient treatment [14, 28, 29].”
BackgroundTutorial on statistical considerations on subgroup analysis in confirmatory clinical trials
Alosh, M., Huque, M. F., Bretz, F., & D’Agostino, R. B. (2016) · Statistics in Medicine · doi:10.1002/sim.7167 · Cited by 81
“Such undisciplined subgroup analyses have led many authors to be critical of subgroup analysis [4,5].”
BackgroundFusion Versus Nonoperative Care for Chronic Low Back Pain: Do Psychological Factors Affect Outcomes?
Daubs, M. D., Norvell, D. C., McGuire, R., Molinari, R. W., Hermsmeyer, J. T., Fourney, D. R., Wolinsky, J., & Brodke, D. S. (2011) · Spine · doi:10.1097/brs.0b013e31822ef6b9 · Cited by 73
Broderick, G. A., Brock, G. B., Roehrborn, C. G., Watts, S. D., Elion‐Mboussa, A., & Viktrup, L. (2010) · Urology · doi:10.1016/j.urology.2009.09.093 · Cited by 68
Dmochowski, R. R., Abrams, P., Marschall‐Kehrel, D., Wang, J. T., & Guan, Z. (2006) · European Urology · doi:10.1016/j.eururo.2006.10.005 · Cited by 64
Metadata
- Authors
- Lagakos, S. W.
- Affiliations
- Harvard University (US)
- Corresponding author
- Lagakos, Stephen W.
- Venue
- New England Journal of Medicine
- Publisher
- Massachusetts Medical Society
- Published
- 2006-04-20
- Volume
- 354
- Issue
- 16
- Pages
- 1667-1669
- ISSN
- 0028-4793 (ISSN-L)
- 1533-4406
- OpenAlex
- W2085477178
- Semantic Scholar
- d03b954bb3525ed19f989e831c39645293c3c984
- Cited by
- 504
- References
- 7
- Citations by year
- 2026: 8 · 2025: 11 · 2024: 23 · 2023: 13 · 2022: 14 · 2021: 13 · 2020: 21 · 2019: 34 · 2018: 21 · 2017: 21 · 2016: 23 · 2015: 29 · 2014: 21 · 2013: 26 · 2012: 29
- FWCI
- 40.60
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- New England Journal of Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- subgroup analysis, subgroup heterogeneity, overinterpretation, reporting bias, treatment effect heterogeneity, randomized controlled trials, primary endpoint
- MeSH terms
- Data Interpretation, Statistical, False Positive Reactions, Probability, Statistics as Topic, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.