The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Credibility of claims of subgroup effects in randomised controlled trials: systematic review
Open accessAbstract
OBJECTIVE: To investigate the credibility of authors' claims of subgroup effects using a representative sample of recently published randomised controlled trials. DESIGN: Systematic review. DATA SOURCE: Core clinical journals, as defined by the National Library of Medicine, in Medline. STUDY SELECTION: Randomised controlled trials published in 2007. Using prespecified criteria, teams of trained reviewers independently judged whether authors claimed subgroup effects and the strength of their claims. Reviewers assessed each of these claims against 10 predefined criteria, developed through a search of existing criteria and a consensus process. RESULTS: Of 207 randomised controlled trials reporting subgroup analyses, 64 (31%) made claims for the primary outcome. Of those, 20 were strong claims and 28 claims of a likely effect. Authors included subgroup variables measured at baseline in 60 (94%) trials, used subgroup variable as a stratification factor at randomisation in 13 (20%), clearly prespecified their hypotheses in 26 (41%), correctly prespecified direction in 4 (6%), tested a small number of hypotheses in 28 (44%), carried out a test of interaction that proved statistically significant in 6 (9%), documented replication of a subgroup effect with previous related studies in 21 (33%), identified consistency of a subgroup effect across related outcomes in 19 (30%), and provided a compelling indirect evidence for the effect in 14 (22%). In the 19 trials making more than one claim, only one (5%) checked the independence of the interaction. Of the 64 claims, 54 (84%) met four or fewer of the 10 criteria. For strong claims, more than 50% failed each of the individual criteria, and only three (15%) met more than five criteria. CONCLUSION: Authors often claim subgroup effects in their trial report. However, the credibility of subgroup effects, even when claims are strong, is usually low. Users of the information should treat claims that fail to meet most criteria with scepticism. Trial researchers should report the conduct of subgroup analyses and provide sufficient evidence when claiming a subgroup effect or suggesting a possible effect.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Self-citationInteraction revisited: the difference between two estimates.
Altman, D. G., & Bland, J. M. (2003) · BMJ · doi:10.1136/bmj.326.7382.219 · Cited by 3,203
Self-citationStatistics in Medicine — Reporting of Subgroup Analyses in Clinical Trials
Wang, R., Lagakos, S. W., Ware, J. H., Hunter, D. J., & Drazen, J. M. (2007) · New England Journal of Medicine · doi:10.1056/nejmsr077003 · Cited by 1,445
Users' guides to the medical literature : a manual for evidence-based clinical practice
Guyatt, G. H., Rennie, D., Meade, M. O., & Cook, D. J. (2002) · Cited by 1,196
Self-citationSubgroup analysis and other (mis)uses of baseline data in clinical trials
Assmann, S. F., Pocock, S. J., Enos, L. E., & Kasten, L. E. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02039-0 · Cited by 1,167
Pocock, S. J., Assmann, S. E., Enos, L. E., & Kasten, L. E. (2002) · Statistics in Medicine · doi:10.1002/sim.1296 · Cited by 1,077
Yusuf, S. (1991) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1991.03470010097038 · Cited by 1,014
Subgroup analysis in randomised controlled trials: importance, indications, and interpretation
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(05)17709-5 · Cited by 942
A consumer's guide to subgroup analyses
Oxman, A. D., & Guyatt, G. H. (1992) · Annals of Internal Medicine · doi:10.7326/0003-4819-116-1-78 · Cited by 914
Self-citationIs a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses
Sun, X., Briel, M., Walter, S. D., & Guyatt, G. H. (2010) · BMJ · doi:10.1136/bmj.c117 · Cited by 808
Self-citation
Cited by
Interpretation of the evidence for the efficacy and safety of statin therapy
Collins, R., Reith, C., Emberson, J., Armitage, J., Baigent, C., Blackwell, L., Blumenthal, R., Danesh, J., Smith, G. D., DeMets, D., et al. (2016) · The Lancet · doi:10.1016/s0140-6736(16)31357-5 · Cited by 1,857
Schandelmaier, S., Briel, M., Varadhan, R., Schmid, C. H., Devasenapathy, N., Hayward, R. A., Gagnier, J., Borenstein, M., van der Heijden, G. J. M. G., Dahabreh, I. J., et al. (2020) · Canadian Medical Association Journal · doi:10.1503/cmaj.200077 · Cited by 749
Self-citationProbiotics for the prevention of Clostridium difficile-associated diarrhea in adults and children
Goldenberg, J. Z., Ma, S. S., Saxton, J. D., Martzen, M. R., Vandvik, P. O., Thorlund, K., Guyatt, G. H., & Johnston, B. C. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006095.pub3 · Cited by 741
Self-citationInterpreting results and drawing conclusions
Schünemann, H. J., Vist, G. E., Higgins, J. P., Santesso, N., Deeks, J. J., Glasziou, P., Akl, E. A., Guyatt, G. H., & on behalf of the Cochrane GRADEing Methods Group (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch15 · Cited by 595
Self-citationProbiotics for the prevention of Clostridium difficile-associated diarrhea in adults and children
Goldenberg, J. Z., Yap, C., Lytvyn, L., Lo, C. K.-F., Beardsley, J., Mertz, D., & Johnston, B. C. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006095.pub4 · Cited by 550
Self-citationHow to Use a Subgroup Analysis: Users’ Guide to the Medical Literature
Sun, X., Ioannidis, J. P. A., Agoritsas, T., Alba, A. C., & Guyatt, G. (2014) · JAMA · doi:10.1001/jama.2013.285063 · Cited by 550
Self-citationPersonalized evidence based medicine: predictive approaches to heterogeneous treatment effects
Kent, D. M., Steyerberg, E., & van Klaveren, D. (2018) · BMJ · doi:10.1136/bmj.k4245 · Cited by 479
The Magic of Randomization versus the Myth of Real-World Evidence
Collins, R., Bowman, L., Landray, M., & Peto, R. (2020) · New England Journal of Medicine · doi:10.1056/nejmsb1901642 · Cited by 478
Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions
Viswanathan, M., Patnode, C. D., Berkman, N. D., Bass, E. B., Chang, S., Hartling, L., Murad, M. H., Treadwell, J. R., & Kane, R. L. (2017) · doi:10.23970/ahrqepcmethguide2 · Cited by 448
Why clinical trial outcomes fail to translate into benefits for patients
Heneghan, C., Goldacre, B., & Mahtani, K. R. (2017) · Trials · doi:10.1186/s13063-017-1870-2 · Cited by 404
McMurray, J. J. V., Jackson, A. M., Lam, C. S. P., Redfield, M. M., Anand, I. S., Ge, J., Lefkowitz, M. P., Maggioni, A. P., Martinez, F., Packer, M., et al. (2020) · Circulation · doi:10.1161/circulationaha.119.044491 · Cited by 380
Bower, P., Kontopantelis, E., Sutton, A., Kendrick, T., Richards, D. A., Gilbody, S., Knowles, S., Cuijpers, P., Andersson, G., Christensen, H., et al. (2013) · BMJ · doi:10.1136/bmj.f540 · Cited by 341
Coventry, P., Lovell, K., Dickens, C., Bower, P., Chew-Graham, C., McElvenny, D., Hann, M., Cherrington, A., Garrett, C., Gibbons, C. J., et al. (2015) · BMJ · doi:10.1136/bmj.h638 · Cited by 325
Knee osteoarthritis phenotypes and their relevance for outcomes: a systematic review
Deveza, L. A., Melo, L., Yamato, T. P., Mills, K., Ravi, V., & Hunter, D. J. (2017) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2017.08.009 · Cited by 331
Comerota, A. J., Kearon, C., Gu, C.-S., Julian, J. A., Goldhaber, S. Z., Kahn, S. R., Jaff, M. R., Razavi, M. K., Kindzelski, A. L., Bashir, R., et al. (2019) · Circulation · doi:10.1161/circulationaha.118.037425 · Cited by 321
Nauth, A., Creek, A. T., Zellar, A., Lawendy, A.-R., Dowrick, A., Gupta, A., Dadi, A., van Kampen, A., Yee, A., de Vries, A. C., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(17)30066-1 · Cited by 319
Self-citationLessons Learned from Alzheimer Disease: Clinical Trials with Negative Outcomes
Cummings, J. (2018) · Clinical and Translational Science · doi:10.1111/cts.12491 · Cited by 298
Three simple rules to ensure reasonably credible subgroup analyses
Burke, J. F., Sussman, J. B., Kent, D. M., & Hayward, R. A. (2015) · BMJ · doi:10.1136/bmj.h5651 · Cited by 312
Welch, V., Petticrew, M., Petkovic, J., Moher, D., Waters, E., White, H., Tugwell, P., Atun, R., Awasthi, S., Barbour, V., et al. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.09.001 · Cited by 297
Advances in Measuring the Environmental and Social Impacts of Environmental Programs
Ferraro, P. J., & Hanauer, M. M. (2014) · Annual Review of Environment and Resources · doi:10.1146/annurev-environ-101813-013230 · Cited by 276
Guirguis-Blake, J. M., Evans, C. V., Senger, C. A., O'Connor, E. A., & Whitlock, E. P. (2016) · Annals of Internal Medicine · doi:10.7326/m15-2113 · Cited by 260
Corticosteroids in Sepsis: An Updated Systematic Review and Meta-Analysis
Rochwerg, B., Oczkowski, S. J., Siemieniuk, R. A. C., Agoritsas, T., Belley-Cote, E., D’Aragon, F., Duan, E., English, S., Gossack-Keenan, K., Alghuroba, M., et al. (2018) · Critical Care Medicine · doi:10.1097/ccm.0000000000003262 · Cited by 258
Self-citationThe lack of statistical power of subgroup analyses in meta-analyses: a cautionary note
Cuijpers, P., Griffin, J. W., & Furukawa, T. A. (2021) · Epidemiology and Psychiatric Sciences · doi:10.1017/s2045796021000664 · Cited by 257
Recommendations for assessing the risk of bias in systematic reviews of health-care interventions
Viswanathan, M., Patnode, C. D., Berkman, N. D., Bass, E. B., Chang, S., Hartling, L., Murad, M. H., Treadwell, J. R., & Kane, R. L. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.12.004 · Cited by 221
Cummings, J., Ritter, A., & Zhong, K. (2018) · Journal of Alzheimer's Disease · doi:10.3233/jad-179901 · Cited by 176
Knopp, J., Knowles, S., Bee, P., Lovell, K., & Bower, P. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2013.08.008 · Cited by 164
Wallach, J. D., Sullivan, P. G., Trepanowski, J. F., Sainani, K. L., Steyerberg, E. W., & Ioannidis, J. P. A. (2017) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.9125 · Cited by 148
Uhlig, K., Leff, B., Kent, D., Dy, S., Brunnhuber, K., Burgers, J. S., Greenfield, S., Guyatt, G., High, K., Leipzig, R., et al. (2014) · Journal of General Internal Medicine · doi:10.1007/s11606-013-2659-y · Cited by 158
Self-citationthe PRISMA-Equity Bellagio group, Welch, V., Petticrew, M., Petkovic, J., Moher, D., Waters, E., White, H., & Tugwell, P. (2015) · International Journal for Equity in Health · doi:10.1186/s12939-015-0219-2 · Cited by 114
Hopewell, S., Collins, G. S., Boutron, I., Yu, L.-M., Cook, J., Shanyinde, M., Wharton, R., Shamseer, L., & Altman, D. G. (2014) · BMJ · doi:10.1136/bmj.g4145 · Cited by 103
Self-citationPalpacuer, C., Laviolle, B., Boussageon, R., Reymann, J. M., Bellissant, E., & Naudet, F. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001924 · Cited by 102
Wallach, J. D., Sullivan, P. G., Trepanowski, J. F., Steyerberg, E. W., & Ioannidis, J. P. A. (2016) · BMJ · doi:10.1136/bmj.i5826 · Cited by 89
Welch, V. A., Akl, E. A., Pottie, K., Ansari, M. T., Briel, M., Christensen, R., Dans, A., Dans, L., Eslava-Schmalbach, J., Guyatt, G., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.01.015 · Cited by 88
Self-citation(2017) · Europe PMC (PubMed Central) · Cited by 61
Raghav, K. P. S., Mahajan, S., Yao, J. C., Hobbs, B. P., Berry, D. A., Pentz, R. D., Tam, A., Hong, W. K., Ellis, L. M., Abbruzzese, J., et al. (2015) · Journal of Clinical Oncology · doi:10.1200/jco.2015.62.4148 · Cited by 59
Fan, J., Song, F., & Bachmann, M. O. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.12.009 · Cited by 31
Wysham, C., Hood, R. C., Warren, M. L., Wang, T., Morwick, T. M., & Jackson, J. A. (2016) · Endocrine Practice · doi:10.4158/ep15959.or · Cited by 29
Omar, H., Tejerina-Arreal, M., & Crawford, M. J. (2014) · Personality and Mental Health · doi:10.1002/pmh.1264 · Cited by 15
Vousoura, E., Gergov, V., Tulbure, B. T., Camilleri, N., Saliba, A., Garcia-Lopez, L., Podina, I. R., Prevendar, T., Löffler-Stastka, H., Chiarenza, G. A., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01788-1 · Cited by 11
“[57], implemented in the study by Knopp et al.”
Method
Metadata
- Authors
- Sun, X., Briel, M., Busse, J. W., You, J. J., Akl, E. A., Mejza, F., Bala, M. M., Bassler, D., Mertz, D., Diaz-Granados, N., Vandvik, P. O., Malaga, G., Srinathan, S. K., Dahm, P., Johnston, B. C., Alonso-Coello, P., Hassouneh, B., Walter, S. D., Heels-Ansdell, D., Bhatnagar, N., Altman, D. G., & Guyatt, G. H.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2012-03-15
- Volume
- 344
- Issue
- mar15 1
- Pages
- e1553-e1553
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- OpenAlex
- W2149974876
- Cited by
- 395
- References
- 27
- FWCI
- 16.28
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.