An Investigation of the Shortcomings of the CONSORT 2010 Statement for the Reporting of Group Sequential Randomised Controlled Trials: A Methodological Systematic Review
Open accessThe DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
BACKGROUND: It can be argued that adaptive designs are underused in clinical research. We have explored concerns related to inadequate reporting of such trials, which may influence their uptake. Through a careful examination of the literature, we evaluated the standards of reporting of group sequential (GS) randomised controlled trials, one form of a confirmatory adaptive design. METHODS: We undertook a systematic review, by searching Ovid MEDLINE from the 1st January 2001 to 23rd September 2014, supplemented with trials from an audit study. We included parallel group, confirmatory, GS trials that were prospectively designed using a Frequentist approach. Eligible trials were examined for compliance in their reporting against the CONSORT 2010 checklist. In addition, as part of our evaluation, we developed a supplementary checklist to explicitly capture group sequential specific reporting aspects, and investigated how these are currently being reported. RESULTS: Of the 284 screened trials, 68(24%) were eligible. Most trials were published in "high impact" peer-reviewed journals. Examination of trials established that 46(68%) were stopped early, predominantly either for futility or efficacy. Suboptimal reporting compliance was found in general items relating to: access to full trials protocols; methods to generate randomisation list(s); details of randomisation concealment, and its implementation. Benchmarking against the supplementary checklist, GS aspects were largely inadequately reported. Only 3(7%) trials which stopped early reported use of statistical bias correction. Moreover, 52(76%) trials failed to disclose methods used to minimise the risk of operational bias, due to the knowledge or leakage of interim results. Occurrence of changes to trial methods and outcomes could not be determined in most trials, due to inaccessible protocols and amendments. DISCUSSION AND CONCLUSIONS: There are issues with the reporting of GS trials, particularly those specific to the conduct of interim analyses. Suboptimal reporting of bias correction methods could potentially imply most GS trials stopping early are giving biased results of treatment effects. As a result, research consumers may question credibility of findings to change practice when trials are stopped early. These issues could be alleviated through a CONSORT extension. Assurance of scientific rigour through transparent adequate reporting is paramount to the credibility of findings from adaptive trials. Our systematic literature search was restricted to one database due to resource constraints.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group* (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-152-11-201006010-00232 · Cited by 3,824
Consort 2010 statement: extension to cluster randomised trials
Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., & for the CONSORT Group (2012) · BMJ · doi:10.1136/bmj.e5661 · Cited by 3,510 · cited 1×
“Extensions to the CON-SORT statement have since been made to accommodate other trial designs and hypotheses, such as: cluster RCTs, non-inferiority and equivalence trials, and pragmatic RCTs [23 – 25].”
A Multiple Testing Procedure for Clinical Trials
O'Brien, P. C., & Fleming, T. R. (1979) · Biometrics · doi:10.2307/2530245 · Cited by 3,311
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755 · cited 1×
“The CONSORT (CONsolidated Standards Of Reporting Trials) statement was first published in 1996, with the aim to enhance adequate reporting of RCTs, [16] and has since been revised in 2001 and 2010 [17,18].”
Piaggio, G., Elbourne, D. R., Pocock, S. J., Evans, S. J. W., Altman, D. G., & CONSORT Group, F. T. (2012) · JAMA · doi:10.1001/jama.2012.87802 · Cited by 2,428 · cited 1×
“Extensions to the CONSORT statement have since been made to accommodate other trial designs and hypotheses, such as: cluster RCTs, non-inferiority and equivalence trials, and pragmatic RCTs [23–25].”
Discrete sequential boundaries for clinical trials
GORDON LAN, K. K., & DEMETS, D. L. (1983) · Biometrika · doi:10.1093/biomet/70.3.659 · Cited by 2,310 · cited 5×
“Of the 44(65%) RCTs that reported stopping rules or boundaries, the most frequently used stopping boundaries were: 15(34%) Lan-DeMets (LD) [34] error spending function mimicking O’Brien-Fleming (OBF) type properties [35], and 12 (27%) OBF.”
InfluentialDiscrete Sequential Boundaries for Clinical Trials
Lan, K. K. G., & DeMets, D. L. (1983) · Biometrika · doi:10.2307/2336502 · Cited by 2,310
Group sequential methods in the design and analysis of clinical trials
POCOCK, S. J. (1977) · Biometrika · doi:10.1093/biomet/64.2.191 · Cited by 1,831 · cited 2×
“Other stopping boundaries or rules that were rarely used were: Pocock [36]; Hay-bittle-Peto [37,38]; Pampallona and Tsiatis [39], in combination with LD error spending function of the OBF type; Pampallona and Tsiatis [39]; Gamma family ( γ = -8) [40]; Rho family ( ρ = 3) [41], in combination with OBF; Wang and Tsiatis…”
Improving the reporting of pragmatic trials: An extension of the CONSORT statement
Zwarenstein, M., Treweek, S., Gagnier, J. J., Altman, D. G., Tunis, S., Haynes, B., Oxman, A. D., Moher, D., & groups, F. T. C. A. P. T. I. H. (. (2008) · BMJ · doi:10.1136/bmj.a2390 · Cited by 1,813
Reducing waste from incomplete or unusable reports of biomedical research
Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., Michie, S., Moher, D., & Wager, E. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62228-x · Cited by 1,319
Self-citationMoher, D., Jones, A., Lepage, L., & for the CONSORT Group (2001) · JAMA · doi:10.1001/jama.285.15.1992 · Cited by 1,027
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
Vrabel, M. (2015) · Oncology Nursing Forum · doi:10.1188/15.onf.552-554 · Cited by 669
Turner, L., Shamseer, L., Altman, D. G., Schulz, K. F., & Moher, D. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-60 · Cited by 628 · cited 1×
“There has been marked general improvement in the conduct and reporting of RCTs since the advent of the first CONSORT statement [19–21], although there are still some suboptimal areas requiring improvements [21,22].”
Adaptive design methods in clinical trials–a review
Chow, S.-C., & Chang, M. (2008) · Orphanet Journal of Rare Diseases · doi:10.1186/1750-1172-3-11 · Cited by 482 · cited 1×
“The potential introduction of operational bias, due to the leaking or knowledge of interim results in adaptive RCTs, has been well described [13,77].”
Study flow diagrams in Cochrane systematic review updates: an adapted PRISMA flow diagram
Stovold, E., Beecher, D., Foxlee, R., & Noel-Storr, A. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-54 · Cited by 356
Approximately Optimal One-Parameter Boundaries for Group Sequential Trials
Wang, S. K., & Tsiatis, A. A. (1987) · Biometrics · doi:10.2307/2531959 · Cited by 341 · cited 1×
“Other stopping boundaries or rules that were rarely used were: Pocock [36]; Haybittle-Peto [37,38]; Pampallona and Tsiatis [39], in combination with LD error spending function of the OBF type; Pampallona and Tsiatis [39]; Gamma family (γ = -8) [40]; Rho family (ρ = 3) [41], in combination with OBF; Wang and Tsiatis (s…”
Value of Flow Diagrams in Reports of Randomized Controlled Trials
Egger, M. (2001) · JAMA · doi:10.1001/jama.285.15.1996 · Cited by 317
Adaptive trial designs: a review of barriers and opportunities
Kairalla, J. A., Coffey, C. S., Thomann, M. A., & Muller, K. E. (2012) · Trials · doi:10.1186/1745-6215-13-145 · Cited by 249
Early stopping of randomized clinical trials for overt efficacy is problematic.
Bassler, D., Montori, V. M., Briel, M., Glasziou, P., & Guyatt, G. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.07.016 · Cited by 130 · cited 1×
“Although much attention has been paid to group sequential RCTs that are stopping early for benefit [62,63,66,68], the consequences could be similar when trials are stopped early for futility, since the evidence can be used to withdraw intervention(s) already in the care pathway.”
Mhaskar, R., Djulbegovic, B., Magazin, A., Soares, H. P., & Kumar, A. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.10.016 · Cited by 102 · cited 3×
“Similar findings on these checklist items were also found in oncology, and moreover, inadequate reporting was associated with exaggerated, biased intervention effects [61].”
Cited by
Adaptive designs in clinical trials: why use them, and how to run and report them
Pallmann, P., Bedding, A. W., Choodari-Oskooei, B., Dimairo, M., Flight, L., Hampson, L. V., Holmes, J., Mander, A. P., Odondi, L., Sydes, M. R., et al. (2018) · BMC Medicine · doi:10.1186/s12916-018-1017-7 · Cited by 829
Self-citationScoping review on interventions to improve adherence to reporting guidelines in health research
Blanco, D., Altman, D., Moher, D., Boutron, I., Kirkham, J. J., & Cobo, E. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-026589 · Cited by 154
Jin, Y., Sanger, N., Shams, I., Luo, C., Shahid, H., Li, G., Bhatt, M., Zielinski, L., Bantoto, B., Wang, M., et al. (2018) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s155103 · Cited by 134
Adaptive designs undertaken in clinical research: a review of registered clinical trials
Hatfield, I., Allison, A., Flight, L., Julious, S. A., & Dimairo, M. (2016) · Trials · doi:10.1186/s13063-016-1273-9 · Cited by 107
“A modification to the CONSORT statement could help with the improvement of the reporting of AD trials [24].”
MethodSelf-citationDimairo, M., Pallmann, P., Wason, J., Todd, S., Jaki, T., Julious, S. A., Mander, A. P., Weir, C. J., Koenig, F., Walton, M. K., et al. (2020) · Trials · doi:10.1186/s13063-020-04334-x · Cited by 58
Self-citation
Metadata
- Authors
- Stevely, A., Dimairo, M., Todd, S., Julious, S. A., Nicholl, J., Hind, D., & Cooper, C. L.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-11-03
- Volume
- 10
- Issue
- 11
- Pages
- e0141104
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W2107091096
- Cited by
- 59
- References
- 20
- FWCI
- 1.40
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.