The reporting quality of randomised controlled trials in surgery: A systematic review
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
OBJECTIVES: In order to examine the reporting quality of urological RCTs, the initial objectives of this study were to evaluate the degree to which RCTs involving urological surgical techniques (as the intervention) published in the years 2000-2003 complied with the CONSORT statement, and to assess trends and patterns of compliance. Following our initial findings in urology, we extended the methodology to a number of other specialties to assess whether our findings in urology could be generalised to other surgical disciplines. METHODS: The Royal Society of Medicine (RSM) Library was commissioned to search the Medline and Cochrane databases for RCTs in compliance with the study inclusion criteria below. Additional analyses of five other specialties (non-urological trials: cardiovascular, gastrointestinal, hepatic, orthopaedic and vascular) were also made. For the non-urological trials, 15 English language trials (from each specialty) were randomly picked from a blast search conducted by the RSM for the year 2003. The RCTs were given a score out of 22, reflecting how many of the 22 CONSORT items were complied with (with each item being given equal weighting), this score was termed the "CONSORT score". RESULTS: Urology trials: In total 122 RCT abstracts were identified which met the inclusion criteria for this study. From these, 32 were excluded as they were follow-up studies, involved a virtual procedure or were a cost analysis, leaving 90 RCTs published across 35 different journals which were analysed (68.2%, 90/122). The average score of 11.1 for urological trials indicates that RCTs in our sample do not comply with the CONSORT statement. No trials reported how they implemented their randomisation process. Only 46% of RCTs stated that they had permission from an ethics review board, 20% had declared their sources of funding, 14% stated whether there were any conflicts of interest and only 1.1% stated their trial registry number. For non-urological studies, an average CONSORT score of 11.2 was obtained. CONCLUSION: Clinical research teams conducting RCTs in urology and other surgical disciplines demonstrate poor compliance with the CONSORT statement. We would recommend that trials should be registered at their outset and that urological and other surgical journals to consider supporting the CONSORT statement and to have compliance 'hard-wired' into their submission, editorial and peer-review processes. Since it seems the best researchers are unable to produce an RCT results which enable surgical techniques to be critically assessed, there is a need for education about the CONSORT statement and its importance at all levels of surgical training. We believe that an open debate is needed on the possible role of other research designs, such as tracker studies. Whether this study actually raises the question of how appropriate RCTs are to surgical techniques, we leave to the reader.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)01085-x · Cited by 3,560
Assessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127
Streptomycin Treatment of Pulmonary Tuberculosis: A Medical Research Council Investigation
(1948) · BMJ · doi:10.1136/bmj.2.4582.769 · Cited by 986
The scandal of poor medical research.
Altman, D. G. (1994) · BMJ · doi:10.1136/bmj.308.6924.283 · Cited by 897
Quality of Reporting of Randomized Trials as a Measure of Methodologic Quality
Huwiler-Müntener, K. (2002) · JAMA · doi:10.1001/jama.287.21.2801 · Cited by 397
Randomization, Statistics, and Causal Inference
Greenland, S. (1990) · Epidemiology · doi:10.1097/00001648-199011000-00003 · Cited by 385
Schulz, K. F., Grimes, D. A., Altman, D. G., & Hayes, R. J. (1996) · BMJ · doi:10.1136/bmj.312.7033.742 · Cited by 163
Hotopf, M., Lewis, G., & Normand, C. (1997) · Journal of Epidemiology and Community Health · doi:10.1136/jech.51.4.354 · Cited by 130
Cited by
Turner, L., Shamseer, L., Altman, D. G., Weeks, L., Peters, J., Kober, T., Dias, S., Schulz, K. F., Plint, A. C., & Moher, D. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000030.pub2 · Cited by 740
Hopewell, S., Dutton, S., Yu, L.-M., Chan, A.-W., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c723 · Cited by 504
The challenges faced in the design, conduct and analysis of surgical randomised controlled trials
Cook, J. A. (2009) · Trials · doi:10.1186/1745-6215-10-9 · Cited by 351
PRISMA 2020 statement: What's new and the importance of reporting guidelines
Sohrabi, C., Franchi, T., Mathew, G., Kerwan, A., Nicola, M., Griffin, M., Agha, M., & Agha, R. (2021) · International Journal of Surgery · doi:10.1016/j.ijsu.2021.105918 · Cited by 329
Self-citationScoping review on interventions to improve adherence to reporting guidelines in health research
Blanco, D., Altman, D., Moher, D., Boutron, I., Kirkham, J. J., & Cobo, E. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-026589 · Cited by 154
Agha, R. A., Fowler, A. J., Limb, C., Whitehurst, K., Coe, R., Sagoo, H., Jafree, D. J., Chandrakumar, C., & Gundogan, B. (2016) · International Journal of Surgery · doi:10.1016/j.ijsu.2016.04.032 · Cited by 99
Self-citationQuality of Reporting in Randomized Trials Published in High-Quality Surgical Journals
Sinha, S., Sinha, S., Ashby, E., Jayaram, R., & Grocott, M. P. W. (2009) · Journal of the American College of Surgeons · doi:10.1016/j.jamcollsurg.2009.07.019 · Cited by 79
How can we improve surgical research and innovation?: The IDEAL framework for action
Hirst, A., Agha, R. A., Rosin, D., & McCulloch, P. (2013) · International Journal of Surgery · doi:10.1016/j.ijsu.2013.09.016 · Cited by 50
Self-citationLiu, X., Zhang, X., Wen, S., Peng, L., Hong, Q., & Kang, D. (2015) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12173 · Cited by 22
Khan, M. S., Shaikh, A., Ochani, R. K., Akhtar, T., Fatima, K., Khan, S. U., Mookadam, F., Murad, M. H., Figueredo, V. M., Doukky, R., et al. (2019) · Circulation: Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.118.005260 · Cited by 21
Metadata
- Authors
- Agha, R., Cooper, D., & Muir, G.
- Venue
- International Journal of Surgery
- Published
- 2007-12
- Volume
- 5
- Issue
- 6
- Pages
- 413-422
- ISSN
- 1743-9159 (ISSN-L)
- 1743-9191
- OpenAlex
- W1967839855
- Cited by
- 142
- References
- 10
- FWCI
- 1.93
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.