Are Reports of Randomized Controlled Trials Improving over Time? A Systematic Review of 284 Articles Published in High-Impact General and Specialized Medical Journals
Open accessAbstract
BACKGROUND: Inadequate reporting undermines findings of randomized controlled trials (RCTs). This study assessed and compared articles published in high-impact general medical and specialized journals. METHODS: Reports of RCTs published in high-impact general and specialized medical journals were identified through a search of MEDLINE from January to March of 1995, 2000, 2005, and 2010. Articles that provided original data on adult patients diagnosed with chronic conditions were included in the study. Data on trial characteristics, reporting of allocation concealment, quality score, and the presence of a trial flow diagram were extracted independently by two reviewers, and discrepancies were resolved by consensus or independent adjudication. Descriptive statistics were used for quantitative variables. Comparisons between general medical and specialized journals, and trends over time were performed using Chi-square tests. RESULTS: Reports of 284 trials were analyzed. There was a significantly higher proportion of RCTs published with adequate reporting of allocation concealment (p = 0.003), presentation of a trial flow diagram (p<0.0001) and high quality scores (p = 0.038) over time. Trials published in general medical journals had higher quality scores than those in specialized journals (p = 0.001), reported adequate allocation concealment more often (p = 0.013), and presented a trial flow diagram more often (p<0.001). INTERPRETATION: We found significant improvements in reporting quality of RCTs published in high-impact factor journals over the last fifteen years. These improvements are likely attributed to concerted international efforts to improve reporting quality such as CONSORT. There is still much room for improvement, especially among specialized journals.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Adherence of authors to the CONSORT guidelines is low and the quality of reporting remains insufficient.
“However, the adherence of authors to these guidelines is low and the quality of reporting remains insufficient [9–21].”
From Impact of an online writing aid tool for writing a randomized trial report: the COBWEB (Consort-based WEB tool) randomized controlled trial (2015) · cites this paper as [16]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Assessing the quality of reports of randomized clinical trials: Is blinding necessary?
Jadad, A. R., Moore, R. A., Carroll, D., Jenkinson, C., Reynolds, D. J. M., Gavaghan, D. J., & McQuay, H. J. (1996) · Controlled Clinical Trials · doi:10.1016/0197-2456(95)00134-4 · Cited by 17,582 · see citing works
Self-citationCONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371 · see citing works
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329 · see citing works
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883 · see citing works
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127 · see citing works
Moher, D., Jones, A., Lepage, L., & for the CONSORT Group (2001) · JAMA · doi:10.1001/jama.285.15.1992 · Cited by 1,027 · see citing works
Plint, A. C., Moher, D., Morrison, A., Schulz, K., Altman, D. G., Hill, C., & Gaboury, I. (2006) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2006.tb00557.x · Cited by 872 · see citing works
Cha, M., Eastwaad, S., Hartan, R., Maher, D., Olkin, I., Pitkin, R., Rennie, D., Schulz, K. F., Simel, D. L., & Straup, D. F. (1996) · Cited by 736 · see citing works
Hopewell, S., Dutton, S., Yu, L.-M., Chan, A.-W., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c723 · Cited by 504 · see citing works
Epidemiology and reporting of randomised trials published in PubMed journals
Chan, A.-W., & Altman, D. G. (2005) · The Lancet · doi:10.1016/s0140-6736(05)71879-1 · Cited by 474 · see citing works
Jadad, A. R., & Enkin, M. W. (2007) · Wiley eBooks · doi:10.1002/9780470691922 · Cited by 461 · see citing works
Self-citationKing, D. L., Delfabbro, P., Griffiths, M. D., & Gradisar, M. (2011) · Clinical Psychology Review · doi:10.1016/j.cpr.2011.06.009 · Cited by 284 · see citing works
Hopewell, S., Altman, D. G., Moher, D., & Schulz, K. F. (2008) · Trials · doi:10.1186/1745-6215-9-20 · Cited by 241 · see citing works
Mills, E. J., Wu, P., Gagnier, J., & Devereaux, P. J. (2005) · Contemporary Clinical Trials · doi:10.1016/j.cct.2005.02.008 · Cited by 216 · see citing works
Pildal, J., Chan, A.-W., Hróbjartsson, A., Forfang, E., Altman, D. G., & Gøtzsche, P. C. (2005) · BMJ · doi:10.1136/bmj.38414.422650.8f · Cited by 209 · see citing works
Improving the quality of reporting of randomized controlled trials. The CONSORT statement
Begg, C. B., Cho, M. K., Eastwood, S., Horton, R. C., Moher, D., Olkin, I., Pitkin, R. M., Rennie, D., Schulz, K. F., Simel, D. L., et al. (1998) · Revista Española de Salud Pública · doi:10.1590/s1135-57271998000100002 · Cited by 178 · see citing works
Hewitt, C., Hahn, S., Torgerson, D. J., Watson, J., & Bland, J. M. (2005) · BMJ · doi:10.1136/bmj.38413.576713.ae · Cited by 176 · see citing works
Randomized Controlled Trials: Questions, Answers and Musings
Jadad, A. R., & Enkin, M. W. (2007) · Medical Entomology and Zoology · Cited by 144 · see citing works
Self-citationConsideration of Multiple Chronic Diseases in Randomized Controlled Trials
Jadad, A. R. (2011) · JAMA · doi:10.1001/jama.2011.1886 · Cited by 98 · see citing works
Self-citationMills, E. J., Wu, P., Gagnier, J. J., Heels‐Ansdell, D., & Montori, V. M. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.01.004 · Cited by 76 · see citing works
Turner, L., Moher, D., Shamseer, L., Weeks, L., Peters, J., Plint, A., Altman, D. G., & Schulz, K. F. (2011) · Trials · doi:10.1186/1745-6215-12-s1-a47 · Cited by 36 · see citing works
Quality of Reporting of Randomized Controlled Trials of Pharmacologic Treatment of Bipolar Disorders
Strech, D., Soltmann, B., Weikert, B., Bauer, M., & Pfennig, A. (2011) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.10r06166yel · Cited by 22 · see citing works
The quality of reporting of randomized controlled trials in pelvic organ prolapse
Cavadas, V., Branco, F., Carvalho, F. L. F., Osório, L., Gomes, M. J., & Silva‐Ramos, M. (2011) · International Urogynecology Journal · doi:10.1007/s00192-011-1426-z · Cited by 12 · see citing works
The quality of reporting of randomised controlled trials in dental public health.
Marshman, Z., & Farid, F. N. (2010) · PubMed · Cited by 11 · see citing works
Quality of reporting randomised controlled trials in dental public health
Richards, D. (2011) · Evidence-Based Dentistry · doi:10.1038/sj.ebd.6400797 · Cited by 11 · see citing works
Flagged first, then by citations
1 of the 26 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Barnes, C., Boutron, I., Giraudeau, B., Porcher, R., Altman, D. G., & Ravaud, P. (2015) · BMC Medicine · doi:10.1186/s12916-015-0460-y · Cited by 142 · see citing works
“However, the adherence of authors to these guidelines is low and the quality of reporting remains insufficient [9–21].”
BackgroundMillar, N., Salager-Meyer, F., & Budgell, B. (2019) · English for Specific Purposes · doi:10.1016/j.esp.2019.02.004 · Cited by 85 · see citing works
da Costa, B. R., Beckett, B., Diaz, A., Resta, N. M., Johnston, B. C., Egger, M., Jüni, P., & Armijo-Olivo, S. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0441-7 · Cited by 62 · see citing works
Fuller, T., Pearson, M., Peters, J., & Anderson, R. (2015) · PLOS ONE · doi:10.1371/journal.pone.0121585 · Cited by 47 · see citing works
Trial registration 10 years on
Weber, W. E. J., Merino, J. G., & Loder, E. (2015) · BMJ · doi:10.1136/bmj.h3572 · Cited by 46 · see citing works
Köhler, M., Haag, S., Biester, K., Brockhaus, A. C., McGauran, N., Grouven, U., Kölsch, H., Seay, U., Horn, H., Moritz, G., et al. (2015) · BMJ · doi:10.1136/bmj.h796 · Cited by 43 · see citing works
Kowark, A., Schmitz, S., Rossaint, R., Schürholz, T., & Coburn, M. (2015) · PLoS ONE · doi:10.1371/journal.pone.0128061 · Cited by 41 · see citing works
Pranić, S. M., & Marušić, A. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.07.007 · Cited by 40 · see citing works
Taieb, V., Ikeoka, H., Ma, F.-F., Borkowska, K., Aballéa, S., Keiko, T. O. N. E., & Hirotsu, N. (2019) · Current Medical Research and Opinion · doi:10.1080/03007995.2019.1584505 · Cited by 39 · see citing works
Role of Placental Glucose Transporters in Determining Fetal Growth
Joshi, N. P., Mane, A. R., Sahay, A. S., Sundrani, D. P., Joshi, S., & Yajnik, C. S. (2021) · Reproductive Sciences · doi:10.1007/s43032-021-00699-9 · Cited by 38 · see citing works
Guidelines for the Reporting of Treatment Trials for Alcohol Use Disorders
Witkiewitz, K. A., Finney, J. W., Harris, A. H. S., Kivlahan, D. R., & Kranzler, H. R. (2015) · Alcoholism Clinical and Experimental Research · doi:10.1111/acer.12797 · Cited by 37 · see citing works
Cramer, H., Langhorst, J., Dobos, G. J., & Lauche, R. (2015) · PLoS ONE · doi:10.1371/journal.pone.0144125 · Cited by 26 · see citing works
Girolamo, N. D., & Reynders, R. M. (2016) · Draw Science · doi:10.7717/peerj.1649 · Cited by 25 · see citing works
Lim, S. Y. M., Alshagga, M. A., Kong, C., Alshawsh, M. A., Alshehade, S. A., & Pan, Y. (2022) · Archives of Toxicology · doi:10.1007/s00204-022-03382-3 · Cited by 21 · see citing works
Carpenter, C. R., & Meisel, Z. F. (2017) · Academic Emergency Medicine · doi:10.1111/acem.13225 · Cited by 18 · see citing works
Hong, S. J., Yoon, D. Y., Cho, Y. K., Yoon, S. J., Moon, J. Y., Baek, S., & Lim, K. J. (2016) · American Journal of Roentgenology · doi:10.2214/ajr.15.15640 · Cited by 19 · see citing works
Devos, F., Ibrahim, N., Foissac, F., Bouazza, N., Ancel, P., Chappuy, H., Elie, C., & Tréluyer, J. (2018) · Worldviews on Evidence-Based Nursing · doi:10.1111/wvn.12329 · Cited by 18 · see citing works
Clinical Research in Surgery: Threats and Opportunities
Ceelen, W. P. (2014) · European Surgical Research · doi:10.1159/000367606 · Cited by 16 · see citing works
Tokalić, R., Viđak, M., Buljan, I., & Marušić, A. (2018) · Implementation Science · doi:10.1186/s13012-018-0828-4 · Cited by 15 · see citing works
A Systematic Review of Reporting in Randomized Controlled Trials in Dermatologic Surgery
Alam, M., Rauf, M., Ali, S. S., Nodzenski, M., & Minkis, K. (2014) · Dermatologic Surgery · doi:10.1097/dss.0000000000000166 · Cited by 15 · see citing works
Carpenter, C. R., & Pinnock, H. (2017) · Journal of the American Geriatrics Society · doi:10.1111/jgs.14938 · Cited by 14 · see citing works
Quality of reporting inflammatory bowel disease randomised controlled trials: a systematic review
Gordon, M. A., Khudr, J., Sinopoulou, V., Lakunina, S., Rane, A., & Akobeng, A. K. (2024) · BMJ Open Gastroenterology · doi:10.1136/bmjgast-2023-001337 · Cited by 13 · see citing works
da Costa, B. R., Resta, N. M., Beckett, B., Israel-Stahre, N., Diaz, A., Johnston, B. C., Egger, M., Jüni, P., & Armijo-Olivo, S. L. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-144 · Cited by 13 · see citing works
Ndounga‐Diakou, L. A., Ntoumi, F., Ravaud, P., & Boutron, I. (2017) · Trials · doi:10.1186/s13063-017-2034-0 · Cited by 13 · see citing works
Purja, S., Park, S., Oh, S., Kim, M., & Kim, E. Y. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.12.010 · Cited by 8 · see citing works
Analysis of congenital heart disease research: Mapping impact, production and global collaboration
Zila-Velasque, J. P., Grados‐Espinoza, P., Cubas, W. S., Diaz-Barrera, M. E., & Pacheco‐Mendoza, J. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e19188 · Cited by 7 · see citing works
A mapping review of randomized controlled trials in the spinal cord injury research literature
McIntyre, A., Benton, B., Janzen, S. E., Iruthayarajah, J., Wiener, J., Eng, J. J., & Teasell, R. W. (2018) · Spinal Cord · doi:10.1038/s41393-018-0155-2 · Cited by 8 · see citing works
Medication development for AUD: A systematic review of clinical trial methodology
Donato, S., Meredith, L. R., Nieto, S. J., Bujarski, S., & Ray, L. A. (2024) · Alcohol · doi:10.1016/j.alcohol.2024.06.007 · Cited by 6 · see citing works
Beyond journal publications – a new format for the publication of clinical trials
Wieseler, B. (2016) · Zeitschrift für Evidenz Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2016.11.003 · Cited by 6 · see citing works
Registration of randomized clinical trials – a challenge
Østervig, R., Sonne, A., & Rasmussen, L. S. (2015) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.12575 · Cited by 6 · see citing works
Are the Most Prestigious Medical Journals Transparent Enough?
Dal‐Ré, R. (2016) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2016.06.004 · Cited by 5 · see citing works
Transparent transdisciplinary reporting in geriatric research using the EQUATOR Network
Carpenter, C. R., & Gill, T. M. (2022) · Journal of the American Geriatrics Society · doi:10.1111/jgs.18097 · Cited by 5 · see citing works
Methodological Characteristics of Clinical Trials: Impact of Mandatory Trial Registration
Kakkar, A. K., Padhy, B. M., Sarangi, S. C., & Gupta, Y. K. (2019) · Journal of Pharmacy & Pharmaceutical Sciences · doi:10.18433/jpps30360 · Cited by 3 · see citing works
Preventing and managing paediatric obesity: a special edition on randomized controlled trials
Ball, G. D. C., & Kebbe, M. (2018) · Pediatric Obesity · doi:10.1111/ijpo.12482 · Cited by 2 · see citing works
Non-existent ClinicalTrials.gov identifiers in abstracts indexed by PubMedPreprint
Carlisle, B. G. (2020) · medRxiv · doi:10.1101/2020.02.24.20027300 · Cited by 2 · see citing works
Reporting Guidelines zur Berichterstattung empirischer Forschung – nützlich oder verfehlt?
Mohler, R. A., & Bartoszek, G. (2014) · Pflege · doi:10.1024/1012-5302/a000394 · Cited by 1 · see citing works
How to Report Exotic Animal Research
Girolamo, N. D., & Winter, A. L. (2017) · Veterinary Clinics of North America Exotic Animal Practice · doi:10.1016/j.cvex.2017.04.010 · Cited by 1 · see citing works
Randomized controlled trials in lung cancer surgery: How are we doing?
Wong, L.-Y., Li, Y., Elliott, I. A., Backhus, L. M., Berry, M. F., Shrager, J. B., & Oh, D. S. (2024) · JTCVS Open · doi:10.1016/j.xjon.2024.01.008 · Cited by 1 · see citing works
Mobile access to virtual randomization for investigator-initiated trials
Deserno, T. M., & Keszei, A. P. (2017) · Clinical Trials · doi:10.1177/1740774517706509 · Cited by 1 · see citing works
Taieb, V., Ikeoka, H., Ma, F.-F., Borkowska, K., Aballéa, S., Keiko, T. O. N. E., & Hirotsu, N. (2020) · Terapevticheskii arkhiv · doi:10.26442/00403660.2020.11.000870 · Cited by 1 · see citing works
Non-existent ClinicalTrials.gov identifiers in abstracts indexed by PubMedPreprint
Carlisle, B. G. (2020) · medRxiv · doi:10.17605/osf.io/hfm9y · Cited by 1 · see citing works
Yurdakul, S., Ayan, G., Özgüler, Y., Hatemi, G., Uğurlu, S., Seyahi, E., & Yazıcı, H. (2016) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2016.09.019
Paul, C. L., Sanson‐Fisher, R., & Carey, M. L. (2011) · Oxford University Press eBooks · doi:10.1093/obo/9780199756797-0047
- Authors
- To, M. J., Jones, J., Emara, M., & Jadad, A. R.
- Affiliations
- University Health Network (CA)
- Corresponding author
- Jadad, Alejandro R.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-12-31
- Volume
- 8
- Issue
- 12
- Pages
- e84779
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3877340
- OpenAlex
- W1982056929
- Cited by
- 46
- References
- 26
- Citations by year
- 2024: 3 · 2023: 1 · 2022: 3 · 2021: 1 · 2020: 3 · 2019: 3 · 2018: 4 · 2017: 6 · 2016: 8 · 2015: 9 · 2014: 4
- FWCI
- 1.43
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- University Health Network
- University of Toronto
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- randomized controlled trials, trial reporting quality, allocation concealment, CONSORT, quality scores, general medical journals, chronic disease
- MeSH terms
- Periodicals as Topic, Research Design, Bibliometrics, Randomized Controlled Trials as Topic, Information Dissemination, Quality Improvement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
