Risk of bias of randomized controlled trials published in orthopaedic journals
Open accessThe DOI matches its Crossref record; 46 of the 48 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The purpose of this study was to assess the quality of methodology in orthopaedics-related randomized controlled trials (RCTs) published from January 2006 to December 2010 in the top orthopaedic journals based on impact scores from the Thompson ISI citation reports (2010). METHODS: Journals included American Journal of Sports Medicine; Journal of Orthopaedic Research; Journal of Bone and Joint Surgery, American; Spine Journal; and Osteoarthritis and Cartilage. Each RCT was assessed on ten criteria (randomization method, allocation sequence concealment, participant blinding, outcome assessor blinding, outcome measurement, interventionist training, withdrawals, intent to treat analyses, clustering, and baseline characteristics) as having empirical evidence for biasing treatment effect estimates when not performed properly. RESULTS: A total of 232 RCTs met our inclusion criteria. The proportion of RCTs in published journals fell from 6% in 2006 to 4% in 2010. Forty-nine percent of the criteria were fulfilled across these journals, with 42% of the criteria not being amendable to assessment due to inadequate reporting. The results of our regression revealed that a more recent publication year was significantly associated with more fulfilled criteria (β = 0.171; CI = -0.00 to 0.342; p = 0.051). CONCLUSION: In summary, very few studies met all ten criteria. Thus, many of these studies likely have biased estimates of treatment effects. In addition, these journals had poor reporting of important methodological aspects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“only 1% of studies in high-impact orthopedic journals reported all ten criteria needed for risk of bias assessment (13).”
From The devil is in the details: Reporting and transparent research practices in sports medicine and orthopedic clinical trials (2021) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In 42% of papers, risk of bias could not be assessed due to incomplete reporting (13).”
From The devil is in the details: Reporting and transparent research practices in sports medicine and orthopedic clinical trials (2021) · cites this paper as [13]
Experimental — accuracy not yet measuredIn the citing paper
“among randomized controlled trials published in the top five orthopedics journals, 60% failed to address the blinding status of the participants and 58% did not specify the number of participants included in the final analysis (13).”
From The devil is in the details: Reporting and transparent research practices in sports medicine and orthopedic clinical trials (2021) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020 · cited 1×
“For example, the most recent of these investigations of reporting quality [19] applied the Consolidated Standards of Reporting Trials (CONSORT) statement [20] to a sample of RCTs, the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement [21] to a sample of case–control, cohort and cr…”
Methodvon Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61602-x · Cited by 21,320
Assessing the quality of reports of randomized clinical trials: Is blinding necessary?
Jadad, A. R., Moore, R. A., Carroll, D., Jenkinson, C., Reynolds, D. J. M., Gavaghan, D. J., & McQuay, H. J. (1996) · Controlled Clinical Trials · doi:10.1016/0197-2456(95)00134-4 · Cited by 17,582 · cited 2×
“Carefully planned and well-executed RCTs give us the best estimates of treatment effect and can thus guide clinical decision making [2,3], although trials that lack methodological rigor cause over- or underestimation of treatment effect sizes due to bias [4-6].”
BackgroundCONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883
Evidence-Based Medicine: How to Practice and Teach EBM
Sackett, D. L. (2000) · Cited by 5,950
Schulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634 · cited 3×
“We can only extrapolate for the extensive literature that has shown this to be true for RCTs in other clinical areas [3,4,23-34].”
BackgroundMethodMoher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835 · cited 2×
“Background Randomized controlled trials (RCTs) provide strong evidence for efficacy of healthcare interventions [1].”
BackgroundCONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group* (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-152-11-201006010-00232 · Cited by 3,824
Research works printed but not linked: 1
- Juni P, Tallon D, Egger M: Proceedings of the 3rd symposium on systematic reviews: beyond the basics. St. Catherine’s College. “Garbage in – garbage out?” Assessment of the quality of controlled trials in meta-analyses published in leading journals. 2000, Oxford: Centre for Statistics in Medicine, 19-
Cited by
Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations
McKiernan, E. C., Schimanski, L. A., Muñoz Nieves, C., Matthias, L., Niles, M. T., & Alperin, J. P. (2019) · eLife · doi:10.7554/elife.47338 · Cited by 347
“Chess & Gagnier (2013) analyzed clinical trial studies for ten312 different indicators of quality, including randomization and blinding, and found that less than 1% of313 studies met all ten quality criteria, while the JIF of the journals did not significantly predict whether314 a larger number of…”
Surgical management of degenerative meniscus lesions: the 2016 ESSKA meniscus consensus
Beaufils, P., Becker, R., Kopf, S., Englund, M., Verdonk, R., Ollivier, M., & Seil, R. (2017) · Knee Surgery, Sports Traumatology, Arthroscopy · doi:10.1007/s00167-016-4407-4 · Cited by 343
Muscogiuri, G., El Ghoch, M., Colao, A., Hassapidou, M., Yumuk, V., Busetto, L., & Obesity Management Task Force (OMTF) of the European Association for the Study of Obesity (EASO) (2021) · Obesity Facts · doi:10.1159/000515381 · Cited by 289
Khan, M., Evaniew, N., Gichuru, M., Habib, A., Ayeni, O. R., Bedi, A., Walsh, M., Devereaux, P. J., & Bhandari, M. (2017) · The American Journal of Sports Medicine · doi:10.1177/0363546516674469 · Cited by 171
“Primary intervention Surgical 18 (38) Shoulder 6 (13) Knee 12 (25) Hip 0 (0) Anesthetic 19 (40) Nonoperative 11 (23) Outcome assessed Imaging 12 (25) Perioperative comfort 10 (21) Clinical assessment 13 (27) Adverse events 13 (27) Year of publication 2005 5 (10) 2006 2 (4) 2007 5 (10) 2008 3 (6) 2009 2 (4) 2010 5 (10)…”
Heterogeneity of the definition of elderly age in current orthopaedic research
Sabharwal, S., Wilson, H., Reilly, P., & Gupte, C. M. (2015) · SpringerPlus · doi:10.1186/s40064-015-1307-x · Cited by 160
Metadata
- Authors
- Chess, L. E., & Gagnier, J.
- Affiliations
- Wayne State University (US)
- University of Michigan (US)
- Corresponding author
- Chess, Laura E
- ORCID
- Gagnier, Joel
- Venue
- BMC Medical Research Methodology
- Published
- 2013-06-09
- Volume
- 13
- Issue
- 1
- Pages
- 76-76
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC3724580
- OpenAlex
- W2009249600
- Cited by
- 83
- References
- 48
- Citations by year
- 2026: 2 · 2024: 3 · 2023: 4 · 2022: 8 · 2021: 6 · 2020: 5 · 2019: 12 · 2018: 9 · 2017: 13 · 2016: 6 · 2015: 12 · 2014: 3
- FWCI
- 7.72
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/2.0 (tdm)
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- risk of bias, randomized controlled trials, orthopaedic research, methodological quality, reporting quality, blinding, randomization methods, intention-to-treat analysis, publication year trends, orthopaedic journals
- MeSH terms
- Female, Humans, Male, Orthopedics, Publishing, Research Design, Bias, Randomized Controlled Trials as Topic, Journal Impact Factor
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Risk of bias of randomized controlled trials published in orthopaedic journals. https://app.cochat.ai/papers/10.1186/1471-2288-13-76
- The paper
- Chess, L. E., & Gagnier, J. (2013). Risk of bias of randomized controlled trials published in orthopaedic journals. BMC Medical Research Methodology. https://doi.org/10.1186/1471-2288-13-76
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

