The Quality of Reporting of Orthopaedic Randomized Trials with Use of a Checklist for Nonpharmacological Therapies
The DOI matches its Crossref record; 18 of the 19 references it cites were found in a registry or an index.
Abstract
BACKGROUND: The Consolidated Standards of Reporting Trials statement for the reporting of randomized controlled trials has been limited by its applicability to surgical trials. In response, a Checklist to Evaluate a Report of a Nonpharmacological Trial was recently developed by the Consolidated Standards of Reporting Trials group to address reporting issues in surgical trials. We aimed (1) to apply the checklist for nonpharmacological therapies to orthopaedic randomized controlled trials across multiple journals from 2004 through 2005, and (2) to survey authors when methodological safeguards itemized in the checklist were not reported to determine whether they actually had been performed. We hypothesized that lack of reporting of a methodological safeguard did not necessarily mean it had not been conducted. METHODS: We searched for relevant orthopaedic randomized controlled trials across eight journals in the period from January 2004 through December 2005. We applied the Checklist to Evaluate a Report of a Nonpharmacological Trial to all eligible studies. We contacted authors to determine what methodological safeguards were actually used, especially when details remained unclear from the publication. RESULTS: We included eighty-seven randomized controlled trials from eighty-five scientific reports. In assessing the randomized controlled trials with the checklist for nonpharmacological therapies, seventy-three studies (84%) had unclear reporting of treatment allocation concealment. Only seventeen studies (20%) mentioned surgeon skill or experience. The blinding of patients, ward staff, rehabilitation staff, clinical outcome assessors, and nonclinical outcome assessors was unclear in forty-eight (55%), sixty-three (72%), sixty-four (74%), forty (46%), and thirty-three studies (38%), respectively. Authors from forty-three randomized controlled trials responded to our survey. The results of the survey showed that 41% (95% confidence interval, 25% to 58%) of the trials had adequate allocation concealment when this had been unclear from the report. Although the surgical experience of the investigators was rarely reported, most authors (70%) acknowledged that they had defined "surgical expertise criteria" such as minimum case criteria, specialized training, and clinical performance. The survey also showed that 28% to 40% of the trials had blinding of relevant groups despite the fact that the reporting of such blinding had been unclear in the publications. CONCLUSIONS: The quality of reporting in the orthopaedic literature was highly variable. Readers should not assume that bias-reducing safeguards that were not reported in a randomized controlled trial did not occur. Our study reinforces the need for the consistent use of a tool like the Checklist to Evaluate a Report of a Nonpharmacological Trial to assess the methodology of surgical trials.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Inadequate reporting is pervasive to almost every area of health research [4-6].”
From Helping editors, peer reviewers and authors improve the clarity, completeness and transparency of reporting health research. (2008) · cites this paper as [5]
Citations
Cites
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127
Moher, D. (2001) · JAMA · doi:10.1001/jama.285.15.1987 · Cited by 2,545
Moher, D., Jones, A., Lepage, L., & for the CONSORT Group (2001) · JAMA · doi:10.1001/jama.285.15.1992 · Cited by 1,027 · cited 2×
“Was the follow-up schedule the same in each group? 1/1 100 (3-100)”
BackgroundPlint, A. C., Moher, D., Morrison, A., Schulz, K., Altman, D. G., Hill, C., & Gaboury, I. (2006) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2006.tb00557.x · Cited by 872
BOUTRON, I., MOHER, D., TUGWELL, P., GIRAUDEAU, B., POIRAUDEAU, S., NIZARD, R., & RAVAUD, P. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.05.004 · Cited by 308
Boutron, I., Tubach, F., Giraudeau, B., & Ravaud, P. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.12.010 · Cited by 300 · cited 1×
“Was the follow-up schedule the same in each group? 1/1 100 (3-100)”
BackgroundDevereaux, P. J. (2001) · JAMA · doi:10.1001/jama.285.15.2000 · Cited by 258 · cited 1×
“Was the follow-up schedule the same in each group? 1/1 100 (3-100)”
BackgroundBHANDARI, M., RICHARDS, R. R., SPRAGUE, S., & SCHEMITSCH, E. H. (2002) · The Journal of Bone and Joint Surgery-American Volume · doi:10.2106/00004623-200203000-00009 · Cited by 243
Self-citationFoley, N. C., Bhogal, S. K., Teasell, R. W., Bureau, Y., & Speechley, M. R. (2006) · Physical Therapy · doi:10.1093/ptj/86.6.817 · Cited by 231
Cited by
CEC’s Standards for Classifying the Evidence Base of Practices in Special Education
Cook, B. G., Buysse, V., Klingner, J., Landrum, T. J., McWilliam, R. A., Tankersley, M., & Test, D. W. (2015) · Remedial and Special Education · doi:10.1177/0741932514557271 · Cited by 361
“In addition, although evidencebased reviewers must evaluate the methodological quality of studies based on the information reported in research reports, researchers often do not fully report all aspects of studies (Chan & Bhandari, 2007; Soares et al., 2004).”
BackgroundThe challenges faced in the design, conduct and analysis of surgical randomised controlled trials
Cook, J. A. (2009) · Trials · doi:10.1186/1745-6215-10-9 · Cited by 351
“However, blinding is not commonly undertaken in surgical trials [59].”
BackgroundMeredick, R. B., Vance, K. J., Appleby, D., & Lubowitz, J. H. (2008) · The American Journal of Sports Medicine · doi:10.1177/0363546508317964 · Cited by 348
“Based on our findings and study limitations, we offer the following recommendations, which could result in the potential for higher quality meta-analyses comparing single-bundle versus double-bundle reconstruction of the ACL in the future: (1) Investigators should become familiar with the Consolidated Standards of Rep…”
MethodStevens, A., Shamseer, L., Weinstein, E., Yazdi, F., Turner, L., Thielman, J., Altman, D. G., Hirst, A., Hoey, J., Palepu, A., et al. (2014) · BMJ · doi:10.1136/bmj.g3804 · Cited by 242
Moher, D., Simera, I., Schulz, K. F., Hoey, J., & Altman, D. G. (2008) · BMC Medicine · doi:10.1186/1741-7015-6-13 · Cited by 153
“Inadequate reporting is pervasive to almost every area of health research [4-6].”
Background
Metadata
- Authors
- Chan, S., & Bhandari, M.
- Affiliations
- Hamilton General Hospital (CA)
- Venue
- The Journal of Bone & Joint Surgery
- Published
- 2007-09
- Volume
- 89
- Issue
- 9
- Pages
- 1970-1978
- ISSN
- 0021-9355 (ISSN-L)
- OpenAlex
- W1973873228
- Cited by
- 85
- References
- 19
- Citations by year
- 2026: 1 · 2022: 2 · 2021: 1 · 2020: 4 · 2019: 4 · 2018: 5 · 2017: 4 · 2016: 6 · 2015: 9 · 2014: 6 · 2013: 6 · 2012: 13
- FWCI
- 7.41
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Bone and Joint Surgery (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- CONSORT statement, non-pharmacological treatment, orthopedic surgery, allocation concealment, blinding, reporting bias, author survey, surgical trials
- MeSH terms
- Authorship, Clinical Competence, Cross-Sectional Studies, Double-Blind Method, Humans, Medical Staff, Patients, Research Design, General Surgery, Bias, Randomized Controlled Trials as Topic, Single-Blind Method, Treatment Outcome, Orthopedic Procedures
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Quality of Reporting of Orthopaedic Randomized Trials with Use of a Checklist for Nonpharmacological Therapies. https://app.cochat.ai/papers/10.2106/jbjs.f.01591
- The paper
- Chan, S., & Bhandari, M. (2007). The Quality of Reporting of Orthopaedic Randomized Trials with Use of a Checklist for Nonpharmacological Therapies. The Journal of Bone & Joint Surgery. https://doi.org/10.2106/jbjs.f.01591
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.