CoChat

The Quality of Reporting of Orthopaedic Randomized Trials with Use of a Checklist for Nonpharmacological Therapies

Chan, Simon, Bhandari, Mohit · 2007 · The Journal of Bone & Joint Surgery

The DOI matches its Crossref record; 18 of the 19 references it cites were found in a registry or an index.

Abstract

BACKGROUND: The Consolidated Standards of Reporting Trials statement for the reporting of randomized controlled trials has been limited by its applicability to surgical trials. In response, a Checklist to Evaluate a Report of a Nonpharmacological Trial was recently developed by the Consolidated Standards of Reporting Trials group to address reporting issues in surgical trials. We aimed (1) to apply the checklist for nonpharmacological therapies to orthopaedic randomized controlled trials across multiple journals from 2004 through 2005, and (2) to survey authors when methodological safeguards itemized in the checklist were not reported to determine whether they actually had been performed. We hypothesized that lack of reporting of a methodological safeguard did not necessarily mean it had not been conducted. METHODS: We searched for relevant orthopaedic randomized controlled trials across eight journals in the period from January 2004 through December 2005. We applied the Checklist to Evaluate a Report of a Nonpharmacological Trial to all eligible studies. We contacted authors to determine what methodological safeguards were actually used, especially when details remained unclear from the publication. RESULTS: We included eighty-seven randomized controlled trials from eighty-five scientific reports. In assessing the randomized controlled trials with the checklist for nonpharmacological therapies, seventy-three studies (84%) had unclear reporting of treatment allocation concealment. Only seventeen studies (20%) mentioned surgeon skill or experience. The blinding of patients, ward staff, rehabilitation staff, clinical outcome assessors, and nonclinical outcome assessors was unclear in forty-eight (55%), sixty-three (72%), sixty-four (74%), forty (46%), and thirty-three studies (38%), respectively. Authors from forty-three randomized controlled trials responded to our survey. The results of the survey showed that 41% (95% confidence interval, 25% to 58%) of the trials had adequate allocation concealment when this had been unclear from the report. Although the surgical experience of the investigators was rarely reported, most authors (70%) acknowledged that they had defined "surgical expertise criteria" such as minimum case criteria, specialized training, and clinical performance. The survey also showed that 28% to 40% of the trials had blinding of relevant groups despite the fact that the reporting of such blinding had been unclear in the publications. CONCLUSIONS: The quality of reporting in the orthopaedic literature was highly variable. Readers should not assume that bias-reducing safeguards that were not reported in a randomized controlled trial did not occur. Our study reinforces the need for the consistent use of a tool like the Checklist to Evaluate a Report of a Nonpharmacological Trial to assess the methodology of surgical trials.

Overview

Integrity signals

Registry

Crossref · 10.2106/jbjs.f.01591

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 18 references

Cited by

See all 85

Metadata

Authors
Chan, S., & Bhandari, M.
Affiliations
Hamilton General Hospital (CA)
Venue
The Journal of Bone & Joint Surgery
Published
2007-09
Volume
89
Issue
9
Pages
1970-1978
ISSN
0021-9355 (ISSN-L)
OpenAlex
W1973873228
Cited by
85
References
19
Citations by year
2026: 1 · 2022: 2 · 2021: 1 · 2020: 4 · 2019: 4 · 2018: 5 · 2017: 4 · 2016: 6 · 2015: 9 · 2014: 6 · 2013: 6 · 2012: 13
FWCI
7.41
Type
journal article
Language
en
Open access
closed
Locations
  • Journal of Bone and Joint Surgery (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords
CONSORT statement, non-pharmacological treatment, orthopedic surgery, allocation concealment, blinding, reporting bias, author survey, surgical trials
MeSH terms
Authorship, Clinical Competence, Cross-Sectional Studies, Double-Blind Method, Humans, Medical Staff, Patients, Research Design, General Surgery, Bias, Randomized Controlled Trials as Topic, Single-Blind Method, Treatment Outcome, Orthopedic Procedures

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.