The DOI matches its Crossref record; 89 of the 89 references it cites were found in a registry or an index; 4 of the 23 sources its claims cite were checked for support.
The devil is in the details: Reporting and transparent research practices in sports medicine and orthopedic clinical trials
Open accessPreprint — published as 10.1136/bmjopen-2021-059347 · Paper page
Abstract
Introduction: While transparent reporting of clinical trials is essential to assess the risk of bias and translate research findings into clinical practice, earlier studies have shown that deficiencies are common. This study examined current clinical trial reporting and transparent research practices in sports medicine and orthopedics. Methods: The sample included clinical trials published in the top 25% of sports medicine and orthopedics journals over nine months. Two independent reviewers assessed pre-registration, open data and criteria related to scientific rigor, the study sample, and data analysis. Results: The sample included 163 clinical trials from 27 journals. While the majority of trials mentioned rigor criteria, essential details were often missing. Sixty percent (confidence interval [CI] 53-68%) of trials reported sample size calculations, but only 32% (CI 25-39%) justified the expected effect size. Few trials indicated the blinding status of all main stakeholders (4%; CI 1-7%). Only 18% (CI 12-24%) included information on randomization type, method, and concealed allocation. Most trials reported participants’ sex/gender (95%; CI 92-98%) and information on inclusion and exclusion criteria (78%; CI 72-84%). Only 20% (CI 14-26%) of trials were pre-registered. No trials deposited data in open repositories. Conclusions: These results will aid the sports medicine and orthopedics community in developing tailored interventions to improve reporting. While authors typically mention blinding, randomization and other factors, essential details are often missing. Greater acceptance of open science practices, like pre-registration and open data, is needed. These practices have been widely encouraged, we discuss systemic interventions that may improve clinical trial reporting. Registration: https://doi.org/10.17605/OSF.IO/9648H
Overview
CoChat Summary
TL;DR
This meta-research study assessed reporting and transparent research practices in 163 clinical trials published in the top 25% of sports medicine and orthopedics journals, finding that while most trials mention rigor criteria like blinding and randomization, essential details needed to assess risk of bias are frequently missing. Only 20% of trials were pre-registered and no trial deposited data in an open repository.
Key findings
- Sixty percent (CI 53–68%) of trials reported sample size calculations, but only 32% (CI 25–39%) justified the expected effect size; among trials reporting a calculation (n=98), 53% (CI 43–63%) included a justification, 39% presented none, and 14% used arbitrary thresholds.
- Few trials indicated the blinding status of all main stakeholders (4%; CI 1–7%), and only 18% (CI 12–24%) included information on randomization type, method, and concealed allocation; 8% (CI 4–12%) provided complete allocation concealment information.
- Most trials reported participants' sex/gender (95%; CI 92–98%) and inclusion/exclusion criteria (78%; CI 72–84%); 75% reported the number of participants included in the final analysis.
- Only 20% (CI 14–26%) of trials were pre-registered (58% of registered trials were retrospective), and no trials deposited data in open repositories; 74% of trials with data availability statements said data were available upon request.
Methods
- Systematic examination of clinical trials published in the top 25% of orthopedics and sports medicine journals (Scimago Journal Rank) over nine months (2019/12–2020/08); 175 articles from 27 journals were screened, and 163 met the ICMJE clinical trial definition and were analyzed.
- Two independent reviewers (RS, GL, RP) performed full-text screening and data abstraction using preformatted Excel spreadsheets against CONSORT-based rigor, sample, data analysis, and transparency criteria; disagreements resolved by consensus.
- Exploratory analysis presenting prevalence as percentages with 95% confidence intervals; odds ratios with 95% CIs examined relationships between reporting completeness and pre-registration, flow charts, or sample size calculations (these analyses were not pre-registered).
Limitations
- Stated: CONSORT-based evaluation criteria for intervention reporting were not optimized for non-exercise or wait-and-see control interventions; odds ratios were exploratory post-hoc calculations requiring larger confirmatory studies; findings from the top 25% of journals may not generalize to journals not indexed by PubMed, lower tier journals, or non-English journals.
- Evident from the text: the study is exploratory and does not use formal sample size calculation (a precision-based estimate of 124–203 trials was used), and the preprint version was not certified by peer review.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
4 Valid · 4 Not valid · 14 Not checked · 70 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In 42% of papers, risk of bias could not be assessed due to incomplete reporting (13).”
What the sources say
- Partly supportedcites [13] 13. Chess LE, Gagnier J. Risk of bias of randomized controlled trials published in orthopaedic journals. BMC Med Res Methodol. 2013;1376. doi:10.1186/1471-2288-13-76 Cited in: PubMed; PMID 23758875. sourceExperimental — accuracy not yet measured
The Results (Table 7) report that 42% of the assessed criteria could not be evaluated because of inadequate reporting. The claim changes the denominator to 42% of papers; the paper does not report that proportion of papers as unassessable. (LLM, conf=0.91)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“only 1% of studies in high-impact orthopedic journals reported all ten criteria needed for risk of bias assessment (13).”
What the sources say
- Partly supportedcites [13] 13. Chess LE, Gagnier J. Risk of bias of randomized controlled trials published in orthopaedic journals. BMC Med Res Methodol. 2013;1376. doi:10.1186/1471-2288-13-76 Cited in: PubMed; PMID 23758875. sourceExperimental — accuracy not yet measured
The Results section of our copy states 'Less than 1% of the included RCTs fulfilled all ten methodological criteria' (232 RCTs from the top five orthopaedic journals: JBJS Am, AJSM, OC, SJ, JOR), which supports the substance of the citing sentence. Two qualifiers are not matched: the figure is 'less than 1%' rather than '1%', and the denominator is randomized controlled trials only, not 'studies' in those journals generally (Abstract > Conclusion and Discussion likewise speak of 'very few studies/RCTs met all ten criteria' within the RCT sample). The Methods > Data extraction section confirms the ten criteria were risk-of-bias items judged on the completeness of reporting, so 'reported all ten criteria' is broadly consistent. (LLM, conf=0.72)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“the odds of obtaining data decline precipitously with time since publication (70).”
What the sources say
- Supportedcites [70] 70. Vines TH, Albert AYK, Andrew RL, Débarre F, Bock DG, Franklin MT, Gilbert KJ, Moore JS, Renaut S, Rennison DJ. The availability of research data declines rapidly with article age. Curr Biol. 2014;24(1):94-7. doi:10.1016/j.cub.2013.11.014 Cited in: PubMed; PMID 24361065. source
The Summary (p. 3) states that 'availability of the data was strongly affected by article age' and that 'for papers where the authors gave the status of their data, the odds of a dataset being extant fell by 17% per year'; the Results (p. 5) give the logistic estimate (OR = 0.83 [0.79-0.90], p < 0.0001) and the Discussion (p. 6-7) describe the 'rapid increase' in lost/inaccessible datasets, with the proportion still existing dropping from 100% in 2011 to 33% in 1991. The paper's title itself frames this as availability declining rapidly with article age, so the paraphrase that the odds of obtaining data decline precipitously with time since publication matches what the source attributes to itself. The only nuance is that the 17%/year odds ratio is conditional on an informative response, but the citing sentence carries no number or qualifier, so nothing is overstated. (LLM, conf=0.87)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Focusing on the magnitude and precision of differences, instead making decisions based on p-value thresholds, reduces the likelihood of spurious findings (72,73).”
What the sources say
- Supportedcites [72] 72. Bernards JR, Sato K, Haff GG, Bazyler CD. Current Research and Statistical Practices in Sport Science and a Need for Change. Sports (Basel). 2017;5(4). doi:10.3390/sports5040087 Cited in: PubMed; PMID 29910447. source
Bernards et al. (2017) make exactly this contrast: in '1. The Problem' they state that under NHST 'p < 0.05 is sufficient to state an intervention was effective without any regard to the magnitude of the effect,' which 'allows researchers to dredge their data' and 'can lead to an inflated rate of Type I errors' — i.e. spurious findings. In '3.6. Counter-Argument against Magnitude-Based Inferences' they assert as fact that, across 500,000 simulated randomised trials, 'magnitude-based inference methods outperformed NHST in respect of inferential error rates,' and also on publishable outcomes and publication bias with suboptimal samples. The precision component is covered in '3.5' (effect size inspected via its confidence interval plus smallest worthwhile change) and in '5. Conclusions' (inclusion of confidence intervals and effect statistics yields more transparent, informative research; a shift to magnitude-based inference 'may be a better fit from ... a reproducibility standpoint'). The Conclusions hedge slightly ('may'), but the error-rate point in 3.6 is stated with at least the confidence of the citing sentence, so the paraphrase is supported. (LLM, conf=0.82)
- Not checked yetcites [73] 73. Riemann BL, Lininger MR. Principles of Statistics: What the Sports Medicine Professional Needs to Know. Clin Sports Med. 2018;37(3):375-86. doi:10.1016/j.csm.2018.03.004 Cited in: PubMed; PMID 29903380. source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“information on inclusion and exclusion criteria (78%; CI 72-84%)”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
42 more claims
Create a free account to see all 92 claims and to verify the 17 references still open.
- Probably not a substantial claim. 17
- It may state two separate claims as one. 32
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Inadequate reporting is also common in published studies [4–6], making it difficult for reviewers to evaluate manuscripts.”
From Is the future of peer review automated? (2022) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“17% (CI 11-22%; n=24/90) reported the reasons for some exclusions”
Reported 17 · recomputed 26.666667
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“Among trials reporting sample size calculations (n=98), 53% (CI 43- It is made available under a CC-BY-ND 4.0 International license . is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) medRxiv preprint doi: https://doi.org/10.1101/2021.07.20.21260565; this version posted September 8, 2021. The copyright holder for this preprint”
Reported 39 · recomputed 32.653061
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“33% (CI 26-41%; n=41/90) did not report any information.”
Reported 33 · recomputed 45.555556
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“Almost all sample size calculations were based on statistical power (93%; CI 8898%; n=96).”
Reported 93 · recomputed 97.959184
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Rigor and Sample Criteria · p. 13
“Blinding : Two-thirds of trials addressed blinding (Figure 2).”
§ Opportunities to improve reporting · p. 19
“Almost all trials addressed blinding, for example, but only 4% reported the blinding status of all main stakeholders.”
Citations
Cites
Rayyan-a web and mobile app for systematic reviews.
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0384-4 · Cited by 26,092
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Hoffmann, T. C., Glasziou, P. P., Boutron, I., Milne, R., Perera, R., Moher, D., Altman, D. G., Barbour, V., Macdonald, H., Johnston, M., et al. (2014) · BMJ · doi:10.1136/bmj.g1687 · Cited by 10,548 · cited 2×
“intervention reporting (52,53) were beyond the scope of this study, previous studies assessed .”
BackgroundResultLakens, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00863 · Cited by 10,303 · cited 1×
“Effect sizes show the magnitude of effects within a study, while standardized effect sizes allow for comparisons across studies (48).”
BackgroundMoher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515 · cited 1×
“Inflated effect sizes were also observed in the psychological science reproducibility project, where replicated effects were generally smaller than those in the initial studies (71).”
ResultPower failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“Category Assessment Rationale and Context Sample Size calculation Was an a priori sample size calculation performed? What type of sample size calculation was performed? Did the authors provide a justification for the expected effect size? - Low power is associated with high rates of spurious findings and overinflated…”
BackgroundWhat Makes Clinical Research Ethical?
Emanuel, E. J. (2000) · JAMA · doi:10.1001/jama.283.20.2701 · Cited by 2,710
Promoting an open research culture
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck, S., Chambers, C. D., Chin, G., Christensen, G., et al. (2015) · Science · doi:10.1126/science.aab2374 · Cited by 2,708
CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000251 · Cited by 2,296
Research works printed but not linked: 5
- ICMJE. Recommendations | Preparing a Manuscript for Submission to a Medical Journal: Methods - statistics [Internet]. 2021 [updated 2021 Apr 14
- SCImago (nd). SJR — SCImago Journal & Country Rank [Portal] [Internet]. 2020 [cited 2021 Feb 22]. Available from: https://.scimagojr.com/journalrank.php?category=2732
- BMC. Advancing peer review at BMC [Internet]. 2021 [updated 2021 Apr 8
- cited 2021 Apr 8]. Available from: https://www.biomedcentral.com/about/advancing-peer-review
- EU Trials Tracker. EU Trials Tracker — Who’s not sharing clinical trial results? [Internet]: Evidence-Based Medicine Data Lab; University of Oxford. 2021 [updated 2021 Apr 12; cited 2021 Apr 12]. Available from: https://eu.trialstracker.net/
Cited by
Is the future of peer review automated?
Schulz, R., Barnett, A., Bernard, R., Brown, N. J. L., Byrne, J. A., Eckmann, P., Gazda, M. A., Kilicoglu, H., Prager, E. M., Salholz-Hillel, M., et al. (2022) · BMC Research Notes · doi:10.1186/s13104-022-06080-6 · Cited by 80
Self-citationSchulz, R., Langen, G., Prill, R., Cassel, M., & Weissgerber, T. L. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059347 · Cited by 23
Self-citationHammami, R., Negra, Y., Nebigh, A., Ramírez‐Campillo, R., Moran, J., & Chaabène, H. (2022) · The Journal of Strength and Conditioning Research · doi:10.1519/jsc.0000000000004394 · Cited by 31
Replacing bar graphs of continuous data with more informative graphics: are we making progress?
Riedel, N., Schulz, R., Kazezian, V., & Weissgerber, T. L. (2022) · Clinical Science · doi:10.1042/cs20220287 · Cited by 19
Self-citation
Metadata
- Authors
- Schulz, R., Langen, G., Prill, R., Cassel, M., & Weissgerber, T.
- Affiliations
- University of Potsdam (DE)
- Medizinische Hochschule Brandenburg Theodor Fontane (DE)
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin (DE)
- Corresponding author
- Weissgerber, Tracey
- Venue
- BMJ Open
- Publisher
- openRxiv
- Published
- 2021-07-23
- Volume
- 12
- Issue
- 8
- Pages
- e059347-e059347
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC9364413
- OpenAlex
- W4290673869
- Semantic Scholar
- 2193515b1a67ed772ea630fb5eb9b0113ac108ea
- Cited by
- 23
- References
- 78
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 2 · 2023: 5 · 2022: 6
- FWCI
- 3.02
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ Open (publishedVersion, cc-by-nc)
- medRxiv (acceptedVersion, cc-by-nd)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Refubium (Universitätsbibliothek der Freien Universität Berlin) (submittedVersion, cc-by-nc)
- Universitätsbibliothek der FU Berlin Hochschulschriftenstelle u. Dokumentenserver
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- trial reporting quality, preregistration, open data, sample size justification, blinding, randomization, open science, research transparency, risk of bias
- MeSH terms
- Humans, Orthopedics, Sports Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The devil is in the details: Reporting and transparent research practices in sports medicine and orthopedic clinical trials. https://app.cochat.ai/papers/10.1101/2021.07.20.21260565
- The paper
- Schulz, R., Langen, G., Prill, R., Cassel, M., & Weissgerber, T. (2021). The devil is in the details: Reporting and transparent research practices in sports medicine and orthopedic clinical trials. BMJ Open. https://doi.org/10.1101/2021.07.20.21260565
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.