Identifying the ‘incredible’! Part 2: Spot the difference - a rigorous risk of bias assessment can alter the main findings of a systematic review
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Systematic reviews are a valuable tool to inform healthcare decision-making.1 2 While a single randomised controlled trial (RCT) is insufficient to definitively guide healthcare decisions, a systematic review synthesising multiple RCTs can overcome this limitation. The results of rigorous systematic reviews possess wide-ranging applicability to numerous stakeholders within the evidence-based medicine ‘ecosystem’. Clinicians consult systematic reviews to inform their clinical decisions.3 Researchers rely on systematic reviews to identify knowledge gaps in existing literature.4 Health policymakers use systematic review evidence to inform practice guidelines and legislation.5 6 Journal editors often prioritise systematic reviews for their impact on readership attention and journal metrics.7 Finally, patients are empowered by systematic reviews that assess the beneficial and harmful patient-important outcomes of available management strategies.8 Evidently, systematic review authors have an important responsibility to ensure their findings provide the most accurate results possible.The biomedical literature expands by 22 systematic reviews daily,9 with no evidence that production is waning. More systematic reviews are desirable if they identify and inform important research questions that improve patient care.10 However, production of this magnitude is problematic when systematic reviews offer ‘extensive redundancy, little value, misleading claims and/or vested interests’.11 As we outlined in part 1, bias is a systematic deviation from the truth in the results of a research study due to limitations in study design, conduct, or analysis.2 Deviations may either overestimate or underestimate a study’s true findings depending of the type and magnitude of bias. As the results of a systematic review are only as valid as the studies it includes, pooling biased results from different studies can compromise the credibility of systematic review findings when no assessment, or a poor assessment, of risk of bias is performed.3 12Inadequate study design, conduct, or analysis …
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
Self-citationWood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538
The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981 · cited 1×
“34 35 For example, quality summary scores presented in the systematic review example ranged from 11/31 to 24/31 (table 4).”
Systematic Reviews: Rationale for systematic reviews
Mulrow, C. D. (1994) · BMJ · doi:10.1136/bmj.309.6954.597 · Cited by 1,677
The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses
IOANNIDIS, J. P. A. (2016) · The Milbank Quarterly · doi:10.1111/1468-0009.12210 · Cited by 1,483
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)00031-w · Cited by 1,308
Savović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00537 · Cited by 1,049
The Medical Review Article: State of the Science
MULROW, C. D. (1987) · Annals of Internal Medicine · doi:10.7326/0003-4819-106-3-485 · Cited by 856 · cited 2×
“Systematic reviews are a valuable tool to inform healthcare decision-making.(1,2) While a single”
BackgroundMethodPage, M. J., Shamseer, L., Altman, D. G., Tetzlaff, J., Sampson, M., Tricco, A. C., Catalá-López, F., Li, L., Reid, E. K., Sarkis-Onofre, R., et al. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1002028 · Cited by 768
Incorporating variations in the quality of individual randomized trials into meta-analysis
Detsky, A. S., Naylor, C. D., O'Rourke, K., McGeer, A. J., & L'Abbé, K. A. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90085-2 · Cited by 640
Cited by
Ardern, C. L., Büttner, F., Andrade, R., Weir, A., Ashe, M. C., Holden, S., Impellizzeri, F. M., Delahunt, E., Dijkstra, H. P., Mathieson, S., et al. (2022) · British Journal of Sports Medicine · doi:10.1136/bjsports-2021-103987 · Cited by 458
Self-citationGalante, J., Friedrich, C., Dawson, A. F., Modrego-Alarcón, M., Gebbing, P., Delgado-Suárez, I., Gupta, R., Dean, L., Dalgleish, T., White, I. R., et al. (2021) · PLOS Medicine · doi:10.1371/journal.pmed.1003481 · Cited by 319
Oral Branched-Chain Amino Acids Supplementation in Athletes: A Systematic Review
Martinho, D. V., Nobari, H., Faria, A., Field, A., Duarte, D., & Sarmento, H. (2022) · Nutrients · doi:10.3390/nu14194002 · Cited by 62
Martinez-Calderon, J., Flores-Cortes, M., Morales-Asencio, J. M., & Luque-Suarez, A. (2021) · Pain Medicine · doi:10.1093/pm/pnaa331 · Cited by 21
Metadata
- Authors
- Büttner, F., Winters, M., Delahunt, E., Elbers, R., Lura, C. B., Khan, K. M., Weir, A., & Ardern, C. L.
- Venue
- British Journal of Sports Medicine
- Publisher
- BMJ
- Published
- 2019-12-23
- Volume
- 54
- Issue
- 13
- Pages
- 801-808
- ISSN
- 0306-3674 (ISSN-L)
- 1473-0480
- OpenAlex
- W2995019734
- Cited by
- 55
- References
- 16
- FWCI
- 1.88
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Identifying the ‘incredible’! Part 2: Spot the difference - a rigorous risk of bias assessment can alter the main findings of a systematic review. https://app.cochat.ai/papers/10.1136/bjsports-2019-101675
- The paper
- Büttner, F., Winters, M., Delahunt, E., Elbers, R., Lura, C. B., Khan, K. M., Weir, A., & Ardern, C. L. (2020). Identifying the ‘incredible’! Part 2: Spot the difference - a rigorous risk of bias assessment can alter the main findings of a systematic review. British Journal of Sports Medicine. https://doi.org/10.1136/bjsports-2019-101675
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.