Cited by 274Meta-analysis and systematic reviewsDelphi Technique in ResearchStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index; 1 of the 21 sources its claims cite were checked for support.

The Cochrane risk of bias assessment tool 2 (RoB 2) versus the original RoB: A perspective on the pros and cons

Open access

Abstract

Background and Aims: Critical appraisal or risk of bias assessment is a fundamental part of systematic reviews that clarifies the degree to which included research articles are qualified and reliable. Version 2 of the Cochrane tool for assessing the risk of bias in randomized trials (RoB 2), the updated version of the first tool, was released in 2019. Here, we have compared these two versions of Cochrane risk of bias assessment tools and highlighted the pros and cons of RoB 2. Methods: Statistical analysis and methodology is not applicable to this article as no new data were created or analyzed in this study. Results: The overall approach in RoB 2 is that by answering some signaling questions after the specification of results, effects of interest, and sources of information, an overall judgment for the quality of each study is reached. Accordingly, in the original version of the Cochrane RoB tool, the judgment can be in three different conclusions, including low, unclear, and high risk of bias. The most prominent difference in bias domains is the removal of “other bias” domain being replaced by “overall bias” judgment. Also, the most common presentation types of Cochrane risk of bias assessments are the “summary” and “graph” which are generated by Review Manager, web‐based applications, or packages in R software. Conclusion: The RoB 2 tool, compared to the original RoB, has improved and is the recommended version by the Cochrane Collaboration for quality assessment of randomized controlled trials. It is recommended to consider funding source, duration of follow‐up, declaration of data availability, the status of baseline characteristics between groups, and sample size calculation methods in further revisions of the Cochrane risk of bias assessment tools.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This perspective article compares the Cochrane risk of bias tool version 2 (RoB 2, released 2019) with the original RoB 1 for randomized trials, arguing RoB 2 is improved and Cochrane-recommended but more complex and time-consuming. The authors recommend that future revisions consider funding source, follow-up duration, data availability, baseline characteristics, and sample size calculation methods.

Key findings

  • RoB 2 assesses five bias domains (randomization process, deviations from intended interventions, missing outcome data, measurement of outcomes, selection of the reported result) plus an "overall bias" judgment, whereas RoB 1 had six domains including "other bias"; the removal of "other bias" is described as the most prominent domain difference and as improving clarity.
  • RoB 2 uses structured signaling questions answered at five levels (yes, probably yes, no, probably no, no information) leading to judgments of low risk, some concerns, or high risk, replacing RoB 1's low/unclear/high judgments; RoB 2 focuses on results rather than outcomes.
  • RoB 2 is more resource-intensive: a systematic review of 113 studies reported an average of 358 min per article, versus 10–60 min per article for the original RoB tool; two studies found no significant inter-rater reliability (IRR: −0.15) and slightly higher IRR (IRR: 0.16) for overall judgment, while use of RoB 2 was significantly associated with improved methodological quality of systematic reviews (p = 0.007).
  • Drawing on Babic et al.'s survey of 768 Cochrane reviews including 11,369 RCTs, 87% of reviews reported the "other bias" domain with 5762 explanations categorized into 31 groups; the five most frequent categories were baseline characteristics (21.4%), funding (15.6%), sample size (8.1%), reporting (7.6%), and conflicts of interest (5.8%). A Cochrane review of 75 articles found industry-funded studies more often had favorable efficacy results (RR: 1.27; 95% CI: 1.17–1.37), suggesting an "industry bias" not assessed by RoB 2.

Methods

  • Perspective article; the authors state that statistical analysis and methodology are not applicable because no new data were created or analyzed.
  • The comparison focuses on RoB 2 for parallel study design versus RoB 1, drawing on previously published studies and the Cochrane Handbook to support the discussion.
  • No primary data were collected; results from prior studies are cited as supporting evidence, and suggestions for future tools are based on previous findings and the authors' opinions.

Limitations

  • Stated by the authors: it is a perspective article that only critiques the pros and cons of RoB 2 versus RoB 1 and offers factors for future iterations; it did not collect or report primary original data; it assessed only Cochrane RoB 2 with a focus on the parallel study design version and did not evaluate other quality assessment tools for clinical trials or observational studies; the suggestions are based on previous findings and authors' opinions, and further research, pilot studies, and expert consensus may be needed to evaluate their implementation.
  • Evident from the text: RoB 2 lacks supplements for some trial types (e.g., trials without a control arm, factorial-design RCTs), and some domains remain subjective, with signaling-question answers (e.g., on deviations from intended intervention) often being "no information" or "probably no" due to unavailable protocols, potentially leading to discrepant judgments.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1002/hsr2.2165

Cites

17 of 17 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

1 of 21 cited sources checked · 1 supported · 0 flagged

  • Premisecitation

    The time for the original RoB tool was between 10 and 60 min per article.

    “While this time for the original RoB tool was between 10 and 60 min per article.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [9] Savović J, Weeks L, Sterne JAC, et al. Evaluation of the Cochrane collaboration's tool for assessing the risk of bias in randomized trials: focus groups, online survey, proposed recommendations and their implementation. Syst Rev. 2014; 3(1): 37. 24731537 PMC4022341 source
  • Premisecitation

    One of the limitations of the RoB 2 is that it takes more time to complete each study.

    “One of the limitations of the RoB2 is that it takes more time to complete each study.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [8] Crocker TF, Lam N, Jordão M, et al. Risk‐of‐bias assessment using RoB2 was useful but challenging and resource‐intensive: observations from a systematic review. J Clin Epidemiol. 2023; 161: 39‐ 45. 37364620 source
    • cites [9] Savović J, Weeks L, Sterne JAC, et al. Evaluation of the Cochrane collaboration's tool for assessing the risk of bias in randomized trials: focus groups, online survey, proposed recommendations and their implementation. Syst Rev. 2014; 3(1): 37. 24731537 PMC4022341 source
  • Premisecitation

    There was a significant association between the use of RoB 2 and the improved methodological quality of the systematic reviews (p = 0.007).

    “Nevertheless, there was a significant association between the use of RoB 2 and the improved methodological quality of the systematic reviews (p = 0.007).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12] Minozzi S, Gonzalez‐Lorenzo M, Cinquini M, et al. Adherence of systematic reviews to Cochrane RoB2 guidance was frequently poor: a meta epidemiological study. J Clin Epidemiol. 2022; 152: 47‐ 55. 36156301 source
  • Interpretationreasoning

    The RoB 2 improved the quality assessment of randomized clinical trials in several ways.

    “The RoB 2 improved the quality assessment of randomized clinical trials in several ways.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The RoB 2 includes supplements for parallel, cluster-randomized, and cross-over designs of RCTs, whereas RoB 1 has one guidance and iteration.
  • Interpretationreasoning

    The most prominent difference in bias domains is that the 'other bias' domain is removed in RoB 2 and replaced by an 'overall bias' judgment based on the five bias domains.

    “The most prominent difference in bias domains is that the “other bias” domain is removed in RoB 2 and instead, an “overall bias” judgment has been added based on the status of the five bias domains.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. 15
  • It may state two separate claims as one. 11
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingnone given

    There were no remarkable changes in presenting the results between RoB 2 and the original RoB assessment tool.

    “There were no remarkable changes in presenting the results between RoB 2 (Figure 1A) and the original version of the RoB assessment tool (Figure 1B).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The most common presentation types of Cochrane risk of bias assessments are the 'summary' and 'graph' generated by Review Manager, web-based applications, or R packages.
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    The RoB 2 is focused on the results instead of outcomes.

    “Also, the RoB 2 is focused on the results instead of outcomes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [5] Ella F, Theresa HM, Isabelle B, et al. Using risk of bias 2 to assess results from randomised controlled trials: guidance from Cochrane. BMJ Evid Based Med. 2023; 28(4): 260. 36693715
  • Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.

    Interpretationnone given

    Omitting the 'other bias' domain has improved the clarity of RoB 2.

    “Meanwhile, omitting the “other bias” domain has improved the clarity of RoB 2.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim.

    Premisecitation

    In the original RoB tool, the overall judgment can be low, unclear, or high risk of bias.

    “However, in the original version of the Cochrane RoB tool, the judgment can be in three different groups, including low, unclear, and high risk based on provided support for judgment.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] Higgins JPT, Altman DG, Gøtzsche PC, et al. The Cochrane collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011; 343: d5928. 22008217 PMC3196245
  • Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.

    Interpretationnone given

    The judgment process and the conclusion of each study's risk of bias have been changed between RoB 2 and original RoB, while the presentation of the assessment is almost similar.

    “The judgment process and the conclusion of each study's risk of bias have been changed, while the presentation of the assessment is almost similar.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Both versions of the Cochrane RoB assessment tools use an option for RoB that cannot be categorized into low or high RoB; the original uses 'unclear' and RoB 2 uses 'some concerns'.
  • Probably not a substantial claim.

    Premisecitation

    The RoB 2 tool assesses five bias domains: bias due to randomization process, deviation from intended intervention, missing outcome data, measurement of outcomes, selection of the reported result, plus an overall risk of bias judgment.

    “In the RoB 2 tool, there are five bias domains, including (1) bias due to the randomization process, (2) deviation from intended intervention, (3) missing outcome data, (4) measurement of outcomes, (5) selection of the reported result, and “overall risk of bias” judgment.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [4] Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019; 366: l4898. 31462531
  • Probably not a substantial claim. The paper may give no reason why it holds.

    Interpretationreasoning

    The presence of signaling questions in RoB 2 (with five levels: yes, probably yes, no, probably no, no information) leading to high risk, low risk, and some concerns for each domain can increase the precision of RoB 2 compared to the original RoB tool.

    “The presence of signaling questions in RoB2, in five levels, including yes, probably yes, no, probably no, and no information, which leads to high risk, low risk, and some concerns for each domain can increase the precision of this revised version compared to the original version of RoB tool.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Interpretationcitation

    The RoB 2 tool, compared to the original RoB, has been improved and is the recommended version by Cochrane for the quality assessment of RCTs.

    “The RoB 2 tool, compared to the original RoB, has been improved and is the recommended version by Cochrane for the quality assessment of RCTs.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [18] Higgins JPT, Savović J, Page MJ, Elbers RG, Sterne JAC. Chapter 8: Assessing risk of bias in a randomized trial. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, eds. Cochrane Handbook for Systematic Reviews of Interventions Version 6.2 (Updated February 2021). Cochrane; 2021.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • Rests on an assumption: The RoB 2 includes supplements for parallel, cluster-randomized, and cross-over designs of RCTs, whereas RoB 1 has one guidance and iteration.
  • Probably not a substantial claim. It may state two separate claims as one.

    Interpretationnone given

    The RoB 2 uses a more comprehensive and structured framework for quality assessment using signaling questions, compared to the original RoB which was based on authors' judgments and quotes.

    “Contrary to the original version of the Cochrane RoB tool which was based on the authors' judgments and used quotes from the papers to support the judgments, the RoB 2 used a more comprehensive and structured framework for quality assessment using the signaling questions.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Premisecitation

    A comparison study between the two versions of the Cochrane RoB assessment tools showed that responses to the question 'Were there deviations from the intended intervention that arose because of the trial context?' are usually 'no information' or 'probably no' due to no available protocol, leading to overall judgments of 'some concerns' or low RoB, respectively, and can lead to discrepancy in the

    “For example, the results of a comparison study between the two versions of the Cochrane RoB assessment tools showed that the responses to “Were there deviations from the intended intervention that arose because of the trial context?” question are usually “no information” or “probably no” due to no available protocol which leads to overall judgments of “some concerns” or low RoB, respectively and”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [13] Richter B, Hemmingsen B. Comparison of the Cochrane Risk of Bias Tool 1 (RoB 1) with the 517 Updated Cochrane Risk of Bias Tool 2 (RoB 2). 2021. https://communitycochraneorg/sites/default/files/uploads/inline-files/RoB1_2_project_220529_BR%20KK%20formattedpdf
  • It may state two separate claims as one.

    Interpretationcitation

    In spite of the improvement in the assessment methods in the RoB2, there are still domains that are subjective and there might be doubt and disagreement about the answer to the signaling questions.

    “In spite of the improvement in the assessment methods in the RoB2, there are still domains that are subjective and there might be doubt and disagreement about the answer to the signaling questions.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Premisecitation

    The study by Babic et al. on 768 Cochrane reviews showed that 87% of reviews reported 'other bias' domain and there were 5,762 types of explanations categorized into 31 groups.

    “The study conducted by Babic et al. on 768 Cochrane reviews which included 11,369 randomized controlled trials (RCTs) showed that 87% of reviews reported “other bias” domain and there were 5,‐762 types of explanations which were categorized into 31 groups.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [6] Babic A, Pijuk A, Brázdilová L, et al. The judgement of biases included in the category “other bias” in Cochrane systematic reviews of interventions: a systematic survey. BMC Med Res Methodol. 2019; 19(1): 77. 30971219 PMC6458756 source
  • It may state two separate claims as one.

    Premisecitation

    Two different studies assessing the RoB 2 showed no significant inter-rater reliability (IRR: −0.15) and slightly higher IRR (IRR: 0.16) for overall judgment.

    “Two different studies assessing the RoB 2 showed no significant inter‐rater reliability (IRR: −0.15) and slightly higher IRR (IRR: 0.16) for overall judgment.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [10] Minozzi S, Dwan K, Borrelli F, Filippini G. Reliability of the revised Cochrane risk‐of‐bias tool for randomised trials (RoB2) improved with the use of implementation instruction. J Clin Epidemiol. 2022; 141: 99‐ 105. 34537386 source
    • cites [11] Minozzi S, Cinquini M, Gianola S, Gonzalez‐Lorenzo M, Banzi R. The revised Cochrane risk of bias tool for randomized trials (RoB 2) showed low interrater reliability and challenges in its application. J Clin Epidemiol. 2020; 126: 37‐ 44. 32562833

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • It may state two separate claims as one.

    Interpretationreasoning

    Overall, the RoB 2 is comprehensive but complex and difficult to use even for experts.

    “Overall, the RoB 2 is comprehensive but complex and difficult to use even for experts.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim.

    Premisecitation

    In RoB 2, the overall approach is that answering signaling questions after specifying results, effects of interest, and sources of information leads to an overall judgment for the quality of each study.

    “The overall approach in RoB2 is that by answering some signaling questions after the specification of results, effects of interest, and sources of information, an overall judgment for the quality of that study is reached.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [4] Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019; 366: l4898. 31462531
  • Probably not a substantial claim.

    Premisecitation

    The original Cochrane RoB tool assesses six bias domains: selection bias, performance bias, detection bias, attrition bias, reporting bias, and other bias.

    “In the original RoB tool, six bias domains (and source[s] of bias) are assessed, including selection bias, performance bias, detection bias, attrition bias, reporting bias, and “other bias.””
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] Higgins JPT, Altman DG, Gøtzsche PC, et al. The Cochrane collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011; 343: d5928. 22008217 PMC3196245
  • It may state two separate claims as one.

    Interpretationnone given

    One of the main limitations of the RoB 2 is that it is complex and needs trained individuals to use this tool.

    “On the other hand, one of the main limitations of the RoB 2 is that it is complex and needs trained individuals to use this tool.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.

    Interpretationnone given

    Still some improvements need to be implemented in RoB 2, such as considering some valuable information addressing the quality of studies.

    “Nevertheless, still some improvements need to be implemented such as considering some valuable information addressing the quality of studies.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.

    Interpretationnone given

    There is still a lack of some supplements for other types of clinical trials like clinical trials without a control arm or RCTs with factorial design.

    “However, there is still a lack of some supplements for other types of clinical trials like clinical trials without a control arm or RCTs with factorial design.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim.

    Interpretationreasoning

    The RoB 2 improved the criteria for judgment on how to reach an overall judgment about the RoB using signaling questions and algorithms.

    “Nevertheless, the RoB 2 improved the criteria for judgment on how to reach an overall judgment about the RoB using the signaling questions and algorithms.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Weibel and colleagues generated a research integrity assessment tool to identify problematic RCTs which evaluates six criteria: ethics approval status, study retraction, prospective trial registration, author group, plausibility of methods, and study results.

    “In this regard, Weibel and colleagues generated a research integrity assessment tool to identify problematic RCTs which evaluates six criteria, including ethics approval status, study retraction, prospective trial registration, author group, plausibility of methods, and study results.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [17] Weibel S, Popp M, Reis S, Skoetz N, Garner P, Sydenham E. Identifying and managing problematic trials: a research integrity assessment tool for randomized controlled trials in evidence synthesis. Res Synt Methods. 2023; 14(3): 357‐ 369. PMC10551123 36054583 source
  • It may state two separate claims as one.

    Premisecitation

    Protocol registration and deviation from registered protocol were reported in 0.3% and 2.5% of the 'other bias' domain in RoB1, respectively.

    “Protocol registration and deviation from registered protocol were reported in 0.3% and 2.5% of the “other bias” domain in the RoB1, respectively.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] Babic A, Pijuk A, Brázdilová L, et al. The judgement of biases included in the category “other bias” in Cochrane systematic reviews of interventions: a systematic survey. BMC Med Res Methodol. 2019; 19(1): 77. 30971219 PMC6458756 source
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    The AMSTAR 2 includes two domains related to the source of funding.

    “The AMSTAR 2, which is a tool for quality assessment of systematic reviews, includes two domains related to the source of funding (Domain 10: “Did the review authors report on the sources of funding for the studies included in the review?” and Domain 16: “Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?”).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non‐randomised studies of healthcare interventions, or both. BMJ. 2017; 358: j4008. 28935701 PMC5833365 source
  • It may state two separate claims as one.

    Interpretationcitation

    There can be an 'industry bias' that is not assessed in the RoB 2, which might be considered for designing future quality assessment tools.

    “As a result, there can be an “industry bias” that is not assessed in the RoB 2 which might be considered for designing future quality assessment tools.”
    Quote in the paperSubstantial claim
24 of 24
  • The paper relies on this without support.

    The implications of the results for researchers and methodologists and how the changes can lead to high-quality systematic reviews were not mentioned in previous publications.

    “However, the implications of the results for researchers and methodologists and how the changes can lead to the production of high‐quality systematic reviews were not mentioned.”
  • The paper relies on this without support.

    Some domains that were omitted from the original version of RoB assessment were not analyzed in depth in previous publications.

    “In addition, some domains that were omitted from the original version of RoB assessment were not analyzed in depth in the previous publications.”
  • The paper relies on this without support.

    The RoB 2 includes supplements for parallel, cluster-randomized, and cross-over designs of RCTs, whereas RoB 1 has one guidance and iteration.

    “Despite the RoB 1 that has one guidance and iteration, the RoB 2 includes other supplements for parallel, cluster‐randomized, and cross‐over designs of RCTs.”
  • The paper relies on this without support.

    Both versions of the Cochrane RoB assessment tools use an option for RoB that cannot be categorized into low or high RoB; the original uses 'unclear' and RoB 2 uses 'some concerns'.

    “Both versions of the Cochrane RoB assessment tools use another option for RoB that cannot be categorized into low or high RoB. In this regard, the original version uses “unclear” and the RoB 2 uses “some concerns.””
  • The paper relies on this without support.

    The RoB 1 and RoB 2 include the 'reporting bias' domain and 'bias in the selection of the reported result,' respectively, to assess reporting bias.

    “The RoB 1 and RoB 2 include the “reporting bias” domain and “bias in the selection of the reported result,” respectively, to assess this outcome.”
  • The paper relies on this without support.

    The most common presentation types of Cochrane risk of bias assessments are the 'summary' and 'graph' generated by Review Manager, web-based applications, or R packages.

    “The most common presentation types of Cochrane risk of bias assessments are the “summary” and “graph” which are generated by Review Manager (RevMan) (Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014) or RevMan Web application.”
6 of 6

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Figure 1 reverses the version labels

    Minor

    The text identifies Figure 1A as RoB 2 and Figure 1B as the original tool, but the figure caption assigns version 1 to A and version 2 to B.

1 of 1

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Higgins JPT, 2021, Cochrane Handbook for Systematic Reviews of Interventions Version 6.2 (Updated February 2021)

Cited by

Sorted by citations

Metadata

Authors
Nejadghaderi, S. A., Balibegloo, M., & Rezaei, N.
Affiliations
  • Kerman University of Medical Sciences (IR)
  • Universal Scientific Education and Research Network (IR)
Corresponding author
Rezaei, Nima
Venue
Health Science Reports
Publisher
Wiley
Published
2024-06
Volume
7
Issue
6
Pages
e2165-e2165
ISSN
2398-8835 (ISSN-L)
OpenAlex
W4399330391
Cited by
274
References
17
Citations by year
2026: 177 · 2025: 89 · 2024: 8
FWCI
33.33
Type
journal article
Language
en
Open access
Locations
  • Health Science Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords
risk of bias assessment, critical appraisal, randomized controlled trials, trial quality assessment, systematic review methodology, funding disclosure, sample size determination

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review