Most Cochrane systematic reviews and protocols did not adhere to the Cochrane's risk of bias 2.0 tool
Open accessAbstract
OBJECTIVE: The aim of this study was to identify the frequency of Cochrane systematic reviews and Cochrane systematic reviews protocols using (or planning to use) the risk of bias 2.0 tool to assess the risk of bias of the included randomized clinical trials. STUDY DESIGN: This is a meta-research study. METHODS: We included Cochrane systematic reviews or Cochrane systematic reviews protocols that planned to include randomized clinical trials. We assessed the Cochrane Database of Systematic Reviews and screened for issues published after the launch of risk of bias 2.0 tool (2019-2022). Two independent investigators performed the study selection and data extraction. RESULTS: We analyzed 440 Cochrane systematic reviews and 536 Cochrane systematic reviews protocols. Overall, 4.8% of the Cochrane systematic reviews and 28.5% of the Cochrane systematic reviews protocols used or planned to use risk of bias 2.0 tool. Although low, adherence is increasing over time. In 2019, 0% of Cochrane systematic reviews used risk of bias 2.0 tool, compared to 24.1% in 2022. In Cochrane systematic reviews protocols, adherence increased from 6.9% in 2019 to 41.5% in 2022. A total of 274 (62.1%) Cochrane systematic reviews had their protocols published before 2018; only one used risk of bias 2.0 tool and reported the change of versions in the "Differences between protocol and revision" section. CONCLUSION: The Cochrane's risk of bias 2.0 tool has low adherence among Cochrane protocols and systematic reviews. Further efforts are necessary to facilitate the implementation of this new tool.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Two independent reviewers used the Cochrane Risk of Bias 2 (RoB 2) tool to assess five domains: randomization, allocation concealment, blinding of participants/personnel, blinding of outcome assessment, incomplete outcome data, and selective reporting.
“Two independent reviewers used the Cochrane Risk of Bias 2 (RoB 2) tool to assess five domains: randomization, allocation concealment, blinding of participants/personnel, blinding of outcome assessment, incomplete outcome data, and selective reporting [19].”
From Ketogenic diet-induced changes in adult lipid metabolism: a comprehensive systematic review and meta-regression of randomized controlled trials (2026) · cites this paper as [19]
The Introduction (page not provided) confirms that RoB 2 is a domain-based risk-of-bias tool, but distinguishes it from the original RoB tool, whose domains include the listed blinding and incomplete-outcome-data categories. The Methods say two independent investigators extracted data for this meta-research study; they do not say those investigators used RoB 2 to assess trials. The paper therefore does not support the claimed reviewer procedure or the stated RoB 2 domain list. (LLM, conf=0.88)
Experimental — accuracy not yet measuredAssessing the risk of bias in studies is an essential step for a systematic review.
“Assessing the risk of bias in studies is an essential step for a systematic review (Martimbianco et al. 2023).”
From Adherence to the CONSORT statement of randomized clinical trials on dentin hypersensitivity treatment using mouthwashes (2025) · cites this paper as Martimbianco et al. 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 33,583 · see citing works
Minozzi, S., Cinquini, M., Gianola, S., Gonzalez-Lorenzo, M., & Banzi, R. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.06.015 · Cited by 471 · see citing works
Risk of bias versus quality assessment of randomised controlled trials: cross sectional study
Hartling, L., Ospina, M., Liang, Y., Dryden, D. M., Hooton, N., Krebs Seida, J., & Klassen, T. P. (2009) · BMJ · doi:10.1136/bmj.b4012 · Cited by 348 · see citing works
The art of quality assessment of RCTs included in systematic reviews
Verhagen, A. P., de Vet, H. C. W., de Bie, R. A., Boers, M., & van den Brandt, P. A. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00360-7 · Cited by 237 · see citing works
The effect of time spent in rehabilitation on activity limitation and impairment after stroke
Clark, B., Whitall, J., Kwakkel, G., Mehrholz, J., Ewings, S. M., & Burridge, J. H. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012612.pub2 · Cited by 84 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [3]Higgins JPT Savović J Page MJ Elbers RG Sterne JAC Chapter 8: assessing risk of bias in a randomized trial Higgins JPT Thomas J Chandler J Cumpston M Li T Page MJ Welch VA Cochrane handbook for systematic reviews of interventions version 6.3 (updated February 2022) Cochrane 2022
Cited by
Sorted by citations
Wang, Y., Keitz, S., Briel, M., Glasziou, P., Brignardello-Petersen, R., Siemieniuk, R. A. C., Zeraatkar, D., Akl, E. A., Armijo-Olivo, S., Bassler, D., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081199 · Cited by 60 · see citing works
A systematic review of suicide risk management strategies in primary care settings
Thangada, M. S., & Kasoju, R. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1440738 · Cited by 26 · see citing works
“The risk of bias evaluates six domains: randomization, bias due to deviation from intended intervention, missing outcome data, measurement of outcome, and bias in the selection of reported results with overall bias (35).”
BackgroundJambawo, S., Owolewa, R., & Jambawo, T. T. (2024) · Schizophrenia Research · doi:10.1016/j.schres.2024.10.006 · Cited by 17 · see citing works
“Furthermore, critical issues like reporting bias were excluded in RoB 2.0 (Martimbianco et al., 2023).”
BackgroundInfluentialComparison of laryngeal mask airway and endotracheal tube in general anesthesia in children
Dong, W. L., Zhang, W., Er, J., Liu, J., & Han, J. (2023) · Experimental and Therapeutic Medicine · doi:10.3892/etm.2023.12253 · Cited by 16 · see citing works
“The quality of the included studies was assessed according to the risk of bias assessment criteria established in the Cochrane Manual (15,16).”
MethodBlinding Assessments in Neonatal Ventilation Meta-Analyses: A Systematic Meta-Epidemiological Review
Kuitunen, I., Räsänen, K., Gualano, M. R., & Luca, D. D. (2024) · Neonatology · doi:10.1159/000539203 · Cited by 14 · see citing works
“The use of RoB 2.0 has been increasing annually, and our data support the importance of implementing this tool in the neonatal epidemiological and clinical research [59].”
InfluentialLuo, R., Wang, Y., Li, R., Ma, Y., Chen, H., Zhang, J., & Shen, J. (2024) · BMC Oral Health · doi:10.1186/s12903-024-04179-9 · Cited by 20 · see citing works
ChatGPT-4o in Risk-of-Bias Assessments in Neonatology: A Validity Analysis
Kuitunen, I., Nyrhi, L., & Luca, D. D. (2025) · Neonatology · doi:10.1159/000544857 · Cited by 11 · see citing works
“A similar low adherence rate to RoB2 has been reported overall in Cochrane Reviews [17], and we will need time to have more RoB2 available to perform this comparison.”
Myonectin and metabolic health: a systematic review
Petro, J. L., Gallo‐Villegas, J., & Calderón, J. C. (2025) · Frontiers in Endocrinology · doi:10.3389/fendo.2025.1557142 · Cited by 11 · see citing works
Chen, W., Miao, J., Wang, Y., Xing, W., Xu, X.-M., & Wu, R. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1212313 · Cited by 8 · see citing works
Brainard, J. S., Swindells, I. C., Wild, J., Hammer, C. C., Hornsey, E., Mahamed, H., & Willet, V. (2024) · BMC Health Services Research · doi:10.1186/s12913-024-11408-y · Cited by 6 · see citing works
Baldauf, B. J., Cemin, R., Biffi, M., Rapacciuolo, A., Zucchelli, G., Bongiorni, M. G., Casorelli, E., Mitacchione, G., Hohendanner, F., Durante‐Mangoni, E., et al. (2025) · Journal of Cardiovascular Development and Disease · doi:10.3390/jcdd12100406 · Cited by 5 · see citing works
“RCTs were evaluated using the Cochrane Risk of Bias 2.0 (RoB 2) [24] tool across five domains.”
InfluentialRiera, R., de Toledo, I. P., Farinasso, C. M., Pacheco, R. L., Silva, R. B., Colpani, V., Martimbianco, A. L. C., Cruz, C. M., Parreira, P. D. C. S., & Latorraca, C. D. O. C. (2025) · The Journal of Clinical Pharmacology · doi:10.1002/jcph.70068 · Cited by 5 · see citing works
Self-citationIncorrect blinding assessments are common in meta‐analyses published in high impact journals
Kuitunen, I., Ponkilainen, V., & Uimonen, M. (2024) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12636 · Cited by 5 · see citing works
Luo, B., Zhong, S., Wang, X., Guo, P., Hou, Y., & Di, W. (2024) · Frontiers in Pharmacology · doi:10.3389/fphar.2024.1440875 · Cited by 4 · see citing works
Effects of Advance Care Planning on the Mental Health of Bereaved Families: A Systematic Review
Tsubaki, M., Aoyagi, H., Ito, Y., Kobayashi, M., & Ushiwata, A. (2025) · Cureus · doi:10.7759/cureus.82403 · Cited by 7 · see citing works
“We chose RoB over the revised version (i.e., the Cochrane risk-of-bias tool for randomized trials (RoB2)) owing to its established ease of use [11].”
Vineesh, A., Shah, S., Shah, K., Hassan, M. Z., Sapkota, A., Khadka, S., Rizwanullah, F., Ibrahim, A. S., Hanumantharayudu, S. K., Surendraiah, P. K. M., et al. (2025) · Cureus · doi:10.7759/cureus.89300 · Cited by 3 · see citing works
Luo, B., Sun, Z., & Luo, H. (2025) · Frontiers in Pharmacology · doi:10.3389/fphar.2025.1584612 · Cited by 3 · see citing works
Jiang, H., Ma, R., Huang, Y., Li, X.-J., & Hao, Y.-F. (2025) · Frontiers in Neurology · doi:10.3389/fneur.2025.1682400 · Cited by 2 · see citing works
Shi, Y., Zhu, Y., Lan, Y., & Xu, L. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1590971 · Cited by 3 · see citing works
“The risk of bias in each randomized controlled trial (RCT) was evaluated according to the Cochrane Risk of Bias 2.0 framework (22), categorizing the risk of bias as low, some concerns, or high based on the predefined criteria.”
InfluentialMickenautsch, S., Rupf, S., & Yengopal, V. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1201951 · Cited by 3 · see citing works
“Besides its poor inter-rater reliability (IRR) and limited evidence base, its complex and demanding application may be the reason why a low adherence to the RoB 2 tool among Cochrane protocols and systematic reviews was observed and why most Cochrane reviewers themselves choose not to use the tool (6).”
MethodXue, X., Song, Q., Yang, X., Kuati, A., Fu, H., & Cui, G. (2024) · PLoS ONE · doi:10.1371/journal.pone.0301820 · Cited by 2 · see citing works
“…(XXL and SQF) independently assessed the methodological quality of each reviewed study by using the Cochrane Risk of Bias tools 2.0 (ROB 2.0) [19] to assess the risk of bias in randomized trials, and differences were resolved through discussion, or a third researcher (CGQ) was consulted if…”
MethodInfluentialExploring long-term psychological effects of bronchiolitis and influenza in school-aged children
Soni, P., & Cheriathu, J. J. (2025) · Frontiers in Pediatrics · doi:10.3389/fped.2025.1536571 · Cited by 2 · see citing works
Pacheco, R. L., & Riera, R. (2025) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.70328 · Cited by 2 · see citing works
“Early investigations revealed low adherence to the tool in both Cochrane [17] and non ‐ Cochrane reviews [18], as well as poor inter ‐ rater reliability [19] and inadequate usage [18].”
Self-citationZhang, X., Liang, N., Zhang, Y., Zhao, C., Yan, L., Hu, Z., Yang, S., Ye, Z., Shi, N., Wang, Y., et al. (2024) · Science of Traditional Chinese Medicine · doi:10.1097/st9.0000000000000042 · Cited by 1 · see citing works
Israyani, I., Rovani, C. A., Chen, S.-F., Gumilar, K. E., Wang, S.-M., Yu, J.-H., & Liu, C. (2025) · Journal of Dental Sciences · doi:10.1016/j.jds.2025.07.015 · Cited by 1 · see citing works
Neto, L. T., Wambier, L. M., & Chibinski, A. C. R. (2025) · The Saudi Dental Journal · doi:10.1007/s44445-025-00017-y
“Assessing the risk of bias in studies is an essential step for a systematic review (Martimbianco et al. 2023).”
Zhao, J., Wei, C., Gong, F., Pang, Z., & Zhao, H. (2026) · BMC Cardiovascular Disorders · doi:10.1186/s12872-026-05799-5
“Two independent reviewers used the Cochrane Risk of Bias 2 (RoB 2) tool to assess five domains: randomization, allocation concealment, blinding of participants/personnel, blinding of outcome assessment, incomplete outcome data, and selective reporting [19].”
Han, Y.-G., & Yi, C. (2024) · Physical Therapy Korea · doi:10.12674/ptk.2024.31.2.91
Arciniegas, S., Acosta-Reyes, J. L., de la Hoz, J. J. E., Arias-Rodriguez, S., Cotes-Mendoza, L. J., & Flórez, I. D. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-102525
“33 36 The RoB tool includes six domains: selection bias, performance bias, detection bias, attrition bias, reporting bias and other bias.”
Rodrigues, J. P., Sá, K. M. M., Lopes, G. C. M., Piovesan, M., Santos, E. M., & Martimbianco, A. L. C. (2025) · Dermatology Practical & Conceptual · doi:10.5826/dpc.1504a5668
Self-citationLuo, R., Wang, Y., Li, R., Ma, Y., Chen, H., Zhang, J., & Shen, J. (2024) · Figshare · doi:10.6084/m9.figshare.25632900
Brainard, J., Swindells, I. C., Wild, J., Hammer, C. C., Hornsey, E., Mahamed, H., & Willet, V. (2024) · Figshare · doi:10.6084/m9.figshare.26768405
Brainard, J., Swindells, I. C., Wild, J., Hammer, C. C., Hornsey, E., Mahamed, H., & Willet, V. (2024) · Figshare · doi:10.6084/m9.figshare.26768352
Bai, T., Jr., 2nd, X. C., Luo, L., Sun, D., Xie, T., & Zeng, X. (2025) · JMIR Research Protocols · doi:10.2196/82153
Liu, A., Xuehong, L., & Zhou, P. (2026) · Journal of Research in Medical Sciences · doi:10.4103/jrms.jrms_1233_25
“[13] The specific areas of RoB 2.0 consist of (1) the randomization process, (2) deviations from planned interventions, (3) absence of outcome data, (4) measurement of outcomes, (5) Two researchers assessed quality independently based on the criteria, rating each domain as low risk, some concerns,…”
InfluentialSimbar, M., Rashidi, F., Hajian, S., Taherian, R., & Kiani, Z. (2026) · Research Square · doi:10.21203/rs.3.rs-9645544/v1
Kılınçel, Ş., Bulut, F., & Kılınçel, O. (2026) · BMC Psychology · doi:10.1186/s40359-026-05010-6
“Pooled odds ratios (ORs) and 95% confidence intervals (CIs) were calculated using a random-effects model (DerSimonian and Laird method) to account for potential heterogeneity across studies [42].”
The impact of walking training on cognitive function in older adults: a meta-analysis
Zhao, S., Zeng, Y., Li, Y., Mu, S., & Zhao, X. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1720121
“The quality of each included randomized trial was evaluated using the Cochrane Risk of Bias version 2.0 (RoB 2) tool (Martimbianco et al., 2023).”
InfluentialXu, Z., Zhang, Y., Huo, Y., Xu, Z., Wang, C., Wang, C., Zhang, Y., & Zhao, Q. (2026) · Frontiers in Immunology · doi:10.3389/fimmu.2026.1854279
A historical review of risk of bias assessment tools for randomised controlled trials
Pacheco, R. L., & Riera, R. (2026) · Journal of the Royal Society of Medicine · doi:10.1177/01410768261494841
Self-citation
- Authors
- Martimbianco, A. L. C., Sá, K. M. M., Santos, G. M., Santos, E. M., Pacheco, R. L., & Riera, R.
- Affiliations
- Universidade Metropolitana de Santos (BR)
- Universidade Metropolitana de Santos, Brazil
- Centro Universitário São Camilo (BR)
- Hospital Sírio-Libanês (BR)
- Corresponding author
- Pacheco, Rafael Leite
- ORCID
- Venue
- Revista da Associação Médica Brasileira
- Publisher
- FapUNIFESP (SciELO)
- Published
- 2023-03
- Volume
- 69
- Issue
- 3
- Pages
- 469-472
- ISSN
- 0004-5241 (ISSN-L)
- 1806-9282
- 0104-4230
- PMC
- PMC10004297
- OpenAlex
- W4321371460
- Cited by
- 39
- References
- 5
- Citations by year
- 2026: 7 · 2025: 17 · 2024: 12 · 2023: 3
- FWCI
- 2.84
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Revista da Associação Médica Brasileira (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- risk of bias assessment, randomized controlled trials, Cochrane Database of Systematic Reviews, meta-research
- MeSH terms
- Systematic Reviews as Topic, Humans, Bias, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
