Revising the JBI quantitative critical appraisal tools to improve their applicability: an overview of methods and the development process
Abstract
JBI offers a suite of critical appraisal instruments that are freely available to systematic reviewers and researchers investigating the methodological limitations of primary research studies. The JBI instruments are designed to be study-specific and are presented as questions in a checklist. The JBI instruments have existed in a checklist-style format for approximately 20 years; however, as the field of research synthesis expands, many of the tools offered by JBI have become outdated. The JBI critical appraisal tools for quantitative studies (eg, randomized controlled trials, quasi-experimental studies) must be updated to reflect the current methodologies in this field. Cognizant of this and the recent developments in risk-of-bias science, the JBI Effectiveness Methodology Group was tasked with updating the current quantitative critical appraisal instruments. This paper details the methods and rationale that the JBI Effectiveness Methodology Group followed when updating the JBI critical appraisal instruments for quantitative study designs. We detail the key changes made to the tools and highlight how these changes reflect current methodological developments in this field.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The integrity of the included studies was appraised critically using the Johanna Briggs Institute checklists for case-control and cross-sectional designs and the Critical Appraisal Skills Programme cohort studies checklist.
“The integrity of the included studies was appraised critically using the critical appraisal tools of the Johanna Briggs Institute checklist for case-controlb and cross-sectionalb designs and the Critical Appraisal Skills Programme cohort studies checklist.26,c”
From Role of Anxiety in Individuals with Generalized Joint Hypermobility: A Systematic Review (2025) · cites this paper as [26]
The methodological quality of the included studies was assessed using the JBI Checklist.
“The methodological quality of the included studies was assessed using the JBI Checklist, with scores of all 109 studies provided in the online supplemental table S3.”
From Opportunities, challenges, and requirements for Artificial Intelligence (AI) implementation in Primary Health Care (PHC): a systematic review (2025) · cites this paper as [60]
The methodological quality of the included publications was assessed by applying the Johanna Briggs Institute (JBI) Checklist critical appraisal tool.
“The methodological quality of the included publications was assessed by applying the Johanna Briggs Institute (JBI) Checklist critical appraisal tool, which is designed to evaluate various study types.”
From Opportunities, challenges, and requirements for Artificial Intelligence (AI) implementation in Primary Health Care (PHC): a systematic review (2025) · cites this paper as [60]
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817 · see citing works
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 33,583 · see citing works
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329 · see citing works
Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n160 · Cited by 11,883 · see citing works
Downs, S. H., & Black, N. (1998) · Journal of Epidemiology and Community Health · doi:10.1136/jech.52.6.377 · Cited by 8,252 · see citing works
Munn, Z., Moola, S., Riitano, D., & Lisy, K. (2014) · International Journal of Health Policy and Management · doi:10.15171/ijhpm.2014.71 · Cited by 1,479 · see citing works
Self-citationThe importance of covariate selection in controlling for selection bias in observational studies.
Steiner, P. M., Cook, T. D., Shadish, W. R., & Clark, M. H. (2010) · Psychological Methods · doi:10.1037/a0018719 · Cited by 490 · see citing works
Catalogue of bias: attrition bias
Nunan, D., Aronson, J., & Bankhead, C. (2018) · BMJ Evidence-Based Medicine · doi:10.1136/ebmed-2017-110883 · Cited by 204 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Barker, T. H., Stone, J. C., Sears, K., Klugar, M., Tufanaru, C., Leonardi-Bee, J., Aromataris, E., & Munn, Z. (2023) · JBI Evidence Synthesis · doi:10.11124/jbies-22-00430 · Cited by 1,225 · see citing works
Self-citationBarker, T. H., Habibi, N., Aromataris, E., Stone, J. C., Leonardi-Bee, J., Sears, K., Hasanoff, S., Klugar, M., Tufanaru, C., Moola, S., et al. (2024) · JBI Evidence Synthesis · doi:10.11124/jbies-23-00268 · Cited by 843 · see citing works
Self-citationMsambwa, M. M., Wen, Z., & Daniel, K. (2025) · European Journal of Education · doi:10.1111/ejed.12909 · Cited by 76 · see citing works
Yousefi, F., Dehnavieh, R., Laberge, M., Gagnon, M.-P., Ghaemi, M. M., Nadali, M., & Azizi, N. (2025) · BMC Primary Care · doi:10.1186/s12875-025-02785-2 · Cited by 48 · see citing works
Sun, Y., Jia, Y., Wang, S., Xu, C., Qu, Y., Hu, M., & Jiang, H. (2024) · BMC Oral Health · doi:10.1186/s12903-024-04931-1 · Cited by 12 · see citing works
Role of Anxiety in Individuals with Generalized Joint Hypermobility: A Systematic Review
van Die-de Vries, J. E., Rameckers, E., Calders, P., Engelbert, R. H., Ramaekers, S. P., Goossens, M. E., & Verbunt, J. A. (2025) · Archives of Rehabilitation Research and Clinical Translation · doi:10.1016/j.arrct.2025.100467 · Cited by 1 · see citing works
- Authors
- Barker, T. H., Stone, J. C., Sears, K., Klugar, M., Leonardi-Bee, J., Tufanaru, C., Aromataris, E., & Munn, Z.
- Affiliations
- The University of Adelaide (AU)
- Queen's University (CA)
- Masaryk University (CZ)
- The Nottingham Centre for Evidence-Based Healthcare: A JBI Centre of Excellence, School of Medicine, University of Nottingham, Nottingham, UK
- JBI, Faculty of Health and Medical Sciences, The University of Adelaide, Adelaide, SA, Australia
- Corresponding author
- Barker, Timothy Hugh
- ORCID
- Venue
- JBI Evidence Synthesis
- Published
- 2022-11-22
- Volume
- 21
- Issue
- 3
- Pages
- 478-493
- ISSN
- 2689-8381 (ISSN-L)
- OpenAlex
- W4296330923
- Cited by
- 471
- References
- 8
- Citations by year
- 2026: 174 · 2025: 157 · 2024: 113 · 2023: 26 · 2022: 1
- FWCI
- 45.01
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- JBI Evidence Synthesis (publishedVersion)
- PubMed (publishedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- risk of bias, systematic review methodology, randomized controlled trials, quasi-experimental design
- MeSH terms
- Humans, Research Design, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
