The DOI matches its Crossref record; 48 of the 49 references it cites were found in a registry or an index.
A comparison of two assessment tools used in overviews of systematic reviews: ROBIS versus AMSTAR-2
Open accessAbstract
BACKGROUND: AMSTAR-2 is a 16-item assessment tool to check the quality of a systematic review and establish whether the most important elements are reported. ROBIS is another assessment tool which was designed to evaluate the level of bias present within a systematic review. Our objective was to compare, contrast and establish both inter-rater reliability and usability of both tools as part of two overviews of systematic reviews. Strictly speaking, one tool assesses methodological quality (AMSTAR-2) and the other assesses risk of bias (ROBIS), but there is considerable overlap between the tools in terms of the signalling questions. METHODS: for weighted analysis). RESULTS: Thirty-one systematic reviews were included. For AMSTAR-2, the median agreement for all questions was 0.61. Eight of the 16 AMSTAR-2 questions had substantial agreement or higher (> 0.61). For ROBIS, the median agreement for all questions was also 0.61. Eleven of the 24 ROBIS questions had substantial agreement or higher. CONCLUSION: ROBIS is an effective tool for assessing risk of bias in systematic reviews and AMSTAR-2 is an effective tool at assessing quality. The median agreement between raters for both tools was identical (0.61). Reviews that included a meta-analysis were easier to rate with ROBIS; however, further developmental work could improve its use in reviews without a formal synthesis. AMSTAR-2 was more straightforward to use; however, more response options would be beneficial.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“As AMSTAR 2 and the ROBIS tool 58 overlap in many sig naling questions, 37 the ROBIS tool was not used in this overview.”
From Acceptance and Commitment Therapy for Chronic Pain: An Overview of Systematic Reviews with Meta-Analysis of Randomized Clinical Trials (2024) · cites this paper as [37]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
ROBIS: A new tool to assess risk of bias in systematic reviews was developed
Whiting, P., Savović, J., Higgins, J. P. T., Caldwell, D. M., Reeves, B. C., Shea, B., Davies, P., Kleijnen, J., & Churchill, R. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.06.005 · Cited by 2,272
Self-citationShea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.009 · Cited by 1,755
Murad, M. H., Asi, N., Alsawas, M., & Alahdab, F. (2016) · Evidence Based Medicine · doi:10.1136/ebmed-2016-110401 · Cited by 1,388
Gwet, K. L. (2014) · Medical Entomology and Zoology · Cited by 1,095
Acupuncture for treating fibromyalgia
Deare, J. C., Zheng, Z., Xue, C. C., Liu, J., Shang, J., Scott, S., & Littlejohn, G. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007070.pub2 · Cited by 271
Lactobacillus reuteri to Treat Infant Colic: A Meta-analysis
Sung, V., D'Amico, F. J., Cabana, M. D., Chau, K., Koren, G., Savino, F., Szajewska, H., Deshpande, G. C., Dupont, C. M., Indrio, F., et al. (2017) · PEDIATRICS · doi:10.1542/peds.2017-1811 · Cited by 235
Research works printed but not linked: 1
- Terhorst L, Schneider M. Complementary and alternative medicine in the treatment of pain in fibromyalgia: a systematic review of randomized controlled trials. Portland: From International Research Congress on Integrative Medicine and Health; 2012. 15-18 May 2012
Cited by
Sorted by citations
Items 101–130 of 130
A MELHORA DA DOR COMO DESFECHO CLÍNICO-CIRÚRGICO NA ENDOMETRIOSE: UMA VISÃO GERAL
Rocha, L. C. A., Silva, M. K. D. T., Erthal, S. D. S., de Souza, N. A., & Vasconcellos, M. (2023) · RECIMA21 - Revista Científica Multidisciplinar · doi:10.47820/recima21.v4i5.3102
Talla, P. K., Allison, P. J., Bussières, A., Giraudeau, N., Komarova, S. V., Basiren, Q., Bergeron, F., & Emami, E. (2023) · medRxiv · doi:10.1101/2023.07.04.23292218
Fernández‐Moreno, M., Pérez‐Santiago, L., & Sabater-Orti, L. (2024) · Cirugía Española (English Edition) · doi:10.1016/j.cireng.2024.04.005
Fernández‐Moreno, M., Pérez‐Santiago, L., & Sabater, L. (2024) · Cirugía Española · doi:10.1016/j.ciresp.2024.04.002
A Systematic Review of the Influence of Anorexic Voice on the Recovery of Anorexia NervosaPreprint
Dev, A. T., & Williams, C. (2024) · Research Square · doi:10.21203/rs.3.rs-4371197/v1
Was ist eine Literaturübersicht?
Seidlein, A., & Kahl-Scholz, M. (2024) · Palliative Care und Forschung · doi:10.1007/978-3-658-43829-6_2
GADELHA, G. C. A., & Júnior, A. M. D. C. (2024) · Revista do Colégio Brasileiro de Cirurgiões · doi:10.1590/0100-6991e-20243757
Grundlagen der Diagnostik und Therapie
(2024) · Elsevier eBooks · doi:10.1016/b978-3-437-22487-4.16001-0
Descatha, A., Mayet, A., Sembajwe, G. N., & Fadel, M. (2025) · Archives of Public Health · doi:10.1186/s13690-025-01622-3 · Cited by 1
Talla, P. K., Djedou, B. Y., Gore, G. C., Nicolau, B. F., & Bush, P. L. (2025) · medRxiv · doi:10.1101/2025.07.30.25332274
Efficacy and Safety of SGLT2 Inhibitors in the Treatment of Type 2 Diabetes: An Umbrella ReviewPreprint
Liu, H., & Ma, R. (2025) · medRxiv · doi:10.1101/2025.11.27.25341177
Matterne, U., Theurich, M. A., Proebstl, S., Pieper, D., Wang, J., Xu, A., & Apfelbacher, C. (2024) · Figshare · doi:10.6084/m9.figshare.26580269
Luo, J., Chen, Z., Liu, D., Li, H., He, S., Zeng, L., Yang, M., Liu, Z., Xiao, X., & Zhang, L. (2023) · Figshare · doi:10.6084/m9.figshare.23812451
Burton, R., Sharpe, C., Bhuptani, S., Jecks, M., Henn, C., Pearce‐Smith, N., Knight, S., Regan, M., & Sheron, N. (2024) · Figshare · doi:10.6084/m9.figshare.25801064
Naderi, M., Ghiyasi, Z., Farjami, P., Yarahmadi, D., Parnianfard, N., Aboutalebi, M., & Parnianfard, N. (2025) · Future Virology · doi:10.1080/17460794.2025.2609517
Henry, M. L., O'Connell, N. E., Riley, R. D., Moons, K. G. M., Shea, B., Hooft, L., Wallwork, S. B., Damen, J. A., Skoetz, N., Appiah, R. P., et al. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-109388
Anestesia Pediátrica y Neurodesarrollo: ¿Qué Nos Dice la Evidencia Científica?
Loor, C. S. A., Fernández, I. A. A., Negrete, E. L. H., & Delgado, M. J. S. (2026) · Estudios y Perspectivas Revista Científica y Académica · doi:10.61384/r.c.a..v6i1.1751
Ma, R., Liu, H., & Li, J. (2026) · doi:10.2196/preprints.95872
Risk of Bias in Rapid Reviews: Omission, Adaptation and TransparencyPreprint
Potter, R., Kenny, R. P. W., Inskip, A., Oliver, E. J., Ray, D., & Norman, G. (2026) · Research Square · doi:10.21203/rs.3.rs-9418335/v1
Storman, M., Storman, D., Świerz, M. J., Jasińska, K., Jemioło, P., & Bała, M. M. (2026) · Obesity Reviews · doi:10.1111/obr.70147
Lee, J., & Lee, Y. (2026) · Journal of Nutrition and Health · doi:10.4163/jnh.2026.59.2.115
Experiences from conducting systematic reviews of systematic reviews
Grooten, W. J. A., Bohman, T., Boström, C., Dedering, Å., Halvorsen, M., Kuster, R. P., Nilsson-Wikmar, L. B., Olsson, C. B., Rovner, G., Tseli, E., et al. (2026) · BMC Medical Research Methodology · doi:10.1186/s12874-026-02880-7
Banatwala, U. E. S. S., Mansoor, A., Fawzy, M., Khan, S. A., Nusrat, N. B., & Khan, K. S. (2026) · International Urology and Nephrology · doi:10.1007/s11255-026-05243-9
Apicella, A., Oliver, S. R., & Tripney, J. (2026) · Campbell Systematic Reviews · doi:10.1177/18911803261461576
Gliske, K., Buchanan, G. J. R., Carlson, D. D., Cai, Q., Waring, M. E., & Zhang, N. (2026) · Mindfulness · doi:10.1007/s12671-026-02917-4
Common Inflammatory Pathways Between Periodontal Disease and Multiple Sclerosis: A Systematic Review
Arcas, V. C., Roman-Filip, I., Moga, D. F. C., Săceleanu, A., Fratila, A. M., Fratila, L. N., & Roman-Filip, C. (2026) · Diseases · doi:10.3390/diseases14080268
Acupuncture for nausea and vomiting during pregnancy: an overview of systematic reviews
Cui, J., Ai, Z., Shen, W., & Han, M. (2026) · Frontiers in Endocrinology · doi:10.3389/fendo.2026.1886667
Zhang, M., Xia, Y., Mo, H., Zhu, Y., Fang, L., Xiang, Q., Shen, J., Wang, L., & Lou, J. (2026) · Epidemiologic Reviews · doi:10.1093/epirev/mxag013
Bebeselea, A.-M., Arcas, V. C., Moga, D. F. C., Fratila, A. M., Teodoru, M., Negrea, M. O., Arcas, A.-M. C., & Maniţiu, I. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15187196
Selecting the optimal study design!
Albabtain, M. A., AlRasheed, M. M., Arafat, A. A., Breitinger, H., Breitinger, U., Elsisi, G. H., Gaber, A., Gumaa, M., Hozien, M., Abdelghany, R. M., et al. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-45595-7.00015-7
Metadata
- Authors
- Perry, R., Whitmarsh, A., Leach, V., & Davies, P.
- Affiliations
- National Institute for Health and Care Research (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Corresponding author
- Perry, R.
- Venue
- Systematic Reviews
- Published
- 2021-10-25
- Volume
- 10
- Issue
- 1
- Pages
- 273-273
- ISSN
- 2046-4053 (ISSN-L)
- PMC
- PMC8543959
- OpenAlex
- W3173323133
- Cited by
- 130
- References
- 49
- Citations by year
- 2026: 22 · 2025: 24 · 2024: 33 · 2023: 37 · 2022: 13
- FWCI
- 0.86
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Systematic Reviews (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- University Hospitals Bristol NHS Foundation Trust
- National Institute for Health and Care Research
- National Institutes of Health
- National Institute for Health Research Applied Research Collaboration West
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- AMSTAR 2, ROBIS, overviews of systematic reviews, risk of bias, methodological quality assessment, inter-rater reliability, reporting quality, meta-analysis
- MeSH terms
- Systematic Reviews as Topic, Humans, Reproducibility of Results, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
