A comparison of two assessment tools used in overviews of systematic reviews: ROBIS versus AMSTAR-2
Open accessAbstract
BACKGROUND: AMSTAR-2 is a 16-item assessment tool to check the quality of a systematic review and establish whether the most important elements are reported. ROBIS is another assessment tool which was designed to evaluate the level of bias present within a systematic review. Our objective was to compare, contrast and establish both inter-rater reliability and usability of both tools as part of two overviews of systematic reviews. Strictly speaking, one tool assesses methodological quality (AMSTAR-2) and the other assesses risk of bias (ROBIS), but there is considerable overlap between the tools in terms of the signalling questions. METHODS: for weighted analysis). RESULTS: Thirty-one systematic reviews were included. For AMSTAR-2, the median agreement for all questions was 0.61. Eight of the 16 AMSTAR-2 questions had substantial agreement or higher (> 0.61). For ROBIS, the median agreement for all questions was also 0.61. Eleven of the 24 ROBIS questions had substantial agreement or higher. CONCLUSION: ROBIS is an effective tool for assessing risk of bias in systematic reviews and AMSTAR-2 is an effective tool at assessing quality. The median agreement between raters for both tools was identical (0.61). Reviews that included a meta-analysis were easier to rate with ROBIS; however, further developmental work could improve its use in reviews without a formal synthesis. AMSTAR-2 was more straightforward to use; however, more response options would be beneficial.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The ROBIS tool was not used in this overview because AMSTAR 2 and the ROBIS tool overlap in many signaling questions.
“As AMSTAR 2 and the ROBIS tool 58 overlap in many sig naling questions, 37 the ROBIS tool was not used in this overview.”
From Acceptance and Commitment Therapy for Chronic Pain: An Overview of Systematic Reviews with Meta-Analysis of Randomized Clinical Trials (2024) · cites this paper as [37]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148 · see citing works
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636 · see citing works
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · see citing works
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374 · see citing works
ROBIS: A new tool to assess risk of bias in systematic reviews was developed
Whiting, P., Savović, J., Higgins, J. P. T., Caldwell, D. M., Reeves, B. C., Shea, B., Davies, P., Kleijnen, J., & Churchill, R. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.06.005 · Cited by 2,272 · see citing works
Self-citationShea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.009 · Cited by 1,755 · see citing works
Murad, M. H., Asi, N., Alsawas, M., & Alahdab, F. (2016) · Evidence Based Medicine · doi:10.1136/ebmed-2016-110401 · Cited by 1,388 · see citing works
Gwet, K. L. (2014) · Medical Entomology and Zoology · Cited by 1,095 · see citing works
Acupuncture for treating fibromyalgia
Deare, J. C., Zheng, Z., Xue, C. C., Liu, J., Shang, J., Scott, S., & Littlejohn, G. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007070.pub2 · Cited by 256 · see citing works
Lactobacillus reuteri to Treat Infant Colic: A Meta-analysis
Sung, V., D'Amico, F. J., Cabana, M. D., Chau, K., Koren, G., Savino, F., Szajewska, H., Deshpande, G. C., Dupont, C. M., Indrio, F., et al. (2017) · PEDIATRICS · doi:10.1542/peds.2017-1811 · Cited by 235 · see citing works
A psychometric study found AMSTAR 2 to be a valid and moderately reliable appraisal tool
Lorenz, R. C., Matthias, K., Pieper, D., Wegewitz, U. E., Morche, J., Nocon, M., Rissling, O., Schirm, J., & Jacobs, A. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.05.028 · Cited by 207 · see citing works
Macfarlane, G. J., El‐Metwally, A., Silva, V. D., Ernst, E. K., Dowds, G. L., & Moots, R. J. (2011) · Lara D. Veeken · doi:10.1093/rheumatology/ker119 · Cited by 145 · see citing works
Acupuncture for fibromyalgia--a systematic review of randomized clinical trials
Mayhew, E., & Ernst, E. (2006) · Lara D. Veeken · doi:10.1093/rheumatology/kel406 · Cited by 127 · see citing works
Complementary and alternative medicine in fibromyalgia and related syndromes
Holdcraft, L. C., Assefi, N. P., & Buchwald, D. (2003) · Best Practice & Research Clinical Rheumatology · doi:10.1016/s1521-6942(03)00037-8 · Cited by 125 · see citing works
Urbańska, M., & Szajewska, H. (2014) · European Journal of Pediatrics · doi:10.1007/s00431-014-2328-0 · Cited by 125 · see citing works
Probiotics to Prevent or Treat Excessive Infant Crying
Sung, V., Collett, S., de Gooyer, T., Hiscock, H., Tang, M. L. K., & Wake, M. A. (2013) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/jamapediatrics.2013.2572 · Cited by 119 · see citing works
Pieper, D., Puljak, L., González‐Lorenzo, M., & Minozzi, S. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.12.004 · Cited by 118 · see citing works
Manipulative therapies for infantile colic
Dobson, D., Lucassen, P. L., Miller, J., Vlieger, A. M., Prescott, P., & Lewith, G. T. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004796.pub2 · Cited by 114 · see citing works
Probiotics for infantile colic: a systematic review
Anabrees, J., Indrio, F., Paes, B. A., & AlFaleh, K. M. (2013) · BMC Pediatrics · doi:10.1186/1471-2431-13-186 · Cited by 102 · see citing works
Burda, B. U., Holmer, H. K., & Norris, S. L. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0237-1 · Cited by 93 · see citing works
Infant Colic—What works: A Systematic Review of Interventions for Breast‐fed Infants
Harb, T., Matsuyama, M., David, M., & Hill, R. J. (2016) · Journal of Pediatric Gastroenterology and Nutrition · doi:10.1097/mpg.0000000000001075 · Cited by 92 · see citing works
Baranowsky, J., Klose, P., Musial, F., Häuser, W., Dobos, G., & Langhorst, J. (2009) · Rheumatology International · doi:10.1007/s00296-009-0977-5 · Cited by 88 · see citing works
Nutritional Supplements and Other Complementary Medicines for Infantile Colic: A Systematic Review
Perry, R. E., Hunt, K. J., & Ernst, E. (2011) · PEDIATRICS · doi:10.1542/peds.2010-2098 · Cited by 87 · see citing works
Self-citationXu, M., Wang, J., Wang, N., Sun, F., Wang, L. Q., & Liu, X. (2015) · PLoS ONE · doi:10.1371/journal.pone.0141445 · Cited by 85 · see citing works
Probiotics for the Treatment of Infantile Colic: A Systematic Review
Bird, A. S., Gregory, P. J., Jalloh, M. A., Cochrane, Z. R., & Hein, D. J. (2016) · Journal of Pharmacy Practice · doi:10.1177/0897190016634516 · Cited by 85 · see citing works
Herbal Medicines for Gastrointestinal Disorders in Children and Adolescents: A Systematic Review
Anheyer, D., Frawley, J., Koch, A. K., Lauche, R., Langhorst, J., Dobos, G., & Cramer, H. (2017) · PEDIATRICS · doi:10.1542/peds.2017-0062 · Cited by 81 · see citing works
Efficacy of Lactobacillus reuteri DSM 17938 for infantile colic
Gutiérrez‐Castrellón, P., Indrio, F., Galvis, A. B., Jiménez-Gutiérrez, C., Jiménez–Escobar, I., & López‐Velázquez, G. (2017) · Medicine · doi:10.1097/md.0000000000009375 · Cited by 67 · see citing works
for the ANRS 1215 study group, Boussari, O., Subtil, F., Genolini, C., Bastard, M., Iwaz, J., Fonton, N., Étard, J.-F., & Écochard, R. (2015) · BMC Medical Research Methodology · doi:10.1186/1471-2288-15-10 · Cited by 66 · see citing works
Komplementäre und alternative Verfahren beim Fibromyalgiesyndrom
Langhorst, J., Häuser, W., Bernardy, K., Lucius, H., Settan, M., Winkelmann, A., & Musial, F. (2012) · Der Schmerz · doi:10.1007/s00482-012-1178-9 · Cited by 63 · see citing works
Silva, V. D., El‐Metwally, A., Ernst, E., Lewith, G. T., & Macfarlane, G. J. (2010) · Lara D. Veeken · doi:10.1093/rheumatology/keq025 · Cited by 63 · see citing works
Terhorst, L. A., Schneider, M., Kim, K. H., Goozdich, L. M., & Stilley, C. S. (2011) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2011.05.006 · Cited by 61 · see citing works
Perry, R. E., Leach, V. A., Davies, P. A., Penfold, C. M., Ness, A., & Churchill, R. C. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0487-6 · Cited by 60 · see citing works
Self-citationMartín-Sánchez, E., Torralba, E., Díaz-Domínguez, E., Barriga-Martín, A., & Martín, J. L. R. (2009) · The Open Rheumatology Journal · doi:10.2174/1874312900903010025 · Cited by 59 · see citing works
Efficacy of acupuncture on fibromyalgia syndrome: a Meta-analysis
Bai, Y., Guo, Y., Wang, H., Chen, B., Wang, Z., Liu, Y., Zhao, X., & Li, Y. (2014) · 中医杂志：英文版 · doi:10.1016/s0254-6272(15)30037-6 · Cited by 58 · see citing works
Probiotics for management of infantile colic: a systematic review of randomized controlled trials
Dryl, R., & Szajewska, H. (2017) · Archives of Medical Science · doi:10.5114/aoms.2017.66055 · Cited by 58 · see citing works
Chiropractic treatment for fibromyalgia: a systematic review
Ernst, E. (2009) · Clinical Rheumatology · doi:10.1007/s10067-009-1217-9 · Cited by 52 · see citing works
Carnes, D., Plunkett, A., Ellwood, J., & Miles, C. (2018) · BMJ Open · doi:10.1136/bmjopen-2017-019040 · Cited by 50 · see citing works
Gleberzon, B. J., Arts, J., Mei, A., & McManus, E. L. (2012) · Europe PMC (PubMed Central) · Cited by 48 · see citing works
A systematic review of homoeopathy for the treatment of fibromyalgia
Perry, R. E., Terry, R. H., & Ernst, E. (2010) · Clinical Rheumatology · doi:10.1007/s10067-009-1361-2 · Cited by 37 · see citing works
Self-citationNascimento, S. D. S., DeSantana, J. M., Nampo, F. K., Ribeiro, Ê. A. N., da Silva, D. L., de Araújo‐Júnior, J. X., Almeida, J. R. G. D. S., Bonjardim, L. R., Araújo, A. A. D. S., & Quintans‐Júnior, L. J. (2013) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2013/149468 · Cited by 36 · see citing works
An overview of systematic reviews of complementary and alternative therapies for infantile colic
Perry, R. E., Leach, V. A., Penfold, C. M., & Davies, P. A. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1191-5 · Cited by 36 · see citing works
Self-citationHomeopathy in the treatment of fibromyalgia—A comprehensive literature-review and meta-analysis
Boehm, K., Raak, C., Cramer, H., Lauche, R., & Ostermann, T. (2014) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2014.06.005 · Cited by 35 · see citing works
Acupoint Stimulation for Fibromyalgia: A Systematic Review of Randomized Controlled Trials
Cao, H., Li, X., Han, M., & Liu, J. (2013) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2013/362831 · Cited by 30 · see citing works
Efficacy of acupuncture on fibromyalgia syndrome：a Meta-analysis
Yang, Bai, ̆, Y., -, G., Hong, H., Wang, Bo, B., Chen, Zhan-kui, Yangyang, et al. (2014) · 中医杂志：英文版 · Cited by 21 · see citing works
Skjeie, H., Skonnord, T., Brekke, M., Klovning, A., Fetveit, A., Landgren, K., Hallström, I., & Brurberg, K. G. (2018) · Scandinavian Journal of Primary Health Care · doi:10.1080/02813432.2018.1426146 · Cited by 17 · see citing works
Quels traitements pour les coliques du nourrisson ?
Bruyas-Bertholon, V., Lachaux, A., Dubois, J.-P., Fourneret, P., & Letrilliart, L. (2012) · La Presse Médicale · doi:10.1016/j.lpm.2012.01.015 · Cited by 12 · see citing works
The efficacy of acupuncture in the treatment of fibromyalgia syndrome
Daya, S. (2007) · Cited by 5 · see citing works
Langhorst, J., Häuser, W., Bernardy, K., Lucius, H., Settan, M., Winkelmann, A., & Musial, F. (2012) · Der Schmerz · Cited by 3 · see citing works
Flagged first, then by citations
1 of the 49 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- [17]Terhorst L Schneider M Complementary and alternative medicine in the treatment of pain in fibromyalgia: a systematic review of randomized controlled trials 2012 Portland From International Research Congress on Integrative Medicine and Health 21875523
Cited by
Sorted by citations
Guidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02255-9 · Cited by 190 · see citing works
Schleider, J. L., Zapata, J. P., Rapoport, A., Wescott, A., Ghosh, A., Kaveladze, B., Szkody, E. M., & Ahuvia, I. L. (2025) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-081423-025033 · Cited by 108 · see citing works
Martinez-Calderon, J., García-Muñoz, C., Rufo-Barbero, C., Matias-Soto, J., & Cano-García, F. J. (2024) · The Journal of Pain · doi:10.1016/j.jpain.2023.09.013 · Cited by 79 · see citing works
Umbrella reviews: a methodological guide
Fernandez, R. S., Sharifnia, A. M., & Khalil, H. (2025) · European Journal of Cardiovascular Nursing · doi:10.1093/eurjcn/zvaf012 · Cited by 79 · see citing works
Benefits of Participation in Clinical Trials: An Umbrella Review
Bouzalmate-Hajjaj, A., Massó Guijarro, P., Khan, K. S., Bueno‐Cavanillas, A., & Ibáñez, N. C. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192215368 · Cited by 71 · see citing works
Oral-health-related quality of life in adolescents: umbrella review
Chimbinha, I. G. M., Ferreira, B. N. C., Miranda, G. P., & Guedes, R. S. (2023) · BMC Public Health · doi:10.1186/s12889-023-16241-2 · Cited by 64 · see citing works
Health Effects of Electronic Cigarettes: An Umbrella Review and Methodological Considerations
Travis, N., Knoll, M., Cadham, C. J., Cook, S., Warner, K. E., Fleischer, N. L., Douglas, C. E., Sánchez‐Romero, L. M., Mistry, R., Meza, R., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19159054 · Cited by 55 · see citing works
Critically appraising the cass report: methodological flaws and unsupported claims
Noone, C., Southgate, A., Ashman, A., Quinn, É., Comer, D., Shrewsbury, D., Ashley, F., Hartland, J., Paschedag, J., Gilmore, J., et al. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02581-7 · Cited by 50 · see citing works
Guidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · JBI Evidence Synthesis · doi:10.11124/jbies-23-00139 · Cited by 53 · see citing works
García‐Muñoz, C., González-García, P., Casuso-Holgado, M. J., Martínez-Calderón, J., & Heredia‐Rizo, A. M. (2022) · Complementary Therapies in Medicine · doi:10.1016/j.ctim.2022.102912 · Cited by 47 · see citing works
Risk Factors for Breast Cancer-Related Lymphedema: An Umbrella Review
Shen, A., Qiang, W., Zhang, L., Bian, J., Zhu, F., Zhang, Z., & Lü, Q. (2023) · Annals of Surgical Oncology · doi:10.1245/s10434-023-14277-7 · Cited by 35 · see citing works
Wafaie, K., Rizk, M. Z., Basyouni, M. E., Daniel, B. K., & Mohammed, H. (2023) · European Journal of Orthodontics · doi:10.1093/ejo/cjad004 · Cited by 33 · see citing works
Lunny, C., Kanji, S., Thabet, P., Haidich, A., Bougioukas, K. I., & Pieper, D. (2024) · BMJ Medicine · doi:10.1136/bmjmed-2023-000604 · Cited by 34 · see citing works
Talla, P. K., Allison, P. J., Bussières, A., Rodrigues, A., Bergeron, F., Giraudeau, N., & Emami, E. (2025) · Journal of Medical Internet Research · doi:10.2196/65211 · Cited by 34 · see citing works
Yan, Z., Dong, M., Lin, L., & Wu, D. (2022) · Journal of Psychiatric and Mental Health Nursing · doi:10.1111/jpm.12883 · Cited by 33 · see citing works
Santis, K. K. D., Pieper, D., Lorenz, R. C., Wegewitz, U. E., Siemens, W., & Matthias, K. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-01879-8 · Cited by 32 · see citing works
Martínez-Calderón, J., Ho, E. K., Ferreira, P. H., García‐Muñoz, C., Villar-Alises, O., & Matias-Soto, J. (2023) · Journal of Orthopaedic and Sports Physical Therapy · doi:10.2519/jospt.2023.11833 · Cited by 30 · see citing works
Luo, J., Chen, Z., Liu, D., Li, H., He, S., Zeng, L., Yang, M., Liu, Z., Xiao, X., & Zhang, L. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-01980-y · Cited by 27 · see citing works
Systematic literature reviews over the years
Smela, B., Toumi, M., Swierk, K., Gawlik, K., Clay, E., & Boyer, L. (2023) · Journal of Market Access & Health Policy · doi:10.1080/20016689.2023.2244305 · Cited by 25 · see citing works
Arora, N. K., Donath, L., Owen, P. J., Miller, C. T., Saueressig, T., Winter, F., Hambloch, M., Neason, C., Karner, V., & Belavý, D. L. (2023) · Sports Medicine · doi:10.1007/s40279-023-01966-2 · Cited by 25 · see citing works
Guidance to best tools and practices for systematic reviews1
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · Journal of Pediatric Rehabilitation Medicine · doi:10.3233/prm-230019 · Cited by 29 · see citing works
Maghraby, H., Elsuity, M. A., Adel, N., Magdi, Y., Abdelbadie, A. S., Rashwan, M., Ahmed, O. Y., Elmahdy, M., Khan, K. S., & Fawzy, M. M. (2024) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.17796 · Cited by 22 · see citing works
Guidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · British Journal of Pharmacology · doi:10.1111/bph.16100 · Cited by 27 · see citing works
Burton, R., Sharpe, C., Bhuptani, S., Jecks, M., Henn, C., Pearce‐Smith, N., Knight, S., Regan, M., & Sheron, N. (2024) · BMC Public Health · doi:10.1186/s12889-024-18599-3 · Cited by 23 · see citing works
Guidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · BMC Infectious Diseases · doi:10.1186/s12879-023-08304-x · Cited by 23 · see citing works
Grammatopoulos, T., Hunter, J., Munn, Z., Stone, J., & Barker, T. H. (2023) · JBI Evidence Synthesis · doi:10.11124/jbies-22-00317 · Cited by 19 · see citing works
Talla, P. K., Allison, P. J., Bussières, A., Giraudeau, N., Komarova, S. V., Basiren, Q., Bergeron, F., & Emami, E. (2024) · PLoS ONE · doi:10.1371/journal.pone.0288677 · Cited by 21 · see citing works
Guidance to Best Tools and Practices for Systematic Reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · JBJS Reviews · doi:10.2106/jbjs.rvw.23.00077 · Cited by 20 · see citing works
Umbrella Reviews: Concepts, Methodological Frameworks, and Step‐by‐Step Implementation
Liu, C., Zhou, D.-Y., Xu, W., Pan, H.-F., Wang, X.-Q., Peng, J.-B., Ji, X., Huang, J., & Zhu, Z. (2025) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.70092 · Cited by 17 · see citing works
Pereira, A. Z., Martins, C. C., Campos, J., Faria, S. S., Notaro, S., Peričić, T. P., Costa, L. C. M., Costa, F. O., & Cota, L. (2023) · Journal of Clinical and Experimental Dentistry · doi:10.4317/jced.60197 · Cited by 15 · see citing works
Xie, K., Guan, S.-N., Jing, H., Ji, W., Kong, X., Du, S., Jia, M., & Wang, H. (2023) · Frontiers in Pharmacology · doi:10.3389/fphar.2023.1227436 · Cited by 15 · see citing works
Salem, A., Trabelsi, K., Jahrami, H., Al-Rasheed, M. M., Boukhris, O. F., Puce, L., Bragazzi, N. L., Ammar, A., Glenn, J. M., & Chtourou, H. (2024) · Journal of the American Nutrition Association · doi:10.1080/27697061.2023.2297899 · Cited by 16 · see citing works
Does valerian work for insomnia? An umbrella review of the evidence
Valente, V., Machado, D., Jorge, S., Drake, C. L., & Marques, D. R. (2024) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2024.01.008 · Cited by 14 · see citing works
Meng, T., You, Y., Li, M., Guo, J.-B., Song, X., Ding, J., Xie, X., Li, A.-Q., Li, S.-J., Yin, X., et al. (2024) · Journal of Chinese Integrative Medicine · doi:10.1016/j.joim.2024.03.003 · Cited by 13 · see citing works
Molla‐Esparza, C., Gómez‐Núñez, M. I., & García, F. J. G. (2025) · International Journal of Educational Research Open · doi:10.1016/j.ijedro.2024.100433 · Cited by 20 · see citing works
Probiotics/prebiotics/synbiotics and human neuropsychiatric outcomes: an umbrella review
Du, Q., Li, Q., Liu, C., Liao, G., Li, J., Yang, J., Zhang, Q., Gong, X., & Li, K. (2017) · Beneficial Microbes · doi:10.1163/18762891-bja00035 · Cited by 13 · see citing works
The Effectiveness of Duloxetine for Knee Osteoarthritis: An Overview of Systematic Reviews
Zhou, Q., Chen, J., Yu, W., Yang, K. Y., Guo, T., Niu, P., Ye, Y., & Liu, A. (2022) · Frontiers in Physiology · doi:10.3389/fphys.2022.906597 · Cited by 11 · see citing works
Karakasis, P., Bougioukas, K. I., Pamporis, K., Fragakis, N., & Haidich, A. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1680 · Cited by 11 · see citing works
Weissfeld, A. S., Baselski, V. S., Cornish, N. E., Kraft, C. S., LaRocco, M., McNult, P., Nachamkin, I., Parrott, J. S., Richter, S. S., Rubinstein, M. L., et al. (2024) · Clinical Microbiology Reviews · doi:10.1128/cmr.00065-18 · Cited by 11 · see citing works
Taxonera, C., García‐Brenes, M. A., Olivares, D., López-García, O. N., Zapater, R., & Alba, C. (2024) · United European Gastroenterology Journal · doi:10.1002/ueg2.12673 · Cited by 12 · see citing works
Xiao, B., Fu, L., Yang, Z., & Yu, G. (2025) · Journal of Health Population and Nutrition · doi:10.1186/s41043-025-00816-3 · Cited by 10 · see citing works
Work Exposome and Related Disorders of Firefighters: An Overview of Systematized Reviews
Descatha, A., Dousseau, H., Pitet, S., Magnolini, F., McMillan, N., Mangelsdorf, N., Swan, R., Steve, J.-M., Pourret, D., & Fadel, M. (2025) · Safety and Health at Work · doi:10.1016/j.shaw.2025.04.003 · Cited by 10 · see citing works
Moore, S. F., Price, S., Konya, J., Blummers, S., Walter, F. M., Neal, R. D., & Abel, G. (2025) · EClinicalMedicine · doi:10.1016/j.eclinm.2025.103297 · Cited by 10 · see citing works
Heredia‐Rizo, A. M., Martínez-Calderón, J., Piña-Pozo, F., González-García, P., & García‐Muñoz, C. (2023) · Disability and Rehabilitation · doi:10.1080/09638288.2023.2233899 · Cited by 10 · see citing works
Riley, S. P., Swanson, B. T., Shaffer, S. M., Somma, M. J., Flowers, D. W., & Sawyer, S. F. (2022) · Journal of Manual & Manipulative Therapy · doi:10.1080/10669817.2022.2110419 · Cited by 10 · see citing works
Lunny, C., Jain, N., Nazari, T., Kosaner-Kließ, M., Santos, L. H. C. C., Goodman, I., Osman, A. A. M., Berrone, S., Dadam, M. N., Brenna, C. T. A., et al. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10032 · Cited by 11 · see citing works
Bauer, B. S., Azcoaga-Lorenzo, A., Agrawal, U., Fagbamigbe, A. F., & McCowan, C. (2022) · PLoS ONE · doi:10.1371/journal.pone.0268070 · Cited by 12 · see citing works
Alkhalaf, A., Aljaroudi, E., Al-Hulmai, M., Gaffar, B. O., & Almas, K. (2023) · Cureus · doi:10.7759/cureus.37631 · Cited by 8 · see citing works
Value of preclinical systematic reviews and meta-analyses in pediatric research
Romantsik, O., Bank, M., Menon, J. M. L., Malhotra, A. K., & Bruschettini, M. (2024) · Pediatric Research · doi:10.1038/s41390-024-03197-1 · Cited by 8 · see citing works
Ngcongo, M. T., & Tekere, M. (2023) · Journal of Water Sanitation and Hygiene for Development · doi:10.2166/washdev.2023.028 · Cited by 9 · see citing works
Shalbaf, N., Sadeghi, S., Homaee, S., & Saberian, F. (2025) · Metabolism Open · doi:10.1016/j.metop.2025.100419 · Cited by 7 · see citing works
Tools to assess the risk of bias of evidence syntheses: a scoping review protocol
Hasanoff, S., Pollock, D., Barker, T. H., & Munn, Z. (2023) · JBI Evidence Synthesis · doi:10.11124/jbies-23-00316 · Cited by 9 · see citing works
An umbrella review of the diagnostic value of next-generation sequencing in infectious diseases
Cao, H., Chen, Y., Ge, L., Kwong, J. S. W., Lai, H., Hu, F., Zhang, R., Zhao, H., Hu, L., He, R., et al. (2024) · International Journal of Clinical Pharmacy · doi:10.1007/s11096-024-01704-2 · Cited by 9 · see citing works
Leite, K. L. D. F., Rodrigues, G. F., Chevitarese, A. B., Magno, M. B., Marañón‐Vásquez, G. A., Pintor, A. V. B., & Maia, L. C. (2024) · Journal of Evidence Based Dental Practice · doi:10.1016/j.jebdp.2024.102010 · Cited by 7 · see citing works
Matterne, U., Theurich, M. A., Proebstl, S., Pieper, D., Wang, J., Xu, A., & Apfelbacher, C. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-01899-4 · Cited by 7 · see citing works
The public consultation is open: Insights from urban green infrastructure planning in Romania
Mitincu, C.-G., Grădinaru, S. R., Iojă, C., Hartel, T., van Lierop, M., & Hossu, C.-A. (2023) · Urban forestry & urban greening · doi:10.1016/j.ufug.2023.127985 · Cited by 7 · see citing works
A mapping review of systematic reviews in orthodontics: a five-year analysis
Ladewig, V. D. M., Stefani, C. M., Canto, G. D. L., Pandis, N., & Flores‐Mir, C. (2025) · European Journal of Orthodontics · doi:10.1093/ejo/cjaf040 · Cited by 6 · see citing works
Martínez-Calderón, J., Villar-Alises, O., García‐Muñoz, C., Pineda-Escobar, S., & Matias-Soto, J. (2024) · Disability and Rehabilitation · doi:10.1080/09638288.2024.2321318 · Cited by 6 · see citing works
Risk factors for acquired weakness in intensive care unit patients: An umbrella review.
Zhang, Y.-M., Hu, Q.-L., Zhou, M., Wang, Y., Yang, J.-R., Jin, X., Zhang, X., & Ma, F. (2025) · Intensive & Critical Care Nursing · doi:10.1016/j.iccn.2025.103940 · Cited by 5 · see citing works
Zhou, Z., Zhang, J., Fan, C., Wei, Z., Wang, Q., & Liu, C. (2025) · Frontiers in Nutrition · doi:10.3389/fnut.2025.1662409 · Cited by 16 · see citing works
Guidance to best tools and practices for systematic reviews
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · Acta Anaesthesiologica Scandinavica · doi:10.1111/aas.14295 · Cited by 10 · see citing works
Aiyar, A., Pedersen, T. K., Resnick, C. M., Nørholt, S. E., Verna, C., & Stoustrup, P. B. (2023) · Orthodontics and Craniofacial Research · doi:10.1111/ocr.12729 · Cited by 5 · see citing works
Xie, K., Jing, H., Guan, S.-N., Kong, X., Ji, W., Du, C., Jia, M., & Wang, H. (2024) · European journal of medical research · doi:10.1186/s40001-024-01837-0 · Cited by 6 · see citing works
Methodological quality of umbrella reviews in endodontics: A cross‐sectional study
Nagendrababu, V., Gopinath, V. K., Narasimhan, S., Acharya, A. B., Dummer, P. M. H., & Faggion, C. M. (2024) · International Endodontic Journal · doi:10.1111/iej.14114 · Cited by 5 · see citing works
Fiddes, R., & McCaffrey, N. (2024) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000007187 · Cited by 5 · see citing works
Improving systematic reviews: guidance on guidance and other options and challenges
Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.05.008 · Cited by 5 · see citing works
Dulisse, B. C., Denis, J., & Connealy, N. T. (2026) · Journal of Money Laundering Control · doi:10.1108/jmlc-12-2025-0241 · Cited by 6 · see citing works
Xuan, B., Sun, D., Song, Y., Li, F., Zhou, Z., Wang, Z., Zhu, J., Chen, K., Zhao, R., Wang, D., et al. (2026) · Mechanobiology in Medicine · doi:10.1016/j.mbm.2026.100206 · Cited by 4 · see citing works
Kurniawan, F., Agustian, H., Dermawan, D., Nurdin, R., Ahmadi, N., & Dinaryanto, O. (2025) · Applied Sciences · doi:10.3390/app151910761 · Cited by 4 · see citing works
Þorsteinsdóttir, F., Walker, K. C., Runstedt, S. E., Jacobsen, R., Maslova, E., Backer, V., Heitmann, B. L., & Händel, M. N. (2022) · Clinical Nutrition · doi:10.1016/j.clnu.2022.06.040 · Cited by 4 · see citing works
Lee, S. W. H. (2023) · Cochrane Evidence Synthesis and Methods · doi:10.1002/cesm.12023 · Cited by 4 · see citing works
Trozić, I., Fischer, L., Deckert, S., & Gmeinwieser, K. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-080542 · Cited by 3 · see citing works
Liu, A., Yu, W., Chen, J., Guo, T., Niu, P., Feng, H., & Jia, Y. (2022) · Stem Cells and Development · doi:10.1089/scd.2022.0060 · Cited by 3 · see citing works
Mo, Q., Xu, S., Hu, F., & Zheng, X. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-086643 · Cited by 4 · see citing works
Mondragon, K. M., Tan‐Lim, C. S. C., Velasco, R., Cordero, C. P., Strebel, H. M. J. S., Palileo‐Villanueva, L. M., & Mantaring, J. B. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.112056 · Cited by 2 · see citing works
AMSTAR-PF: a critical appraisal tool for systematic reviews of prognostic factor studies
Henry, M. L., O'Connell, N. E., Riley, R. D., Moons, K. G. M., Shea, B., Hooft, L., Damen, J. A., Skoetz, N., Wallwork, S. B., & Moseley, G. L. (2025) · BMJ · doi:10.1136/bmj-2025-085718 · Cited by 2 · see citing works
Quality of systematic reviews and meta‐analyses in dermatology
Muthiah, A., Lee, L. K., Koh, J., Liu, A., & Tan, A. (2024) · Cochrane Evidence Synthesis and Methods · doi:10.1002/cesm.12056 · Cited by 2 · see citing works
GADELHA, G. C. A., & Júnior, A. M. D. C. (2024) · Revista do Colégio Brasileiro de Cirurgiões · doi:10.1590/0100-6991e-20243757-en · Cited by 2 · see citing works
Matthias, K., Honekamp, I., & Santis, K. K. D. (2022) · JMIR Research Protocols · doi:10.2196/40538 · Cited by 2 · see citing works
The Evidence for Screening Older Adults for Nutrition Risk in Primary Care: An Umbrella Review
Mills, C. M., & Trinca, V. (2023) · Canadian Journal of Dietetic Practice and Research · doi:10.3148/cjdpr-2022-043 · Cited by 2 · see citing works
PASS: A checklist for assessing the quality of systematic reviews of prevalence studies
Walker, J., Hobbs, H., Magill, N., van Niekerk, M., & Sharpe, M. (2022) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2022.01.003 · Cited by 2 · see citing works
Talla, P. K., Allison, P. J., Bussières, A., Rodrigues, A., Bergeron, F., Giraudeau, N., & Emami, E. (2024) · doi:10.2196/preprints.65211 · Cited by 2 · see citing works
Michael, H., O'Connell, N. E., Riley, R. D., Moons, K. G. M., Shea, B., Hooft, L., Wallwork, S. B., Damen, J. A., Skoetz, N., Appiah, R. P., et al. (2025) · medRxiv · doi:10.1101/2025.04.10.25325555 · Cited by 2 · see citing works
Suicidal Ideation in South Korean: A Systematic Review
Been, S., & Byeon, H. (2023) · Iranian Journal of Public Health · doi:10.18502/ijph.v52i6.12951 · Cited by 1 · see citing works
Msuya, M. A., & Srivastava, G. (2024) · Cosmoderma · doi:10.25259/csdm_265_2023 · Cited by 1 · see citing works
Overview of systematic reviews of diagnostic test accuracy studies in physical therapy
Kaizik, M. A., Hancock, M. J., Choi, J., & Herbert, R. (2024) · Physical Therapy Reviews · doi:10.1080/10833196.2024.2402602 · Cited by 1 · see citing works
Climate change-related exposures, low birthweight, and preterm birth: Overview of reviews protocol.
Ludwig-Borycz, E. F., Rojas, A. I., Sunuwar, D., Aaron, B., Jayakumar, G., Moyer, C. A., Waljee, A., Baylin, A., & Agrawal, A. (2025) · Environment International · doi:10.1016/j.envint.2025.109704 · Cited by 1 · see citing works
Jiang, Y., Zhong, C. C., Wang, B. H., Xu, S., Ho, F. F., Kwong, M. H., Ho, L. T. F., Zhou, J., Lam, K. C., Liu, J.-P., et al. (2025) · Journal of Chinese Integrative Medicine · doi:10.1016/j.joim.2025.07.005 · Cited by 1 · see citing works
Acupuncture for smoking cessation: an overview of systematic reviews
Fan, S., Wang, J., Wang, X., Zhang, H., Li, H., Wang, J., Zhang, W., & Wang, J. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1677231 · Cited by 1 · see citing works
Cho, M., & Kim, M. Y. (2026) · Journal of Korean Biological Nursing Science · doi:10.7586/jkbns.25.078 · Cited by 1 · see citing works
Luijk, A., Bricca, A., Mortensen, S. R., Holm, P., Zanger, G., Simonÿ, C., Skou, S., Lund, T., & Tang, L. (2026) · British Journal of Sports Medicine · doi:10.1136/bjsports-2025-110444
Wang, W.-L., Ge, L., Xiong, Y.-Q., Chen, J., Chen, C., Ren, Y., Thabane, L., Sun, X., & Tan, J. (2026) · Systematic Reviews · doi:10.1186/s13643-026-03235-5 · Cited by 1 · see citing works
Khan, M. N., Jha, A., Kumar, A., Mahajan, R., & Reddy, P. (2026) · Food and Humanity · doi:10.1016/j.foohum.2026.101416 · Cited by 1 · see citing works
Zergaw, M., Elgendy, M., Billey, A., Saleem, A., Zeeshan, B., Dissanayake, G., & Nassar, S. T. (2024) · Cureus · doi:10.7759/cureus.67913 · Cited by 2 · see citing works
Clinical risk factors for pancreatic cancer: protocol for an umbrella review
Moore, S. F., Price, S., Dongo, G., Walter, F. M., Neal, R. D., & Abel, G. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-089008 · Cited by 1 · see citing works
Critical Appraisal Tools to Aid Pharmacists in Evidence-Based Practice: A Narrative Review
Blanc, A., Ho, V., & Cameron, J. D. (2023) · The Canadian Journal of Hospital Pharmacy · doi:10.4212/cjhp.3281 · Cited by 1 · see citing works
Muqsith, M. F., Priyadi, Y., & Astuti, W. (2023) · International Conference on Electronic and Electrical Engineering and Intelligent System (ICE3IS) · doi:10.1109/ice3is59323.2023.10335296 · Cited by 1 · see citing works
Sachse, T., Mathes, T., Dorando, E., Heß, S., Thürmann, P., Schmiedl, S., Kanji, S., Lunny, C., Thabet, P., & Pieper, D. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4052254 · Cited by 1 · see citing works
Xie, K., Jing, H., Guan, S.-N., Kong, X., Ji, W., Du, S., Jia, M., & Wang, H. (2023) · Research Square · doi:10.21203/rs.3.rs-2949787/v1 · Cited by 1 · see citing works
Quality of Systematic Reviews/Meta-Analyses in Coronavirus Disease 2019 and Venous Thromboembolism
Alkhamesi, S., Almohammadi, A. T., Alahwal, H. M., Barefah, A., Bahashwan, S., & Radhwi, O. (2022) · Journal of Applied Hematology · doi:10.4103/joah.joah_34_22
- Authors
- Perry, R., Whitmarsh, A., Leach, V., & Davies, P.
- Affiliations
- National Institute for Health and Care Research (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Corresponding author
- Perry, R.
- Venue
- Systematic Reviews
- Published
- 2021-10-25
- Volume
- 10
- Issue
- 1
- Pages
- 273-273
- ISSN
- 2046-4053 (ISSN-L)
- PMC
- PMC8543959
- OpenAlex
- W3173323133
- Cited by
- 130
- References
- 49
- Citations by year
- 2026: 22 · 2025: 24 · 2024: 33 · 2023: 37 · 2022: 13 · 2017: 1
- FWCI
- 0.86
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Systematic Reviews (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- University Hospitals Bristol NHS Foundation Trust
- National Institute for Health and Care Research
- National Institutes of Health
- National Institute for Health Research Applied Research Collaboration West
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- AMSTAR 2, ROBIS, overviews of systematic reviews, risk of bias, methodological quality assessment, inter-rater reliability, reporting quality, meta-analysis
- MeSH terms
- Systematic Reviews as Topic, Humans, Reproducibility of Results, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
