The DOI matches its Crossref record, with a small difference in title or year; 21 of the 21 references it cites were found in a registry or an index.
Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach
Abstract
AIMS: With the increase in the number of systematic reviews available, a logical next step to provide decision makers in healthcare with the evidence they require has been the conduct of reviews of existing systematic reviews. Syntheses of existing systematic reviews are referred to by many different names, one of which is an umbrella review. An umbrella review allows the findings of reviews relevant to a review question to be compared and contrasted. An umbrella review's most characteristic feature is that this type of evidence synthesis only considers for inclusion the highest level of evidence, namely other systematic reviews and meta-analyses. A methodology working group was formed by the Joanna Briggs Institute to develop methodological guidance for the conduct of an umbrella review, including diverse types of evidence, both quantitative and qualitative. The aim of this study is to describe the development and guidance for the conduct of an umbrella review. METHODS: Discussion and testing of the elements of methods for the conduct of an umbrella review were held over a 6-month period by members of a methodology working group. The working group comprised six participants who corresponded via teleconference, e-mail and face-to-face meeting during this development period. In October 2013, the methodology was presented in a workshop at the Joanna Briggs Institute Convention. Workshop participants, review authors and methodologists provided further testing, critique and feedback on the proposed methodology. RESULTS: This study describes the methodology and methods developed for the conduct of an umbrella review that includes published systematic reviews and meta-analyses as the analytical unit of the review. Details are provided regarding the essential elements of an umbrella review, including presentation of the review question in a Population, Intervention, Comparator, Outcome format, nuances of the inclusion criteria and search strategy. A critical appraisal tool with 10 questions to help assess risk of bias in systematic reviews and meta-analyses was also developed and tested. Relevant details to extract from included reviews and how to best present the findings of both quantitative and qualitative systematic reviews in a reader friendly format are provided. CONCLUSIONS: Umbrella reviews provide a ready means for decision makers in healthcare to gain a clear understanding of a broad topic area. The umbrella review methodology described here is the first to consider reviews that report other than quantitative evidence derived from randomized controlled trials. The methodology includes an easy to use and informative summary of evidence table to readily provide decision makers with the available, highest level of evidence relevant to the question posed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Because of the overlap in some of the component studies included in the various meta-analyses captured in this umbrella review, a meta-analysis of the results of the meta-analyses was not appropriate, given that it would count results from some component studies several times, violating meta-analysis principles.33 Rather, a narrative synthesis was used, 34”
From Effectiveness of wearable activity trackers to increase physical activity and improve health: a systematic review of systematic reviews and meta-analyses. (2022) · cites this paper as [34]
In the citing paper
“Overviews of reviews summarize existing evidence from systematic reviews without further analyses and are therefore a rapid means to inform guidelines.15”
In the citing paper
“The quality of the included studies was assessed using the Joanna Briggs Institute (JBI) checklist [20], with two authors independently evaluating each study.”
From Impact of social media on cognitive development of children and young adults: a systematic review (2025) · cites this paper as [20]
In the citing paper
“This umbrella review followed the Joanna Briggs Institute’s (JBI) methodology for umbrella reviews [21] and is reported in accordance with the Preferred Reporting Items for Overviews of Reviews (PRIOR) [22] (Supplementary file 1).”
From School-based healthy eating interventions for adolescents aged 10–19 years: an umbrella review (2024) · cites this paper as [21]
In the citing paper
“The included reviews were appraised using the standard JBI critical appraisal tool by two independent reviewers (NS and FN). Seventy per cent of these were checked by another researcher (LB or DP).”
From School-based healthy eating interventions for adolescents aged 10–19 years: an umbrella review (2024) · cites this paper as [21]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.005 · Cited by 19,575
A typology of reviews: an analysis of 14 review types and associated methodologies
Grant, M. J., & Booth, A. (2009) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2009.00848.x · Cited by 10,817
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
ROBIS: A new tool to assess risk of bias in systematic reviews was developed
Whiting, P., Savović, J., Higgins, J. P. T., Caldwell, D. M., Reeves, B. C., Shea, B., Davies, P., Kleijnen, J., & Churchill, R. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.06.005 · Cited by 2,272
Evidence summaries: the evolution of a rapid review approach
Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-10 · Cited by 1,232 · cited 1×
“Conduct of an umbrella review may also offer a means for a rapid review of the evidence to address a broad and high-quality evidence base in relation to a topic.(5) Umbrella reviews are conducted to provide an overall examination of the body of information that is available for a given topic, and to compare and contra…”
Methodology in conducting a systematic review of systematic reviews of healthcare interventions
Smith, V., Devane, D., Begley, C. M., & Clarke, M. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-15 · Cited by 1,227 · cited 2×
“Alongside the presentation of quantitative data, the number of reviews that inform the outcome, number of participants (from included reviews) and statistical heterogeneity should also be reported.(4) An example of the table of findings for an umbrella review titled ‘ Effectiveness of non-pharmacological strategies to…”
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188 · cited 1×
“One recent estimate suggests that 11 systematic reviews are published every day.(1) With the increased number of systematic reviews available, a logical and appropriate next step has been the conduct of reviews of existing systematic reviews, allowing the findings of separate reviews to be compared and contrasted, the…”
Developing the Review Question and Inclusion Criteria
Stern, C., Jordan, Z., & McArthur, A. (2014) · AJN, American Journal of Nursing · doi:10.1097/01.naj.0000445689.67800.86 · Cited by 629 · cited 4×
“The inclusion criteria indicate the basis on which available reviews will be considered for inclusion into the umbrella review.(9) These criteria provide a guide for the reader to clearly understand what is proposed by the umbrella review authors and, more importantly, a guide for the reviewers themselves to base deci…”
InfluentialThe Systematic Review: An Overview
Aromataris, E., & Pearson, A. (2014) · AJN, American Journal of Nursing · doi:10.1097/01.naj.0000444496.24228.2c · Cited by 598 · cited 2×
“As a result, many syntheses published today are of original qualitative research.(7) Similarly, we may find that umbrella reviews inevitably ask questions that direct the reviewer predominantly to existing qualitative reviews.”
Self-citation
Cited by
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636
Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0468-4 · Cited by 1,164
“These include integrative reviews, scoping reviews [47], evidence maps [48], realist syntheses [49], rapid reviews [50], umbrella reviews (systematic reviews of reviews) [51], mixed methods reviews [52], concept analyses [53] and others.”
BackgroundSelf-citationDinu, M., Pagliai, G., Casini, A., & Sofi, F. (2018) · European Journal of Clinical Nutrition · doi:10.1038/ejcn.2017.58 · Cited by 1,016
Coffee consumption and health: umbrella review of meta-analyses of multiple health outcomes
Poole, R., Kennedy, O. J., Roderick, P., Fallowfield, J. A., Hayes, P. C., & Parkes, J. (2017) · BMJ · doi:10.1136/bmj.j5024 · Cited by 835
“11 We conducted a review of coffee consumption and multiple health outcomes by systematically searching for meta-analyses in which coffee consumption was all or part of the exposure of interest or where coffee consumption had been part of a subgroup analysis.”
BackgroundSocial media use and its impact on adolescent mental health: An umbrella review of the evidence.
Valkenburg, P. M., Meier, A., & Beyens, I. (2022) · Current opinion in psychology · doi:10.1016/j.copsyc.2021.08.017 · Cited by 767
Metadata
- Authors
- Aromataris, E., Fernandez, R., Godfrey, C. M., Holly, C., Khalil, H., & Tungpunkom, P.
- Affiliations
- Queen's University (CA)
- University of Wollongong (AU)
- Rutgers, The State University of New Jersey (US)
- Monash University (AU)
- Chiang Mai University (TH)
- Corresponding author
- Aromataris, Edoardo
- ORCID
- Venue
- International Journal of Evidence-Based Healthcare
- Published
- 2015-09
- Volume
- 13
- Issue
- 3
- Pages
- 132-140
- ISSN
- 1744-1595 (ISSN-L)
- 1744-1609
- OpenAlex
- W2364450135
- Cited by
- 2,587
- References
- 20
- Citations by year
- 2026: 434 · 2025: 414 · 2024: 452 · 2023: 385 · 2022: 313 · 2021: 235 · 2020: 159 · 2019: 90 · 2018: 62 · 2017: 31 · 2016: 9
- FWCI
- 5.63
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Evidence-Based Healthcare (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- umbrella review, systematic reviews, meta-analysis, methodological recommendations, critical appraisal tools, risk of bias, search strategies, inclusion criteria, evidence synthesis, healthcare decision-making
- MeSH terms
- Systematic Reviews as Topic, Humans, Research Design, Review Literature as Topic, Meta-Analysis as Topic, Bias, Guidelines as Topic, Evidence-Based Medicine, Empirical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. https://app.cochat.ai/papers/10.1097/xeb.0000000000000055
- The paper
- Aromataris, E., Fernandez, R., Godfrey, C. M., Holly, C., Khalil, H., & Tungpunkom, P. (2015). Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-Based Healthcare. https://doi.org/10.1097/xeb.0000000000000055
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.