A descriptive analysis of overviews of reviews published between 2000 and 2011.
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Overviews of systematic reviews compile data from multiple systematic reviews (SRs) and are a new method of evidence synthesis. OBJECTIVES: To describe the methodological approaches in overviews of interventions. DESIGN: Descriptive study. METHODS: We searched 4 databases from 2000 to July 2011; we handsearched Evidence-based Child Health: A Cochrane Review Journal. We defined an overview as a study that: stated a clear objective; examined an intervention; used explicit methods to identify SRs; collected and synthesized outcome data from the SRs; and intended to include only SRs. We did not restrict inclusion by population characteristics (e.g., adult or children only). Two researchers independently screened studies and applied eligibility criteria. One researcher extracted data with verification by a second. We conducted a descriptive analysis. RESULTS: From 2,245 citations, 75 overviews were included. The number of overviews increased from 1 in 2000 to 14 in 2010. The interventions were pharmacological (n = 20, 26.7%), non-pharmacological (n = 26, 34.7%), or both (n = 29, 38.7%). Inclusion criteria were clearly stated in 65 overviews. Thirty-three (44%) overviews searched at least 2 databases. The majority reported the years and databases searched (n = 46, 61%), and provided key words (n = 58, 77%). Thirty-nine (52%) overviews included Cochrane SRs only. Two reviewers independently screened and completed full text review in 29 overviews (39%). Methods of data extraction were reported in 45 (60%). Information on quality of individual studies was extracted from the original SRs in 27 (36%) overviews. Quality assessment of the SRs was performed in 28 (37%) overviews; at least 9 different tools were used. Quality of the body of evidence was assessed in 13 (17%) overviews. Most overviews provided a narrative or descriptive analysis of the included SRs. One overview conducted indirect analyses and the other conducted mixed treatment comparisons. Publication bias was discussed in 18 (24%) overviews. CONCLUSIONS: This study shows considerable variation in the methods used for overviews. There is a need for methodological rigor and consistency in overviews, as well as empirical evidence to support the methods employed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“There remains no standardized method to evaluate the certainty of evidence in overviews of reviews (15,22), so a summary percentage scores were calculated to demonstrate the quality of evidence contributing to each conclusion.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
3 retracted
Ventre, K., & Randolph, A. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000181.pub3 · Cited by 312
RetractedSurfactant therapy for bronchiolitis in critically ill infants
Ventre, K. M., Haroon, M., & Davison, C. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005150.pub2 · Cited by 72
RetractedImmunoglobulin treatment for respiratory syncytial virus infection
Fuller, H. L., & Mar, C. D. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004883.pub2 · Cited by 54
RetractedLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · cited 2×
“In Appendix S1, we list the mandatory standards developed by The Cochrane Collaboration [13] and discuss their applicability to overviews, and we present the PRISMA reporting guidelines for SRs and meta-analyses [15] and their applicability to overviews.”
BackgroundMethodDermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables
Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., Norris, S., Falck-Ytter, Y., Glasziou, P., & deBeer, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.026 · Cited by 10,681 · cited 1×
“[20] Few overviews in our sample described the quality of evidence, likely because this process has been implemented more recently.”
BackgroundMoher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883 · cited 1×
“[14,15] Similar guidelines need to be established and followed for overviews.”
BackgroundDevelopment of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
(2008) · Cited by 3,790
Norderyd, O. M., Koch, G., Papias, A., Köhler, A. A., Helkimo, A. N., Brahm, C., Lindmark, U., Lindfors, N., Mattsson, A., Rolander, B., et al. (2015) · PubMed · Cited by 3,584
Cited by
Aromataris, E., Fernandez, R., Godfrey, C. M., Holly, C., Khalil, H., & Tungpunkom, P. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000055 · Cited by 2,587
Synthesizing information systems knowledge: A typology of literature reviews
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015) · Information & Management · doi:10.1016/j.im.2014.08.008 · Cited by 1,987
“By pooling statistically significant findings with non-significant findings from all relevant studies included in the review, meta-analyses are able to calculate more precise estimates of the effects”
BackgroundSystematic review finds overlapping reviews were not mentioned in every other overview
Pieper, D., Antoine, S.-L., Mathes, T., Neugebauer, E. A. M., & Eikermann, M. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.11.007 · Cited by 1,318
Gates, M., Gates, A., Pieper, D., Fernandes, R. M., Tricco, A. C., Moher, D., Brennan, S. E., Li, T., Pollock, M., Lunny, C., et al. (2022) · BMJ · doi:10.1136/bmj-2022-070849 · Cited by 856
Self-citationSquires, J. E., Sullivan, K., Eccles, M. P., Worswick, J., & Grimshaw, J. M. (2014) · Implementation Science · doi:10.1186/s13012-014-0152-6 · Cited by 368
Hunt, H., Pollock, A., Campbell, P., Estcourt, L., & Brunton, G. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0695-8 · Cited by 314
“Overviews have evolved to address a growing need to filter the information overload, improve access to targeted information and inform healthcare decision-making [2, 3].”
BackgroundInfluentialO’Donoghue, G., Kennedy, A., Puggina, A., Aleksovska, K., Buck, C., Burns, C., Cardon, G., Carlin, A., Ciarapica, D., Colotto, M., et al. (2018) · PLOS ONE · doi:10.1371/journal.pone.0190737 · Cited by 310
“The aim of an umbrella review is not to repeat the searches, assessment of study eligibility, assessment of risk of bias or meta-analyses from the included reviews, but rather to provide an overall picture of findings for particular questions or phenomenon [14].”
Graphical Representation of Overlap for OVErviews: GROOVE tool
Bracchiglione, J., Meza, N., Bangdiwala, S. I., Niño de Guzmán, E., Urrútia, G., Bonfill, X., & Madrid, E. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1557 · Cited by 299
“4 An overview can be defined as a SR of SRs instead of primary studies, and constitutes a relatively new approach for broad evidence synthesis.”
BackgroundWhat Are the Effects of Teaching Evidence-Based Health Care (EBHC)? Overview of Systematic Reviews
Young, T., Rohwer, A., Volmink, J., & Clarke, M. (2014) · PLoS ONE · doi:10.1371/journal.pone.0086706 · Cited by 254
Meta‐analysis of meta‐analyses of anterior cruciate ligament injury reduction training programs
Webster, K. E., & Hewett, T. E. (2018) · Journal of Orthopaedic Research · doi:10.1002/jor.24043 · Cited by 253
“Therefore, overviews, or reviews of reviews, have recently gained interest as a new type of synthesis.(22,23) These compare and combine the findings of several reviews and provide an overall summary in one accessible source.”
BackgroundLunny, C., Pieper, D., Thabet, P., & Kanji, S. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01269-y · Cited by 252
Bougioukas, K. I., Liakos, A., Tsapas, A., Ntzani, E., & Haidich, A.-B. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.002 · Cited by 251
“The search was restricted to English language and humans using a modified algorithm based on a previous work [10] (Appendix 1-S2).”
MethodFernandes, R. M., Cary, M., Duarte, G., Jesus, G., Alarcão, J., Torre, C., Costa, S., Costa, J., & Carneiro, A. V. (2017) · BMC Public Health · doi:10.1186/s12889-017-4210-2 · Cited by 236
Jaschinski, T., Mosch, C., Eikermann, M., & Neugebauer, E. A. (2015) · BMC Gastroenterology · doi:10.1186/s12876-015-0277-3 · Cited by 233
Pollock, M., Fernandes, R. M., Pieper, D., Tricco, A. C., Gates, M., Gates, A., & Hartling, L. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1252-9 · Cited by 228
Self-citationSilver diamine fluoride for managing carious lesions: an umbrella review
Seifo, N., Cassie, H., Radford, J. R., & Innes, N. P. T. (2019) · BMC Oral Health · doi:10.1186/s12903-019-0830-5 · Cited by 217
Umbrella reviews: what they are and why we need them
Papatheodorou, S. (2019) · European Journal of Epidemiology · doi:10.1007/s10654-019-00505-6 · Cited by 201
Pollock, M., Fernandes, R. M., Becker, L. A., Featherstone, R., & Hartling, L. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0367-5 · Cited by 195
Self-citationLunny, C., Brennan, S. E., McDonald, S., & McKenzie, J. E. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0617-1 · Cited by 184
“5 Define the objectives using PICO elements (or equivalent) to develop an answerable question Baker 2014 [29]; Becker 2008 [1]; Cooper 2012 [32]; Hartling 2012 [35]; JBI 2015 [40, 41]; Li 2012 [44]; Ryan 2009 [53, 54]; Smith 2011 [57]; Robinson 2015 [48–52]; Thomson 2010 [58] CMIMG Comparing Multiple Interventions Met…”
BackgroundMethodInfluentialMatwiejczyk, L., Mehta, K., Scott, J., Tonkin, E., & Coveney, J. (2018) · Nutrients · doi:10.3390/nu10030293 · Cited by 177
“Umbrella reviews are becoming relatively common [31,32] as a means of providing an overall examination of a broad range of topics within a similar area of interest [33,34].”
BackgroundNyadanu, S. D., Dunne, J., Tessema, G. A., Mullins, B., Kumi-Boateng, B., Lee Bell, M., Duko, B., & Pereira, G. (2022) · Environmental Pollution · doi:10.1016/j.envpol.2022.119465 · Cited by 162
Systematic review found AMSTAR, but not R(evised)-AMSTAR, to have good measurement properties
Pieper, D., Buechter, R. B., Li, L., Prediger, B., & Eikermann, M. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.12.009 · Cited by 161
Introduction to Umbrella Reviews as a Useful Evidence-Based Practice
Choi, G. J., & Kang, H. (2023) · Journal of Lipid and Atherosclerosis · doi:10.12997/jla.2023.12.1.3 · Cited by 164
Overviews in Education Research: A Systematic Review and Analysis
Polanin, J. R., Maynard, B. R., & Dell, N. A. (2017) · Review of Educational Research · doi:10.3102/0034654316631117 · Cited by 131
Lunny, C., Brennan, S. E., McDonald, S., & McKenzie, J. E. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0784-8 · Cited by 139
“3 (Re)-assess RoB of some or all primary studies CMIMG 2012 [47]; Cooper 2012 [6]; Hartling 2012 [53]; Jadad 1997 [29]; Moja 2012 [63]; Thomson 2010 [26] ▪ When two different tools are used, then assess the primary studies using one tool ▪ When two different tools are used (e.”
InfluentialAcupuncture for Low Back Pain: An Overview of Systematic Reviews
Liu, L., Skinner, M. A., McDonough, S., Mabire, L., & Baxter, D. G. (2015) · Evidence-based Complementary and Alternative Medicine · doi:10.1155/2015/328196 · Cited by 154
Ballard, M., & Montgomery, P. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1229 · Cited by 136
“…Oxman, 2011; Caird et al., 2015; Cochrane Comparing Multiple Interventions Methods Group, 2013; Conn and Coon Sells, 2014; Cooper and Koenka, 2012; Hartling et al., 2012; Hartling et al., 2014; Ioannidis, 2009; Li et al., 2012; Pieper et al., 2014a; Pieper et al., 2014c; Pieper et al., 2012; Piso…”
BackgroundResultInfluentialBucci, R., D’Antò, V., Rongo, R., Valletta, R., Martina, R., & Michelotti, A. (2016) · Journal of Oral Rehabilitation · doi:10.1111/joor.12393 · Cited by 152
Overviews of systematic reviews: great promise, greater challenge
McKenzie, J. E., & Brennan, S. E. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0582-8 · Cited by 134
Selecting and implementing overview methods: implications from five exemplar overviews
Pollock, A., Campbell, P., Brunton, G., Hunt, H., & Estcourt, L. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0534-3 · Cited by 134
Pieper, D., Puljak, L., González‐Lorenzo, M., & Minozzi, S. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.12.004 · Cited by 130
“decade, A Measurement Tool to Assess Systematic Reviews (AMSTAR) [1] has become the most common tool to assess the methodological quality of SRs [2,3].”
MethodTricco, A. C., Zarin, W., Ghassemi, M., Nincic, V., Lillie, E., Page, M. J., Shamseer, L., Antony, J., Rios, P., Hwee, J., et al. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.014 · Cited by 122
Self-citationBougioukas, K. I., Vounzoulaki, E., Mantsiou, C. D., Savvides, E. D., Karakosta, C., Diakonidis, T., Tsapas, A., & Haidich, A.-B. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.12.004 · Cited by 117
Gates, M., Gates, A., Guitard, S., Pollock, M., & Hartling, L. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01509-0 · Cited by 115
Self-citationPollock, M., Fernandes, R. M., & Hartling, L. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0325-5 · Cited by 107
“In addition, we found that the AMSTAR assessments obtained for the SRs in our sample of seven overviews fell within the range of scores observed in a broader sample of overviews (all relevant overviews identified by [4] and [5] and contained within issue 12, 2016 of the CDSR).”
MethodSelf-citationCURRIER, B. S., D’SOUZA, A. C., SINGH, M. A. F., LOWISZ, C. V., RAWSON, E. S., SCHOENFELD, B. J., SMITH-RYAN, A. E., STEEN, J. P., THOMAS, G. A., TRIPLETT, N. T., et al. (2026) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000003897 · Cited by 105
“…a systematic collection and assessment of available evidence that provides a comprehensive, user-friendly summary of a research topic, enabling practitioners and professionals to make evidence-based decisions without assimilating the results of numerous systematic reviews and meta-analyses (14,15).”
Martínez-Calderón, J., Flores-Cortés, M., Morales‐Asencio, J. M., & Luque-Suárez, A. (2020) · Clinical Journal of Pain · doi:10.1097/ajp.0000000000000838 · Cited by 113
Pollock, A., Campbell, P., Deery, R., Fleming, M. P., Rankin, J., Sloan, G., & Cheyne, H. (2017) · Journal of Advanced Nursing · doi:10.1111/jan.13253 · Cited by 104
“Had the included reviews presented meta-analyses exploring the effectiveness of clinical supervision on any of our pre-stated outcomes of interest, we planned to summarise these pooled data within tables using recommended methods (Hartling et al. 2012, Smith et al. 2011).”
MethodImpact of financial inclusion in low- and middle-income countries: A systematic review of reviews
Duvendack, M., & Mader, P. (2019) · Campbell Systematic Reviews · doi:10.4073/csr.2019.2 · Cited by 131
Database combinations to retrieve systematic reviews in overviews of reviews: a methodological study
Goossen, K., Hess, S., Lunny, C., & Pieper, D. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-00983-3 · Cited by 85
Nonallergic comorbidities of atopic eczema: an overview of systematic reviews
Deckert, S., Kopkow, C., & Schmitt, J. (2014) · Allergy · doi:10.1111/all.12246 · Cited by 97
Methodology for jbi umbrella reviews
Aromataris, E., Fernandez, R. S., Godfrey, C. M., Holly, C. M., Khalil, H., & Tungpunkom, P. (2014) · Research Online (University of Wollongong) · Cited by 95
What Influences the Implementation of Shared Decision Making: An Umbrella Review
Alsulamy, N. A., Lee, A. C., Thokala, P., & Alessa, T. (2020) · Patient Education and Counseling · doi:10.1016/j.pec.2020.08.009 · Cited by 100
Treating child and adolescent anxiety effectively: Overview of systematic reviews
BENNETT, K. J., Manassis, K., Duda, S. J., Bagnell, A. L., Bernstein, G. A., Garland, E., Miller, L. D., Newton, A. S., Thabane, L., & Wilansky, P. (2016) · Clinical Psychology Review · doi:10.1016/j.cpr.2016.09.006 · Cited by 102
Pussegoda, K., Turner, L., Garritty, C., Mayhew, A., Skidmore, B., Stevens, A., Boutron, I., Sarkis-Onofre, R., Bjerre, L. M., Hróbjartsson, A., et al. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0507-6 · Cited by 75
Umbrella Reviews: What They Are and Why We Need Them
Papatheodorou, S., & Εvangelou, Ε. (2021) · Methods in molecular biology · doi:10.1007/978-1-0716-1566-9_8 · Cited by 86
How is AMSTAR applied by authors – a call for better reporting
Pieper, D., Könsgen, N., Breuing, J., Ge, L., & Wegewitz, U. E. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0520-z · Cited by 85
“Nevertheless, some of our sample characteristics are comparable to those of other studies in terms of quality/risk of bias assessment [1] and number of included studies per review/overview [2, 3, 16].”
Resuming the discussion of AMSTAR: What can (should) be made better?
Wegewitz, U. E., Weikert, B., Fishta, A., Jacobs, A., & Pieper, D. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0183-6 · Cited by 83
Effectiveness and Safety of Acupuncture for Migraine: An Overview of Systematic Reviews
Li, Y., Xiao, X., Zhong, D., Luo, L.-J., Yang, H., Zhou, J., He, M., Shi, L., Li, J., Zheng, H., et al. (2020) · Pain Research and Management · doi:10.1155/2020/3825617 · Cited by 83
The risk of bias in systematic reviews tool showed fair reliability and good construct validity
Bühn, S., Mathes, T., Prengel, P., Wegewitz, U. E., Ostermann, T., Robens, S., & Pieper, D. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.06.019 · Cited by 82
Gates, M., Gates, A., Duarte, G. S., Cary, M., Becker, M., Prediger, B., Vandermeer, B., Fernandes, R. M., Pieper, D., & Hartling, L. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.04.026 · Cited by 79
Self-citationStorman, M., Storman, D., Jasinska, K. W., Swierz, M. J., & Bala, M. M. (2020) · Obesity Reviews · doi:10.1111/obr.12994 · Cited by 70
Lunny, C., Brennan, S. E., Reid, J., McDonald, S., & McKenzie, J. E. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.09.025 · Cited by 71
Hartling, L., Vandermeer, B., & Fernandes, R. M. (2014) · Evidence-Based Child Health: A Cochrane Review Journal · doi:10.1002/ebch.1968 · Cited by 67
Self-citationGates, A., Gates, M., Duarte, G. S., Cary, M., Becker, M., Prediger, B., Vandermeer, B., Fernandes, R. M., Pieper, D., & Hartling, L. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0746-1 · Cited by 70
Self-citation50 years of social marketing: seeding solutions for the future
Dietrich, T., Hurley, E., Carins, J., Kassirer, J., Rundle-Thiele, S. R., Palmatier, R. W., Merritt, R., Weaven, S. K., & Lee, N. (2022) · European Journal of Marketing · doi:10.1108/ejm-06-2021-0447 · Cited by 67
Can AMSTAR also be applied to systematic reviews of non-randomized studies?
Pieper, D., Mathes, T., & Eikermann, M. (2014) · BMC Research Notes · doi:10.1186/1756-0500-7-609 · Cited by 62
“reviews of reviews) found both instruments to be used frequently in this context [8,9].”
BackgroundPieper, D., Lorenz, R. C., Rombey, T., Jacobs, A., Rissling, O., Freitag, S., & Matthias, K. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.09.046 · Cited by 62
Nambiema, A., Fouquet, J.-L., Guilloteau, J., & Descatha, A. (2021) · Archives des maladies professionnelles et de médecine du travail/Archives des maladies professionnelles et de l'environnement · doi:10.1016/j.admp.2021.03.004 · Cited by 86
Lunny, C., McKenzie, J. E., & McDonald, S. J. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.12.002 · Cited by 66
Verdonschot, A., Follong, B., Collins, C. E., de Vet, E., Haveman‐Nies, A., & Bucher, T. (2022) · Nutrition Reviews · doi:10.1093/nutrit/nuac057 · Cited by 64
Rosendahl, J., Alldredge, C. T., & Haddenhorst, A. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1330238 · Cited by 104
Baker, P. R. A., Costello, J. T., Dobbins, M., & B. Waters, E. (2014) · Journal of Public Health · doi:10.1093/pubmed/fdu050 · Cited by 53
DEFINING RAPID REVIEWS: A MODIFIED DELPHI CONSENSUS APPROACH
Kelly, S. E., Moher, D., & Clifford, T. J. (2016) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462316000489 · Cited by 50
A scoping approach to systematically review published reviews: Adaptations and recommendations
Schultz, A. S. H., Goertzen, L. N., Rothney, J., Wener, P. F., Enns, J. E., Halas, G., & Katz, A. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1272 · Cited by 53
Conneely, M., Leahy, S., Dore, L., Trépel, D., Robinson, K., Jordan, F., & Galvin, R. (2022) · BMC Geriatrics · doi:10.1186/s12877-022-03007-5 · Cited by 57
Bucci, R., Montanaro, D., Rongo, R., Valletta, R., Michelotti, A., & D’Antò, V. (2019) · Journal of Oral Rehabilitation · doi:10.1111/joor.12766 · Cited by 51
A Critical Review of Literature Review Methodologies
Farrukh, A., & Sajjad, A. (2023) · Review of management literature · doi:10.1108/s2754-586520230000002006 · Cited by 60
Faillie, J., Ferrer, P., Gouverneur, A., Driot, D., Berkemeyer, S., Vidal, X., Martínez‐Zapata, M. J., Huerta, C., Cervelló, X. C., Rottenkolber, M., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.04.023 · Cited by 57
Bougioukas, K. I., Vounzoulaki, E., Mantsiou, C. D., Papanastasiou, G. D., Savvides, E. D., Ntzani, E. E., & Haidich, A.-B. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.03.019 · Cited by 46
An Overview of Reviews: Organizational Change Management Architecture
Ros, A. D., Vainieri, M., & Bellè, N. (2023) · Journal of Change Management · doi:10.1080/14697017.2023.2197451 · Cited by 50
Towards Nuts and Bolts of Conducting Literature Review: A Typology of Literature Review
Taherdoost, H. (2023) · Electronics · doi:10.3390/electronics12040800 · Cited by 56
Lunny, C., Kanji, S., Thabet, P., Haidich, A., Bougioukas, K. I., & Pieper, D. (2024) · BMJ Medicine · doi:10.1136/bmjmed-2023-000604 · Cited by 43
Overview of systematic reviews - a new type of study. Part II
Silva, V., Grande, A. J., Carvalho, A. P. V. D., Martimbianco, A. L. C., & Riera, R. (2014) · Sao Paulo Medical Journal · doi:10.1590/1516-3180.2013.8150015 · Cited by 41
Sadoyu, S., Tanni, K. A., Punrum, N., Paengtrai, S., Kategaew, W., Promchit, N., Lai, N. M., Thakkinstian, A., Ngorsuraches, S., Bangpan, M., et al. (2022) · PLOS ONE · doi:10.1371/journal.pone.0269009 · Cited by 40
“The number of systematic review (SR) and umbrella reviews (UR) and meta-analysis (MAs) has increased dramatically over recent years [1].”
BackgroundMethodResultInfluentialMethodological approaches in conducting overviews: current state in HTA agencies
Pieper, D., Antoine, S., Morfeld, J., Mathes, T., & Eikermann, M. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1107 · Cited by 39
“…related to the problems and advantages of overviews (Becker and Oxman, 2008, Cooper and Koenka, 2012, Smith et al., 2011, Thomson et al., 2010, Hartling et al., 2012, Pieper et al., 2012), we developed an analytical framework to guide the presentation of results and subsequent discussion.”
MethodBüchter, R. B., Weise, A., & Pieper, D. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01438-z · Cited by 39
Płaszewski, M., & Bettany‐Saltikov, J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0110254 · Cited by 45
Lunny, C., Neelakant, T., Chen, A., Shinger, G., Stevens, A., Tasnim, S., Sadeghipouya, S., Adams, S., Zheng, Y. W., Lin, L., et al. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1530 · Cited by 40
Déry, J., Ouellet, B., de Guise, É., Bussières, È.-L., & Lamontagne, M. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02284-4 · Cited by 37
Kittle, J. M., Amendum, S. J., & Budde, C. M. (2024) · Educational Psychology Review · doi:10.1007/s10648-024-09942-6 · Cited by 39
“A systematic review of systematic reviews allows researchers to summarize evidence from multiple empirical research synthe-ses to provide an overall summary of a body of knowledge on a particular topic (Hartling et al., 2012).”
Conroy, T., Heuzenroeder, L., & Feo, R. (2020) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzaa064 · Cited by 37
Huff, N. R., Dunderdale, L., Kellogg, A. J., & Isbell, L. M. (2024) · Clinical Psychology Review · doi:10.1016/j.cpr.2024.102504 · Cited by 36
Bilican, S., Irfan, M., Cox, A., Salaets, H., Sabbe, M. B., & Schoenmakers, B. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-096267 · Cited by 35
Mattu, A. T., Ghali, B., Linton, V., Zheng, A., & Pike, I. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19084648 · Cited by 42
“Therefore, umbrella reviews, otherwise known as “overviews”, “review of reviews”, and “metareviews”, have increased in popularity as a form of evidence synthesis to provide userfriendly summaries for healthcare providers [19].”
BackgroundNonviolent Discipline Options for Caregivers and Teachers: A Systematic Overview of the Evidence
Quail, K. R., & Ward, C. L. (2020) · Trauma Violence & Abuse · doi:10.1177/1524838020967340 · Cited by 35
“Because of vast heterogeneity between reviews, and as is usual for an overview, extracted data are presented as a narrative synthesis using text and tables, without further statistical analysis (Hartling et al., 2012; Thomson et al., 2013).”
BackgroundMadani, M. T., Madani, L., Ghogomu, E. T., Dahrouge, S., Hébert, P. C., Juandó‐Prats, C., Mulligan, K., & Welch, V. A. (2022) · BMC Public Health · doi:10.1186/s12889-022-14667-8 · Cited by 31
Umbrella Reviews, Overviews of Reviews, and Meta-epidemiologic Studies: Similarities and Differences
Tsagris, M., & Fragkos, K. C. (2016) · Springer eBooks · doi:10.1007/978-3-319-25655-9_4 · Cited by 35
Protogerou, C., McHugh, R. K., & Johnson, B. T. (2019) · Health Psychology Review · doi:10.1080/17437199.2019.1707104 · Cited by 30
Pieper, D., Pollock, M., Fernandes, R. M., Büchter, R. B., & Hartling, L. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0468-9 · Cited by 25
Self-citationMcClatchey, K., Murray, J., Chouliara, Z., & Rowat, A. M. (2018) · Archives of Suicide Research · doi:10.1080/13811118.2018.1480983 · Cited by 26
Déry, J., de Guise, É., Bussières, È.-L., & Lamontagne, M. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01810-6 · Cited by 26
Driot, D., Bismuth, M., Maurel, A., Soulié-Albouy, J., Birebent, J., Oustric, S., & Dupouy, J. (2017) · La Presse Médicale · doi:10.1016/j.lpm.2017.10.010 · Cited by 25
Drovandi, A. D., Wong, S. A., Seng, L. S. D., Crowley, B. J., Alahakoon, C., Banwait, J., Fernando, M. E., & Golledge, J. (2021) · Journal of Diabetes Science and Technology · doi:10.1177/19322968211012456 · Cited by 28
Are systematic reviews and meta-analyses still useful research? No
Chevret, S., Ferguson, N. D., & Bellomo, R. (2018) · Intensive Care Medicine · doi:10.1007/s00134-018-5066-3 · Cited by 24
Lunny, C., Thirugnanasampanthar, S. S., Kanji, S., Ferri, N., Pieper, D., Whitelaw, S., Tasnim, S., Nelson, H. H., Reid, E. K., Zhang, J. H., et al. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01750-2 · Cited by 24
Sanchez‐Ramos, L., Lin, L., Vilchez-Lagos, G., Duncan, J. R., Condon, N. A., Wheatley, J., & Kaunitz, A. M. (2023) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2022.01.005 · Cited by 23
Thomson, D., Foisy, M., Oleszczuk, M., Wingert, A., Chisholm, A., & Hartling, L. (2013) · Evidence-Based Child Health: A Cochrane Review Journal · doi:10.1002/ebch.1897 · Cited by 22
“45) 62% Enuresis alarm versus dry bed training 108 (3) 37% 1.”
BackgroundResultInfluentialSelf-citationGasparetto, M., Narula, P., Wong, C., Ashton, J. J., Kammermeier, J., Pierik, M. J., Kopylov, U., & Arebi, N. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2024.12.007 · Cited by 26
From Qualitative Reviews to Umbrella Reviews
Ortega, A., Briz, E. L., & Fraga‐Fuentes, M. D. (2016) · Springer eBooks · doi:10.1007/978-3-319-25655-9_3 · Cited by 27
Metadata
- Authors
- Hartling, L., Chisholm, A., Thomson, D., & Dryden, D. M.
- Affiliations
- University of Alberta (CA)
- Corresponding author
- Hartling, Lisa
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2012-11-15
- Volume
- 7
- Issue
- 11
- Pages
- e49667
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3499476
- OpenAlex
- W2003178123
- Cited by
- 241
- References
- 28
- Citations by year
- 2026: 7 · 2025: 13 · 2024: 20 · 2023: 28 · 2022: 30 · 2021: 30 · 2020: 21 · 2019: 12 · 2018: 12 · 2017: 19 · 2016: 26 · 2015: 5 · 2014: 14 · 2013: 4
- FWCI
- 8.39
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- overviews of systematic reviews, methodological quality, evidence synthesis, systematic reviews, search strategy reporting, publication bias, network meta-analysis, indirect analysis, child health interventions, pharmacological interventions, non-pharmacological treatment, narrative synthesis, evidence quality assessment
- MeSH terms
- Child, Child Welfare, Humans, Publications, Review Literature as Topic, Evidence-Based Medicine, History, 21st Century
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

