Advancing knowledge of rapid reviews: an analysis of results, conclusions and recommendations from published review articles examining rapid reviews
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Rapid review (RR) products are inherently appealing as they are intended to be less time-consuming and resource-intensive than traditional systematic reviews (SRs); however, there is concern about the rigor of methods and reliability of results. In 2013 to 2014, a workgroup comprising representatives from the Agency for Healthcare Research and Quality's Evidence-based Practice Center Program conducted a formal evaluation of RRs. This paper summarizes results, conclusions, and recommendations from published review articles examining RRs. METHODS: A systematic literature search was conducted and publications were screened independently by two reviewers. Twelve review articles about RRs were identified. One investigator extracted data about RR methods and how they compared with standard SRs. A narrative summary is presented. RESULTS: A cross-comparison of review articles revealed the following: 1) ambiguous definitions of RRs, 2) varying timeframes to complete RRs ranging from 1 to 12 months, 3) limited scope of RR questions, and 4) significant heterogeneity between RR methods. CONCLUSIONS: RR definitions, methods, and applications vary substantially. Published review articles suggest that RRs should not be viewed as a substitute for a standard SR, although they have unique value for decision-makers. Recommendations for RR producers include transparency of methods used and the development of reporting standards.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The conduct of SRs can be a time-consuming, cost- and resource-intensive task, which may take on average from 6 months to several years [4–6].”
From How to conduct systematic reviews more expeditiously? (2015) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The January 2015 issue of Systematic Reviews has published a thematic collection of articles highlighting important developments in the RR methodology which will likely help addressing the issues related to timely production of SRs [6, 26, 54].”
From How to conduct systematic reviews more expeditiously? (2015) · cites this paper as [6]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“RRs are better suited for narrowly defined research questions where one or more SR steps may be reduced or omitted [4, 6, 21, 22, 24, 25].”
From How to conduct systematic reviews more expeditiously? (2015) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“The majority of RR methodology overviews represent surveys that either describe or compare the methods and processes used for conducting RRs and SRs [4, 6, 21, 22, 24–26].”
From How to conduct systematic reviews more expeditiously? (2015) · cites this paper as [6]
In the citing paper
“It has been estimated that rapid reviews take around half the time of a typical SR [13].”
From Making progress with the automation of systematic reviews: principles of the International Collaboration for the Automation of Systematic Reviews (ICASR) (2018) · cites this paper as [13]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · cited 4×
“Only one included review article [15] followed methods consistent with a systematic review, as defined by the Cochrane Handbook for SRs [2].”
BackgroundMethodInfluentialA typology of reviews: an analysis of 14 review types and associated methodologies
Grant, M. J., & Booth, A. (2009) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2009.00848.x · Cited by 10,817 · cited 8×
“Grant and Booth emphasized the care needed to develop questions that are well-suited to a RR methodology [22].”
BackgroundMethodResultInfluentialCochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,178
Finding What Works in Health Care: Standards for Systematic Reviews
Eden, J., Levit, L. A., Berg, A. O., & Morton, S. C. (2013) · Cited by 948
Expediting systematic reviews: methods and implications of rapid reviews
Ganann, R., Ciliska, D., & Thomas, H. (2010) · Implementation Science · doi:10.1186/1748-5908-5-56 · Cited by 752 · cited 8×
“Authors of review articles described RRs as ‘varying widely in terms of the language used to describe them’ [15].”
BackgroundMethodInfluentialFinding What Works in Health Care: Standards for Systematic Reviews
Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, Board on Health Care Services, & Institute of Medicine (2011) · doi:10.17226/13059 · Cited by 289
Watt, A., Cameron, A., Sturm, L., Lathlean, T., Babidge, W., Blamey, S., Facey, K., Hailey, D., Norderhaug, I., & Maddern, G. (2008) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462308080185 · Cited by 226
The art and science of knowledge synthesis
Tricco, A. C., Tetzlaff, J., & Moher, D. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.11.007 · Cited by 195 · cited 1×
“Eight of those publications were subsequently excluded from the final analysis because they discussed a research method that did not fall under our broad definition of RRs [7-10], were not a review article [4,11,12], or were not published in English [13].”
BackgroundRAPID VERSUS FULL SYSTEMATIC REVIEWS: VALIDITY IN CLINICAL PRACTICE?
Watt, A., Cameron, A., Sturm, L., Lathlean, T., Babidge, W., Blamey, S., Facey, K., Hailey, D., Norderhaug, I., & Maddern, G. (2008) · ANZ Journal of Surgery · doi:10.1111/j.1445-2197.2008.04730.x · Cited by 163 · cited 8×
“between 1 and 6 months [14,16,18], but Dennett and Chojecki described products taking as little as 1 week [21].”
BackgroundMethodInfluentialRAPID REVIEW: AN EMERGING APPROACH TO EVIDENCE SYNTHESIS IN HEALTH TECHNOLOGY ASSESSMENT
Khangura, S., Polisena, J., Clifford, T. J., Farrah, K., & Kamel, C. (2014) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462313000664 · Cited by 138 · cited 1×
“Rapid review (RR) products are intended to synthesize available evidence and meet the time constraints of healthcare decision-makers [1].”
BackgroundHarker, J., & Kleijnen, J. (2012) · International Journal of Evidence-Based Healthcare · doi:10.1111/j.1744-1609.2012.00290.x · Cited by 137 · cited 8×
“Of the 46 RRs reviewed by Harker and Kleijnen, 47% (n = 23) did not have a clear question, and only one RR used the Patient, Intervention, Comparison, Outcomes (PICO) criteria to formulate their question [16].”
BackgroundMethodInfluentialRapid evidence assessments of research to inform social policy: taking stock and moving forward
Thomas, J., Newman, M., & Oliver, S. (2013) · Evidence and Policy · doi:10.1332/174426413x662572 · Cited by 109 · cited 9×
“Multiple reviews found that RR questions were narrowly focused [6,22] and that time to complete the RR was relative to the complexity of the guiding question [17].”
BackgroundMethodInfluentialIssues in conducting and disseminating brief reviews of evidence
Abrami, P. C., Borokhovski, E., Bernard, R. M., Wade, C. A., Tamim, R., Persson, T., Bethel, E. C., Hanz, K., & Surkes, M. A. (2010) · Evidence & Policy · doi:10.1332/174426410x524866 · Cited by 83 · cited 3×
“Twelve review articles about RRs were included in the final analysis [6,14-24] (see Figure 1 for flow of publications through the screening process).”
BackgroundMethodOverview of systematic reviews - a new type of study: part I: why and for whom?
Silva, V., Grande, A. J., Martimbianco, A. L. C., Riera, R., & de Carvalho, A. P. V. (2012) · Sao Paulo Medical Journal · doi:10.1590/s1516-31802012000600007 · Cited by 83 · cited 1×
“Eight of those publications were subsequently excluded from the final analysis because they discussed a research method that did not fall under our broad definition of RRs [7-10], were not a review article [4,11,12], or were not published in English [13].”
BackgroundLehoux, P., Tailliez, S., Denis, J.-L., & Hivon, M. (2004) · International Journal of Technology Assessment in Health Care · doi:10.1017/s026646230400114x · Cited by 66 · cited 2×
“Eight of those publications were subsequently excluded from the final analysis because they discussed a research method that did not fall under our broad definition of RRs [7-10], were not a review article [4,11,12], or were not published in English [13].”
BackgroundEPC Methods: An Exploration of Methods and Context for the Production of Rapid Reviews
Hartling, L., Guise, J., Kato, E. U., Anderson, J., Aronson, N., Belinson, S., Berliner, E., Dryden, D. M., Featherstone, R. M., Foisy, M., et al. (2015) · Cited by 62
Self-citationMartelli, N., Lelong, A.-S., Prognon, P., & Pineau, J. (2013) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462313000019 · Cited by 45 · cited 1×
“Eight of those publications were subsequently excluded from the final analysis because they discussed a research method that did not fall under our broad definition of RRs [7-10], were not a review article [4,11,12], or were not published in English [13].”
BackgroundA preliminary survey on the influence of rapid health technology assessments
Hailey, D. (2009) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462309990067 · Cited by 34 · cited 7×
“Reviewers concluded that RRs were not viable alternatives to SRs [15] and that decision-makers would lose the detail and assurance provided by a standard SR [24].”
BackgroundMethodInfluentialHTA responses and the classic HTA report
Milne, R., Clegg, A. J., & Stevens, A. J. (2003) · Journal of Public Health · doi:10.1093/pubmed/fdg023 · Cited by 14 · cited 1×
“Eight of those publications were subsequently excluded from the final analysis because they discussed a research method that did not fall under our broad definition of RRs [7-10], were not a review article [4,11,12], or were not published in English [13].”
BackgroundThe efficient production of high quality evidence reviews is important for the public good
Elliott, J., Mavergames, C., Becker, L. A., Meerpohl, J. J., Thomas, J., Gruen, R. L., & Tovey, D. I. (2013) · BMJ · doi:10.1136/bmj.f846 · Cited by 5
Summaries of evidence: an adjunct to knowledge translation?
Kendall, S. A. (2013) · Primary Health Care Research & Development · doi:10.1017/s146342361300008x · Cited by 2 · cited 1×
“Eight of those publications were subsequently excluded from the final analysis because they discussed a research method that did not fall under our broad definition of RRs [7-10], were not a review article [4,11,12], or were not published in English [13].”
Background[Rapid reviews for evidence-based decision support. (Restricting) requirements].
Ziegler, S., Lühmann, D., Raspe, H. H., & Windeler, J. (2001) · PubMed · Cited by 2
Research works printed but not linked: 2
- 20. Aidelsburger P, Felder S, Siebert U, Wasem J. Health economic short HTA reports - systematic review of methods and implementation. In Health economic short HTA reports - systematic review of methods and implementation, vol. 6. Deutsches Institut fur Medizinische Dokumentation und Information; 2003.
- 23. Scott A, Harstall C. Utilizing diverse HTA products in the Alberta Health Technologies Decision Process: work in process. In Utilizing diverse HTA products in the Alberta Health Technologies Decision Process: Work in process. Institute of Health Economics; 2012.
Cited by
A scoping review of rapid review methods
Tricco, A. C., Antony, J., Zarin, W., Strifler, L., Ghassemi, M., Ivory, J., Perrier, L., Hutton, B., Moher, D., & Straus, S. E. (2015) · BMC Medicine · doi:10.1186/s12916-015-0465-6 · Cited by 1,146
“The authors summarized evidence from 12 review articles of rapid reviews [120], as well as 35 different rapid reviews produced by 20 different organizations [119].”
BackgroundA review of medical image data augmentation techniques for deep learning applications
Chlap, P., Min, H., Vandenberg, N., Dowling, J., Holloway, L., & Haworth, A. (2021) · Journal of Medical Imaging and Radiation Oncology · doi:10.1111/1754-9485.13261 · Cited by 956
“The area of deep learning is quickly evolving, so to ensure relevance of the articles the concept of a rapid review was adopted.(11) Appropriate constraints were placed on the literature search and article inclusion criteria to provide a snapshot of data augmentation techniques that are currently used in radiology and…”
BackgroundAdams, J., Hillier-Brown, F. C., Moore, H. J., Lake, A. A., Araujo-Soares, V., White, M., & Summerbell, C. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0337-y · Cited by 486
Haby, M. M., Chapman, E., Clark, R., Barreto, J., Reveiz, L., & Lavis, J. N. (2016) · Health Research Policy and Systems · doi:10.1186/s12961-016-0155-7 · Cited by 409
“[31, 32] also included the four previous systematic reviews included in this rapid review [18, 19, 21, 30, 33].”
BackgroundThe impact of COVID-19 school closure on child and adolescent health: a rapid systematic review
Chaabane, S., Doraiswamy, S., Chaabna, K., Mamtani, R., & Cheema, S. (2021) · Children · doi:10.3390/children8050415 · Cited by 288
“In such a situation where the evidence is urgently needed to guide decision-making during the current pandemic, carrying out a rapid systematic review has emerged as a useful approach to provide actionable and relevant evidence in a timely and cost-effective manner compared with a standard systematic review [28].”
Background
Metadata
- Authors
- Featherstone, R. M., Dryden, D. M., Foisy, M., Guise, J.-M., Mitchell, M. D., Paynter, R. A., Robinson, K. A., Umscheid, C. A., & Hartling, L.
- Affiliations
- University of Alberta (CA)
- Oregon Health & Science University (US)
- ECRI Institute (US)
- Portland VA Research Foundation (US)
- Johns Hopkins University (US)
- Corresponding author
- Featherstone, Robin M
- ORCID
- Venue
- Systematic Reviews
- Published
- 2015-04-17
- Volume
- 4
- Issue
- 1
- Pages
- 50-50
- ISSN
- 2046-4053 (ISSN-L)
- PMC
- PMC4415284
- OpenAlex
- W2148913386
- Cited by
- 140
- References
- 21
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 12 · 2023: 12 · 2022: 16 · 2021: 20 · 2020: 18 · 2019: 11 · 2018: 11 · 2017: 12 · 2016: 13 · 2015: 6
- FWCI
- 12.77
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Systematic Reviews (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- ERA: Education and Research Archive (University of Alberta)
- Funders
- University of Pennsylvania
- Johns Hopkins University
- University of Alberta
- Agency for Healthcare Research and Quality (290-2012-00007-I, 290-2012-00013-I)
- U.S. Public Health Service (290-2012-00010-I, 290-2012-00013-I, 290-2012-00004-C, 290-2012-00011-I, 290-2012-00015-I)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- rapid review, review methodology, methodological quality, reporting guidelines, transparency, evidence-based practice
- MeSH terms
- Clinical Protocols, Decision Making, Humans, Publishing, Research Design, Review Literature as Topic, Reproducibility of Results, Knowledge, Evidence-Based Practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
