Fit for purpose: perspectives on rapid reviews from end-user interviews
Open accessAbstract
BACKGROUND: There is increasing demand for rapid reviews and timely evidence synthesis. The goal of this project was to understand end-user perspectives on the utility and limitations of rapid products including evidence inventories, rapid responses, and rapid reviews. METHODS: Interviews were conducted with key informants representing: guideline developers (n = 3), health care providers/health system organizations (n = 3), research funders (n = 1), and payers/health insurers (n = 1). We elicited perspectives on important characteristics of systematic reviews, acceptable methods to streamline reviews, and uses of rapid products. We analyzed content of the interview transcripts and identified themes and subthemes. RESULTS: Key informants identified the following as critical features of evidence reviews: (1) originating from a reliable source (i.e., conducted by experienced reviewers from an established research organization), (2) addressing clinically relevant questions, and (3) trusted relationship between the user and producer. Key informants expressed strong preference for the following review methods and characteristics: use of evidence tables, quality rating of studies, assessments of total evidence quality/strength, and use of summary tables for results and conclusions. Most acceptable trade-offs to increase efficiencies were limiting the literature search (e.g., limiting search dates or language) and performing single screening of citations and full texts for relevance. Key informants perceived rapid products (particularly evidence inventories and rapid responses) as useful interim products to inform downstream investigation (e.g., whether to proceed with a full review or guideline, direction for future research). Most key informants indicated that evidence analysis/synthesis and quality/strength of evidence assessments were important for decision-making. They reported that rapid reviews in particular were useful for guideline development on narrow topics, policy decisions when a quick turn-around is needed, decision-making for practicing clinicians in nuanced clinical settings, and decisions about coverage by payers/health insurers. Rapid reviews may be more relevant within specific clinical settings or health systems; whereas, broad/national guidelines often need a traditional systematic review. CONCLUSIONS: Key informants interviewed in our study indicated that evidence inventories, rapid responses, and rapid reviews have utility in specific decisions and contexts. They indicated that the credibility of the review producer, relevance of key questions, and close working relationship between the end-user and producer are critical for any rapid product. Our findings are limited by the sample size which may have been too small to reach saturation for the themes described.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Evidence indicates that policymakers are increasingly using rapid reviews in their daily decision-making.
“Evidence suggests that policymakers are increasingly using RRs in their daily decision-making [[9], [10], [11], [12]].”
From Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews (2021) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(1995) · Medical Entomology and Zoology · Cited by 2,472 · see citing works
Systematic Reviews: Identifying relevant studies for systematic reviews
Dickersin, K., Scherer, R. W., & Lefebvre, C. (1994) · BMJ · doi:10.1136/bmj.309.6964.1286 · Cited by 2,255 · see citing works
Shea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.009 · Cited by 1,755 · see citing works
Expediting systematic reviews: methods and implications of rapid reviews
Ganann, R., Ciliska, D., & Thomas, H. (2010) · Implementation Science · doi:10.1186/1748-5908-5-56 · Cited by 752 · see citing works
What contributions do languages other than English make on the results of meta-analyses?
Moher, D., Pham, Klassen, T. P., Schulz, K. F., Berlin, J. A., Jadad, A. R., & Liberati, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00188-8 · Cited by 453 · see citing works
Checking reference lists to find additional studies for systematic reviews
Horsley, T., Dingwall, O., & Sampson, M. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000026.pub2 · Cited by 446 · see citing works
Watt, A., Cameron, A., Sturm, L., Lathlean, T., Babidge, W., Blamey, S., Facey, K., Hailey, D., Norderhaug, I., & Maddern, G. (2008) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462308080185 · Cited by 226 · see citing works
Google Scholar is not enough to be used alone for systematic reviews
Giustini, D., & Kamel Boulos, M. N. (2013) · Online Journal of Public Health Informatics · doi:10.5210/ojphi.v5i2.4623 · Cited by 187 · see citing works
RAPID VERSUS FULL SYSTEMATIC REVIEWS: VALIDITY IN CLINICAL PRACTICE?
Watt, A., Cameron, A., Sturm, L., Lathlean, T., Babidge, W., Blamey, S., Facey, K., Hailey, D., Norderhaug, I., & Maddern, G. (2008) · ANZ Journal of Surgery · doi:10.1111/j.1445-2197.2008.04730.x · Cited by 163 · see citing works
Featherstone, R. M., Dryden, D. M., Foisy, M., Guise, J.-M., Mitchell, M. D., Paynter, R. A., Robinson, K. A., Umscheid, C. A., & Hartling, L. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0040-4 · Cited by 140 · see citing works
Self-citationOxman, A. D., Schünemann, H. J., & Fretheim, A. (2006) · Health Research Policy and Systems · doi:10.1186/1478-4505-4-20 · Cited by 116 · see citing works
Systematic reviews can be produced and published faster
Sampson, M., Shojania, K. G., Garritty, C., Horsley, T., Ocampo, M., & Moher, D. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.02.004 · Cited by 103 · see citing works
A taxonomy of rapid reviews links report types and methods to specific decision-making contexts
Hartling, L., Guise, J., Kato, E. U., Anderson, J., Belinson, S., Berliner, E., Dryden, D. M., Featherstone, R. M., Mitchell, M. D., Motu’apuaka, M. L., et al. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.05.036 · Cited by 75 · see citing works
Self-citationEPC Methods: An Exploration of Methods and Context for the Production of Rapid Reviews
Hartling, L., Guise, J., Kato, E. U., Anderson, J., Aronson, N., Belinson, S., Berliner, E., Dryden, D. M., Featherstone, R. M., Foisy, M., et al. (2015) · Cited by 62 · see citing works
Self-citationBushman, B. J., & Wells, G. L. (2001) · Personality and Social Psychology Bulletin · doi:10.1177/0146167201279005 · Cited by 58 · see citing works
Centre for Reviews and Dissemination databases: Value, content, and developments
Booth, A. M., Wright, K. E., & Outhwaite, H. (2010) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462310000978 · Cited by 51 · see citing works
Selph, S., Ginsburg, A. D., & Chou, R. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-107 · Cited by 50 · see citing works
Incorporating evidence review into quality improvement: meeting the needs of innovators
Danz, M. S., Hempel, S., Lim, Y.-W., Shanman, R., Motala, A., Stockdale, S. E., Shekelle, P., & Rubenstein, L. V. (2013) · BMJ Quality & Safety · doi:10.1136/bmjqs-2012-001722 · Cited by 25 · see citing works
Self-citationSystematic Reviews of Complex Multicomponent Health Care Interventions
Guise, J., Chang, C., Viswanathan, M., Glick, S., Treadwell, J. R., Umscheid, C. A., Whitlock, E. P., Fu, R., Berliner, E., Paynter, R. A., et al. (2014) · Cited by 25 · see citing works
Self-citationEPC Methods: AHRQ End-User Perspectives of Rapid Reviews
Hartling, L., Guise, J., Hempel, S., Featherstone, R. M., Mitchell, M. D., Motu’apuaka, M. L., Robinson, K. A., Schoelles, K. M., Totten, A. M., Whitlock, E. P., et al. (2016) · Europe PMC (PubMed Central) · Cited by 13 · see citing works
Self-citationVenous Thromboembolism Prophylaxis in Orthopedic Surgery
Sobieraj, D. M. (2014) · Europe PMC (PubMed Central) · doi:10.7301/z0h12zxh · Cited by 11 · see citing works
Jayakumar, K. L., Lavenberg, J. G., Mitchell, M. D., Doshi, J. A., Leas, B. F., Goldmann, D. R., Williams, K., Brennan, P. J., & Umscheid, C. A. (2015) · Journal of Hospital Medicine · doi:10.1002/jhm.2498 · Cited by 9 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews
Garritty, C., Gartlehner, G., Nussbaumer-Streit, B., King, V. J., Hamel, C., Kamel, C., Affengruber, L., & Stevens, A. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.10.007 · Cited by 1,141 · see citing works
Moya, B., Eaton, S., Pethrick, H., Hayden, A., Brennan, R., Wiens, J., & McDermott, B. (2024) · Canadian Perspectives on Academic Integrity · doi:10.55016/ojs/cpai.v7i3.78123 · Cited by 48 · see citing works
The use of rapid review methods for the U.S. Preventive services task force
Patnode, C. D., Eder, M. L., Walsh, E. S., Viswanathan, M., & Lin, J. S. (2018) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2017.07.024 · Cited by 40 · see citing works
Garritty, C., Hamel, C., Hersi, M., Butler, C., Monfaredi, Z., Stevens, A., Nussbaumer-Streit, B., Cheng, W., & Moher, D. (2020) · Health Research Policy and Systems · doi:10.1186/s12961-020-00624-7 · Cited by 21 · see citing works
Griebler, U., Dobrescu, A., Ledinger, D., Klingenstein, P., Sommer, I., Emprechtinger, R., Persad, E., Gadinger, A., Trivella, M., Klerings, I., et al. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1656 · Cited by 5 · see citing works
- Authors
- Hartling, L., Guise, J.-M., Hempel, S., Featherstone, R., Mitchell, M. D., Motu’apuaka, M. L., Robinson, K. A., Schoelles, K., Totten, A., Whitlock, E., Wilt, T. J., Anderson, J., Berliner, E., Gozu, A., Kato, E., Paynter, R., & Umscheid, C. A.
- Affiliations
- University of Alberta (CA)
- Portland VA Research Foundation (US)
- RAND Corporation (US)
- University of Pennsylvania Health System (US)
- Johns Hopkins University (US)
- ECRI Institute (US)
- Kaiser Permanente (US)
- University of Minnesota (US)
- Agency for Healthcare Research and Quality (US)
- University of Pennsylvania (US)
- Corresponding author
- Hartling, Lisa
- ORCID
- Venue
- Systematic Reviews
- Published
- 2017-02-17
- Volume
- 6
- Issue
- 1
- Pages
- 32-32
- ISSN
- 2046-4053 (ISSN-L)
- PMC
- PMC5316162
- OpenAlex
- W2589073600
- Cited by
- 65
- References
- 22
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 10 · 2023: 9 · 2022: 9 · 2021: 3 · 2020: 18 · 2019: 7 · 2018: 2 · 2017: 2
- FWCI
- 7.10
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Systematic Reviews (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Agency for Healthcare Research and Quality (290-2012-00013-I)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- rapid review, rapid response, end-user perspective, guideline development, evidence synthesis, evidence quality assessment, evidence tables, healthcare decision-making, clinical practice
- MeSH terms
- Delivery of Health Care, Health Personnel, Humans, Insurance, Health, Interviews as Topic, Review Literature as Topic, Time Factors, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
