Overcoming challenges in conducting systematic reviews in implementation science: a methods commentary
Open accessAbstract
Consolidation of the literature using systematic reviews is a critical way to advance a discipline and support evidence-based decision-making in healthcare. However, unique challenges exist that impact the conduct of systematic reviews in implementation science. In this commentary, we reflect on our combined experience to describe five key challenges unique to systematic reviews of primary implementation research. These challenges include (1) descriptors used in implementation science publications, (2) distinction between evidence-based interventions and implementation strategies, (3) assessment of external validity, (4) synthesis of implementation studies with substantial clinical and methodological diversity, and (5) variability in defining implementation ‘success’. We outline possible solutions and highlight resources that can be used by authors of primary implementation research, as well as systematic review and editorial teams, to overcome the identified challenges and optimise the utility of future systematic reviews in implementation science.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Heterogeneity in population characteristics, study designs, methods, and outcomes has made it difficult to quantitatively compare which strategies work and under which conditions.
“Heterogeneity in population characteristics, study designs, methods, and outcomes have made it difficult to quantitatively compare which strategies work and under which conditions [21].”
From A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022 (2024) · cites this paper as [21]
There remains significant variability in how researchers operationalize, apply, and report strategies and outcomes.
“Moreover, there remains significant variability in how researchers operationalize, apply, and report strategies (individually or in combination) and outcomes [21, 22].”
From A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022 (2024) · cites this paper as [21]
Implementation outcomes were often inconsistently defined and poorly reported.
“Similarly, because implementation outcomes were often inconsistently defined and poorly reported [28–30], we broadened our scope to include studies that identified team constructs as implementation determinants (i.e., barriers/facilitators) without explicitly defining and measuring an implementation outcome.”
From Teamwork and implementation of innovations in healthcare and human service settings: a systematic review (2024) · cites this paper as [30]
Heterogeneity preventing quantitative synthesis is common in implementation science systematic reviews.
“Because of the heterogeneity in team constructs and implementation outcomes, we were unable to quantitatively synthesize results using meta-analysis or formally investigate heterogeneity; this challenge is common in implementation science systematic reviews [30].”
From Teamwork and implementation of innovations in healthcare and human service settings: a systematic review (2024) · cites this paper as [30]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 33 · Cited by 36
Research works printed but not linked: 2
- [2]Rabin BA, Brownson RC. Terminology for Dissemination and Implementation Research. In: Brownson RC, Colditz GA, Proctor EK, eds. Dissemination and Implementation Research in Health: Translating Science to Practice. 2nd ed. New York: Oxford University Press; 2018. p. 19–45.
- [30]McKenzie JE, Brennan SE. Chapter 12: Synthesizing and presenting findings using other methods. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, eds. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane. 2022. Available from www.training.cochrane.org/handbook.
- Authors
- Chapman, A., Rankin, N. M., Jongebloed, H., Yoong, S. L., White, V., Livingston, P. M., Hutchinson, A. M., & Ugalde, A.
- Affiliations
- Deakin University (AU)
- The University of Melbourne (AU)
- Barwon Health (AU)
- Corresponding author
- Chapman, Anna
- ORCID
- Venue
- Systematic Reviews
- Published
- 2023-07-07
- Volume
- 12
- Issue
- 1
- Pages
- 116-116
- ISSN
- 2046-4053 (ISSN-L)
- PMC
- PMC10327144
- OpenAlex
- W4383534865
- Cited by
- 36
- References
- 33
- Citations by year
- 2026: 8 · 2025: 8 · 2024: 18 · 2023: 2
- FWCI
- 1.83
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Systematic Reviews (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- implementation science, evidence-based interventions, implementation strategies, generalizability
- MeSH terms
- Implementation Science, Systematic Reviews as Topic, Humans, Research Design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
