Qualitative evidence synthesis for complex interventions and guideline development: clarification of the purpose, designs and relevant methods
Open accessAbstract
This paper is one of a series exploring the implications of complexity for systematic reviews and guideline development, commissioned by the WHO. The paper specifically explores the role of qualitative evidence synthesis. Qualitative evidence synthesis is the broad term for the group of methods used to undertake systematic reviews of qualitative research evidence. As an approach, qualitative evidence synthesis is increasingly recognised as having a key role to play in addressing questions relating to intervention or system complexity, and guideline development processes. This is due to the unique role qualitative research can play in establishing the relative importance of outcomes, the acceptability, fidelity and reach of interventions, their feasibility in different settings and potential consequences on equity across populations. This paper outlines the purpose of qualitative evidence synthesis, provides detail of how qualitative evidence syntheses can help establish understanding and explanation of the complexity that can occur in relation to both interventions and systems, and how qualitative evidence syntheses can contribute to evidence to decision frameworks. It provides guidance for the choice of qualitative evidence synthesis methods in the context of guideline development for complex interventions, giving 'real life' examples of where this has occurred. Information to support decision-making around choice qualitative evidence synthesis methods in the context of guideline development is provided. Approaches for reporting qualitative evidence syntheses are discussed alongside mechanisms for assessing confidence in the findings of a review.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Inductive thematic analysis is the most appropriate method for synthesizing a mixed research design.
“Inductive thematic analysis was considered the most appropriate method for synthesizing a mixed research design (Flemming et al., 2019).”
From Outcome domains and assessment measures in school-based youth mentoring programs: a systematic review (2026) · cites this paper as Flemming et al., 2019
The QES methods section describes thematic synthesis as one option and says method choice depends on the review question and context. The thematic synthesis subsection says it may work well for questions about sociocultural acceptability, but does not recommend inductive thematic analysis as the most appropriate method for a mixed research design. Pages are not specified in our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Methods for the thematic synthesis of qualitative research in systematic reviews
Thomas, J., & Harden, A. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-45 · Cited by 9,312 · see citing works
Qualitative data analysis for applied policy research
(2002) · Analyzing Qualitative Data · doi:10.4324/9780203413081-14 · Cited by 4,964 · see citing works
Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ
Tong, A., Flemming, K., McInnes, E., Oliver, S., & Craig, J. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-181 · Cited by 3,569 · see citing works
Self-citationNoblit, G., & Hare, R. (1988) · doi:10.4135/9781412985000 · Cited by 2,723 · see citing works
Meta-Ethnography: Synthesizing Qualitative Studies
Noblit, G. W., & Hare, R. D. (1988) · Medical Entomology and Zoology · Cited by 2,191 · see citing works
Dixon-Woods, M., Cavers, D., Agarwal, S., Annandale, E., Arthur, A., Harvey, J., Hsu, R., Katbamna, S., Olsen, R., Smith, L., et al. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-35 · Cited by 2,053 · see citing works
Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series
Lewin, S., Booth, A., Glenton, C., Munthe-Kaas, H., Rashidian, A., Wainwright, M., Bohren, M. A., Tunçalp, Ö., Colvin, C. J., Garside, R., et al. (2018) · Implementation Science · doi:10.1186/s13012-017-0688-3 · Cited by 1,231 · see citing works
Self-citationUsing meta ethnography to synthesise qualitative research: a worked example
Britten, N., Campbell, R., Pope, C., Donovan, J., Morgan, M., & Pill, R. (2002) · Journal of Health Services Research & Policy · doi:10.1258/135581902320432732 · Cited by 1,110 · see citing works
Clarifying differences between review designs and methods
Gough, D., Thomas, J., & Oliver, S. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-28 · Cited by 1,063 · see citing works
Pfadenhauer, L. M., Gerhardus, A., Mozygemba, K., Lysdahl, K. B., Booth, A., Hofmann, B., Wahlster, P., Polus, S., Burns, J., Brereton, L., et al. (2017) · Implementation Science · doi:10.1186/s13012-017-0552-5 · Cited by 1,016 · see citing works
Self-citationQualitative Data Analysis for Applied Policy Research
Ritchie, J., & Spencer, L. (2002) · Sage eBooks · doi:10.4135/9781412986274.n12 · Cited by 920 · see citing works
Lewin, S., Glenton, C., Munthe-Kaas, H., Carlsen, B., Colvin, C. J., Gülmezoglu, M., Noyes, J., Booth, A., Garside, R., & Rashidian, A. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001895 · Cited by 866 · see citing works
Self-citationBohren, M. A., Hunter, E. C., Munthe-Kaas, H. M., Souza, J. P., Vogel, J. P., & Gülmezoglu, A. M. (2014) · Reproductive Health · doi:10.1186/1742-4755-11-71 · Cited by 823 · see citing works
“Best fit” framework synthesis: refining the method
Carroll, C., Booth, A., Leaviss, J., & Rick, J. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-37 · Cited by 649 · see citing works
Dixon‐Woods, M., McNicol, S., & Martin, G. P. (2012) · BMJ Quality & Safety · doi:10.1136/bmjqs-2011-000760 · Cited by 615 · see citing works
Southern, D. A., Norris, C. M., Quan, H., Shrive, F. M., Galbraith, P. D., Humphries, K. H., Gao, M., Knudtson, M. L., Ghali, W. A., & the APPROACH Investigators (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-1 · Cited by 503 · see citing works
Noyes, J., Booth, A., Moore, G., Flemming, K., Tunçalp, Ö., & Shakibazadeh, E. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-000893 · Cited by 497 · see citing works
Self-citationUsing framework-based synthesis for conducting reviews of qualitative studies
Dixon‐Woods, M. (2011) · BMC Medicine · doi:10.1186/1741-7015-9-39 · Cited by 496 · see citing works
Noyes, J., Booth, A., Flemming, K., Garside, R., Harden, A., Lewin, S., Pantoja, T., Hannes, K., Cargo, M., & Thomas, J. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.06.020 · Cited by 480 · see citing works
Self-citationCarroll, C., Booth, A., & Cooper, K. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-29 · Cited by 461 · see citing works
Self-citationGlenton, C., Colvin, C., Carlsen, B., Swartz, A., Lewin, S., Noyes, J., & Rashidian, A. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010414.pub2 · Cited by 454 · see citing works
Self-citationRosewilliam, S., Roskell, C., & Pandyan, A. (2011) · Clinical Rehabilitation · doi:10.1177/0269215510394467 · Cited by 373 · see citing works
Using logic models to capture complexity in systematic reviews: Logic Models in Systematic Reviews
Anderson, L. M., Petticrew, M., Rehfuess, E., Armstrong, R., Ueffing, E., Baker, P., Francis, D., & Tugwell, P. (2011) · Research Synthesis Methods · doi:10.1002/jrsm.32 · Cited by 372 · see citing works
A multidimensional conceptual framework for analysing public involvement in health services research
Oliver, S. R., Rees, R. W., Clarke‐Jones, L., Milne, R., Oakley, A., Gabbay, J., Stein, K., Buchanan, P., & Gyte, G. M. (2008) · Health Expectations · doi:10.1111/j.1369-7625.2007.00476.x · Cited by 344 · see citing works
Petticrew, M. P., Rehfuess, E. A., Noyes, J., Higgins, J. P. T., Mayhew, A., Tomás Pantoja, C., Shemilt, I., & Sowden, A. J. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.06.005 · Cited by 292 · see citing works
Self-citationNoyes, J., Booth, A., Lewin, S. A., Carlsen, B., Glenton, C., Colvin, C. J., Garside, R., Bohren, M. A., Rashidian, A., Wainwright, M., et al. (2018) · Implementation Science · doi:10.1186/s13012-017-0693-6 · Cited by 266 · see citing works
Self-citationBooth, A., & Carroll, C. (2015) · BMJ Quality & Safety · doi:10.1136/bmjqs-2014-003642 · Cited by 262 · see citing works
Self-citationFormulating questions to explore complex interventions within qualitative evidence synthesis
Booth, A., Noyes, J., Flemming, K., Moore, G., Tunçalp, Ö., & Shakibazadeh, E. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-001107 · Cited by 258 · see citing works
Self-citationDowne, S. M., Finlayson, K. W., Tunçalp, Ö., & Gülmezoglu, A. M. (2015) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.13819 · Cited by 242 · see citing works
Self-citationRehfuess, E. A., Stratil, J. M., Scheel, I. B., Portela, A. G., Norris, S. L., & Baltussen, R. M. P. M. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-000844 · Cited by 209 · see citing works
Wong, S. S.-L., Wilczynski, N. L., & Haynes, R. B. (2004) · PubMed · Cited by 203 · see citing works
Petticrew, M. P., Knai, C., Thomas, J., Rehfuess, E. A., Noyes, J., Gerhardus, A., Grimshaw, J., Rutter, H., & McGill, E. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-000899 · Cited by 201 · see citing works
Self-citationBooth, A., Noyes, J., Flemming, K., Gerhardus, A., Wahlster, P., van der Wilt, G. J., Mozygemba, K., Refolo, P., Sacchini, D., Tummers, M., et al. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.03.003 · Cited by 198 · see citing works
Self-citationA methodological systematic review of what’s wrong with meta-ethnography reporting
France, E. F., Ring, N., Thomas, R., Noyes, J., Maxwell, M., & Jepson, R. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-119 · Cited by 183 · see citing works
Self-citationHarris, J. L., Booth, A., Cargo, M., Hannes, K., Harden, A., Flemming, K., Garside, R., Pantoja, T., Thomas, J., & Noyes, J. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.023 · Cited by 165 · see citing works
Self-citationBooth, A. (2016) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.13140/rg.2.1.2318.0562 · Cited by 151 · see citing works
Self-citationFlemming, K., Graham, H. M., Heirs, M. K., Fox, D., & Sowden, A. J. (2012) · Journal of Advanced Nursing · doi:10.1111/jan.12066 · Cited by 138 · see citing works
Self-citationFlemming, K., Booth, A., Hannes, K., Cargo, M., & Noyes, J. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.022 · Cited by 128 · see citing works
Self-citationFlemming, K., McCaughan, D., Angus, K., & Graham, H. M. (2014) · Journal of Advanced Nursing · doi:10.1111/jan.12580 · Cited by 116 · see citing works
Self-citationBaxter, S. K., Killoran, A., Kelly, M. P., & Goyder, E. (2010) · Public Health · doi:10.1016/j.puhe.2010.01.002 · Cited by 116 · see citing works
Qualitative evidence synthesis to improve implementation of clinical guidelines
Carroll, C. (2017) · BMJ · doi:10.1136/bmj.j80 · Cited by 90 · see citing works
Protocol-developing meta-ethnography reporting guidelines (eMERGe)
France, E. F., Ring, N., Noyes, J., Maxwell, M., Jepson, R. G., Duncan, E., Turley, R. L., Jones, D., & Uny, I. (2015) · BMC Medical Research Methodology · doi:10.1186/s12874-015-0068-0 · Cited by 82 · see citing works
Self-citationGarside, R., Britten, N., & Stein, K. (2008) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2008.04750.x · Cited by 68 · see citing works
Self-citationThe role of systematic reviews of qualitative evidence in evaluating interventions: a case study
Lorenc, T., Pearson, M., Jamal, F., Cooper, C., & Garside, R. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1036 · Cited by 61 · see citing works
Self-citationNoyes, J., Hendry, M., Booth, A., Chandler, J., Lewin, S. A., Glenton, C., & Garside, R. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.12.009 · Cited by 58 · see citing works
Self-citationKelly, M. P., Noyes, J., Kane, R. L., Chang, C., Uhl, S., Robinson, K. A., Springs, S., Butler, M. E., & Guise, J.-M. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.06.012 · Cited by 49 · see citing works
Self-citationThe synthesis of qualitative research and evidence-based nursing
Flemming, K. (2007) · Evidence-Based Nursing · doi:10.1136/ebn.10.3.68 · Cited by 45 · see citing works
Self-citationNoyes, J., Hendry, M., Lewin, S. A., Glenton, C., Chandler, J., & Rashidian, A. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.01.009 · Cited by 25 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- [35]World Health Organization (WHO). WHO recommendations: optimizing health worker roles to improve access to key maternal and newborn health interventions through task shifting. 2012 http://apps.who.int/iris/bitstream/10665/77764/1/9789241504843_eng.pdf. 23844452
- [40]WHO. WHO recommendations on antenatal care for a positive pregnancy experience. 2016 http://apps.who.int/iris/bitstream/10665/250796/1/9789241549912-eng.pdf (accessed 23 Oct 17). 28079998
- [42]WHO. WHO recommendations for the prevention and management of tobacco use and second-hand smoke exposure in pregnancy. 2013 http://apps.who.int/iris/bitstream/10665/94555/1/9789241506076_eng.pdf?ua=1. 24649520
Cited by
Sorted by citations
The ABC of systematic literature review: the basic methodological guidance for beginners
Mohamed Shaffril, H. A., Samsuddin, S. F., & Abu Samah, A. (2021) · Quality & Quantity · doi:10.1007/s11135-020-01059-6 · Cited by 656 · see citing works
Noyes, J., Booth, A., Flemming, K., Garside, R., Harden, A., Lewin, S., Pantoja, T., Hannes, K., Cargo, M., & Thomas, J. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.06.020 · Cited by 480 · see citing works
Self-citationNoyes, J., Booth, A., Moore, G., Flemming, K., Tunçalp, Ö., & Shakibazadeh, E. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-000893 · Cited by 497 · see citing works
Self-citationSynthesising quantitative evidence in systematic reviews of complex health interventions
Higgins, J. P. T., López-López, J. A., Becker, B. J., Davies, S. R., Dawson, S., Grimshaw, J. M., McGuinness, L. A., Moore, T. H. M., Rehfuess, E. A., Thomas, J., et al. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-000858 · Cited by 322 · see citing works
Formulating questions to explore complex interventions within qualitative evidence synthesis
Booth, A., Noyes, J., Flemming, K., Moore, G., Tunçalp, Ö., & Shakibazadeh, E. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-001107 · Cited by 258 · see citing works
Self-citationNiño de Guzmán Quispe, E., Martínez García, L., Orrego Villagrán, C., Heijmans, M., Sunol, R., Fraile-Navarro, D., Pérez-Bracchiglione, J., Ninov, L., Salas-Gama, K., Viteri García, A., et al. (2021) · The Patient - Patient-Centered Outcomes Research · doi:10.1007/s40271-021-00514-2 · Cited by 54 · see citing works
Afferri, A., Allen, H., Booth, A., Dierickx, S., Pacey, A., & Balen, J. (2022) · Human Reproduction Update · doi:10.1093/humupd/dmab040 · Cited by 41 · see citing works
Self-citationTiraieyari, N., & Krauss, S. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1518053
- Authors
- Flemming, K., Booth, A., Garside, R., Tunçalp, Ö., & Noyes, J.
- Affiliations
- University of York (GB)
- University of Sheffield (GB)
- University of Exeter (GB)
- World Health Organization (CH)
- Bangor University (GB)
- Corresponding author
- Flemming, Kate
- Venue
- BMJ Global Health
- Publisher
- BMJ
- Published
- 2019-01
- Volume
- 4
- Issue
- Suppl 1
- Pages
- e000882
- ISSN
- 2059-7908 (ISSN-L)
- PMC
- PMC6350756
- OpenAlex
- W2912370207
- Cited by
- 389
- References
- 48
- Citations by year
- 2026: 31 · 2025: 51 · 2024: 66 · 2023: 51 · 2022: 79 · 2021: 40 · 2020: 49 · 2019: 21 · 2017: 1
- Type
- conference abstract
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ Global Health (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
- White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Open Research Exeter (University of Exeter) (submittedVersion, cc-by)
- Funders
- United States Agency for International Development
- Department of Maternal, Newborn, Child and Adolescent Health, World Health Organization
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- qualitative evidence synthesis, complex interventions, clinical guideline development, intervention complexity, system complexity, evidence-to-decision framework, equity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review