Cited by 497Meta-analysis and systematic reviewsHealth Policy Implementation ScienceClinical practice guidelines implementation

Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods

Open access
Noyes, Jane, Booth, Andrew, Moore, Graham, Flemming, Kate, Tunçalp, Özge, Shakibazadeh, Elham·2019·BMJ Global Health

Abstract

Guideline developers are increasingly dealing with more difficult decisions concerning whether to recommend complex interventions in complex and highly variable health systems. There is greater recognition that both quantitative and qualitative evidence can be combined in a mixed-method synthesis and that this can be helpful in understanding how complexity impacts on interventions in specific contexts. This paper aims to clarify the different purposes, review designs, questions, synthesis methods and opportunities to combine quantitative and qualitative evidence to explore the complexity of complex interventions and health systems. Three case studies of guidelines developed by WHO, which incorporated quantitative and qualitative evidence, are used to illustrate possible uses of mixed-method reviews and evidence. Additional examples of methods that can be used or may have potential for use in a guideline process are outlined. Consideration is given to the opportunities for potential integration of quantitative and qualitative evidence at different stages of the review and guideline process. Encouragement is given to guideline commissioners and developers and review authors to consider including quantitative and qualitative evidence. Recommendations are made concerning the future development of methods to better address questions in systematic reviews and guidelines that adopt a complexity perspective.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/bmjgh-2018-000893

Cites

63 of 63 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The first author read all charted data before coding each study's findings to preserve its original context and ensure that its findings could be fully understood without misinterpretation.

    “the first author read all the charted data (including study aims, methods, and samples) before coding each study’s findings to preserve its original context and ensure that its findings could be fully understood without misinterpretation [39].”

    From Experiences of Using Digital Mindfulness-Based Interventions: Rapid Scoping Review and Thematic Synthesis (2023) · cites this paper as [39]

    The ‘Types of findings produced by specific methods’ section says that third-order interpretations preserve studies’ original context, and ‘Data extraction’ notes that a synthesis framework may arise from reading included studies early in a review. Neither passage specifies that a first author should read all charted data before coding, nor says that doing so ensures findings are understood without misinterpretation. No page numbers were available in our copy. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. [2]de Savigny D, Borghi J, Windish R. Chapter 3. Systems thinging: applying a systems perspective to design and evaluate health systems interventions. In: de Savigny D, Tagheed A, eds. Systems thinging for health systems strengtheing. Geneva: World Health Organisation, 2009.
  2. [12]Fletcher A, Jamal F, Moore G, et al. Realist complex intervention science: applying realist principles across all phases of the medical research council framework for developing and evaluating complex interventions. Evaluation 2016; 22: 286– 303. 10.1177/1356389016652743 27478401 PMC4946011 · DOI · Paper page

Cited by

Sorted by citations

Authors
Noyes, J., Booth, A., Moore, G., Flemming, K., Tunçalp, Ö., & Shakibazadeh, E.
Affiliations
  • Bangor University (GB)
  • University of Sheffield (GB)
  • Cardiff University (GB)
  • University of York (GB)
  • World Health Organization (CH)
  • Tehran University of Medical Sciences (IR)
Corresponding author
Noyes, Jane
Venue
BMJ Global Health
Publisher
BMJ
Published
2019-01
Volume
4
Issue
Suppl 1
Pages
e000893
ISSN
2059-7908 (ISSN-L)
OpenAlex
W2906715785
Cited by
497
References
63
Citations by year
2026: 70 · 2025: 80 · 2024: 83 · 2023: 66 · 2022: 51 · 2021: 83 · 2020: 49 · 2019: 13 · 2015: 1 · 2012: 1
Type
conference abstract
Language
en
Open access
Locations
  • BMJ Global Health (publishedVersion, cc-by-nc)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
  • ORCA Online Research @Cardiff (Cardiff University) (acceptedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
Funding provided by the World Health Organization Department of Maternal, Newborn, Child and Adolescent Health through a grant received from the United States Agency for International Development.
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
Keywords
complex interventions, evidence synthesis, clinical guideline development, WHO guidelines, complexity perspective

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review