Cited by 180Mental Health and Patient InvolvementMeta-analysis and systematic reviewsClinical practice guidelines implementation

Stakeholder involvement in systematic reviews: a scoping review

Open access
Pollock, Alex, Campbell, Pauline, Struthers, Caroline, Synnot, Anneliese, Nunn, Jack, Hill, Sophie, Goodare, Heather, Morris, Jacqui +2 more·2018·Systematic Reviews

Abstract

BACKGROUND: There is increasing recognition that it is good practice to involve stakeholders (meaning patients, the public, health professionals and others) in systematic reviews, but limited evidence about how best to do this. We aimed to document the evidence-base relating to stakeholder involvement in systematic reviews and to use this evidence to describe how stakeholders have been involved in systematic reviews. METHODS: We carried out a scoping review, following a published protocol. We searched multiple electronic databases (2010-2016), using a stepwise searching approach, supplemented with hand searching. Two authors independently screened and discussed the first 500 abstracts and, after clarifying selection criteria, screened a further 500. Agreement on screening decisions was 97%, so screening was done by one reviewer only. Pre-planned data extraction was completed, and the comprehensiveness of the description of methods of involvement judged. Additional data extraction was completed for papers judged to have most comprehensive descriptions. Three stakeholder representatives were co-authors for this systematic review. RESULTS: We included 291 papers in which stakeholders were involved in a systematic review. Thirty percent involved patients and/or carers. Thirty-two percent were from the USA, 26% from the UK and 10% from Canada. Ten percent (32 reviews) were judged to provide a comprehensive description of methods of involving stakeholders. Sixty-nine percent (22/32) personally invited people to be involved; 22% (7/32) advertised opportunities to the general population. Eighty-one percent (26/32) had between 1 and 20 face-to-face meetings, with 83% of these holding ≤ 4 meetings. Meetings lasted 1 h to ½ day. Nineteen percent (6/32) used a Delphi method, most often involving three electronic rounds. Details of ethical approval were reported by 10/32. Expenses were reported to be paid to people involved in 8/32 systematic reviews. DISCUSSION/CONCLUSION: We identified a relatively large number (291) of papers reporting stakeholder involvement in systematic reviews, but the quality of reporting was generally very poor. Information from a subset of papers judged to provide the best descriptions of stakeholder involvement in systematic reviews provide examples of different ways in which stakeholders have been involved in systematic reviews. These examples arguably currently provide the best available information to inform and guide decisions around the planning of stakeholder involvement within future systematic reviews. This evidence has been used to develop online learning resources. SYSTEMATIC REVIEW REGISTRATION: The protocol for this systematic review was published on 21 April 2017. Publication reference: Pollock A, Campbell P, Struthers C, Synnot A, Nunn J, Hill S, Goodare H, Watts C, Morley R: Stakeholder involvement in systematic reviews: a protocol for a systematic review of methods, outcomes and effects. Research Involvement and Engagement 2017, 3:9. https://doi.org/10.1186/s40900-017-0060-4 .

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13643-018-0852-0

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Engaging patients in designing the review protocol and reviewing the findings increases applicability and relevance of the findings.

    “Engaging patients in the process of designing the review protocol and in reviewing the findings increase the applicability and relevance of the findings of this study.”

    From Systematic review of mind‐body interventions to treat myalgic encephalomyelitis/chronic fatigue syndrome (2021) · cites this paper as [37]

    The Background says stakeholder involvement is widely accepted as beneficial to the relevance of health research, and the Results describe examples of patients contributing to review planning and discussion of findings. But the Results also say none of the 32 studies formally evaluated the impact of involvement. The source therefore does not establish that these activities increased the applicability and relevance of this study’s findings. (LLM, conf=0.86)

    Experimental — accuracy not yet measured
  • Stakeholder engagement underlines the concept of integrated knowledge translation.

    “In fact, it underlines the concept of integrated knowledge translation [37, 38].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. [15]Cochrane Consumer Network. Cochrane consumer network structure and function review, 2015. https://consumers.cochrane.org/ccnet-structure-and-function-review. Accessed 6 Nov 2018.
  2. [60]Involving people: a learning resource for systematic review authors. http://training.cochrane.org/involving-people. Accessed 6 Nov 2018.

Cited by

Sorted by citations

Authors
Pollock, A., Campbell, P., Struthers, C., Synnot, A., Nunn, J., Hill, S., Goodare, H., Morris, J., Watts, C., & Morley, R.
Affiliations
  • Glasgow Caledonian University (GB)
  • University of Oxford (GB)
  • La Trobe University (AU)
  • University of Dundee (GB)
  • Cochrane (GB)
Corresponding author
Pollock, Alex
Venue
Systematic Reviews
Published
2018-11-24
Volume
7
Issue
1
Pages
208-208
ISSN
2046-4053 (ISSN-L)
OpenAlex
W2900615641
Cited by
180
References
56
Citations by year
2026: 20 · 2025: 13 · 2024: 43 · 2023: 29 · 2022: 20 · 2021: 28 · 2020: 17 · 2019: 8 · 2018: 2
FWCI
2.50
Type
journal article
Language
en
Open access
Locations
  • Systematic Reviews (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Discovery Research Portal (University of Dundee) (acceptedVersion, cc-by)
  • ResearchOnline (Glasgow Caledonian University) (acceptedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Cochrane Central Executive (n/a)
Topics
  • Health Sciences › Health Professions › General Health Professions › Mental Health and Patient Involvement
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
Keywords
stakeholder engagement, patient and public involvement, co-authorship, Delphi method, reporting quality, research ethics approval, healthcare workers, online learning resources, methodological transparency
MeSH terms
Stakeholder Participation, Systematic Reviews as Topic, Humans, Patients, Research Design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review