The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Stakeholder involvement in systematic reviews: a scoping review
Open accessAbstract
BACKGROUND: There is increasing recognition that it is good practice to involve stakeholders (meaning patients, the public, health professionals and others) in systematic reviews, but limited evidence about how best to do this. We aimed to document the evidence-base relating to stakeholder involvement in systematic reviews and to use this evidence to describe how stakeholders have been involved in systematic reviews. METHODS: We carried out a scoping review, following a published protocol. We searched multiple electronic databases (2010-2016), using a stepwise searching approach, supplemented with hand searching. Two authors independently screened and discussed the first 500 abstracts and, after clarifying selection criteria, screened a further 500. Agreement on screening decisions was 97%, so screening was done by one reviewer only. Pre-planned data extraction was completed, and the comprehensiveness of the description of methods of involvement judged. Additional data extraction was completed for papers judged to have most comprehensive descriptions. Three stakeholder representatives were co-authors for this systematic review. RESULTS: We included 291 papers in which stakeholders were involved in a systematic review. Thirty percent involved patients and/or carers. Thirty-two percent were from the USA, 26% from the UK and 10% from Canada. Ten percent (32 reviews) were judged to provide a comprehensive description of methods of involving stakeholders. Sixty-nine percent (22/32) personally invited people to be involved; 22% (7/32) advertised opportunities to the general population. Eighty-one percent (26/32) had between 1 and 20 face-to-face meetings, with 83% of these holding ≤ 4 meetings. Meetings lasted 1 h to ½ day. Nineteen percent (6/32) used a Delphi method, most often involving three electronic rounds. Details of ethical approval were reported by 10/32. Expenses were reported to be paid to people involved in 8/32 systematic reviews. DISCUSSION/CONCLUSION: We identified a relatively large number (291) of papers reporting stakeholder involvement in systematic reviews, but the quality of reporting was generally very poor. Information from a subset of papers judged to provide the best descriptions of stakeholder involvement in systematic reviews provide examples of different ways in which stakeholders have been involved in systematic reviews. These examples arguably currently provide the best available information to inform and guide decisions around the planning of stakeholder involvement within future systematic reviews. This evidence has been used to develop online learning resources. SYSTEMATIC REVIEW REGISTRATION: The protocol for this systematic review was published on 21 April 2017. Publication reference: Pollock A, Campbell P, Struthers C, Synnot A, Nunn J, Hill S, Goodare H, Watts C, Morley R: Stakeholder involvement in systematic reviews: a protocol for a systematic review of methods, outcomes and effects. Research Involvement and Engagement 2017, 3:9. https://doi.org/10.1186/s40900-017-0060-4 .
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In fact, it underlines the concept of integrated knowledge translation [37, 38].”
From Scoping reviews: reinforcing and advancing the methodology and application (2021) · cites this paper as [37]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Engaging patients in the process of designing the review protocol and in reviewing the findings increase the applicability and relevance of the findings of this study.”
From Systematic review of mind‐body interventions to treat myalgic encephalomyelitis/chronic fatigue syndrome (2021) · cites this paper as [37]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scoping studies: towards a methodological framework
Arksey, H., & O'Malley, L. (2005) · International Journal of Social Research Methodology · doi:10.1080/1364557032000119616 · Cited by 36,742
Scoping studies: advancing the methodology
Levac, D., Colquhoun, H., & O'Brien, K. K. (2010) · Implementation Science · doi:10.1186/1748-5908-5-69 · Cited by 15,268
Scoping reviews: Time for clarity in definition, methods, and reporting
Colquhoun, H. L., Levac, D., O'Brien, K. K., Straus, S., Tricco, A. C., Perrier, L., Kastner, M., & Moher, D. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.03.013 · Cited by 2,783
Staniszewska, S., Brett, J., Simera, I., Seers, K., Mockford, C., Goodlad, S., Altman, D. G., Moher, D., Barber, R., Denegri, S., et al. (2017) · BMJ · doi:10.1136/bmj.j3453 · Cited by 1,983
Patient engagement in research: a systematic review
Domecq, J. P., Prutsky, G., Elraiyah, T., Wang, Z., Nabhan, M., Shippee, N., Brito, J. P., Boehmer, K., Hasan, R., Firwana, B., et al. (2014) · BMC Health Services Research · doi:10.1186/1472-6963-14-89 · Cited by 1,773
Staniszewska, S., Brett, J., Simera, I., Seers, K., Mockford, C., Goodlad, S., Altman, D. G., Moher, D., Barber, R., Denegri, S., et al. (2017) · Research Involvement and Engagement · doi:10.1186/s40900-017-0062-2 · Cited by 1,643 · cited 1×
“stakeholder involvement in research has recently been updated (GRIPP2, [62]); however, this guidance has not been specifically tested for use with systematic reviews and has lacked international input during development.”
BackgroundA New Taxonomy for Stakeholder Engagement in Patient-Centered Outcomes Research
Concannon, T. W., Meißner, P., Grunbaum, J. A., McElwee, N. E., Guise, J., Santa, J. S., Conway, P. H., Daudelin, D. H., Morrato, E. H., & Leslie, L. K. (2012) · Journal of General Internal Medicine · doi:10.1007/s11606-012-2037-1 · Cited by 532
Concannon, T. W., Fuster, M., Saunders, T. S., Patel, K., Wong, J. B., Leslie, L. K., & Lau, J. (2014) · Journal of General Internal Medicine · doi:10.1007/s11606-014-2878-x · Cited by 470
Systematic review of tools to measure outcomes for young children with autism spectrum disorder
McConachie, H., Parr, J. R., Glod, M., Hanratty, J., Livingstone, N., Oono, I. P., Robalino, S., Baird, G., Beresford, B., Charman, T., et al. (2015) · Health Technology Assessment · doi:10.3310/hta19410 · Cited by 403
Staniszewska, S., Brett, J., Mockford, C., & Barber, R. (2011) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462311000481 · Cited by 369
Higginson, I. J., Evans, C., Grande, G., Preston, N. J., Morgan, M., McCrone, P., Lewis, P. J., Fayers, P. M., Harding, R., Hotopf, M. H., et al. (2013) · BMC Medicine · doi:10.1186/1741-7015-11-111 · Cited by 338
Involving the public in NHS public health, and social care research: briefing notes for researchers
Hanley, B., Bradburn, J., Barnes, M., Evans, C., Goodare, H., Kelson, M., Kent, A., Oliver, S. R., Thomas, S. E., & Wallcraft, J. (2004) · Cited by 283
Self-citationPearson, M., Chilton, R., Wyatt, K. M., Abraham, C., Ford, T. J., Woods, H., & Anderson, R. (2015) · Implementation Science · doi:10.1186/s13012-015-0338-6 · Cited by 247
Liu, J., Davidson, E., Bhopal, R., White, M., Johnson, M., Netto, G., Deverill, M., & Sheikh, A. (2012) · Health Technology Assessment · doi:10.3310/hta16440 · Cited by 237 · cited 5×
“which they were encouraged to share or reflect on their experiences and perspectives [31, 33, 47]; and two had”
BackgroundMethodInfluentialCamden, C., Shikako‐Thomas, K., Nguyen, T., Graham, E. L., Thomas, A., Sprung, J., Morris, C., & Russell, D. J. (2014) · Disability and Rehabilitation · doi:10.3109/09638288.2014.963705 · Cited by 232
Boote, J. D., Wong, R., & Booth, A. (2012) · Health Expectations · doi:10.1111/hex.12007 · Cited by 216
How to incorporate patient and public perspectives into the design and conduct of research
Hoddinott, P., Pollock, A., O’Cathain, A., Boyer, I., Taylor, J., MacDonald, C., Oliver, S. R., & Donovan, J. (2018) · F1000Research · doi:10.12688/f1000research.15162.1 · Cited by 207
Self-citationCollaborative research and the co-production of knowledge for practice: an illustrative case study
Heaton, J., Day, J., & Britten, N. (2015) · Implementation Science · doi:10.1186/s13012-016-0383-9 · Cited by 202
Survival of patients with breast cancer attending Bristol Cancer Help Centre
Chilvers, C. E. D., Bagenal, F. S., Easton, D. F., Harris, E., & McElwain, T. J. (1990) · The Lancet · doi:10.1016/0140-6736(90)93402-b · Cited by 132
Boote, J. D., Baird, W. O., & Sutton, A. (2011) · Health Policy · doi:10.1016/j.healthpol.2011.05.002 · Cited by 131
Wallace, J., Nwosu, B., & Clarke, M. J. (2012) · BMJ Open · doi:10.1136/bmjopen-2012-001220 · Cited by 116
Smith, E., Ross, F., Donovan, S., Manthorpe, J., Brearley, S., Sitzia, J., & Beresford, P. (2008) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2006.09.010 · Cited by 113
Morgan, H. M., Hoddinott, P., Thomson, G., Crossland, N., Farrar, S., Yi, D., Hislop, J., Morán, V. H., MacLennan, G., Dombrowski, S. U., et al. (2015) · Health Technology Assessment · doi:10.3310/hta19300 · Cited by 110
Pollock, A., Campbell, P., Deery, R., Fleming, M. P., Rankin, J., Sloan, G., & Cheyne, H. (2017) · Journal of Advanced Nursing · doi:10.1111/jan.13253 · Cited by 104
Self-citationDudley, L., Gamble, C. L., Allam, A., Bell, P. A., Buck, D., Goodare, H., Hanley, B., Preston, J., Walker, A., Williamson, P. R., et al. (2015) · Trials · doi:10.1186/s13063-015-0667-4 · Cited by 99
Self-citationHow stakeholder participation can contribute to systematic reviews of complex interventions
Harris, J., Croot, L., Thompson, J., & Springett, J. (2015) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2015-205701 · Cited by 98
Consumer involvement in systematic reviews of comparative effectiveness research
Kreis, J., Puhan, M. A., Schünemann, H. J., & Dickersin, K. (2013) · Health Expectations · doi:10.1111/j.1369-7625.2011.00722.x · Cited by 88
Co-production from proposal to paper
Hickey, G., Richards, T., & Sheehy, J. (2018) · Nature · doi:10.1038/d41586-018-06861-9 · Cited by 80
McGinn, C. A., Gagnon, M., Shaw, N. T., Sicotte, C., Mathieu, L., Leduc, Y., Grenier, S., Duplantie, J., Abdeljelil, A. B., & Légaré, F. (2012) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-12-105 · Cited by 76
Serrano-Aguilar, P., Trujillo-Martín, M. M., Ramos-Goñi, J. M., Mahtani‐Chugani, V., Perestelo‐Pérez, L., & de la Paz, M. P. (2009) · Social Science & Medicine · doi:10.1016/j.socscimed.2009.07.005 · Cited by 74
Emerging from out of the shadows? Service user and carer involvement in systematic reviews
Braye, S., & Preston‐Shoot, M. (2005) · Evidence & Policy · doi:10.1332/1744264053730743 · Cited by 63
What are funders doing to minimise waste in research?
Nasser, M., Clarke, M. J., Chalmers, I., Brurberg, K. G., Nykvist, H., Lund, H., & Glasziou, P. (2017) · The Lancet · doi:10.1016/s0140-6736(17)30657-8 · Cited by 62
Pollock, A., Campbell, P., Struthers, C., Synnot, A., Nunn, J., Hill, S., Goodare, H., Watts, C., & Morley, R. (2017) · Research Involvement and Engagement · doi:10.1186/s40900-017-0060-4 · Cited by 52
Self-citationMorley, R., Norman, G., Golder, S., & Griffith, P. (2016) · Research Involvement and Engagement · doi:10.1186/s40900-016-0049-4 · Cited by 46
Self-citationHyde, C., Dunn, K. M., Higginbottom, A., & Chew-Graham, C. A. (2016) · Health Expectations · doi:10.1111/hex.12458 · Cited by 45
Edwards, D. J., Evans, N., Gillen, E., Longo, M., Pryjmachuk, S., Trainor, G., & Hannigan, B. (2015) · Child and Adolescent Psychiatry and Mental Health · doi:10.1186/s13034-015-0087-y · Cited by 42
Pollock, A., Campbell, P., Baer, G. D., Choo, P. L., Morris, J. H., & Förster, A. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0023-5 · Cited by 41
Self-citationPatient involvement in a qualitative meta-synthesis: lessons learnt
Bayliss, K., Starling, B., Raza, K., Johansson, E. C., Zăbălan, C., Moore, S., Skingle, D., Jasinski, T., Thomas, S., & Stack, R. J. (2016) · Research Involvement and Engagement · doi:10.1186/s40900-016-0032-0 · Cited by 37
Rees, R. W., Kavanagh, J., Burchett, H., Shepherd, J., Brunton, G., Harden, A., Thomas, J., Oliver, S. R., & Oakley, A. (2004) · IOE EPrints · Cited by 35
Oliver, K. A., Rees, R. W., Brady, L., Kavanagh, J., Oliver, S. R., & Thomas, J. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1145 · Cited by 34
Measuring a caring culture in hospitals: a systematic review of instruments
Hesselink, G., Kuis, E., Pijnenburg, M. A. M., & Wollersheim, H. C. H. (2013) · BMJ Open · doi:10.1136/bmjopen-2013-003416 · Cited by 28
Bunn, F., Sworn, K., Brayne, C., Iliffe, S., Robinson, L. A., & Goodman, C. M. (2013) · Health Expectations · doi:10.1111/hex.12162 · Cited by 26
Jepson, R. G., Harris, F., MacGillivray, S. A., Kearney, N., & Rowa‐Dewar, N. (2006) · Stirling Online Research Repository (University of Stirling) · Cited by 25
Martin, S., Fleming, J., Cullum, S. J., Dening, T., Rait, G., Fox, C., Katona, C., Brayne, C., & Lafortune, L. (2015) · BMC Geriatrics · doi:10.1186/s12877-015-0100-6 · Cited by 24
Coon, J. T., Gwernan‐Jones, R., Moore, D. A., Richardson, M. A., Shotton, C., Pritchard, W., Morris, C., Stein, K., & Ford, T. J. (2015) · Health Expectations · doi:10.1111/hex.12400 · Cited by 23
Jamal, F., Langford, R. M., Daniels, P., Thomas, J., Harden, A., & Bonell, C. P. (2014) · Health Expectations · doi:10.1111/hex.12312 · Cited by 23
Vale, C. L., Nightingale, A. L., Spera, N., Whelan, A., Hanley, B., & Tierney, J. F. (2010) · Clinical Oncology · doi:10.1016/j.clon.2010.05.017 · Cited by 19
Hayden, J. A., Killian, L., Zygmunt, A., Babineau, J., Martin‐Misener, R., Jensen, J. L., & Carter, A. J. E. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-7 · Cited by 18
Bond, K. S., Chalmers, K., Jorm, A. F., Kitchener, B. A., & Reavley, N. J. (2015) · BMC Health Services Research · doi:10.1186/s12913-015-0868-2 · Cited by 17
What is known about communication with parents about newborn bloodspot screening
Stewart, R. J., & Oliver, S. R. (2003) · Cited by 11
Boelens, P. G., Taylor, C., Henning, G., de Mheen, P. J. M., Espín, E., Wiggers, T., Gore-Booth, J., Moss, B., Valentini, V., & van de Velde, C. J. �. (2014) · Patient · doi:10.1007/s40271-014-0061-4 · Cited by 11
Fighting Spirit: the Stories of Women in the Bristol Breast Cancer Survey
Harris, M., & Harris, V. S. (2002) · Cited by 10
Caries preventive measures in orthodontic practice: the development of a clinical practice guideline
Oosterkamp, B. C. M., van der Sanden, W. J. M., Frencken, J. E., & Kuijpers‐Jagtman, A. M. (2015) · Orthodontics and Craniofacial Research · doi:10.1111/ocr.12107 · Cited by 10
McCusker, J., Yaffe, M., Sussman, T., Kates, N., Mulvale, G., Jayabarathan, A., Law, S., & Haggerty, J. (2013) · Canadian journal of psychiatry. Revue canadienne de psychiatrie · doi:10.1177/070674371305800306 · Cited by 9
Recording and Accounting for Stakeholder Involvement in Systematic Reviews
Saan, M. C., Boeije, H., Sattoe, J. N. T., Bal, M. I., Missler, M. A., & van Wesel, F. (2015) · Health Information & Libraries Journal · doi:10.1111/hir.12099 · Cited by 8
Liabo, K. (2013) · IOE EPrints · Cited by 4
Research works printed but not linked: 2
- Cochrane Consumer Network. Cochrane consumer network structure and function review, 2015. https://consumers.cochrane.org/ccnet-structure-and-function-review . Accessed 6 Nov 2018.
- Involving people: a learning resource for systematic review authors. http://training.cochrane.org/involving-people . Accessed 6 Nov 2018.
Cited by
Sorted by citations
Scoping reviews: reinforcing and advancing the methodology and application
Peters, M. D. J., Marnie, C., Colquhoun, H., Garritty, C. M., Hempel, S., Horsley, T., Langlois, E. V., Lillie, E., O’Brien, K. K., Tunçalp, Ӧ., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01821-3 · Cited by 887
Campbell, F., Tricco, A. C., Munn, Z., Pollock, D., Saran, A., Sutton, A., White, H., & Khalil, H. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02178-5 · Cited by 345
Artificial intelligence and the conduct of literature reviews
Wagner, G., Lukyanenko, R., & Paré, G. (2021) · Journal of Information Technology · doi:10.1177/02683962211048201 · Cited by 364
Petkovic, J., Riddle, A. Y., Akl, E. A., Khabsa, J., Lytvyn, L., Atwere, P., Campbell, P., Chalkidou, K., Chang, S., Crowe, S., et al. (2020) · Systematic Reviews · doi:10.1186/s13643-020-1272-5 · Cited by 241
Self-citationInnovations in framework synthesis as a systematic review method
Brunton, G., Oliver, S. R., & Thomas, J. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1399 · Cited by 254
Metadata
- Authors
- Pollock, A., Campbell, P., Struthers, C., Synnot, A., Nunn, J., Hill, S., Goodare, H., Morris, J., Watts, C., & Morley, R.
- Affiliations
- Glasgow Caledonian University (GB)
- University of Oxford (GB)
- La Trobe University (AU)
- University of Dundee (GB)
- Cochrane (GB)
- Corresponding author
- Pollock, Alex
- ORCID
- Venue
- Systematic Reviews
- Published
- 2018-11-24
- Volume
- 7
- Issue
- 1
- Pages
- 208-208
- ISSN
- 2046-4053 (ISSN-L)
- PMC
- PMC6260873
- OpenAlex
- W2900615641
- Cited by
- 180
- References
- 56
- Citations by year
- 2026: 20 · 2025: 13 · 2024: 43 · 2023: 29 · 2022: 20 · 2021: 28 · 2020: 17 · 2019: 8 · 2018: 2
- FWCI
- 2.50
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Systematic Reviews (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Discovery Research Portal (University of Dundee) (acceptedVersion, cc-by)
- ResearchOnline (Glasgow Caledonian University) (acceptedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Cochrane Central Executive (n/a)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mental Health and Patient Involvement
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- stakeholder engagement, patient and public involvement, co-authorship, Delphi method, reporting quality, research ethics approval, healthcare workers, online learning resources, methodological transparency
- MeSH terms
- Stakeholder Participation, Systematic Reviews as Topic, Humans, Patients, Research Design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
