Exploring areas of consensus and conflict around values underpinning public involvement in health and social care research: a modified Delphi study
Open accessAbstract
OBJECTIVE: There is growing interest in the potential benefits of public involvement (PI) in health and social care research. However, there has been little examination of values underpinning PI or how these values might differ for different groups with an interest in PI in the research process. We aimed to explore areas of consensus and conflict around normative, substantive and process-related values underpinning PI. DESIGN: Mixed method, three-phase, modified Delphi study, conducted as part of a larger multiphase project. SETTING: The UK health and social care research community. PARTICIPANTS: Stakeholders in PI in research, defined as: clinical and non-clinical academics, members of the public, research managers, commissioners and funders; identified via research networks, online searches and a literature review. RESULTS: We identified high levels of consensus for many normative, substantive and process-related issues. However, there were also areas of conflict in relation to issues of bias and representativeness, and around whether the purpose of PI in health and social care research is to bring about service change or generate new knowledge. There were large differences by group in the percentages endorsing the ethical justification for PI and the argument that PI equalises power imbalances. With regard to practical implementation of PI, research support infrastructures were reported as lacking. Participants reported shortcomings in the uptake and practice of PI. Embedding PI practice and evaluation in research study designs was seen as fundamental to strengthening the evidence base. CONCLUSIONS: Our findings highlight the extent to which PI is already embedded in research. However, they also highlight a need for 'best practice' standards to assist research teams to understand, implement and evaluate PI. These findings have been used in developing a Public Involvement Impact Assessment Framework (PiiAF), which offers guidance to researchers and members of the public involved in the PI process.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Conflict or tension, while posing difficulties to resolve, is not necessarily undesirable and, indeed, is evidence of engagement, so much so that where it was managed well, through, for example, clear guidelines, dialogue and consensus, it was seen to add to the strength of the case study in terms of how embedded PPI was.
“Conflict or tension, while posing difficulties to resolve, is not necessarily undesirable and, indeed, is evidence of engagement, 105,174 so much so that where it was managed well, through, for example, clear guidelines, dialogue and consensus, it was seen to add to the strength of the case study in terms of how embedded PPI was.”
From ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study (2015) · cites this paper as [174]
Different value sets are seen to lead to diverse constructions of PPI and can affect whether or not and how PPI is evaluated.
“However, whichever labels are used to define value sets, it is clear that different value sets are seen to lead to diverse constructions of PPI, 28,174 and can affect whether or not and how PPI is evaluated.52”
From ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study (2015) · cites this paper as [174]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
An Experimental Application of the DELPHI Method to the Use of Experts
Dalkey, N., & Helmer, O. (1963) · Management Science · doi:10.1287/mnsc.9.3.458 · Cited by 6,061 · see citing works
Research guidelines for the Delphi survey technique
Hasson, F. (2000) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2000.01567.x · Cited by 5,951 · see citing works
the delphi method: techniques and applications
Linstone, H. A., & Turoff, M. (1975) · Bibliothèque et Archives nationales du Québec (Québec government) · Cited by 5,236 · see citing works
Qualitative data analysis for applied policy research
(2002) · Analyzing Qualitative Data · doi:10.4324/9780203413081-14 · Cited by 4,964 · see citing works
Qualitative data analysis for applied policy research
Ritchie, J., & Spencer, L. (2010) · Analyzing qualitative data · doi:10.4324/9780203413081_chapter_9 · Cited by 4,800 · see citing works
Qualitative Research: Consensus methods for medical and health services research
Jones, J., & Hunter, D. (1995) · BMJ · doi:10.1136/bmj.311.7001.376 · Cited by 3,471 · see citing works
Increasing response rates to postal questionnaires: systematic review
Edwards, P. J., Roberts, I., Clarke, M. J., DiGuiseppi, C. G., Pratap, S., Wentz, R., & Kwan, I. (2002) · BMJ · doi:10.1136/bmj.324.7347.1183 · Cited by 1,915 · see citing works
Currivan, D. B., & Gilbert, N. (1994) · Teaching Sociology · doi:10.2307/1318574 · Cited by 1,736 · see citing works
Crisp, J., Pelletier, D., Duffield, C. M., Adams, A. M. N., & Nagy, S. A. (1997) · Nursing Research · doi:10.1097/00006199-199703000-00010 · Cited by 1,388 · see citing works
A critical review of the Delphi technique as a research methodology for nursing
Keeney, S., Hasson, F., & McKenna, H. (2001) · International Journal of Nursing Studies · doi:10.1016/s0020-7489(00)00044-4 · Cited by 1,333 · see citing works
The Delphi technique: a worthwhile research approach for nursing?
McKenna, H. (1994) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.1994.tb01207.x · Cited by 1,151 · see citing works
Research governance framework for health and social care
Taylor, M. (2002) · Health & Social Care in the Community · doi:10.1046/j.0966-0410.2001.00336.x · Cited by 798 · see citing works
McColl, E., Jacoby, A., Thomas, L., Soutter, J. L., Bamford, C. H., Steen, N., Thomas, R., Harvey, E., Garratt, A., & Bond, J. B. (2001) · Health Technology Assessment · doi:10.3310/hta5310 · Cited by 779 · see citing works
Self-citationResearch methods used in developing and applying quality indicators in primary care
Campbell, S. (2002) · BMJ Quality & Safety · doi:10.1136/qhc.11.4.358 · Cited by 689 · see citing works
The Delphi Method: Techniques and Applications.
Moore, P. G., Lingstone, H. A., & Turoff, M. (1977) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344913 · Cited by 622 · see citing works
Nilsen, E. S., Myrhaug, H. T., Johansen, M., Oliver, S. R., & Oxman, A. D. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004563.pub2 · Cited by 565 · see citing works
The Nominal Group Technique: A Research Tool for General Practice?
Gallagher, M., Hares, T., SPENCER, J., Bradshaw, C., & WEBB, I. (1993) · Family Practice · doi:10.1093/fampra/10.1.76 · Cited by 538 · see citing works
Consumer involvement in health research: a review and research agenda
Boote, J. D., Telford, R., & Cooper, C. L. (2002) · Health Policy · doi:10.1016/s0168-8510(01)00214-7 · Cited by 506 · see citing works
Response rate in patient satisfaction research: an analysis of 210 published studies.
Sitzia, J., & Wood, N. (1998) · International Journal for Quality in Health Care · doi:10.1093/intqhc/10.4.311 · Cited by 430 · see citing works
Applying the Delphi technique in a study of GPs’ information requirements
Green, B. F., Jones, M. L., Hughes, D., et al. (1999) · Health & Social Care in the Community · doi:10.1046/j.1365-2524.1999.00176.x · Cited by 384 · see citing works
Lay perspectives: advantages for health research
Entwistle, V. A., Renfrew, M. J., Yearley, S., Forrester, J. M., & Lamont, T. (1998) · BMJ · doi:10.1136/bmj.316.7129.463 · Cited by 376 · see citing works
Staniszewska, S., Brett, J., Mockford, C., & Barber, R. (2011) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462311000481 · Cited by 369 · see citing works
A multidimensional conceptual framework for analysing public involvement in health services research
Oliver, S. R., Rees, R. W., Clarke‐Jones, L., Milne, R., Oakley, A., Gabbay, J., Stein, K., Buchanan, P., & Gyte, G. M. (2008) · Health Expectations · doi:10.1111/j.1369-7625.2007.00476.x · Cited by 344 · see citing works
The Delphi technique: a comparison of results obtained using two expert panels
Duffield, C. M. (1993) · International Journal of Nursing Studies · doi:10.1016/0020-7489(93)90033-q · Cited by 338 · see citing works
User Involvement in Research and Evaluation: Liberation or Regulation?
Beresford, P. (2002) · Social Policy and Society · doi:10.1017/s1474746402000222 · Cited by 304 · see citing works
Involving the public in NHS public health, and social care research: briefing notes for researchers
Hanley, B., Bradburn, J., Barnes, M., Evans, C., Goodare, H., Kelson, M., Kent, A., Oliver, S. R., Thomas, S. E., & Wallcraft, J. (2004) · Cited by 283 · see citing works
Patients and professionals as research partners: Challenges, practicalities, and benefits
HEWLETT, S. E., de Wit, M. P. T., Richards, P., Quest, E. M., Hughes, R., Heiberg, T., & Kirwan, J. (2006) · Arthritis Care & Research · doi:10.1002/art.22091 · Cited by 272 · see citing works
Boote, J., Baird, W., & Beecroft, C. (2010) · Health Policy · doi:10.1016/j.healthpol.2009.11.007 · Cited by 234 · see citing works
The Delphi method: a useful tool for the allied health researcher
Walker, A. D. M., & Selfe, J. (1996) · British Journal of Therapy and Rehabilitation · doi:10.12968/bjtr.1996.3.12.14731 · Cited by 226 · see citing works
Values associated with public involvement in health and social care research: a narrative review
Gradinger, F., Britten, N., Wyatt, K., Froggatt, K., Gibson, A., Jacoby, A., Lobban, F., Mayes, D., Snape, D., Rawcliffe, T., et al. (2013) · Health Expectations · doi:10.1111/hex.12158 · Cited by 177 · see citing works
Self-citationBoote, J. D., Baird, W. O., & Sutton, A. (2011) · Health Policy · doi:10.1016/j.healthpol.2011.05.002 · Cited by 131 · see citing works
Disabled People, Service Users, User Involvement and Representation
Beresford, P., & Campbell, J. (1994) · Disability & Society · doi:10.1080/09687599466780361 · Cited by 129 · see citing works
Can the impact of public involvement on research be evaluated? A mixed methods study
Barber, R., Boote, J. D., Parry, G. D., Cooper, C. L., Yeeles, P., & Cook, S. (2011) · Health Expectations · doi:10.1111/j.1369-7625.2010.00660.x · Cited by 109 · see citing works
Caron-Flinterman, J. F., Broerse, J. E. W., Teerling, J., & Bunders, J. (2005) · Health Expectations · doi:10.1111/j.1369-7625.2005.00337.x · Cited by 108 · see citing works
Boote, J., Barber, R., & Cooper, C. (2006) · Health Policy · doi:10.1016/j.healthpol.2005.03.012 · Cited by 105 · see citing works
The impact of service user involvement in research
Minogue, V., Boness, J., Brown, A., & Girdlestone, J. (2005) · International Journal of Health Care Quality Assurance · doi:10.1108/09526860510588133 · Cited by 104 · see citing works
Hardy, D. J., O’Brien, A. P., Gaskin, C. J., O'Brien, A. J., Morrison‐Ngatai, E., Skews, G., Ryan, T., & McNulty, N. (2004) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2003.02969.x · Cited by 103 · see citing works
Nursing Administration Research Priorities
Lynn, M. R., Layman, E. L., & Englebardt, S. P. (1998) · JONA The Journal of Nursing Administration · doi:10.1097/00005110-199805000-00002 · Cited by 101 · see citing works
Quality indicators for primary care mental health services
Shield, T. (2003) · BMJ Quality & Safety · doi:10.1136/qhc.12.2.100 · Cited by 95 · see citing works
Abma, T., & Widdershoven, G. (2005) · Evaluation & the Health Professions · doi:10.1177/0163278704273080 · Cited by 78 · see citing works
User involvement in research: Exploring the challenges
Beresford, P. (2003) · Journal of research in nursing · doi:10.1177/136140960300800106 · Cited by 73 · see citing works
Yih, J.-P. (1995) · BMJ · doi:10.1136/bmj.311.7000.276 · Cited by 58 · see citing works
Patient and clinician collaboration in the design of a national randomized breast cancer trial
Marsden, J., & Bradburn, J. (2004) · Health Expectations · doi:10.1111/j.1369-7625.2004.00232.x · Cited by 58 · see citing works
Wyatt, K. M., Carter, M., Mahtani, V. M., Barnard, A., Hawton, A., & Britten, N. (2008) · Family Practice · doi:10.1093/fampra/cmn019 · Cited by 51 · see citing works
Self-citationHandbook of Service User Involvement in Nursing and Healthcare Research
Morrow, E. M., Boaz, A., Brearley, S. A., & Ross, F. (2011) · Wiley eBooks · doi:10.1002/9781444347050 · Cited by 46 · see citing works
Panellist consistency in the assessment of medical appropriateness
McDonnell, J., Meijler, A. P., Kahan, J. P., et al. (1996) · Health Policy · doi:10.1016/s0168-8510(96)90021-4 · Cited by 39 · see citing works
Successive surveys of an expert panel: research in decision‐making with health visitors
Lemmer, B. (1998) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.1998.00544.x · Cited by 31 · see citing works
Investigating Consumer Perspectives on Evaluating Health Technologies
Oliver, S. R., Milne, R., Bradburn, J., et al. (2001) · Evaluation · doi:10.1177/13563890122209847 · Cited by 26 · see citing works
A Delphi study to identify healthcare users’ priorities for cancer care in Greece
Efstathiou, N., Ameen, J., & Coll, A.-M. (2008) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2008.04.010 · Cited by 24 · see citing works
The case for user involvement in research: the research priorities of cancer patients
Wright, D. N. M., Corner, J. L., Hopkinson, J. B., et al. (2007) · Breast Cancer Research · doi:10.1186/bcr1801 · Cited by 19 · see citing works
McKenzie, A., & Hanley, B. (2012) · UWA Profiles and Research Repository (UWA) · Cited by 17 · see citing works
Research priorities in vascular nursing
Hatton, J. M., & Nunnelee, J. D. (1995) · Journal of Vascular Nursing · doi:10.1016/s1062-0303(05)80046-9 · Cited by 15 · see citing works
(2017) · Bristol University Press eBooks · doi:10.2307/j.ctt9qgshb.28 · Cited by 14 · see citing works
Sully, P. (2002) · Journal of Interprofessional Care · doi:10.1080/1356182021000008346 · Cited by 13 · see citing works
The Nurse Executive Role in Hong Kong's Transforming Healthcare Environment
Wong, F. K. Y., & Wah, K. Y. (1998) · JONA The Journal of Nursing Administration · doi:10.1097/00005110-199802000-00003 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [26]Dalkey N Helmer O Delphi technique: characteristics and sequence model to the use of experts. Manag Sci 1963; 9: 458– 67
- [57]Peterson RA Albaum G Kerin RA A note on alternative contact strategies in mail surveys. J Market Res Soc 1989; 31: 409– 18
Cited by
Sorted by citations
Greenhalgh, T., Hinton, L., Finlay, T., Macfarlane, A., Fahy, N., Clyde, B., & Chant, A. (2019) · Health Expectations · doi:10.1111/hex.12888 · Cited by 1,059 · see citing works
Evaluating patient and stakeholder engagement in research: moving from theory to practice
Esmail, L., Moore, E., & Rein, A. (2015) · Journal of Comparative Effectiveness Research · doi:10.2217/cer.14.79 · Cited by 502 · see citing works
ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study
Wilson, P., Mathie, E., Keenan, J., McNeilly, E., Goodman, C., Howe, A., Poland, F., Staniszewska, S., Kendall, S., Munday, D., et al. (2015) · Health Services and Delivery Research · doi:10.3310/hsdr03380 · Cited by 297 · see citing works
“Values were a key mechanism in shaping PPI within a research project and, at funder level, a finding consistent with a previous study.(174) Funding organisations often drew a focus on either the moral or the methodological rationale for PPI, and this shaped requirements for PPI within research projects.”
Crocker, J. C., Boylan, A., Bostock, J., & Locock, L. (2017) · Health Expectations · doi:10.1111/hex.12479 · Cited by 194 · see citing works
Ceallaigh, T. J. Ó., O’ Brien, E., Tømte, C., Kulaksız, T., & Connolly, C. (2025) · European Journal of Teacher Education · doi:10.1080/02619768.2025.2563696 · Cited by 14 · see citing works
- Authors
- Snape, D., Kirkham, J., Preston, J., Popay, J., Britten, N., Collins, M., Froggatt, K., Gibson, A., Lobban, F., Wyatt, K., & Jacoby, A.
- Affiliations
- University of Liverpool (GB)
- Lancaster University (GB)
- University of Exeter (GB)
- Corresponding author
- Jacoby, A
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2014-01
- Volume
- 4
- Issue
- 1
- Pages
- e004217
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC3902382
- OpenAlex
- W2110674337
- Cited by
- 88
- References
- 55
- Citations by year
- 2026: 5 · 2025: 7 · 2024: 5 · 2023: 4 · 2022: 9 · 2021: 5 · 2020: 7 · 2019: 9 · 2018: 8 · 2017: 8 · 2016: 6 · 2015: 10 · 2014: 4 · 2013: 1
- FWCI
- 16.86
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- BMJ Open (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Lancaster EPrints (Lancaster University) (acceptedVersion, cc-by)
- UWE Research Repository (UWE Bristol) (submittedVersion, cc-by)
- HighWire Press Open Archive (submittedVersion)
- Research Explorer (The University of Manchester) (submittedVersion)
- Funders
- McGill University
- Medical Research Council
- Topics
- Health Sciences › Health Professions › General Health Professions › Mental Health and Patient Involvement
- Health Sciences › Health Professions › General Health Professions › Community Health and Development
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Keywords
- public involvement in research, health and social care research, research value, stakeholder perspectives, consensus and conflict, representativeness, research bias, power imbalance, best practice standards
- MeSH terms
- Community Participation, Delphi Technique, United Kingdom, Health Services Research, Humans, Social Values, Guidelines as Topic, Consensus, Community-Based Participatory Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review