Public involvement in research: assessing impact through a realist evaluation
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Background: This study was concerned with developing the evidence base for public involvement in research in health and social care. There now is significant support for public involvement within the National Institute for Health Research, and researchers applying for National Institute for Health Research grants are expected to involve the public. Despite this policy commitment, evidence for the benefits of public involvement in research remains limited. This study addressed this need through a realist evaluation. Aim and objectives: The aim was to identify the contextual factors and mechanisms that are regularly associated with effective public involvement in research. The objectives included identifying a sample of eight research projects and their desired outcomes of public involvement, tracking the impact of public involvement in these case studies, and comparing the associated contextual factors and mechanisms. Design: The research design was based on the application of realist theory of evaluation, which argues that social programmes are driven by an underlying vision of change – a ‘programme theory’ of how the intervention is supposed to work. The role of the evaluator is to compare theory and practice. Impact can be understood by identifying regularities of context, mechanism and outcome. Thus the key question for the evaluator is ‘What works for whom in what circumstances . . . and why?’ (Pawson R.The Science of Evaluation. London: Sage; 2013). We therefore planned a realist evaluation based on qualitative case studies of public involvement in research. Setting and participants: Eight diverse case studies of research projects in health and social care took place over the calendar year 2012 with 88 interviews from 42 participants across the eight studies: researchers, research managers, third-sector partners and research partners (members of the public involved in research). Results: Case study data supported the importance of some aspects of our theory of public involvement in research and led us to amend other elements. Public involvement was associated with improvements in research design and delivery, particularly recruitment strategies and materials, and data collection tools. This study identified the previously unrecognised importance of principal investigator leadership as a key contextual factor leading to the impact of public involvement; alternatively, public involvement might still be effective without principal investigator leadership where there is a wider culture of involvement. In terms of the mechanisms of involvement, allocating staff time to facilitate involvement appeared more important than formal budgeting. Another important new finding was that many research proposals significantly undercosted public involvement. Nurturing good interpersonal relationships was crucial to effective involvement. Payment for research partner time and formal training appeared more significant for some types of public involvement than others. Feedback to research partners on the value of their contribution was important in maintaining motivation and confidence. Conclusions: A revised theory of public involvement in research was developed and tested, which identifies key regularities of context, mechanism and outcome in how public involvement in research works. Implications for future research include the need to further explore how leadership on public involvement might be facilitated, methodological work on assessing impact and the development of economic analysis of involvement. Funding details: The National Institute for Health Research Health Service and Delivery programme.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Ladder Of Citizen Participation
Arnstein, S. R. (1969) · Journal of the American Institute of Planners · doi:10.1080/01944366908977225 · Cited by 17,064
Qualitative data analysis for applied policy research
Ritchie, J., & Spencer, L. (2010) · Analyzing qualitative data · doi:10.4324/9780203413081_chapter_9 · Cited by 4,800
Pawson, R. (2006) · doi:10.4135/9781849209120 · Cited by 1,791
Brett, J., Staniszewska, S., Mockford, C., Herron‐Marx, S., Hughes, J., Tysall, C., & Suleman, R. (2014) · Health Expectations · doi:10.1111/j.1369-7625.2012.00795.x · Cited by 1,598 · cited 4×
“Brett et al.(30) have subsequently updated their discussion of their review in their 2012 journal paper but their analysis of the state of the art remains substantially the same, that is, that the evidence base is still relatively weak DOI: 10.”
BackgroundInfluentialUsing evidence: How research can inform public services
Nutley, S. M., Walter, I., & Davies, H. T. O. (2007) · doi:10.2307/j.ctt9qgwt1 · Cited by 808
The impact of patient and public involvement on UK NHS health care: a systematic review
Mockford, C., Staniszewska, S., Griffiths, F., & Herron-Marx, S. (2012) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzr066 · Cited by 570
How Do You Modernize a Health Service? A Realist Evaluation of Whole‐Scale Transformation in London
GREENHALGH, T., HUMPHREY, C., HUGHES, J., MACFARLANE, F., BUTLER, C., & PAWSON, R. (2009) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2009.00562.x · Cited by 380
Health researchers’ attitudes towards public involvement in health research
Thompson, J., Barber, R., Ward, P. R., Boote, J. D., Cooper, C. L., Armitage, C. J., & Jones, G. (2009) · Health Expectations · doi:10.1111/j.1369-7625.2009.00532.x · Cited by 208
Staniszewska, S., Adebajo, A., Barber, R., Beresford, P., Brady, L.-M., Brett, J., Elliott, J., Evans, D., Haywood, K. L., Jones, D., et al. (2011) · International Journal of Consumer Studies · doi:10.1111/j.1470-6431.2011.01020.x · Cited by 182
Self-citationWard, P. R., Thompson, J., Barber, R., Armitage, C. J., Boote, J. D., Cooper, C. L., & Jones, G. L. (2010) · Journal of Sociology · doi:10.1177/1440783309351771 · Cited by 164 · cited 3×
“have conducted a qualitative study of health researchers’ attitudes towards public involvement in health research.(14,35) They found researchers held a range of varying constructions of public involvement in research, with a number of benefits being identified but also some feelings of apprehension expressed.”
BackgroundThompson, J., Bissell, P., Cooper, C., Armitage, C. J., & Barber, R. (2012) · Health: An Interdisciplinary Journal for the Social Study of Health, Illness and Medicine · doi:10.1177/1363459312441008 · Cited by 111 · cited 1×
“The nature and implications of professionalised research partners is only just beginning to be explored in the literature.(26) This diversity of extent of professionalisation was also reflected in our case studies, although overall our own research partners were probably nearer the professionalised end of the spectrum…”
ResultStaley, K., Buckland, S. A., Hayes, H., & Tarpey, M. (2014) · Health Expectations · doi:10.1111/hex.12017 · Cited by 94 · cited 2×
“published a paper calling for the application of realist evaluation to the study of the impact of public involvement in research.(20) The development of our realist theory of public involvement in research over the course of our study is described in more detail in Chapter 3.”
BackgroundMethod
Cited by
Greenhalgh, T., Hinton, L., Finlay, T., Macfarlane, A., Fahy, N., Clyde, B., & Chant, A. (2019) · Health Expectations · doi:10.1111/hex.12888 · Cited by 1,059
Crocker, J. C., Ricci-Cabello, I., Parker, A., Hirst, J. A., Chant, A., Petit-Zeman, S., Evans, D., & Rees, S. (2018) · BMJ · doi:10.1136/bmj.k4738 · Cited by 624
Self-citationManafo, E., Petermann, L., Mason-Lai, P., & Vandall-Walker, V. (2018) · Health Research Policy and Systems · doi:10.1186/s12961-018-0282-4 · Cited by 482
Wong, G., Westhorp, G., Greenhalgh, J., Manzano, A., Jagosh, J., & Greenhalgh, T. (2017) · Health Services and Delivery Research · doi:10.3310/hsdr05280 · Cited by 196
Crocker, J. C., Boylan, A., Bostock, J., & Locock, L. (2017) · Health Expectations · doi:10.1111/hex.12479 · Cited by 194
Metadata
- Authors
- Evans, D., Coad, J., Cottrell, K., Dalrymple, J., Davies, R., Donald, C., Laterza, V., Long, A., Longley, A., Moule, P., Pollard, K., Powell, J., Puddicombe, A., Rice, C., & Sayers, R.
- Venue
- Health Services and Delivery Research
- Published
- 2014-10
- Volume
- 2
- Issue
- 36
- Pages
- 1-128
- ISSN
- 2050-4349 (ISSN-L)
- 2050-4357
- OpenAlex
- W2063740259
- Cited by
- 88
- References
- 12
- FWCI
- 10.45
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.