The DOI matches its Crossref record; 67 of the 68 references it cites were found in a registry or an index.
Evaluating patient and stakeholder engagement in research: moving from theory to practice
Open accessAbstract
Despite the growing demand for research that engages stakeholders, there is limited evidence in the literature to demonstrate its value – or return on investment. This gap indicates a general lack of evaluation of engagement activities. To adequately inform engagement activities, we need to further investigate the dividends of engaged research, and how to evaluate these effects. This paper synthesizes the literature on hypothesized impacts of engagement, shares what has been evaluated and identifies steps needed to reduce the gap between engagement's promises and the underlying evidence supporting its practice. This assessment provides explicit guidance for better alignment of engagement's promised benefits with evaluation efforts and identifies specific areas for development of evaluative measures and better reporting processes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Despite the rise in popularity and continued growth of youth engagement in health research, systematic and deliberate theory‐building has been limited.”
From A systematic review of theories, models and frameworks used for youth engagement in health research (2024) · cites this paper as [61]
In the citing paper
“Previous studies indicated that engaging the target group increases study enrollment and may enhance the uptake and the acceptance of interventions [67, 68].”
Citations
Cites
A Typology of Public Engagement Mechanisms
Rowe, G., & Frewer, L. J. (2005) · Science Technology & Human Values · doi:10.1177/0162243904271724 · Cited by 1,860
Carman, K. L., Dardess, P., Maurer, M. E., Sofaer, S., Adams, K., Bechtel, C., & Sweeney, J. (2013) · Health Affairs · doi:10.1377/hlthaff.2012.1133 · Cited by 1,770
Brett, J., Staniszewska, S., Mockford, C., Herron‐Marx, S., Hughes, J., Tysall, C., & Suleman, R. (2014) · Health Expectations · doi:10.1111/j.1369-7625.2012.00795.x · Cited by 1,598
Development of a theory of implementation and integration: Normalization Process Theory
May, C. R., Mair, F., Finch, T., MacFarlane, A., Dowrick, C., Treweek, S., Rapley, T., Ballini, L., Ong, B. N., Rogers, A., et al. (2009) · Implementation Science · doi:10.1186/1748-5908-4-29 · Cited by 1,409
Nothing about us without us: disability oppression and empowerment
Charlton, J. I. (1998) · Choice Reviews Online · doi:10.5860/choice.36-0640 · Cited by 1,124
Stakeholder Engagement: Beyond the Myth of Corporate Responsibility
Greenwood, M. (2007) · Journal of Business Ethics · doi:10.1007/s10551-007-9509-y · Cited by 1,061
Abelson, J., Forest, P., Eyles, J. D., Smith, P., Martin, E., & Gauvin, F. (2003) · Social Science & Medicine · doi:10.1016/s0277-9536(02)00343-x · Cited by 983
Community-based participatory research: assessing the evidence.
Viswanathan, M., Ammerman, A., Eng, E. M., Garlehner, G., Lohr, K. N., Griffith, D. M., Rhodes, S. D., Samuel‐Hodge, C. D., Maty, S. C., Lux, L. J., et al. (2004) · PubMed Central · Cited by 959
Evaluating Public-Participation Exercises: A Research Agenda
Rowe, G., & Frewer, L. J. (2004) · Science Technology & Human Values · doi:10.1177/0162243903259197 · Cited by 882
Patient and service user engagement in research: a systematic review and synthesized framework
Shippee, N. D., Garces, J. P. D., López, G., Wang, Z., Elraiyah, T. A., Nabhan, M., Brito, J. P., Boehmer, K. B., Hasan, R., Firwana, B., et al. (2013) · Health Expectations · doi:10.1111/hex.12090 · Cited by 630
Beal, A. C. (2012) · JAMA · doi:10.1001/jama.2012.500 · Cited by 625
A New Taxonomy for Stakeholder Engagement in Patient-Centered Outcomes Research
Concannon, T. W., Meißner, P., Grunbaum, J. A., McElwee, N. E., Guise, J., Santa, J. S., Conway, P. H., Daudelin, D. H., Morrato, E. H., & Leslie, L. K. (2012) · Journal of General Internal Medicine · doi:10.1007/s11606-012-2037-1 · Cited by 532
Consumer involvement in health research: a review and research agenda
Boote, J. D., Telford, R., & Cooper, C. L. (2002) · Health Policy · doi:10.1016/s0168-8510(01)00214-7 · Cited by 523
Community engagement in research: Frameworks for education and peer review
Ahmed, S. M., & Palermo, A.-G. S. (2010) · American Journal of Public Health · doi:10.2105/ajph.2009.178137 · Cited by 509
Concannon, T. W., Fuster, M., Saunders, T. S., Patel, K., Wong, J. B., Leslie, L. K., & Lau, J. (2014) · Journal of General Internal Medicine · doi:10.1007/s11606-014-2878-x · Cited by 470
De Las Nueces, D., Hacker, K., DiGirolamo, A., & Hicks, L. S. (2012) · Health Services Research · doi:10.1111/j.1475-6773.2012.01386.x · Cited by 400
Public participation in health care priority setting: A scoping review
Mitton, C., Smith, N., Peacock, S. J., Evoy, B. E., & Abelson, J. (2009) · Health Policy · doi:10.1016/j.healthpol.2009.01.005 · Cited by 399
Staniszewska, S., Brett, J., Mockford, C., & Barber, R. (2011) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462311000481 · Cited by 369
A multidimensional conceptual framework for analysing public involvement in health services research
Oliver, S. R., Rees, R. W., Clarke‐Jones, L., Milne, R., Oakley, A., Gabbay, J., Stein, K., Buchanan, P., & Gyte, G. M. (2008) · Health Expectations · doi:10.1111/j.1369-7625.2007.00476.x · Cited by 360
Deverka, P., Lavallee, D., Desai, P., Esmail, L., Ramsey, S., Veenstra, D., & Tunis, S. (2012) · Journal of Comparative Effectiveness Research · doi:10.2217/cer.12.7 · Cited by 307
Self-citationTritter, J. Q. (2009) · Health Expectations · doi:10.1111/j.1369-7625.2009.00564.x · Cited by 302
Facey, K. M., Boivin, A., Gracia, J., Hansen, H. P., Scalzo, A. L., Mossman, J., & Single, A. N. V. (2010) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462310000395 · Cited by 266
Priority setting: what constitutes success? A conceptual framework for successful priority setting
Sibbald, S. L., SINGER, P. A., Upshur, R. E. G., & Martin, D. K. (2009) · BMC Health Services Research · doi:10.1186/1472-6963-9-43 · Cited by 259
Barreteau, O., Bots, P. W. G., & Daniell, K. A. (2010) · Ecology and Society · doi:10.5751/es-03186-150201 · Cited by 243
Abma, T., Nierse, C. J., & Widdershoven, G. (2009) · Qualitative Health Research · doi:10.1177/1049732309331869 · Cited by 226
Boote, J. D., Wong, R., & Booth, A. (2012) · Health Expectations · doi:10.1111/hex.12007 · Cited by 216
Sandoval, J. A., Lucero, J. E., OETZEL, J. G., Avila, M., Belone, L., Mau, M. L. M., Pearson, C. R., Tafoya, G., Duran, B., Rios, L. I., et al. (2011) · Health Education Research · doi:10.1093/her/cyr087 · Cited by 214
What does it mean to involve consumers successfully in NHS research? A consensus study
Telford, R., Boote, J. D., & Cooper, C. L. (2004) · Health Expectations · doi:10.1111/j.1369-7625.2004.00278.x · Cited by 182
Cook, W. K. (2008) · Journal of Epidemiology & Community Health · doi:10.1136/jech.2007.067645 · Cited by 181
An Interactive and Contextual Model of Community-University Collaborations for Research and Action
Suarez‐Balcazar, Y., Harper, G. W., & Lewis, R. K. (2005) · Health Education & Behavior · doi:10.1177/1090198104269512 · Cited by 174
Nothing About Us Without Us: Disability Oppression and Empowerment
Stonnington, H. H. (1998) · JAMA · doi:10.1001/jama.280.16.1456-a · Cited by 169
Assessing the Impacts of Public Participation: Concepts, Evidence and Policy ImplicationsPreprint
Abelson, J., & Gauvin, F. (2006) · RePEc: Research Papers in Economics · Cited by 159
Brett, J., Staniszewska, S. H., Mockford, C., Seers, K., Herron‐Marx, S., & Bayliss, H. (2010) · University of Warwick · doi:10.82444/warw.33667912 · Cited by 151
Boote, J. D., Baird, W. O., & Sutton, A. (2011) · Health Policy · doi:10.1016/j.healthpol.2011.05.002 · Cited by 131
Callard, F. J., Rose, D. S., & Wykes, T. (2011) · Health Expectations · doi:10.1111/j.1369-7625.2011.00681.x · Cited by 130
User involvement in the development of a research bid: barriers, enablers and impacts1
Staniszewska, S., Jones, N., Newburn, M., & Marshall, S. (2007) · Health Expectations · doi:10.1111/j.1369-7625.2007.00436.x · Cited by 112
Hoffman, A., Montgomery, R., Aubry, W. M., & Tunis, S. (2010) · Health Affairs · doi:10.1377/hlthaff.2010.0675 · Cited by 112
Can the impact of public involvement on research be evaluated? A mixed methods study
Barber, R., Boote, J. D., Parry, G. D., Cooper, C. L., Yeeles, P., & Cook, S. (2011) · Health Expectations · doi:10.1111/j.1369-7625.2010.00660.x · Cited by 109
Evaluating the impact of service user involvement on research: a prospective case study
Barber, R., Beresford, P., Boote, J., Cooper, C., & Faulkner, A. (2011) · International Journal of Consumer Studies · doi:10.1111/j.1470-6431.2011.01017.x · Cited by 107
Boote, J., Barber, R., & Cooper, C. (2006) · Health Policy · doi:10.1016/j.healthpol.2005.03.012 · Cited by 105
The impact of service user involvement in research
Minogue, V., Boness, J., Brown, A., & Girdlestone, J. (2005) · International Journal of Health Care Quality Assurance · doi:10.1108/09526860510588133 · Cited by 104
Keeping Service User Involvement in Research Honest
McLaughlin, H. (2009) · The British Journal of Social Work · doi:10.1093/bjsw/bcp064 · Cited by 97
Consumer Participation in Mental Health Research: Articulating a Model to Guide Practice
Happell, B., & Roper, C. (2007) · Australasian Psychiatry · doi:10.1080/10398560701320113 · Cited by 91
Snape, D., Kirkham, J., Preston, J., Popay, J., Britten, N., Collins, M., Froggatt, K., Gibson, A., Lobban, F., Wyatt, K., et al. (2014) · BMJ Open · doi:10.1136/bmjopen-2013-004217 · Cited by 88
Lindenmeyer, A., Hearnshaw, H., Sturt, J. A., Ormerod, R., & Aitchison, G. (2007) · Health Expectations · doi:10.1111/j.1369-7625.2007.00451.x · Cited by 88
Beresford, P. (2007) · Evidence & Policy · doi:10.1332/174426407781738074 · Cited by 65
Saunders, C., Crossing, S., Penman, A., Butow, P. N., & Girgis, A. (2007) · Australia and New Zealand Health Policy · doi:10.1071/hp070413 · Cited by 59
Engaging Stakeholders To Identify and Prioritize Future Research Needs
O’Haire, C., McPheeters, M. L., Nakamoto, E., LaBrant, L., Most, C., Lee, K., Graham, E., Cottrell, E., & Guise, J. (2011) · Europe PMC (PubMed Central) · Cited by 59
Stakeholder engagement in comparative effectiveness research: how will we measure success?
Lavallee, D. C., Williams, C. J., Tambor, E. S., & Deverka, P. A. (2012) · Journal of Comparative Effectiveness Research · doi:10.2217/cer.12.44 · Cited by 57
Wyatt, K. M., Carter, M., Mahtani, V. M., Barnard, A., Hawton, A., & Britten, N. (2008) · Family Practice · doi:10.1093/fampra/cmn019 · Cited by 56
Using a Collaboratory Model to Translate Social Work Research Into Practice and Policy
Allen‐Meares, P., Hudgins, C. A., Engberg, M. E., & Lessnau, B. (2004) · Research on Social Work Practice · doi:10.1177/1049731504272345 · Cited by 44
McKevitt, C., Fudge, N., & Wolfe, C. D. A. (2009) · Health Expectations · doi:10.1111/j.1369-7625.2009.00573.x · Cited by 40
User involvement leads to more ethically sound research
Staley, K., & Minogue, V. (2006) · Clinical Ethics · doi:10.1258/147775006777254489 · Cited by 40
Newhouse, R. P. (2017) · doi:10.25302/67.2017 · Cited by 34
Professionals and the public: power or partnership in health research?
Robinson, L. J., Newton, J. L., & Dawson, P. (2010) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2010.01572.x · Cited by 33
Involving the public in priority setting: a case study using discrete choice experiments
Watson, V., Carnon, A., Ryan, M., & Cox, D. (2011) · Journal of Public Health · doi:10.1093/pubmed/fdr102 · Cited by 32
Howe, A. C., MacDonald, H., Barrett, B., & Little, B. (2006) · Primary Health Care Research & Development · doi:10.1191/1463423606pc269oa · Cited by 26
Williamson, T., Brogden, J., Jones, E., & Ryan, J. (2010) · International Journal of Consumer Studies · doi:10.1111/j.1470-6431.2010.00909.x · Cited by 25
Research, Engagement, and Public Bioethics: Promoting Socially Robust Science
Pickersgill, M. (2011) · Journal of Medical Ethics · doi:10.1136/jme.2010.041954 · Cited by 25
Patient and public involvement in translative healthcare research
Savory, C. (2010) · Clinical Governance An International Journal · doi:10.1108/14777271011063823 · Cited by 24
Participating in community research: exploring the experiences of lay researchers in Bradford
Newell, C., & South, J. (2009) · Community Work & Family · doi:10.1080/13668800802627934 · Cited by 23
Unresolved tensions in consumer engagement in CER: a US research perspective
Workman, T. A., Maurer, M. E., & Carman, K. L. (2013) · Journal of Comparative Effectiveness Research · doi:10.2217/cer.13.6 · Cited by 20
Public involvement in the design and conduct of clinical trials: a narrative review of case examples
Boote, J. D., Baird, W. O., & Sutton, A. (2011) · Trials · doi:10.1186/1745-6215-12-s1-a82 · Cited by 20
A potential model for the first all Wales mental health service user and carer‐led research group
Wilson, C., Fothergill, A., & Rees, H. (2009) · Journal of Psychiatric and Mental Health Nursing · doi:10.1111/j.1365-2850.2009.01473.x · Cited by 16
Gliklich, R. E., Leavy, M. B., Velentgas, P., Dreyer, N. A., Tunis, S., Mohr, P. E., Messner, D. A., Moloney, R., Karkare, S. U., Dubois, R. W., et al. (2012) · Journal of Comparative Effectiveness Research · doi:10.2217/cer.12.25 · Cited by 10
Involving and engaging patients in cancer and palliative care research: workshop presentation
Johnston, B. M., Forbat, L., & Hubbard, G. (2008) · International Journal of Palliative Nursing · doi:10.12968/ijpn.2008.14.11.31760 · Cited by 6
Science and the community: a collaborative model for integration of research within the community.
Weglicki, L. S., Rice, V. H., & Hammad, A. S. (2005) · PubMed · Cited by 3
Flagged first, then by citations
1 of the 68 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Krumholz H . National priorities and research agenda . In : Patient-Centered Outcomes Research Institute National Patient and Stakeholder Dialogue . Patient-Centered Outcomes Research Institute , Washington, DC, USA ( 2012 ).
- Garces JPD , Lopez GJP , Wang Z Eliciting Patient Perspective in Patient-Centered Outcomes Research: a Meta Narrative Systematic Review . Patient-Centered Outcomes Research Institute (PCORI) , Washington, DC, USA , 1 – 163 ( 2012 ).
- Mullins CD . Integrating Patients’ Voices in Study Design Elements with a Focus on Hard-to-reach Populations . Patient-Centered Outcomes Research Institute (PCORI) , Washington, DC, USA ( 2012 ).
- Nass PL , Levine S , Yancy C . Methods for Involving Patients in Topic Generation for Patient-Centered Comparative Effectiveness Research – an International Perspective . Patient-Centered Outcomes Research Institute (PCORI) , Washington, DC, USA ( 2012 ).
Cited by
Sorted by citations
Research co-design in health: a rapid overview of reviews
Slattery, P., Saeri, A. K., & Bragge, P. (2020) · Health Research Policy and Systems · doi:10.1186/s12961-020-0528-9 · Cited by 1,313
Greenhalgh, T., Hinton, L., Finlay, T., Macfarlane, A., Fahy, N., Clyde, B., & Chant, A. (2019) · Health Expectations · doi:10.1111/hex.12888 · Cited by 1,059
Manafò, E., Petermann, L., Mason-Lai, P., & Vandall‐Walker, V. (2018) · Health Research Policy and Systems · doi:10.1186/s12961-018-0282-4 · Cited by 483
How to engage stakeholders in research: design principles to support improvement
Boaz, A., Hanney, S. R., Borst, R. A. J., O’Shea, A., & Kok, M. O. (2018) · Health Research Policy and Systems · doi:10.1186/s12961-018-0337-6 · Cited by 471
The PCORI Engagement Rubric: Promising Practices for Partnering in Research
Sheridan, S. E., Schrandt, S., Forsythe, L. P., Hilliard, T. S., & Paez, K. (2017) · The Annals of Family Medicine · doi:10.1370/afm.2042 · Cited by 360
Metadata
- Authors
- Esmail, L., Moore, E., & Rein, A.
- Affiliations
- AcademyHealth (US)
- Venue
- Journal of Comparative Effectiveness Research
- Publisher
- Becaris Publishing Limited
- Published
- 2015-03
- Volume
- 4
- Issue
- 2
- Pages
- 133-145
- ISSN
- 2042-6305 (ISSN-L)
- 2042-6313
- OpenAlex
- W2096611776
- Cited by
- 502
- References
- 68
- Citations by year
- 2026: 32 · 2025: 43 · 2024: 65 · 2023: 59 · 2022: 58 · 2021: 57 · 2020: 52 · 2019: 41 · 2018: 45 · 2017: 31 · 2016: 14 · 2015: 3
- FWCI
- 5.62
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Journal of Comparative Effectiveness Research (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mental Health and Patient Involvement
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Community Health and Development
- Keywords
- patient engagement, stakeholder engagement, engagement outcomes, return on investment
- MeSH terms
- Humans, Patient Participation, Professional-Patient Relations, Comparative Effectiveness Research, Patient Outcome Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review