CoChat
Sign Up
Cited by 138Mental Health and Patient InvolvementMobile Health and mHealth ApplicationsHealth Literacy and Information Accessibility

Meaningful patient and public involvement in digital health innovation, implementation and evaluation: A systematic review

Open access
Baines, Rebecca, Bradwell, Hannah, Edwards, Katie, Stevens, Sebastian, Prime, Samantha, Tredinnick‐Rowe, John, Sibley, Miles, Chatterjee, Arunangsu·2022·Health Expectations
DOI 10.1111/hex.13506PMC PMC9327849
Open this paper in CoChat — free accountOpen PDF

Abstract

INTRODUCTION: The importance of meaningfully involving patients and the public in digital health innovation is widely acknowledged, but often poorly understood. This review, therefore, sought to explore how patients and the public are involved in digital health innovation and to identify factors that support and inhibit meaningful patient and public involvement (PPI) in digital health innovation, implementation and evaluation. METHODS: Searches were undertaken from 2010 to July 2020 in the electronic databases MEDLINE, EMBASE, PsycINFO, CINAHL, Scopus and ACM Digital Library. Grey literature searches were also undertaken using the Patient Experience Library database and Google Scholar. RESULTS: Of the 10,540 articles identified, 433 were included. The majority of included articles were published in the United States, United Kingdom, Canada and Australia, with representation from 42 countries highlighting the international relevance of PPI in digital health. 112 topic areas where PPI had reportedly taken place were identified. Areas most often described included cancer (n = 50), mental health (n = 43), diabetes (n = 26) and long-term conditions (n = 19). Interestingly, over 133 terms were used to describe PPI; few were explicitly defined. Patients were often most involved in the final, passive stages of an innovation journey, for example, usability testing, where the ability to proactively influence change was severely limited. Common barriers to achieving meaningful PPI included data privacy and security concerns, not involving patients early enough and lack of trust. Suggested enablers were often designed to counteract such challenges. CONCLUSIONS: PPI is largely viewed as valuable and essential in digital health innovation, but rarely practised. Several barriers exist for both innovators and patients, which currently limits the quality, frequency and duration of PPI in digital health innovation, although improvements have been made in the past decade. Some reported barriers and enablers such as the importance of data privacy and security appear to be unique to PPI in digital innovation. Greater efforts should be made to support innovators and patients to become meaningfully involved in digital health innovations from the outset, given its reported benefits and impacts. Stakeholder consensus on the principles that underpin meaningful PPI in digital health innovation would be helpful in providing evidence-based guidance on how to achieve this. PATIENT OR PUBLIC CONTRIBUTION: This review has received extensive patient and public contributions with a representative from the Patient Experience Library involved throughout the review's conception, from design (including suggested revisions to the search strategy) through to article production and dissemination. Other areas of patient and public contributor involvement include contributing to the inductive thematic analysis process, refining the thematic framework and finalizing theme wording, helping to ensure relevance, value and meaning from a patient perspective. Findings from this review have also been presented to a variety of stakeholders including patients, patient advocates and clinicians through a series of focus groups and webinars. Given their extensive involvement, the representative from the Patient Experience Library is rightly included as an author of this review.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/hex.13506

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Fragmented care coordination and lack of shared governance structures limit the effective deployment of AI across healthcare levels.

    “Additionally, fragmented care coordination and lack of shared governance structures were seen as limiting factors in the effective deployment of AI across healthcare levels [101,102].”

    From Barriers and Facilitators to Artificial Intelligence Implementation in Diabetes Management from Healthcare Workers’ Perspective: A Scoping Review (2025) · cites this paper as [102]

    The Results and Discussion passages address barriers to patient and public involvement in digital health, including power imbalances and a disconnect between developers and patients. They do not address fragmented care coordination, shared governance across healthcare levels, or AI deployment. Searches for those terms and close paraphrases found no passage supporting the attributed claim; page numbers were unavailable in our copy. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 71

  1. [2]Mjøsund NH, Eriksson M, Espnes GA, et al. Service user involvement enhanced the research quality in a study using interpretative phenomenological analysis–the power of multiple perspectives. J Adv Nurs. 2017; 73(1): 265‐ 278. 27509545
  2. [3]Rolfe DE, Ramsden VR, Banner D, Graham ID. Using qualitative health research methods to improve patient and public involvement and engagement in research. Res Involv Engagem. 2018; 4(1): 1‐ 8. 30564459 PMC6293564
  3. [4]Mockford C, Staniszewska S, Griffiths F, Herron‐Marx S. The impact of patient and public involvement on UK NHS health care: a systematic review. Int J Qual Health Care. 2012; 24(1): 28‐ 38. 22109631
  4. [5]Locock L, Boylan AM, Snow R, Staniszewska S. The power of symbolic capital in patient and public involvement in health research. Health Expect. 2017; 20(5): 836‐ 844. 27885770 PMC5600212
  5. [7]Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2015; 18(5): 1151‐ 1166. 23731468 PMC5060820
  6. [8]Lalani M, Baines R, Bryce M, et al. Patient and public involvement in medical performance processes: a systematic review. Health Expect. 2019; 22(2): 149‐ 161. 30548359 PMC6433319
  7. [9]Dijk SW, Duijzer EJ, Wienold M. Role of active patient involvement in undergraduate medical education: a systematic review. BMJ Open. 2020; 10(7): e037217. PMC7389514 32718925
  8. [10]Baines RL, Regan de Bere S. Optimizing patient and public involvement (PPI): identifying its “essential” and “desirable” principles using a systematic review and modified Delphi methodology. Health Expect. 2018; 21(1): 327‐ 335. 28929554 PMC5750770
  9. [11]Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q. 2012; 90(2): 311‐ 346. 22709390 PMC3460206
  10. [12]Locock L, Kirkpatrick S, Brading L, et al. Involving service users in the qualitative analysis of patient narratives to support healthcare quality improvement. Res Involv Engagem. 2019; 5(1): 1. 30788147 PMC6318884
  11. [13]Staniszewska S, Jones N, Newburn M, Marshall S. User involvement in the development of a research bid: barriers, enablers and impacts. Health Expect. 2007; 10(2): 173‐ 183. 17524010 PMC5060394
  12. [15]Locock L, Boaz A. Drawing straight lines along blurred boundaries: qualitative research, patient and public involvement in medical research, co‐production and co‐design. Evid Policy. 2019; 15(3): 409‐ 421.
  13. [16]Pizzo E, Doyle C, Matthews R, Barlow J. Patient and public involvement: how much do we spend and what are the benefits? Health Expect. 2015; 18(6): 1918‐ 1926. 24813243 PMC5810684
  14. [17]Alderson H, Brown R, Smart D, Lingam R, Dovey‐Pearce G. ‘You've come to children that are in care and given us the opportunity to get our voices heard’: the journey of looked after children and researchers in developing a patient and public involvement group. Health Expect. 2019; 22(4): 657‐ 665. 31115138 PMC6737768
  15. [18]Gillard S, Borschmann R, Turner K, Goodrich‐Purnell N, Lovell K, Chambers M. ‘What difference does it make?’ Finding evidence of the impact of mental health service user researchers on research into the experiences of detained psychiatric patients. Health Expect. 2010; 13(2): 185‐ 194. 20536538 PMC5060533
  16. [19]Gillard S, Simons L, Turner K, Lucock M, Edwards C. Patient and public involvement in the coproduction of knowledge: reflection on the analysis of qualitative data in a mental health study. Qual Health Res. 2012; 22(8): 1126‐ 1137. 22673090
  17. [20]MacDonald C. Understanding participatory action research: a qualitative research methodology option. Can J Action Res. 2012; 13(2): 34‐ 50.
  18. [21]Reason P, Bradbury H. Handbook of Action Research: Concise Paperback Edition. Sage; 2005.
  19. [22]Tangvald‐Pedersen O, Bongaardt R. Towards a tinkering participatory research method in mental health. Scand J Disab Res. 2017; 19(1): 7‐ 17.
  20. [23]Baum F, MacDougall C, Smith D. Participatory action research. J Epidemiol Commun Health. 2006; 60(10): 854‐ 857. PMC2566051 16973531
  21. [24]Smith L, Bratini L, Chambers D‐A, Jensen RV, Romero L. Between idealism and reality: meeting the challenges of participatory action research. Action Res. 2010; 8(4): 407‐ 425.
  22. [25]Waterman H, Tillen D, Dickson R, De Koning K. Action research: a systematic review and guidance for assessment. Health Technol Assess. 2001; 5(23): iii‐ 157. 11785749
  23. [26]Manteghinejad A, Javanmard SH. Challenges and opportunities of digital health in a post‐COVID19 world. J Res Med Sci. 2021; 26: 11. 34084190 PMC8103966
  24. [29]Murphy E, Tierney E, Ní Shé É, et al. COVID‐19: public and patient involvement, now more than ever. HRB Open Res. 2020; 3: 3. 32666039 PMC7327728
  25. [30]Richards T, Scowcroft H. Patient and public involvement in Covid‐19 policy making. BMJ. 2020: ​ m2575. 32611571
  26. [31]Birnbaum F, Lewis D, Rosen RK, Ranney ML. Patient engagement and the design of digital health. Acad Emerg Med. 2015; 22(6): 754‐ 756. 25997375 PMC4674428
  27. [32]Greenhalgh T, Hinton L, Finlay T, et al. Frameworks for supporting patient and public involvement in research: systematic review and co‐design pilot. Health Expect. 2019; 22(4): 785‐ 801. 31012259 PMC6737756
  28. [33]Papoutsi C, Wherton J, Shaw S, Morrison C, Greenhalgh T. Putting the social back into sociotechnical: case studies of co‐design in digital health. J Am Med Inform Assoc. 2021; 28(2): 284‐ 293. 33043359 PMC7883972
  29. [34]Bird M, McGillion M, Chambers EM, et al. A generative co‐design framework for healthcare innovation: development and application of an end‐user engagement framework. Res Involv Engagem. 2021; 7(1): 1‐ 12. 33648588 PMC7923456
  30. [35]Baker TB, Gustafson DH, Shah D. How can research keep up with eHealth? Ten strategies for increasing the timeliness and usefulness of eHealth research. J Med Internet Res. 2014; 16(2): e36. 24554442 PMC3961695
  31. [36]Ting DSW, Carin L, Dzau V, Wong TY. Digital technology and COVID‐19. Nature Med. 2020; 26(4): 459‐ 461. 32284618 PMC7100489
  32. [37]Chu LF, Shah AG, Rouholiman D, Riggare S, Gamble JG. Patient‐centric strategies in digital health. In: Rivas H, Wac K, eds. Digital Health. Springer; 2018: 43‐ 54.
  33. [38]Towle A, Bainbridge L, Godolphin W, et al. Active patient involvement in the education of health professionals. Med Educ. 2010; 44(1): 64‐ 74. 20078757
  34. [39]Baggott R. A funny thing happened on the way to the forum? Reforming patient and public involvement in the NHS in England. Public Adm. 2005; 83(3): 533‐ 551.
  35. [40]Stickley T. Should service user involvement be consigned to history? A critical realist perspective. J Psychiatr Ment Health Nurs. 2006; 13(5): 570‐ 577. 16965476
  36. [41]Tritter JQ. Revolution or evolution: the challenges of conceptualizing patient and public involvement in a consumerist world. Health Expect. 2009; 12(3): 275‐ 287. 19754691 PMC5060496
  37. [44]Farhad F, Samadbeik M, Kazemi A. What is digital health? Review of definition. In: Alpo Varri J, Delgado P, Gallos M, Hagglund K, Hayrinen U‐MK, eds. Integrated Citizen Centered Digital Health and Social Care: Citizens as Data Producers and Service Co‐Creators. IOS Press; 2020: 67‐ 72.
  38. [45]Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence‐based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009; 62: 62‐ 52. 19230612
  39. [47]Pappas C, Williams I. Grey literature: its emerging importance. J Hosp Librariansh. 2011; 11(3): 228‐ 234.
  40. [49]Norman DA. User Centered System Design: New Perspectives on Human‐Computer Interaction. CRC Press; 1986.
  41. [50]Ashmore LA, Stewart H, Hutton D, Evans K. Digital support for living with and beyond gynaecological cancer. Radiography. 2020; 26: 270. 32439292
  42. [51]Borosund E, Clark MM, Ehlers SL, Andrykowski MA, Slevel HR, et al. EHealth, stress management and cancer: testing an app for use, usability and preliminary impact on health‐related quality of life and self‐regulatory capacity. Psychooncology. 2018; 27: 11.
  43. [52]Bogale B, Mørkrid K, O'Donnell B, et al. Development of a targeted client communication intervention to women using an electronic maternal and child health registry: a qualitative study. BMC Med Inform Decis Mak. 2020; 20(1): 1. 31906929 PMC6945530
  44. [53]Bhattacharyya O, Mossman K, Gustafsson L, Schneider EC. Using human‐centered design to build a digital health advisor for patients with complex needs: persona and prototype development. J Med Internet Res. 2019; 21(5): 10318. PMC6532339 31094334
  45. [54]Birnie KA, Campbell F, Nguyen C, et al. ICanCope PostOp: user‐centered design of a smartphone‐based app for self‐management of postoperative pain in children and adolescents. J Med Internet Res. 2019; 21(4): e12028. PMC6658275 31008704
  46. [55]Avila‐Garcia P, Hernandez‐Ramos R, Nouri SS, et al. Engaging users in the design of an mHealth, text message‐based intervention to increase physical activity at a safety‐net health care system. JAMIA Open. 2019; 2(4): 489‐ 497. 32025646 PMC6994014
  47. [56]Cnossen IC, van Uden‐Kraan CF, Eerenstein SE, et al. A participatory design approach to develop a web‐based self‐care program supporting early rehabilitation among patients after total laryngectomy. Folia Phoniatr Logop. 2015; 67(4): 193‐ 201. 26771305
  48. [57]Camerini L, Camerini A‐L, Schulz PJ. Do participation and personalization matter? A model‐driven evaluation of an Internet‐based patient education intervention for fibromyalgia patients. Patient Educ Couns. 2013; 92(2): 229‐ 234. 23664234
  49. [58]Aryana B, Brewster L. Design for mobile mental health: exploring the informed participation approach. Health Inform J. 2019; 26(2): 1208‐ 1224. 31566470
  50. [59]Adam M, McMahon SA, Prober C, Barnighausen T. Human‐centered design of video‐based health education: an iterative, collaborative, community‐based approach. J Med Internet Res. 2019; 21(1): e12128. 30698531 PMC6372941
  51. [60]Caswell N, Kuru K, Ansell D, et al. Patient engagement in medical device design: refining the essential attributes of a wearable, pre‐void, ultrasound alarm for nocturnal enuresis. Pharmaceut Med. 2020; 34(1): 39‐ 48. 31970684
  52. [61]Berg M, Adolfsson A, Ranerup A, Berg M, Sparud‐Lundin C. Person‐centered web support to women with type 1 diabetes in pregnancy and early motherhood—the development process. Diabetes Technol Ther. 2013; 15: 20‐ 25. 23297670
  53. [62]Band R, Bradbury K, Morton K, et al. Intervention planning for a digital intervention for self‐management of hypertension: a theory‐, evidence‐ and person‐based approach. Implement Sci. 2017; 12(25): 25. 28231840 PMC5324312
  54. [63]Corbett T, Walsh JC, Groarke A, Moss‐Morris R, Morrissey E, McGuire BE. Cancer‐related fatigue in post‐treatment cancer survivors: theory‐based development of a web‐based intervention. JMIR Cancer. 2017; 3(2): e8. 28676465 PMC5516102
  55. [64]Te Morenga L, Pekepo C, Corrigan C, et al. Co‐designing an mHealth tool in the New Zealand Māori community with a “Kaupapa Māori” approach. AlterNative. 2018; 14(1): 90‐ 99.
  56. [65]Wilson J, Arshad F, Nnamoko N, Whiteman A, Ring J, Roy B. Patient‐reported outcome measures: an on‐line system empowering patient choice. J Am Med Inform Assoc. 2014; 21(4): 725‐ 729. 24013090 PMC4078267
  57. [67]Iribarren SJ, Beck SL, Pearce PF, Chirico C, Etchevarria M, Rubinstein F. mHealth intervention development to support patients with active tuberculosis. J Mob Technol in Med. 2014; 3(2): 16‐ 27. 26246859 PMC4523302
  58. [68]Calvillo‐Arbizu J, Roa‐Romero LM, Estudillo‐Valderrama MA, et al. User‐centred design for developing e‐Health system for renal patients at home (AppNephro). Int J Med Inform. 2019; 125: 47‐ 54. 30914180
  59. [69]Lustria MLA, Kazmer MM, Glueckauf RL, et al. Participatory design of a health informatics system for rural health practitioners and disadvantaged women. J Am Soc Inform Sci Technol. 2010; 61(11): 2243‐ 2255.
  60. [70]Goodyear‐Smith F, Jackson C, Greenhalgh T. Co‐design and implementation research: challenges and solutions for ethics committees. BMC Med Ethics. 2015; 16(1): 1‐ 5. 26573410 PMC4647576
  61. [71]Alaoui M, Lewkowicz M. Practical issues related to the implication of elderlies in the design process—the case of a Living Lab approach for designing and evaluating social TV services. IRBM. 2015; 36(5): 259‐ 265.
  62. [72]Buus N, Juel A, Haskelberg H, et al. User involvement in developing the MYPLAN mobile phone safety plan app: case study. J Med Internet Res. 2019; 21(4): e11965. PMC6488960 30990456
  63. [73]Clemensen J, Rothmann MJ, Smith AC, Caffery LJ, Danbjorg DB. Participatory design methods in telemedicine research. J Telemed Telecare. 2017; 23(9): 780‐ 785. 28027678
  64. [74]Bauer AM, Hodsdon S, Bechtel JM, Fortney JC. Applying the principles for digital development: case study of a smartphone app to support collaborative care for rural patients with posttraumatic stress disorder or bipolar disorder. J Med Internet Res. 2018; 20(6): e10048. 29875085 PMC6010837
  65. [75]Baek H, Suh JW, Kang SH, et al. Enhancing user experience through user study: design of an mhealth tool for self‐management and care engagement of cardiovascular disease patients. J Med Internet Res. 2018; 20(2): e3. 31758783 PMC6857966
  66. [76]Chhoun P, Kaplan KC, Wieten C, et al. Using participatory methods to build an mHealth intervention for female entertainment workers in Cambodia: the development of the Mobile Link project. Mhealth. 2019; 5: 24. 31559269 PMC6737450
  67. [77]Abrines Jaume N, Abbiss M, Wray J, Ashworth J, Brown KL, Cairns J. CHILDSPLA: a collaboration between children and researchers to design and animate health states. Child Care Health Dev. 2015; 41(6): 1140‐ 1151. 26227090
  68. [79]Jones EL, Williams‐Yesson BA, Hackett RC, Staniszewska SH, Evans D, Francis NK. Quality of reporting on patient and public involvement within surgical research: a systematic review. Ann Surg. 2015; 261(2): 243‐ 250. 24950279
  69. [80]Brett J, Staniszewska S, Simera I, et al. Reaching consensus on reporting patient and public involvement (PPI) in research: methods and lessons learned from the development of reporting guidelines. BMJ Open. 2017; 7(10): e016948. PMC5665282 29061613
  70. [81]Williams O, Sarre S, Papoulias SC, et al. Lost in the shadows: reflections on the dark side of co‐production. Health Res Policy Syst. 2020; 18: 1‐ 10. 32380998 PMC7204208
  71. [82]Boylan AM, Locock L, Thomson R, Staniszewska S. “About sixty per cent I want to do it”: health researchers' attitudes to, and experiences of, patient and public involvement (PPI)—a qualitative interview study. Health Expect. 2019; 22(4): 721‐ 730. 30927334 PMC6737750

Cited by

Sorted by citations

Authors
Baines, R., Bradwell, H., Edwards, K., Stevens, S., Prime, S., Tredinnick‐Rowe, J., Sibley, M., & Chatterjee, A.
Affiliations
  • University of Plymouth (GB)
  • Patient Experience Library Macfin UK
Corresponding author
Baines, Rebecca
ORCID
Venue
Health Expectations
Publisher
Wiley
Published
2022-05-08
Volume
25
Issue
4
Pages
1232-1245
ISSN
  • 1369-6513 (ISSN-L)
  • 1369-7625
DOI
10.1111/hex.13506
PMC
PMC9327849
OpenAlex
W4229335321
Cited by
138
References
7
Citations by year
2026: 37 · 2025: 42 · 2024: 38 · 2023: 19 · 2022: 2
FWCI
4.87
Type
journal article
Language
en
Open access
Licence
Locations
  • Health Expectations (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
  • PEARL (University of Plymouth) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • PEARL (University of Plymouth) (submittedVersion)
Topics
  • Health Sciences › Health Professions › General Health Professions › Mental Health and Patient Involvement
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
Keywords
patient and public involvement, digital health innovation, digital health implementation, barriers and facilitators, data privacy and security, trust, usability testing, cancer, mental health, diabetes mellitus, chronic disease, stakeholder consensus
MeSH terms
Australia, Canada, Community Participation, United Kingdom, Health Plan Implementation, Humans, Mental Health, Patient Participation, United States, Program Development, Telemedicine, Meaningful Use

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review