CoChat

Methods for Human-Centered eHealth Development: Narrative Scoping Review

Open access
Kip, Hanneke, Keizer, Julia, da Silva, Marcia C, Beerlage-de Jong, Nienke, Köhle, Nadine, Kelders, Saskia M · 2022 · Journal of Medical Internet Research

Published version of a preprint

The DOI matches its Crossref record; 187 of the 187 references it cites were found in a registry or an index; 45 of the 145 sources its claims cite were checked for support.

Abstract

Background: Thorough holistic development of eHealth can contribute to a good fit among the technology, its users, and the context. However, despite the availability of frameworks, not much is known about specific research activities for different aims, phases, and settings. This results in researchers having to reinvent the wheel. Consequently, there is a need to synthesize existing knowledge on research activities for participatory eHealth development processes. Objective: The 3 main goals of this review are to create an overview of the development strategies used in studies based on the CeHRes (Center for eHealth Research) Roadmap, create an overview of the goals for which these methods can be used, and provide insight into the lessons learned about these methods. Methods: We included eHealth development studies that were based on the phases and/or principles of the CeHRes Roadmap. This framework was selected because of its focus on participatory, iterative eHealth design in context and to limit the scope of this review. Data were extracted about the type of strategy used, rationale for using the strategy, research questions, and reported information on lessons learned. The most frequently mentioned lessons learned were summarized using a narrative, inductive approach. Results: In the included 160 papers, a distinction was made between overarching development methods (n=10) and products (n=7). Methods are used to gather new data, whereas products can be used to synthesize previously collected data and support the collection of new data. The identified methods were focus groups, interviews, questionnaires, usability tests, literature studies, desk research, log data analyses, card sorting, Delphi studies, and experience sampling. The identified products were prototypes, requirements, stakeholder maps, values, behavior change strategies, personas, and business models. Examples of how these methods and products were applied in the development process and information about lessons learned were provided. Conclusions: This study shows that there is a plethora of methods and products that can be used at different points in the development process and in different settings. To do justice to the complexity of eHealth development, it seems that multiple strategies should be combined. In addition, we found no evidence for an optimal single step-by-step approach to develop eHealth. Rather, researchers need to select the most suitable research methods for their research objectives, the context in which data are collected, and the characteristics of the participants. This study serves as a first step toward creating a toolkit to support researchers in applying the CeHRes Roadmap to practice. In this way, they can shape the most suitable and efficient eHealth development process.

Overview

CoChat Summary

TL;DR

This narrative scoping review synthesized 160 eHealth development studies based on the CeHRes Roadmap, identifying 10 overarching methods and 7 products used across contextual inquiry, value specification, and design phases. It concludes there is no single optimal step-by-step approach; researchers should combine multiple strategies selected for the research objective, context, and participant characteristics.

Key findings

  • In 160 included papers, the review identified 10 methods (focus groups, interviews, questionnaires, usability tests, literature studies, desk research, log data analyses, card sorting, Delphi studies, experience sampling) and 7 products (prototypes, requirements, stakeholder maps, values, behavior change strategies, personas, business models).
  • Most development studies used a multi-method approach (90/160, 56.3%); web-based interventions were the most common technology (74/160, 46.3%), followed by mobile apps (34/160, 21.3%), VR (3/160, 1.9%), and no clear technology description (27/160, 16.9%).
  • Most frequently reported methods were focus groups (71/160, 44.4%), interviews (70/160, 43.8%), questionnaires (51/160, 31.9%), usability tests (51/160, 31.9%), and literature studies (43/160, 26.9%); most frequently reported products were prototypes (32/160, 20%), requirements (11/160, 6.9%), stakeholder maps (10/160, 6.3%), and values (7/160, 4.4%).
  • Lessons learned emphasized that methods and products should fit the context, participant characteristics and skills, and earlier findings; multiple methods should be combined; and there was no evidence for an optimal single step-by-step approach. Experience sampling (1/160, 0.6%), Delphi studies (3/160, 1.9%), personas (5/160, 3.1%), and behavior change strategies (5/160, 3.1%) were underrepresented.

Methods

  • Narrative scoping review of eHealth development studies based on the CeHRes Roadmap phases and/or principles; inclusion required reference to the original 2011 CeHRes paper in the introduction or methods and use of ≥1 phase or principle, while implementation/summative-evaluation-only studies, abstract guideline papers, non-peer-reviewed records, and records not in English, Dutch, German, or Portuguese were excluded.
  • Searched Scopus, Google Scholar, and Web of Science up to June 2021, plus snowball sampling for records coauthored by the CeHRes Roadmap founder; 1713 unique records were screened, 377 remained after title/abstract screening, and 160 were included after full-text screening.
  • Data were extracted by 3 researchers on strategy type, rationale, research questions, and lessons learned; methods and products were categorized by consensus, and lessons learned were summarized using a narrative, inductive approach. Phases were deduced by the authors when not explicitly reported.

Limitations

  • Stated: focusing only on the CeHRes Roadmap means other suitable methods or products may have been overlooked, so the overview is neither exhaustive nor unconditionally generalizable; lessons learned are not exhaustive; no quality assessment of included studies was performed; and the qualitative approach is prone to researcher bias, though screening and summarizing were systematic and conducted by 3 researchers with coauthor discussion.
  • Stated: not all relevant development information may have been published, and reporting clarity and extensiveness varied across studies; suboptimal or incomplete reflection by authors resulted in some lessons learned remaining general.
  • Evident from the text: CeHRes phases were explicitly mentioned in only 30% (48/160) of studies, so phase categorization was often deduced by the authors, which may introduce misclassification; products also appeared less clearly reported than methods.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/31858

Cites

187 of 187 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 The Center for eHealth Research Roadmap [2,11,18].
Figure 1 The Center for eHealth Research Roadmap [2,11,18].CC BY · PubMed Central
Figure 2 Search strategy and results. CeHRes: Center for eHealth Research.
Figure 2 Search strategy and results. CeHRes: Center for eHealth Research.CC BY · PubMed Central

Verification

Claims the paper makes

22 Valid · 0 Not valid · 55 Not checked · 111 Own results and interpretations

81 more claims · 3 with their cited sources checked

Create a free account to see all 188 claims and to verify the 100 references still open.

See all claims — free account
  • It may state two separate claims as one. 40
  • Probably not a substantial claim. 79

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Research works printed but not linked: 21

  1. Van DykLWentzelMLimburg, vanMGemert-Pijnen, vanJEWCSchutteCSBusiness models for sustained ehealth implementation: Lessons from two continentsProceedings of International Conference on Computers and Industrial Engineering2012Proceedings of International Conference on Computers and Industrial EngineeringJul 16-18, 2012Cape Town, South Africa
  2. KulykOVan Gemert-PijnenLDaas DenCDavidSHow persuasive are serious games, social media and mHealth technologies for vulnerable young adults? Design factors for health behavior and lifestyle change support: sexual health caseProceedings of the 3rd International Workshop on Behavior Change Support Systems, BCSS 2015 - co-located with the 10th International Conference on Persuasive Technology, PERSUASIVE 201520153rd International Workshop on Behavior Change Support Systems, BCSS 2015 - co-located with the 10th International Conference on Persuasive Technology, PERSUASIVE 2015Jun 3, 2015Chicago, United States
  3. Kulyk, O, 2014, Advances Life Sci, 6, 300
  4. Beerlage-de Jong, N, 2014, Int J Advances Life Sci, 6, 262
  5. LimburgMOssebaardHWentzelMvan Gemert-PijnenJCo-creating with stakeholders: ideating eHealth applications to support antibiotic stewardship in hospitalsProceedings of the eTELEMED 2016: The Eighth International Conference on eHealth, Telemedicine, and Social Medicine 20162016eTELEMED 2016: The Eighth International Conference on eHealth, Telemedicine, and Social Medicine 2016Apr 24 - 28, 2016Venice, Italy
  6. McGrath, SP, 2017, AMIA Annu Symp Proc, 2017, 1282
  7. SpanMHettingaMSmitsCGroen-van de VenLJukemaJVernooij-DassenMEefstingJDeveloping a supportive tool to facilitate shared decision making in dementiaProceedings of the The Sixth International Conference on eHealth, Telemedicine, and Social Medicine2014The Sixth International Conference on eHealth, Telemedicine, and Social MedicineMar 23-27, 2014Barcelona, Spain
  8. Span, M, 2014, Int J Adv Life Sci, 6, 338
  9. Imhemed, S, 2017, J Eng Applied Sci, 12, 559
  10. Verhoeven, F, 2008, Hygiene Medizin, 33, 412
  11. Cruz-MartínezRSieverinkFWesselinkRvan Gemert-PijnenJTowards data-driven persuasive coaching in a heart failure telemonitoring technologyProceedings of the Sixth International Workshop on Behavior Change Support Systems (BCSS 2018). In conjunction with the 13th International Conference on Persuasive Technology2018Sixth International Workshop on Behavior Change Support Systems (BCSS 2018). In conjunction with the 13th International Conference on Persuasive TechnologyApr 16-19, 2018University of Waterloo, Canada
  12. Nobakht, Z, 2018, Iran J Child Neurol, 12, 65
  13. Span, M, 2014, Int J Adv Life Sci, 6, 107
  14. Ihmhemed, SM, 2017, J Advanced Sci Eng Res, 7, 27
  15. Gude, WT, 2012, Stud Health Technol Inform, 180, 1209
  16. Walters, LE, 2016, Stud Health Technol Inform, 231, 143
  17. OrganDDickSHurleyCHeavinCLinehanCDockraySDavorenMBryneMA systematic review of user-centred design practices in illicit substance use interventions for higher education studentsProceedings of the European Conference on Information Systems 2018: Beyond Digitization-Facets of Socio-Technical Change2018European Conference on Information Systems 2018: Beyond Digitization-Facets of Socio-Technical ChangeJun 23 - 28, 2018Portsmouth, UK
  18. WentzelMKarremanJvanGJvan VanGemert-PijneJEOssebaardHHamalainenPTowards an internet-based infectious disease management platform to increase patient safetyProceedings of the Third International Conference on eHealth, Telemedicine, and Social Medicine2011Third International Conference on eHealth, Telemedicine, and Social MedicineFeb 23-28, 2011Guadeloupe, France
  19. TschimmelKDesign thinking as an effective toolkit for innovationProceedings of the XXIII ISPIM Conference: Action for Innovation: Innovating from Experience2012XXIII ISPIM Conference: Action for Innovation: Innovating from ExperienceJun 17-20, 2012Barcelona, Spain
  20. Kip, H, 2018, eHealth Research, Theory and Development: A Multi-Disciplinary Approach
  21. 2014, Guidelines for Reporting Health Research: A User's Manual

Cited by

See all 73

Metadata

Authors
Kip, H., Keizer, J., da Silva, M. C., Beerlage-de Jong, N., Köhle, N., & Kelders, S. M.
Affiliations
  • University of Twente (NL)
  • North-West University (ZA)
Corresponding author
Kip, Hanneke
Venue
Journal of Medical Internet Research
Published
2022-01-27
Volume
24
Issue
1
Pages
e31858
ISSN
1438-8871 (ISSN-L)
OpenAlex
W3213720648
Cited by
67
References
183
Citations by year
2026: 4 · 2025: 15 · 2024: 20 · 2023: 19 · 2022: 7
FWCI
2.48
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • University of Twente Research Information (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • University of Twente Research Information (submittedVersion, cc-by)
Topics
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
Keywords
participatory design, human-centered design, iterative design, development methods, lessons learned, stakeholder engagement, personas, behavior change strategies, business models, prototype development, usability testing, stakeholder mapping
MeSH terms
Behavior Therapy, Humans, Surveys and Questionnaires, Research Design, Focus Groups, Telemedicine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.