Factors Determining the Success and Failure of eHealth Interventions: Systematic Review of the Literature
Open accessPublished version of a preprint
The DOI matches its Crossref record; 251 of the 251 references it cites were found in a registry or an index.
Abstract
BACKGROUND: eHealth has an enormous potential to improve healthcare cost, effectiveness, and quality of care. However, there seems to be a gap between the foreseen benefits of research and clinical reality. OBJECTIVE: Our objective was to systematically review the factors influencing the outcome of eHealth interventions in terms of success and failure. METHODS: We searched the PubMed database for original peer-reviewed studies on implemented eHealth tools that reported on the factors for the success or failure, or both, of the intervention. We conducted the systematic review by following the patient, intervention, comparison, and outcome framework, with 2 of the authors independently reviewing the abstract and full text of the articles. We collected data using standardized forms that reflected the categorization model used in the qualitative analysis of the outcomes reported in the included articles. RESULTS: Among the 903 identified articles, a total of 221 studies complied with the inclusion criteria. The studies were heterogeneous by country, type of eHealth intervention, method of implementation, and reporting perspectives. The article frequency analysis did not show a significant discrepancy between the number of reports on failure (392/844, 46.5%) and on success (452/844, 53.6%). The qualitative analysis identified 27 categories that represented the factors for success or failure of eHealth interventions. A quantitative analysis of the results revealed the category quality of healthcare (n=55) as the most mentioned as contributing to the success of eHealth interventions, and the category costs (n=42) as the most mentioned as contributing to failure. For the category with the highest unique article frequency, workflow (n=51), we conducted a full-text review. The analysis of the 23 articles that met the inclusion criteria identified 6 barriers related to workflow: workload (n=12), role definition (n=7), undermining of face-to-face communication (n=6), workflow disruption (n=6), alignment with clinical processes (n=2), and staff turnover (n=1). CONCLUSIONS: The reviewed literature suggested that, to increase the likelihood of success of eHealth interventions, future research must ensure a positive impact in the quality of care, with particular attention given to improved diagnosis, clinical management, and patient-centered care. There is a critical need to perform in-depth studies of the workflow(s) that the intervention will support and to perceive the clinical processes involved.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Addressing these issues will likely contribute to development of more user friendly, meaningful interventions for patients and can potentially improve treatment outcomes for patients living with chronic pain [27,39,44,46].”
From A User-Centered Approach to an Evidence-Based Electronic Health Pain Management Intervention for People With Chronic Pain: Design and Development of EPIO (2020) · cites this paper as [46]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Three Approaches to Qualitative Content Analysis
Hsieh, H.-F., & Shannon, S. E. (2005) · Qualitative Health Research · doi:10.1177/1049732305276687 · Cited by 44,601 · cited 1×
“A qualitative analysis [16,17] was carried out by 2 of the authors (CG and WJ) to classify the outcomes reported in the articles’ abstract according to the following 3 levels.”
MethodGraneheim, U. H., & Lundman, B. (2004) · Nurse Education Today · doi:10.1016/j.nedt.2003.10.001 · Cited by 20,772 · cited 1×
“A qualitative analysis [16,17] was carried out by 2 of the authors (CG and WJ) to classify the outcomes reported in the articles’ abstract according to the following 3 levels.”
MethodCochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,178
Utilization of the PICO framework to improve searching PubMed for clinical questions
Schardt, C., Adams, M. B., Owens, T., Keitz, S., & Fontelo, P. (2007) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-7-16 · Cited by 3,053 · cited 1×
“the search terms and define the search string following the patient, intervention, comparison, and outcome (PICO) framework [15].”
BackgroundAgeing populations: the challenges ahead
Christensen, K., Doblhammer, G., Rau, R., & Vaupel, J. W. (2009) · The Lancet · doi:10.1016/s0140-6736(09)61460-4 · Cited by 3,011 · cited 1×
“Additionally, global demographic trends, such as the increasing number of elderly people, have been changing healthcare delivery due to a growing demand for long-term care and increasing costs [2-5].”
BackgroundKawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F. (2005) · BMJ · doi:10.1136/bmj.38398.500764.8f · Cited by 2,703
Personal Health Records: Definitions, Benefits, and Strategies for Overcoming Barriers to Adoption
Tang, P. C., Ash, J. S., Bates, D. W., Overhage, J. M., & Sands, D. Z. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2025 · Cited by 1,437 · cited 1×
“Multimedia Appendix 1 [19-239] presents the outcomes.”
Patient care information systems and health care work: a sociotechnical approach
Berg, M. (1999) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(99)00011-8 · Cited by 786 · cited 2×
“Considering the aforementioned reports on the failure of eHealth interventions [8-11], it appears reasonable to assume that the promises represented by these three pillars are not often accomplished.”
Background
Research works printed but not linked: 33
- Ågren, G, 2006, Healthy Ageing -- A Challenge for Europe
- Goldberg, HS, 2001, Stud Health Technol Inform, 84, 1455
- 2011, ISO/IEC/IEEE 42010:2011(E) (Revision of ISO/IEC 42010:2007 and IEEE Std 1471-2000)
- Ehrler, F, 2014, Stud Health Technol Inform, 205, 667
- Sølling, IK, 2014, Stud Health Technol Inform, 201, 219
- Zhang, X, 2014, Stud Health Technol Inform, 204, 176
- El-Mahalli, AA, 2012, Perspect Health Inf Manag, 9, 1
- Schnall, R, 2009, Stud Health Technol Inform, 146, 248
- Saranummi, N, 2016, Stud Health Technol Inform, 224, 9
- Rossos, PG, 2015, Stud Health Technol Inform, 209, 121
- Barrett, M, 2009, Rural Remote Health, 9, 1154
- Foldy, S, 2007, WMJ, 106, 120
- Mattocks, K, 2007, Conn Med, 71, 27
- Ayatollahi, H, 2014, Perspect Health Inf Manag, 11, 1c
- Alkhaldi, B, 2014, Stud Health Technol Inform, 205, 875
- Alsulame, K, 2015, Stud Health Technol Inform, 213, 233
- Owens, DK, 1998, West J Med, 168, 166
- Doupi, P, 2016, Stud Health Technol Inform, 222, 220
- Bond, CS, 2014, Stud Health Technol Inform, 206, 1
- Griebel, L, 2013, Stud Health Technol Inform, 192, 965
- Schwarz, F, 2014, Rural Remote Health, 14, 2871
- Hanna, L, 2011, Inform Prim Care, 19, 17
- Lober, WB, 2008, AMIA Annu Symp Proc, 434
- Taylor, J, 2015, Nurs Times, 111, 18
- Vimarlund, V, 2013, Stud Health Technol Inform, 192, 933
- Meyer, R, 2010, Yearb Med Inform, 21
- Mars, M, 2010, Yearb Med Inform, 87
- Ruxwana, N, 2014, Health Inf Manag, 43, 26
- Anvari, M, 2004, Semin Laparosc Surg, 11, 123
- Dadd, M, 2002, J Telemed Telecare, 8 Suppl 3, S3:11
- Courtney, KL, 2008, Inform Prim Care, 16, 195
- Gardner, RM, 2004, Respir Care, 49, 378
- Bowens, FM, 2010, Perspect Health Inf Manag, 7, 1d
Cited by
Patient Satisfaction With Telemedicine During the COVID-19 Pandemic: Retrospective Cohort Study
Ramaswamy, A., Yu, M., Drangsholt, S., Ng, E., Culligan, P. J., Schlegel, P. N., & Hu, J. C. (2020) · Journal of Medical Internet Research · doi:10.2196/20786 · Cited by 475
“This is problematic because patient satisfaction has been cited as the most important factor in the success of telemedicine initiatives [33].”
BackgroundeHealth for improving quality of life in breast cancer patients: A systematic review
Triberti, S., Savioni, L., Sebri, V., & Pravettoni, G. (2019) · Cancer Treatment Reviews · doi:10.1016/j.ctrv.2019.01.003 · Cited by 184
Dugstad, J., Eide, T., Nilsen, E. R., & Eide, H. (2019) · BMC Health Services Research · doi:10.1186/s12913-019-4191-1 · Cited by 174
“This breach is traditionally considered to be a major barrier to implementation [22, 39].”
BackgroundTossaint-Schoenmakers, R., Versluis, A., Chavannes, N. H., Talboom-Kamp, E. P. W. A., & Kasteleyn, M. J. (2021) · Journal of Medical Internet Research · doi:10.2196/27180 · Cited by 232
“Inflexibility and complexity of the technology comes at the expense of effective daily processes and their quality [53,54].”
BackgroundTelemedicine in Orthopaedic Surgery: Challenges and Opportunities
Makhni, M. C., Riew, G. J., & Sumathipala, M. G. (2020) · Journal of Bone and Joint Surgery · doi:10.2106/jbjs.20.00452 · Cited by 156
Metadata
- Authors
- Granja, C., Janssen, W., & Johansen, M. A.
- Affiliations
- UiT The Arctic University of Norway (NO)
- Corresponding author
- Granja, Conceição
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2018-05-01
- Volume
- 20
- Issue
- 5
- Pages
- e10235
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC5954232
- OpenAlex
- W2801287189
- Cited by
- 578
- References
- 250
- Citations by year
- 2026: 40 · 2025: 47 · 2024: 68 · 2023: 64 · 2022: 128 · 2021: 91 · 2020: 76 · 2019: 48 · 2018: 15
- FWCI
- 10.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- NORA - Norwegian Open Research Archives (submittedVersion, other-oa)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- digital health interventions, implementation success factors, implementation barriers, workflow disruption, workload, role definition, face-to-face communication, employee turnover, quality of care, implementation costs, clinical process
- MeSH terms
- Humans, Medical Informatics, Telemedicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
