Methods to Evaluate the Effects of Internet-Based Digital Health Interventions for Citizens: Systematic Review of Reviews
Open accessPublished version of a preprint
The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Digital health can empower citizens to manage their health and address health care system problems including poor access, uncoordinated care and increasing costs. Digital health interventions are typically complex interventions. Therefore, evaluations present methodological challenges. OBJECTIVE: The objective of this study was to provide a systematic overview of the methods used to evaluate the effects of internet-based digital health interventions for citizens. Three research questions were addressed to explore methods regarding approaches (study design), effects and indicators. METHODS: We conducted a systematic review of reviews of the methods used to measure the effects of internet-based digital health interventions for citizens. The protocol was developed a priori according to Preferred Reporting Items for Systematic review and Meta-Analysis Protocols and the Cochrane Collaboration methodology for overviews of reviews. Qualitative, mixed-method, and quantitative reviews published in English or French from January 2010 to October 2016 were included. We searched for published reviews in PubMed, EMBASE, The Cochrane Database of Systematic Reviews, CINHAL and Epistemonikos. We categorized the findings based on a thematic analysis of the reviews structured around study designs, indicators, types of interventions, effects and perspectives. RESULTS: A total of 20 unique reviews were included. The most common digital health interventions for citizens were patient portals and patients' access to electronic health records, covered by 10/20 (50%) and 6/20 (30%) reviews, respectively. Quantitative approaches to study design included observational study (15/20 reviews, 75%), randomized controlled trial (13/20 reviews, 65%), quasi-experimental design (9/20 reviews, 45%), and pre-post studies (6/20 reviews, 30%). Qualitative studies or mixed methods were reported in 13/20 (65%) reviews. Five main categories of effects were identified: (1) health and clinical outcomes, (2) psychological and behavioral outcomes, (3) health care utilization, (4) system adoption and use, and (5) system attributes. Health and clinical outcomes were measured with both general indicators and disease-specific indicators and reported in 11/20 (55%) reviews. Patient-provider communication and patient satisfaction were the most investigated psychological and behavioral outcomes, reported in 13/20 (65%) and 12/20 (60%) reviews, respectively. Evaluation of health care utilization was included in 8/20 (40%) reviews, most of which focused on the economic effects on the health care system. CONCLUSIONS: Although observational studies and surveys have provided evidence of benefits and satisfaction for patients, there is still little reliable evidence from randomized controlled trials of improved health outcomes. Future evaluations of digital health interventions for citizens should focus on specific populations or chronic conditions which are more likely to achieve clinically meaningful benefits and use high-quality approaches such as randomized controlled trials. Implementation research methods should also be considered. We identified a wide range of effects and indicators, most of which focused on patients as main end users. Implications for providers and the health system should also be included in evaluations or monitoring of digital health interventions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, these critical reflections consistently stop at that point, providing no concrete, actionable solutions to address the limitations they revealed [54-57].”
From Use of the Principles of Design Thinking to Address Limitations of Digital Mental Health Interventions for Youth: Viewpoint (2019) · cites this paper as [57]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223 · cited 2×
“A systematic review of reviews [18] focused on the methods used to measure the effects of internet-based digital health interventions for citizens was conducted in accordance with the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) [19].”
MethodShamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & the PRISMA-P Group (2015) · BMJ · doi:10.1136/bmj.g7647 · Cited by 13,120 · cited 1×
“This includes a checklist of recommended items to be addressed in a systematic review protocol [20].”
BackgroundA typology of reviews: an analysis of 14 review types and associated methodologies
Grant, M. J., & Booth, A. (2009) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2009.00848.x · Cited by 10,817 · cited 1×
“The general characteristics of the studies, including the type of review [45], population, intervention, setting, and quality, were summarized by the information extracted from the analysis of the full-text articles (Table 1).”
MethodDevelopment of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-10 · Cited by 4,374
Eysenbach, G., & CONSORT-EHEALTH Group (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1923 · Cited by 1,863
Free, C., Phillips, G., Galli, L., Watson, L., Felix, L., Edwards, P., Patel, V., & Haines, A. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001362 · Cited by 1,766 · cited 1×
“Patient portals [4], mobile health applications [5] and patients’ access to electronic health records (EHR) are common examples of digital health interventions.”
Murray, E., Treweek, S., Pope, C., MacFarlane, A., Ballini, L., Dowrick, C., Finch, T., Kennedy, A., Mair, F., O'Donnell, C., et al. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-63 · Cited by 1,478 · cited 1×
“[48].”
Mays, N., Pope, C., & Popay, J. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308576 · Cited by 1,375
Methodology in conducting a systematic review of systematic reviews of healthcare interventions
Smith, V., Devane, D., Begley, C. M., & Clarke, M. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-15 · Cited by 1,227 · cited 1×
“A systematic review of reviews [18] focused on the methods used to measure the effects of internet-based digital health interventions for citizens was conducted in accordance with the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) [19].”
MethodPeters, D. H., Adam, T., Alonge, O., Agyepong, I. A., & Tran, N. (2014) · British Journal of Sports Medicine · doi:10.1136/bmj.f6753 · Cited by 1,109
Research works printed but not linked: 4
- Mold, F, 2012, Inform Prim Care, 20, 271
- 2016, Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment
- Lau, F, 2007, Healthc Q, 10, 112
- Ammenwerth, E, 2011, Stud Health Technol Inform, 165, 63
Cited by
Barteit, S., Lanfermann, L., Bärnighausen, T., Neuhann, F., & Beiersmann, C. (2021) · JMIR Serious Games · doi:10.2196/29080 · Cited by 547
The State of Evidence in Patient Portals: Umbrella Review
Antonio, M. G., Petrovskaya, O., & Lau, F. (2020) · Journal of Medical Internet Research · doi:10.2196/23851 · Cited by 146
“In addition to focusing on portal technology, both meta-level reviews [19,20] and several SRs [21-23] reported combined findings on a variety of eHealth tools, which may limit the reader’s ability to discern portal-specific effects.”
BackgroundInfluentialScholten, H., & Granic, I. (2019) · Journal of Medical Internet Research · doi:10.2196/11528 · Cited by 138
“However, these critical reflections consistently stop at that point, providing no concrete, actionable solutions to address the limitations they revealed [54-57].”
Marthick, M., McGregor, D., Alison, J. A., Cheema, B. S., Dhillon, H. M., & Shaw, T. J. (2021) · Journal of Medical Internet Research · doi:10.2196/24722 · Cited by 139
“Digital health has demonstrated potential in engaging people in their care [15], including as a tool for the treatment and self-management training of chronically ill patients [16,17].”
Zanaboni, P., Kummervold, P. E., Sørensen, T., & Johansen, M. A. (2020) · Journal of Medical Internet Research · doi:10.2196/16144 · Cited by 101
“Future evaluations of PAEHRs should focus on specific populations or chronic conditions that are more likely to achieve clinically meaningful benefits and use randomized controlled trials or implementation research methods [37,47].”
Self-citationIs research on patient portals attuned to health equity? A scoping review
Antonio, M. G., Petrovskaya, O., & Lau, F. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz054 · Cited by 98
“Across multiple reviews, enhanced communication offered through patient portals was a frequent finding;(26,28,30,31) whereas evidence was mixed as to whether portals improved patient health outcomes, and insufficient that portals support patient empowerment(25) and reduce cost and health care utilization.(27,31) A sub…”
BackgroundWani, C., McCann, L., Lennon, M., & Radu, C. (2024) · Journal of Medical Internet Research · doi:10.2196/51376 · Cited by 100
Effects of Digital Technologies on Older People’s Access to Health and Social Care: Umbrella Review
Kunonga, P., Spiers, G., Beyer, F. R., Hanratty, B., Boulton, E., Hall, A., Bower, P., Todd, C. J., & Craig, D. (2021) · Journal of Medical Internet Research · doi:10.2196/25887 · Cited by 86
“A recent review of approaches to the evaluation of digital health interventions identified little evidence from randomized controlled trials and carried out measurement of service utilization in only a minority of the studies [45].”
BackgroundLorca-Cabrera, J., Ferré‐Grau, C., Arques, R. M., Raigal‐Aran, L., Falcó‐Pegueroles, A., & Albacar-Riobóo, N. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.104003 · Cited by 80
Loughnane, C., Laiti, J., O’Donovan, R., & Dunne, P. J. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1536416 · Cited by 76
Digital Patient Experience: Umbrella Systematic Review
Wang, T., Giunti, G., Melles, M., & Goossens, R. (2022) · Journal of Medical Internet Research · doi:10.2196/37952 · Cited by 77
“The latter is commonly mentioned [48,53,70], with some authors suggesting that patients with unstable chronic diseases might benefit the most [47,93].”
BackgroundImpact of digital services on healthcare and social welfare: An umbrella review
Härkönen, H., Lakoma, S., Verho, A., Torkki, P. M., Leskelä, R., Pennanen, P., Laukka, E., & Jansson, M. M. (2024) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2024.104692 · Cited by 80
Adult patient access to electronic health records
Ammenwerth, E., Neyer, S., Hörbst, A., Mueller, G., Siebert, U., & Schnell‐Inderst, P. (2021) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012707.pub2 · Cited by 61
Ahmadvand, A., Kavanagh, D., Clark, M., Drennan, J., & Nissen, L. (2019) · Journal of Medical Internet Research · doi:10.2196/10477 · Cited by 53
“Over the past two decades, various digital health options, technologies, and innovations have been introduced; many of them are still being investigated and evaluated by researchers all around the globe [7].”
Kelley, L., Fujioka, J. K., Liang, K., Cooper, M., Jamieson, T., & Desveaux, L. (2020) · JMIR Public Health and Surveillance · doi:10.2196/20579 · Cited by 61
“Most health professionals seek randomized controlled trials as proof of value [24,25].”
BackgroundHoll, F., Kircher, J., Hertelendy, A. J., Sukums, F. K., & Swoboda, W. J. (2024) · Journal of Medical Internet Research · doi:10.2196/52150 · Cited by 51
mHealth applications: potentials, limitations, current quality and future directions
Messner, E.-M., Probst, T., O’Rourke, T., Stoyanov, S., & Baumeister, H. (2019) · Studies in neuroscience, psychology and behavioral economics · doi:10.1007/978-3-030-31620-4_15 · Cited by 49
Ferrand, J. L., Hockensmith, R., Houghton, R. F., & Buhi, E. R. (2020) · Journal of Medical Internet Research · doi:10.2196/19018 · Cited by 51
“While considerable research has been conducted assessing digital health approaches, such as text message–based approaches and mobile apps for a range of health behaviors [7-9], few have included smart assistant responses to health-related queries.”
BackgroundEasy data, same old platforms? A systematic review of digital activism methodologies
Özkula, S. M., Reilly, P., & Hayes, J. (2022) · Information Communication & Society · doi:10.1080/1369118x.2021.2013918 · Cited by 47
“Although systematic reviews often include methodological re fl ections (e.g., Zanaboni et al., 2018; Zhu et al., 2018), purely methodologically oriented reviews remain rare.”
BackgroundHall, C. L., Sanderson, C., Brown, B. J., Andrén, P., Bennett, S. D., Chamberlain, L. R., Davies, E. B., Khan, K., Kouzoupi, N., Mataix‐Cols, D., et al. (2020) · Trials · doi:10.1186/s13063-020-04902-1 · Cited by 37
Implementierung und partizipative Gestaltung digitaler Gesundheitsinterventionen
Hochmuth, A., Exner, A.-K., & Dockweiler, C. (2020) · Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz · doi:10.1007/s00103-019-03079-6 · Cited by 33
Lorca-Cabrera, J., Arques, R. M., Albacar-Riobóo, N., Raigal‐Aran, L., Roldán-Merino, J. F., & Ferré‐Grau, C. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104310 · Cited by 34
Farmer, C., O’Connor, D. A., Lee, H., McCaffery, K. J., Maher, C. G., Newell, D., Cashin, A. G., Byfield, D. C., Jarvik, J. G., & Buchbinder, R. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-049938 · Cited by 31
“Common and usually non-serious terms in lumbar spine imaging reports are poorly understood by the general population and may contribute to the burden of LBP. Incorporating clear explanations about the implications of these findings may reduce unwarranted anxiety and reduce low-value care.”
Digital health in oncology in Africa: A scoping review and cross-sectional survey
Kabukye, J. K., Kakungulu, E., de Keizer, N. F., & Cornet, R. (2021) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2021.104659 · Cited by 31
Avdagovska, M., Menon, D., & Stafinski, T. (2020) · Journal of Medical Internet Research · doi:10.2196/24568 · Cited by 28
“The following sections summarize the studies according to the QA dimensions (Multimedia Appendices 3-6) [27,43,44,46-48, 53,55-58,60-63,65,67,68,72-103,105-114,119,126,127,129].”
MethodKröner, S., Schreiweis, B., Strotbaum, V., Brandl, L. C., Pobiruchin, M., & Wiesner, M. (2025) · BMC Health Services Research · doi:10.1186/s12913-024-12175-6 · Cited by 21
“Technological progress drives the introduction of digital health technologies [1].”
Gagnon, M., Ndiaye, M. A., Larouche, A., Chabot, G., Chabot, C., Buyl, R., Fortin, J., Giguere, A. M. C., LeBlanc, A., Légaré, F., et al. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-028554 · Cited by 19
Self-citationAssociation of Dental Caries with Use of Internet and Social Media among 12 and 15-Year-Olds
Almoddahi, D., Machuca Vargas, C., & Sabbah, W. (2022) · Acta Odontologica Scandinavica · doi:10.1080/00016357.2021.1951349 · Cited by 17
“Hence, internet-based health interventions have considerable potential in changing people’s behaviours towards health management and empowerment [27].”
BackgroundEvaluation of Patient-Facing Mobile Apps to Support Physiotherapy Care: Systematic Review
Merolli, M., Francis, J., Vallance, P., Bennell, K. L., Malliaras, P., & Hinman, R. S. (2024) · JMIR mhealth and uhealth · doi:10.2196/55003 · Cited by 17
Evaluating digital health attributes for users’ satisfaction: an application of the Kano model
Cavacece, Y., Maggiore, G., Resciniti, R., & Tartaglione, A. M. (2024) · The TQM Journal · doi:10.1108/tqm-09-2023-0301 · Cited by 23
Jia, H. (2024) · BMC Public Health · doi:10.1186/s12889-024-20730-3 · Cited by 17
Bonet, L., Torous, J. B., Arce, D., Blanquer, I., & Sanjuán, J. (2020) · Early Intervention in Psychiatry · doi:10.1111/eip.12960 · Cited by 16
“A third limitation is the impact of digital health interventions on clinicians' workload and clinical efficiency remains understudied (Gitlow et al., 2017; Hoerbst & Schweitzer, 2015; Zanaboni et al., 2018).”
BackgroundYe, L., Yang, S., Jiang, B., Liu, C., Jin, X., & Chang, P. (2024) · PeerJ · doi:10.7717/peerj.18281 · Cited by 16
Merolli, M., Hinman, R. S., Lawford, B. J., Choo, D., & Gray, K. M. (2021) · JMIR Research Protocols · doi:10.2196/25177 · Cited by 18
“This is further supported by Zanaboni et al [28], who advise that to build a reliable evidence base about digital health use and health outcomes, greater focus needs to be placed on clinical research in the form of high-quality randomized controlled trials.”
MethodAlsaqqa, H. H., & Alwawi, A. A. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1142443 · Cited by 11
“This is further confirmed by Zanaboni et al. (59), who state that more emphasis should be given to clinical research in the form of high-quality randomized controlled trials in order to run a credible evidence base about the use of digital health and health results.”
BackgroundFreitag, B., Uncovska, M., Meister, S., Prinz, C., & Fehring, L. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01324-0 · Cited by 11
Bjerkan, J., Kane, B. T., Uhrenfeldt, L., Veie, M., & Fossum, M. (2020) · JMIR Research Protocols · doi:10.2196/16781 · Cited by 11
“Both the lack of appropriate user interfaces and economic barriers have, until recently, also restricted the system adoption [11].”
BackgroundWeirauch, V., Soehnchen, C., Burmann, A., & Meister, S. (2024) · Journal of Medical Internet Research · doi:10.2196/55714 · Cited by 11
“Although RCTs are considered as the gold standard [6], especially in the context of evaluating DHIs, they are a much-discussed topic.”
BackgroundMethodResultInfluentialDöring, N. M., & Lehmann, S. (2023) · Zeitschrift für Sexualforschung · doi:10.1055/a-2055-3160 · Cited by 10
Digital Health Technology Implementation: Is It Effective in a Healthy Healthcare Perspective?
Stein, A. T., Ben, Â. J., Pachito, D. V., Cazella, S. C., van Dongen, J. M., & Bosmans, J. E. (2020) · Springer eBooks · doi:10.1007/978-3-030-59467-1_9 · Cited by 10
Strategies to improve recruitment in mental health clinical trials: a scoping review (RE-MIND study)
Iflaifel, M., Hall, C. L., Green, H. R., Willis, A., Rennick‐Egglestone, S., Juszczak, E., Townsend, M., Martin, J. L., & Sprange, K. (2024) · Trials · doi:10.1186/s13063-024-08665-x · Cited by 9
Hua, J., Sun, D., Wang, H., Chang, J., Guo, F., Zhou, Q. H., Hui, N., Gu, D., & Zhu, L. (2025) · Digital Health · doi:10.1177/20552076251387050 · Cited by 8
Othman, M. O., & Menon, K. V. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.03.019 · Cited by 8
Schlieter, H. (2021) · Monitor Versorgungsforschung · doi:10.24945/mvf.02.21.1866-0533.2301 · Cited by 9
Pasat, Z., Sinn, C. J., Rahman, B., Gayowsky, A., Lokker, C., Tarride, J., Alarakhia, M., & Costa, A. P. (2024) · PLoS ONE · doi:10.1371/journal.pone.0299005 · Cited by 8
“Digital health technologies can enhance patient experience and patient-provider communication by supporting accessible and timely information flow and team-based processes [11– 14].”
BackgroundResultPerspectives of New Zealand patients and GPs at the beginning of patient portal implementation
Wells, S. J., Mahony, F., Huang, Y., & Day, K. (2019) · Journal of Primary Health Care · doi:10.1071/hc19016 · Cited by 7
Gityamwi, N., Armes, J., Harris, J., Ream, E., Green, R., Ahankari, A. S., Callwood, A., Ip, A., Cockle‐Hearne, J., Grosvenor, W., et al. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000806 · Cited by 6
“Consequently, RCTs have been widely employed when testing digital health interventions [5,6].”
Quality of the digital gp visits and characteristics of the users: retrospective observational study
Lakoma, S., Pasanen, H., Lahdensuo, K., Pehkonen, J., Viinikainen, J., & Torkki, P. M. (2024) · Scandinavian Journal of Primary Health Care · doi:10.1080/02813432.2024.2380921 · Cited by 8
“Patients with chronic diseases significantly benefit from using digital services, such as patient portals [10], but long-term patients form a significant minority in primary health care relative to occasional users [20].”
BackgroundDigital Health Interventions: New opportunities for health science librarians
Murphy, J. (2021) · Health Information & Libraries Journal · doi:10.1111/hir.12374 · Cited by 7
mHealth Applications: Potentials, Limitations, Current Quality and Future Directions
Meßner, E.-M., Probst, T., O’Rourke, T., Stoyanov, S. R., & Baumeister, H. (2022) · Studies in neuroscience, psychology and behavioral economics · doi:10.1007/978-3-030-98546-2_18 · Cited by 6
Islam, S., Elaiho, C. R., Arniella, G., Rivera, S., & Vangeepuram, N. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph191912286 · Cited by 7
“There are also privacy concerns that have been emerging in the digital realm that should be considered [65,66].”
BackgroundPerrin, C., Bediang, G., Randriambelonoro, M., & Geissbühler, A. (2019) · Frontiers in Public Health · doi:10.3389/fpubh.2019.00188 · Cited by 6
Burger, S.-A., Poots, A. J., Perris, A., Crump, H., Thorne, H., Hughes, S., & West, J. A. (2020) · Patient Experience Journal · doi:10.35680/2372-0247.1418 · Cited by 6
Basholli, A., Λάγκας, Θ., Bath, P. A., & Eleftherakis, G. (2021) · Health Informatics Journal · doi:10.1177/1460458220979350 · Cited by 5
Alichniewicz, K., Hampton, S., Romaniuk, M., Bennett, D., & Guindalini, C. (2025) · JMIR Formative Research · doi:10.2196/60868 · Cited by 5
“Through such evaluations, policy makers, researchers, and support organizations can gain valuable insights into the most beneficial strategies for veterans [15].”
Nursing Documentation in Digital Solutions
Grogan, L., Reed, A., & Fennelly, O. (2021) · Health informatics · doi:10.1007/978-3-030-58740-6_7 · Cited by 3
Evaluating Digitally Enabled Health Services
Gray, C. S., Shen, N., Phạm, Q., & Wiljer, D. (2025) · Springer eBooks · doi:10.1007/978-3-031-87869-5_17 · Cited by 3
Ekman, B., Thulesius, H. O., Wilkens, J., & Arvidsson, E. (2022) · medRxiv · doi:10.1101/2022.06.28.22277001 · Cited by 2
Substance Use Disorder Among Low-Income People in the Deep SouthPreprint
Albright, D. L., Suntai, Z., McDaniel, J. T., Johnson, K. A., Henson, H., Robertson, E., & McIntosh, S. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4133118 · Cited by 2
Digital Patient Experience: Umbrella Systematic Review (Preprint)Preprint
Wang, T., Giunti, G., Melles, M., & Goossens, R. (2022) · doi:10.2196/preprints.37952 · Cited by 2
Shen, K., Hong, Y. A., Zhang, Y., Han, H., Lin, J., & Hepburn, K. W. (2025) · Alzheimer s & Dementia · doi:10.1002/alz.70663 · Cited by 1
Introducing a digital portal that enables patients to access their health records
Balsdon, H. (2021) · Nursing Management · doi:10.7748/nm.2021.1994 · Cited by 1
McCarthy, S. E. (2021) · Advances in media, entertainment and the arts (AMEA) book series · doi:10.4018/978-1-7998-8630-3.ch004 · Cited by 1
Bellinghausen, M., Waerdt, L. M., & Baumeister, H. (2022) · Springer eBooks · doi:10.1007/978-3-658-37903-2_11 · Cited by 1
Kip, H. (2021) · doi:10.3990/1.9789036551311 · Cited by 1
Independently testing prosocial interventions: Methods and recommendations from 31 researchersPreprint
Grüning, D. J., Kamin, J., Saltz, E., Acosta, T., DiFranzo, D. J., Goldberg, B., Leavitt, A., Menczer, F., Musgrave, T., Wang, Y., et al. (2025) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/wvfjq_v2 · Cited by 1
Designing and testing a game intervention to help youth quit smoking
Scholten, H. (2020) · Radboud Repository (Radboud University)
Helsepersonells forventninger til e-konsultasjon – en kvalitativ studie
Thorshov, T. C., Moen, A., & Børøsund, E. (2021) · Sykepleien Forskning · doi:10.4220/sykepleienf.2021.86348
Perrin, C., Randriambelonoro, M., & Geissbühler, A. (2019) · doi:10.2196/preprints.15611
Nassih, O., Sénécal, S., Coursaris, C. K., Léger, P., & Rochon, L.-J. (2022) · doi:10.2196/preprints.39458
The Development of Health Technologies in the Digitalization Era
Prihantari, A., & Tahawy, L. T. L. (2023) · Advances in health sciences research/Advances in Health Sciences Research · doi:10.2991/978-94-6463-206-4_30
Instrumente zur Evaluation der Nützlichkeit und Wirksamkeit digitaler Technologien
Held, P., & Lindwedel, U. (2024) · Springer eBooks · doi:10.1007/978-3-662-68415-3_55
Ekman, B. (2024) · medRxiv · doi:10.1101/2024.12.13.24318993
Bélanger, V., Cousineau, M., Moreault, M.-P., & Motulsky, A. (2025) · BMC Digital Health · doi:10.1186/s44247-025-00189-x
“Reviewing 27 studies of symptom checkers, the evidence on their utility was weak and many methodological challenges to evaluating digital health interventions were highlighted, as already observed by others [15–18].”
Determinants of Quality in the Design, Development and Evaluation of Digital Health Programs
Stoyanov, S. R. (2025) · Queensland University of Technology · doi:10.5204/thesis.eprints.261406
Arques, R. M. (2019) · Rovira & Virgili repository
Sroge, R. A. (2023) · ORCA Online Research @Cardiff (Cardiff University) · Cited by 1
Marthick, M. (2021) · The Sydney eScholarship Repository (The University of Sydney)
Instruments for Evaluating the Usefulness and Effectiveness of Digital Technologies
Held, P., & Lindwedel, U. (2025) · Springer eBooks · doi:10.1007/978-3-662-72298-5_55
Chen, A., Smirnova, A., Chen, F. T., Madu, R., Liverpool, S., Khadjesari, Z., Pereira, E., Trovati, M., & Karkou, V. (2026) · Mental Health and Digital Technologies · doi:10.1108/mhdt-07-2025-0045
Öhl, N., Strack, D., Kusnik, S., Klinger, I., Hofmann, M., Steigleder, T., Weisser, A., Ostgathe, C., Trollmann, R., & Heckel, M. (2026) · Journal of Medical Internet Research · doi:10.2196/83009
“Observations have proven particularly relevant in the context of health technologies [51], as they allow researchers to move beyond subjective intended use and capture actual practices.”
Evaluation of the Barriers to Digital Transformation in Healthcare Organizations With AHP Method
Köse, İ., ŞAHİN, K., & Gedıklı, E. (2026) · IGI Global eBooks · doi:10.4018/979-8-2600-1026-6.ch005
Eriksson, P., Thulesius, H. O., Bergman, P., & Hammar, T. (2026) · Research Square · doi:10.21203/rs.3.rs-10849007/v1
Lange, M., Barisch‐Fritz, B., Baumann, H., Eckert, K. G., Gabrys, L., Heub, A., Kernebeck, S., Maaß, L., Morf, H., Richter, J. G., et al. (2026) · Das Gesundheitswesen · doi:10.1055/a-2930-2321
“Diese Unterschiede führen nicht nur zu eingeschränkter Vergleichbarkeit [78, 79], sondern erschweren zudem eine systematische Qualitätsentwicklung [80, 81].”
Mobile health and non-communicable diseases
Anagnostopoulou, A. (2026) · Global Journal of Health Sciences and Research · doi:10.25259/gjhsr_44_2025
Metadata
- Authors
- Zanaboni, P., Ngangue, P., Mbemba, G. I. C., Schopf, T. R., Bergmo, T. S., & Gagnon, M.-P.
- Affiliations
- University Hospital of North Norway (NO)
- Université du Québec en Abitibi-Témiscamingue (CA)
- Université Laval (CA)
- Corresponding author
- Zanaboni, Paolo
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2018-06-07
- Volume
- 20
- Issue
- 6
- Pages
- e10202
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC6013714
- OpenAlex
- W2804205035
- Cited by
- 79
- References
- 51
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 14 · 2023: 3 · 2022: 9 · 2021: 16 · 2020: 11 · 2019: 10 · 2018: 1
- FWCI
- 1.73
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- digital health interventions, online intervention, assessment methods, patient portals, electronic health records, research design, health outcomes, health care utilization, patient satisfaction, patient-provider communication, system adoption, complex interventions, health system challenges, health care access, care coordination, health care costs, patient empowerment, evaluation metrics, evaluation perspectives
- MeSH terms
- Delivery of Health Care, Humans, Patient Satisfaction, Internet, Qualitative Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
