Developing and Evaluating Digital Public Health Interventions Using the Digital Public Health Framework DigiPHrame: A Framework Development Study
Open accessPublished version of a preprint
Abstract
Background: Digital public health (DiPH) interventions may help us tackle substantial public health challenges and reach historically underserved populations, in addition to presenting valuable opportunities to improve and complement existing services. However, DiPH interventions are often triggered through technological advancements and opportunities rather than public health needs. To develop and evaluate interventions designed to serve public health needs, a comprehensive framework is needed that systematically covers all aspects with relevance for public health. This includes considering the complexity of the technology, the context in which the technology is supposed to operate, its implementation, and its effects on public health, including ethical, legal, and social aspects. Objective: We aimed to develop such a DiPH framework with a comprehensive list of core principles to be considered throughout the development and evaluation process of any DiPH intervention. Methods: The resulting digital public health framework (DigiPHrame) was based on a scoping review of existing digital health and public health frameworks. After extracting all assessment criteria from these frameworks, we clustered the criteria. During a series of multidisciplinary meetings with experts from the Leibniz ScienceCampus Digital Public Health, we restructured each domain to represent the complexity of DiPH. In this paper, we used a COVID-19 contact–tracing app as a use case to illustrate how DigiPHrame may be applied to assess DiPH interventions. Results: The current version of DigiPHrame consists of 182 questions nested under 12 domains. Domain 1 describes the current status of health needs and existing interventions; domains 2 and 3, the DiPH technology under assessment and aspects related to human-computer interaction, respectively; domains 4 and 5, structural and process aspects, respectively; and domains 6-12, contextual conditions and the outcomes of the DiPH intervention from broad perspectives. In the CWA use case, a number of questions relevant during its development but also important for assessors once the CWA was available were highlighted. Conclusions: DigiPHrame is a comprehensive framework for the development and assessment of digital technologies designed for public health purposes. It is a living framework and will, therefore, be updated regularly and as new public health needs and technological advancements emerge.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The adoption of wearables is shaped by the economic interests of wearable producers and commercial factors.
“The adoption of wearables is also shaped by the economic interests of wearable producers and commercial factors.”
From Sociodemographics and Digital Health Literacy in Using Wearables for Health Promotion and Disease Prevention: Cross-Sectional Nationwide Survey in Germany (2025) · cites this paper as Jahnel et al., 2024
The Introduction (Background; page unavailable) says that the cost of purchasing an app may influence its adoption, supporting an economic factor in digital-health uptake. It does not address wearables specifically or say that wearable producers’ economic interests shape their adoption. (LLM, conf=0.76)
Experimental — accuracy not yet measuredWearable companies often determine product design, pricing, accessibility, and affordability.
“It is also important to consider that these wearable companies often determine product design, pricing, accessibility, and affordability (Jahnel et al., 2024).”
From Sociodemographics and Digital Health Literacy in Using Wearables for Health Promotion and Disease Prevention: Cross-Sectional Nationwide Survey in Germany (2025) · cites this paper as Jahnel et al., 2024
The Introduction > Background discusses app purchase costs and factors affecting the design of digital public health interventions. The Results > Use Case: Corona-Warn-App discusses accessibility and costs, but neither section says that wearable companies often determine product design, pricing, accessibility, or affordability. Searches for wearable companies and manufacturers found no passage addressing that claim; page numbers were not provided in our copy. (LLM, conf=0.73)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0850 · Cited by 43,565 · see citing works
Principles of Biomedical Ethics
(2012) · Thieves of Virtue · doi:10.7551/mitpress/9079.003.0009 · Cited by 13,453 · see citing works
Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz
(2013) · Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz · doi:10.1007/103.1437-1588 · Cited by 11,769 · see citing works
Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8775 · Cited by 2,887 · cited 1× · see citing works
“Thus, these aspects may influence the design of a DiPH intervention and need to be considered from the beginning of the development process [10].”
Digital technologies in the public-health response to COVID-19
Budd, J., Miller, B. S., Manning, E. M., Lampos, V., Zhuang, M., Edelstein, M., Rees, G., Emery, V. C., Stevens, M. M., Keegan, N., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1011-4 · Cited by 1,195 · cited 1× · see citing works
“Digital platforms of health authorities and national agencies played a critical role in rapidly engaging and educating the population through prompt dissemination of trusted and tailored public health information, while limiting the visibility of information from unreliable sources [5].”
Borghouts, J., Eikey, E., Mark, G., De Leon, C., Schueller, S. M., Schneider, M., Stadnick, N., Zheng, K., Mukamel, D., & Sorkin, D. H. (2021) · Journal of Medical Internet Research · doi:10.2196/24387 · Cited by 1,069 · cited 1× · see citing works
“Individual decisions around the initial use, adoption, rejection, and continued use of an app might be influenced by concerns regarding data security and data protection issues, costs to purchase the app, or user-friendliness for different user groups [8,9].”
Digital health for aging populations
Chen, C., Ding, S., & Wang, J. (2023) · Nature Medicine · doi:10.1038/s41591-023-02391-8 · Cited by 525 · see citing works
Mummah, S. A., Robinson, T. N., King, A. C., Gardner, C. D., & Sutton, S. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5927 · Cited by 388 · see citing works
Whitelaw, S., Pellegrini, D. M., Mamas, M. A., Cowie, M., & Van Spall, H. G. C. (2021) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztab005 · Cited by 241 · see citing works
The HTA Core Model: A novel method for producing and reporting health technology assessments
Lampe, K., Mākelä, M., Garrido, M. V., et al. (2009) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462309990638 · Cited by 220 · see citing works
Deriving a practical framework for the evaluation of health apps
Henson, P., David, G., Albright, K., & Torous, J. B. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30013-5 · Cited by 182 · see citing works
Lorencatto, F., Charani, E., Sevdalis, N., Tarrant, C., & Davey, P. G. (2018) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dky222 · Cited by 152 · see citing works
mHealth Assessment: Conceptualization of a Global Framework
Bradway, M., Carrion, C., Vallespin, B., Saadatfard, O., Puigdomènech, E., Espallargues, M., & Kotzeva, A. (2017) · JMIR mHealth and uHealth · doi:10.2196/mhealth.7291 · Cited by 142 · see citing works
Unsworth, H., Dillon, B., Collinson, L., Powell, H., Salmon, M., Oladapo, T., Ayiku, L., Shield, G., Holden, J., Patel, N., et al. (2021) · Digital Health · doi:10.1177/20552076211018617 · Cited by 137 · see citing works
Conceptual Framework for Development of Comprehensive e-Health Evaluation Tool
Khoja, S., Durrani, H., Scott, R. E., Sajwani, A., & Piryani, U. (2012) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2012.0073 · Cited by 128 · see citing works
A design and evaluation framework for digital health interventions
Kowatsch, T., Otto, L., Harperink, S., Cotti, A., & Schlieter, H. (2019) · it - Information Technology · doi:10.1515/itit-2019-0019 · Cited by 118 · see citing works
Rodriguez-Villa, E., & Torous, J. B. (2019) · BMC Medicine · doi:10.1186/s12916-019-1447-x · Cited by 71 · see citing works
Digital public health and COVID-19
Murray, C. J. L., Alamro, N., Hwang, H. Y., & Lee, U. (2020) · The Lancet Public Health · doi:10.1016/s2468-2667(20)30187-0 · Cited by 70 · see citing works
Digital Public Health – ein Überblick
Bremen, L.-W. D. P. H., Zeeb, H., Pigeot, I., & Schüz, B. (2020) · Bundesgesundheitsblatt - Gesundheitsforschung - Gesundheitsschutz · doi:10.1007/s00103-019-03078-7 · Cited by 59 · see citing works
Biviji, R., Vest, J. R., Dixon, B. E., Cullen, T., & Harle, C. A. (2020) · JMIR mhealth and uhealth · doi:10.2196/15663 · Cited by 56 · see citing works
Stratil, J. M., Baltussen, R. M. P. M., Scheel, I. B., Nacken, A., & Rehfuess, E. A. (2020) · Cost Effectiveness and Resource Allocation · doi:10.1186/s12962-020-0203-6 · Cited by 47 · see citing works
Pozo-Martin, F., Beltrán, M., Müller, S. A., Diaconu, V., Weil, K., & Bcheraoui, C. E. (2023) · European Journal of Epidemiology · doi:10.1007/s10654-023-00963-z · Cited by 47 · see citing works
Health technology assessment: a framework
Joore, M. A., Grimm, S. E., Boonen, A. E. R. C. H., et al. (2020) · RMD Open · doi:10.1136/rmdopen-2020-001289 · Cited by 46 · see citing works
A Conceptual Framework for Life-Cycle Health Technology Assessment
Kirwin, E. M., Round, J. A., Bond, K., & McCabe, C. J. (2022) · Value in Health · doi:10.1016/j.jval.2021.11.1373 · Cited by 41 · see citing works
A Contextual Factors Framework to Inform Implementation and Evaluation of Public Health Initiatives
Vanderkruik, R. C., & McPherson, M. E. (2016) · American Journal of Evaluation · doi:10.1177/1098214016670029 · Cited by 40 · see citing works
Digital health for climate change mitigation and response: a scoping review
Rahimi‐Ardabili, H., Magrabi, F., & Coiera, E. W. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac134 · Cited by 40 · see citing works
Cybercare 2.0: meeting the challenge of the global burden of disease in 2030
Rosen, J. M., Kun, L. G., Mosher, R. E., Grigg, E., Merrell, R. C., Macedonia, C. R., Klaudt-Moreau, J., Price-Smith, A. T., & Geiling, J. (2016) · Health and Technology · doi:10.1007/s12553-016-0132-8 · Cited by 33 · see citing works
The German COVID-19 Digital Contact Tracing App: A Socioeconomic Evaluation
Ellmann, S., Maryschok, M., Schöffski, O., & Emmert, M. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192114318 · Cited by 15 · see citing works
Organised by: Leibniz ScienceCampus Digital Public Health Bremen LSC (Germany), E.-D., & Chair persons: Laura Maass (Germany), C.-C. P. (. (2022) · European Journal of Public Health · doi:10.1093/eurpub/ckac129.147 · Cited by 2 · see citing works
Jahnel, T., Pan, C.-C., Barnils, N. P., Jürgens, D., Müllmann, S., Janetzki, S., Kolschen, J., Freye, M., Dassow, H.-H., Lange, O., et al. (2023) · Das Gesundheitswesen · doi:10.1055/s-0043-1770419 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Pan, C.-C., De Santis, K. K., Muellmann, S., Hoffmann, S., Spallek, J., Barnils, N. P., Ahrens, W., Zeeb, H., & Schüz, B. (2025) · Journal of Prevention · doi:10.1007/s10935-024-00821-y · Cited by 23 · see citing works
Self-citation
- Authors
- Jahnel, T., Pan, C.-C., Pedros Barnils, N., Muellmann, S., Freye, M., Dassow, H.-H., Lange, O., Reinschluessel, A. V., Rogowski, W., & Gerhardus, A.
- Affiliations
- University of Bremen (DE)
- Leibniz Institute for Prevention Research and Epidemiology - BIPS (DE)
- Corresponding author
- Jahnel, Tina
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2024-09-12
- Volume
- 26
- Pages
- e54269
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC11427851
- OpenAlex
- W4400099731
- Cited by
- 15
- References
- 30
- Citations by year
- 2026: 7 · 2025: 6 · 2024: 2
- FWCI
- 8.81
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- PUBLISSO (German National Library of Medicine) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- KOPS (University of Konstanz) (submittedVersion)
- Funders
- Freie Hansestadt Bremen
- Leibniz-Gemeinschaft (W4/2018)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Physical Sciences › Computer Science › Information Systems › COVID-19 Digital Contact Tracing
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- digital public health, framework development, digital health interventions, intervention development, intervention evaluation, COVID-19 contact tracing apps, public health needs, human-computer interaction, contextual factors
- MeSH terms
- COVID-19, Humans, Public Health, Telemedicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
