Developing a digital health competency assessment framework for public health services: a Delphi-AHP approach
Open accessAbstract
Objective: To construct a scientifically validated and practical digital health competency assessment framework for public health services, addressing gaps in current evaluation paradigms that fail to comprehensively assess individuals' proactive engagement with digital health tools. Methods: The study employed a mixed-methods approach, beginning with a systematic literature review and structured interviews to draft an initial indicator system. Two rounds of Delphi consultations with 15 experts were conducted to refine the framework, followed by quantification of indicators using the Analytic Hierarchy Process (AHP). Results: The final framework comprises 3 primary indicators (Health Literacy, Information Literacy, Digital Competency), 9 secondary indicators, and 19 tertiary indicators. Health Literacy and Information Literacy were weighted highest (0.4545 each), reflecting their foundational roles, while Digital Competency (0.0909) served as a complementary enabler. Conclusion: This study addresses a critical gap in digital health research by providing a multidimensional assessment tool that integrates technological, cognitive, and behavioral dimensions. The framework aligns with global digital health strategies and responds to societal challenges like the "digital divide." Its practical applications include guiding policy-making, optimizing digital health services, and tailoring health education programs to improve public health outcomes.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The expert panel size (N = 10) is consistent with established Delphi methodology in educational program development but remains modest.
“First, the expert panel size (N = 10), while consistent with established Delphi methodology in educational program development [24,33], remains modest”
From Toward Sustainable AI Literacy in Higher Education: A Delphi–AHP Approach to Identifying Key Components of AI/Data Extracurricular Programs (2026) · cites this paper as [33]
The Methods > Expert selection passage says research suggests a minimum of 12 panellists for Delphi consensus, contrary to citing this paper as support for a panel of 10 being consistent with that methodology. The source used 15 experts; it does not establish a different standard for educational program development. No page number is available in our copy. (LLM, conf=0.89)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe AHP hierarchy was constructed by drawing all evaluation criteria and alternatives directly from the Delphi consensus, rather than introducing externally derived criteria.
“Following the methodological logic of Tang [24] and Yao et al. [33], the AHP hierarchy was constructed by drawing all evaluation criteria and alternatives directly from the Delphi consensus, rather than introducing externally derived criteria.”
From Toward Sustainable AI Literacy in Higher Education: A Delphi–AHP Approach to Identifying Key Components of AI/Data Extracurricular Programs (2026) · cites this paper as [33]
The Methods > AHP method section (page not provided) says the hierarchical model was based on indicators determined after two Delphi rounds. It does not say that alternatives were drawn from the Delphi consensus. The Methods > Indicator selection section also says the initial indicators were screened from a literature review, so the source does not support an unqualified exclusion of externally derived criteria. (LLM, conf=0.84)
Experimental — accuracy not yet measuredPanel composition adhered to Green's recommendation that Delphi panels comprise individuals with substantive expertise and to the suggested minimum panel size for educational Delphi studies (8-15 experts).
“Panel composition adhered to Green's [22] recommendation that Delphi panels comprise individuals with substantive expertise on the topic, and to the suggested minimum panel size for educational Delphi studies (8-15 experts) [24,33].”
From Toward Sustainable AI Literacy in Higher Education: A Delphi–AHP Approach to Identifying Key Components of AI/Data Extracurricular Programs (2026) · cites this paper as [33]
The Methods > Expert selection passage says panel size should reflect project scale and disciplinary coverage, cites a suggested minimum of 12 panellists, and reports that this study engaged 15 experts. It does not give an 8–15-expert recommendation specifically for educational Delphi studies. The expertise recommendation is attributed to Green [22], not to this source. (LLM, conf=0.84)
Experimental — accuracy not yet measuredThe combined Delphi-AHP approach is particularly well-suited to educational program design contexts where existing frameworks are insufficient and components must be inductively elicited from expert consensus and where relative importance must be quantified to inform resource allocation and design priorities.
“The combined Delphi-AHP approach is particularly well-suited to educational program design contexts where (a) existing frameworks are insufficient and components must be inductively elicited from expert consensus and (b) the relative importance of components must be quantified to inform resource allocation and design priorities [24,33].”
From Toward Sustainable AI Literacy in Higher Education: A Delphi–AHP Approach to Identifying Key Components of AI/Data Extracurricular Programs (2026) · cites this paper as [33]
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredThe same expert panel participated in all three phases, introducing a potential circularity in which the panel that generated the framework also evaluated its validity.
“the same expert panel participated in all three phases of the study-Delphi consensus-building, AHP prioritization, and CVR verification.While this design choice preserved judgmental consistency across phases and reflects common practice in Delphi-AHP research [24,33], it introduces a potential circularity in which the panel that generated the framework also evaluated its validity.”
From Toward Sustainable AI Literacy in Higher Education: A Delphi–AHP Approach to Identifying Key Components of AI/Data Extracurricular Programs (2026) · cites this paper as [33]
The Methods sections on expert selection and AHP describe a 15-expert Delphi consultation whose ratings were used to construct the AHP judgment matrix. This supports the paper as an example of using expert input across Delphi and AHP, but our copy does not describe a third, CVR-verification phase or establish that the same panel participated in one. The potential circularity is the citing authors’ interpretation, not a finding attributed to this source. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
eHealth Literacy: Essential Skills for Consumer Health in a Networked World.
Norman, C. D., & Skinner, H. A. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.2.e9 · Cited by 2,643 · see citing works
A review of fuzzy AHP methods for decision-making with subjective judgements
Liu, Y., Eckert, C. M., & Earl, C. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113738 · Cited by 1,088 · see citing works
van der Vaart, R., & Drossaert, C. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6709 · Cited by 687 · see citing works
A systematic review on digital literacy
Tınmaz, H., Lee, Y.-T., Fanea‐Ivanovici, M., & Baber, H. (2022) · Smart Learning Environments · doi:10.1186/s40561-022-00204-y · Cited by 592 · see citing works
How to Fight an Infodemic: The Four Pillars of Infodemic Management
Eysenbach, G. (2020) · Journal of Medical Internet Research · doi:10.2196/21820 · Cited by 551 · see citing works
Kickbusch, I., Piselli, D., Agrawal, A., Balicer, R. D., Banner, O., Adelhardt, M., Capobianco, E., Fabian, C., Gill, A. S., Lupton, D. A., et al. (2021) · The Lancet · doi:10.1016/s0140-6736(21)01824-9 · Cited by 455 · see citing works
A Delphi study to build consensus on the definition and use of big data in obesity research
Vogel, C., Zwolinsky, S., Griffiths, C., Hobbs, M., Henderson, E. J., & Wilkins, E. L. (2019) · International Journal of Obesity · doi:10.1038/s41366-018-0313-9 · Cited by 452 · see citing works
Digital transformation in healthcare – architectures of present and future information technologies
Gopal, G., Suter‐Crazzolara, C., Toldo, L., & Eberhardt, W. (2018) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2018-0658 · Cited by 396 · see citing works
Estacio, E. V., Whittle, R., & Protheroe, J. (2017) · Journal of Health Psychology · doi:10.1177/1359105317695429 · Cited by 315 · see citing works
Ivanitskaya, L. V., O’Boyle, I., & Casey, A. M. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.2.e6 · Cited by 269 · see citing works
Kemp, E., Trigg, J. L., Beatty, L., Christensen, C., Dhillon, H. M., Maeder, A. J., Williams, P. A. H., & Koczwara, B. (2020) · Health Promotion Journal of Australia · doi:10.1002/hpja.387 · Cited by 259 · see citing works
The promise of digital healthcare technologies
Yeung, A. W. K., Torkamani, A., Butte, A. J., Glicksberg, B. S., Schuller, B. W., Rodríguez, B., Ting, D. S. W., Bates, D. W., Schaden, E., Peng, H., et al. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1196596 · Cited by 254 · see citing works
Ji, H., Dong, J., Pan, W., & Yu, Y. (2024) · International Journal for Equity in Health · doi:10.1186/s12939-024-02150-2 · Cited by 133 · see citing works
Western, M. J., Smit, E. S., Gültzow, T., Neter, E., Sniehotta, F. F., Malkowski, O. S., Wright, C., Busse, H., Peuters, C., Rehackova, L., et al. (2025) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2025.2493139 · Cited by 116 · see citing works
Approaches to Improvement of Digital Health Literacy (eHL) in the Context of Person-Centered Care
Busse, T. S., Nitsche, J., Kernebeck, S., Jux, C., Weitz, J., Ehlers, J. P., & Bork, U. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19148309 · Cited by 103 · see citing works
Electronic Health Literacy Across the Lifespan: Measurement Invariance Study
Paige, S. R., Miller, M. D., Krieger, J. L., Stellefson, M., & Cheong, J. (2018) · Journal of Medical Internet Research · doi:10.2196/10434 · Cited by 93 · see citing works
A Delphi consensus statement for digital surgery
Lam, K., Abràmoff, M. D., Balibrea, J. M., Bishop, S. M., Brady, R. R. W., Callcut, R. A., Chand, M., Collins, J. W., Diener, M. K., Eisenmann, M., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00641-6 · Cited by 87 · see citing works
Transactional eHealth Literacy: Developing and Testing a Multi-Dimensional Instrument
Paige, S. R., Stellefson, M., Krieger, J. L. R., Miller, M. D., Cheong, J., & Anderson‐Lewis, C. (2019) · Journal of Health Communication · doi:10.1080/10810730.2019.1666940 · Cited by 86 · see citing works
Cingolani, M., Scendoni, R., Fedeli, P., & Cembrani, F. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2022.1095001 · Cited by 75 · see citing works
Ernstmann, N., Halbach, S. M., Kowalski, C., Pfaff, H., & Ansmann, L. (2017) · Zeitschrift für Evidenz Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2016.12.008 · Cited by 75 · see citing works
Development and Validation of Digital Health Technology Literacy Assessment Questionnaire
Yoon, J., Lee, M., Ahn, J. S., Oh, D., Shin, S.-Y., Chang, Y. J., & Cho, J. (2022) · Journal of Medical Systems · doi:10.1007/s10916-022-01800-8 · Cited by 74 · see citing works
Dijkman, E. M., ter Brake, W. W. M., Drossaert, C. H. C., & Doggen, C. J. M. (2023) · Healthcare · doi:10.3390/healthcare12010011 · Cited by 70 · see citing works
Wang, C., Kane, R. L., Xu, D., & Meng, Q. (2015) · BMC Women s Health · doi:10.1186/s12905-015-0190-5 · Cited by 69 · see citing works
Wieczorek, M., Meier, C., Vilpert, S., Reinecke, R., Borrat-Besson, C., Maurer, J., & Kliegel, M. (2023) · BMC Public Health · doi:10.1186/s12889-023-15136-6 · Cited by 51 · see citing works
The causal relationship between digital literacy and students’ academic achievement: a meta-analysis
Li, F., Cheng, L., Wang, X., Shen, L., Ma, Y., & Islam, A. Y. M. A. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-04399-6 · Cited by 38 · cited 1× · see citing works
“It encompasses a wide range of aspects such as access to digital information, evaluation, management, and security awareness, and it is a way to safeguard the needs of a digital life, enjoy the convenience of digital society, and the basic ability to integrate into digital society (47).”
Mäder, M., Schönfelder, T., Heinrich, R., Militzer-Horstmann, C., & Timpel, P. (2025) · Archives of Public Health · doi:10.1186/s13690-024-01474-3 · Cited by 37 · see citing works
Pant, S., Garg, P., Kumar, A., Ram, M., Sharma, H. K., Sharma, H. K., & Klochkov, Y. (2023) · International Journal of Systems Assurance Engineering and Management · doi:10.1007/s13198-023-01904-5 · Cited by 37 · see citing works
Price‐Haywood, E. G., Arnold, C. L., Harden-Barrios, J., & Davis, T. C. (2023) · Ochsner Journal · doi:10.31486/toj.22.0101 · Cited by 36 · see citing works
The role of social media literacy in infodemic management: a systematic review
Ziapour, A., Malekzadeh, R., Darabı, F., Yıldırım, M., Montazeri, N., Kianipour, N., & NeJhaddadgar, N. (2024) · Frontiers in Digital Health · doi:10.3389/fdgth.2024.1277499 · Cited by 33 · see citing works
Diana, M. G., Mascia, M. L., Tomczyk, Ł., & Penna, M. P. (2025) · Sustainability · doi:10.3390/su17041718 · Cited by 33 · see citing works
Choosing the Best Digital Health Literacy Measure for Research: Mixed Methods Study
Thorup, C. B., Uitto, M., Butler-Henderson, K. A., Andersson, S. W., Hoffrén‐Mikkola, M., Thoft, D. S., Hæsum, L. K. E., Irrazábal, G., Pruneda, L., & Valkama, K. (2025) · Journal of Medical Internet Research · doi:10.2196/59807 · Cited by 21 · see citing works
Chen, S., Qin, B., Zhu, J., & Liu, G. (2025) · BMC Public Health · doi:10.1186/s12889-025-23003-9 · Cited by 20 · see citing works
Gao, Y.-Q., Yan, K., Yan, X., Xi, N., Gao, J., & Ren, H. (2022) · Nursing Open · doi:10.1002/nop2.1566 · Cited by 19 · see citing works
Jiang, N., Liu, W. B., Zong, Y., Yu, L., & Shuhua, C. (2024) · BMC Nursing · doi:10.1186/s12912-024-02349-2 · Cited by 18 · see citing works
Liu, X., Gan, X., Ren, G., Mao, Z., Hu, J., Sha, C., & Wu, J. (2025) · BMC Geriatrics · doi:10.1186/s12877-025-05952-3 · Cited by 16 · see citing works
Kim, S., Austin, L., Liu, B. F., & Jin, Y. (2022) · Public Relations Review · doi:10.1016/j.pubrev.2022.102204 · Cited by 12 · see citing works
Cheng, K., Xu, Z., Zhang, Y., Chen, R., & Gui, L. (2025) · BMC Public Health · doi:10.1186/s12889-025-22549-y · Cited by 7 · see citing works
Wang, B., Sun, W., Liu, F., Zheng, T., Quan, L., Liu, G., Lin, H., Qiao, Y., Cai, J., & Huang, X. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-12676-y · Cited by 7 · see citing works
Jiang, B. (2024) · American Journal of Translational Research · doi:10.62347/asab6587 · Cited by 4 · see citing works
Negera, D., Zewdie, A., Kera, A. M., & Degefa, G. H. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-067540 · Cited by 3 · see citing works
Sharma, S. (2016) · International Journal of Information Dissemination and Technology · Cited by 2 · see citing works
Fu, J., Jin, L., Shang, X., Zhang, C. D., & Chen, L. (2025) · BMC Nursing · doi:10.1186/s12912-025-03056-2 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Bae, J.-S., Ko, K.-M., & Park, H.-J. (2026) · Sustainability · doi:10.3390/su18168050 · Cited by 3 · see citing works
Maaß, L., Stark, A. L., Tokgöz, P., Fischer, F., Schulz, H., & Albrecht, J. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1799393 · Cited by 1 · see citing works
Edumadze, J. K. E., Edumadze, G. E., Arkaifie, R. K., Kissiedu, A. N. T., Biney, B. A., Essel, D. D., Osei-Gyasi, A., Ankomah, E., Addai, O.-D., & Amenumey, D. E. K. (2026) · Journal of Digital Educational Technology · doi:10.29333/jdet/18536
Gea, N. Y. K., Simamora, R. S., & Purnomo, R. (2026) · JKEP · doi:10.32668/jkep.v11i1.2328
Development of Digital Competency Framework for Vaccine Manufacturing Industry
Ompi, C. P., Saedudin, R. R., & Rahmad, B. (2026) · Equivalent Jurnal Ilmiah Sosial Teknik · doi:10.59261/jequi.v8i3.392
Yushida: an IoT-based smart injection system for long-term management of chronic diseases
Lei, L., Ma, R., Lin, F., Yang, L., Peng, J., Yang, Y., Yin, Y., & Yang, M. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1900971
Mtshali, S., & Dhanpat, N. (2026) · Frontiers in Organizational Psychology · doi:10.3389/forgp.2026.1823243
Ying, L. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1978109
- Authors
- Yao, Y., Cao, J., Tan, Z., & Chen, K.
- Affiliations
- Jilin University (CN)
- Corresponding author
- Tan, Zheng
- ORCID
- Venue
- Frontiers in Public Health
- Publisher
- Frontiers Media SA
- Published
- 2025-07-21
- Volume
- 13
- Pages
- 1634261-1634261
- ISSN
- 2296-2565 (ISSN-L)
- PMC
- PMC12319002
- OpenAlex
- W4412506891
- Cited by
- 5
- References
- 42
- Citations by year
- 2026: 5
- FWCI
- 6.31
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Public Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Physical Sciences › Computer Science › Information Systems › Digital literacy in education
- Health Sciences › Health Professions › Health Information Management › Health, Technology, Consumer Behavior
- Keywords
- digital health competencies, public health services, health literacy, information literacy, digital literacy, digital divide, indicator system, expert consensus, analytic hierarchy process
- MeSH terms
- Digital Health, Delphi Technique, Humans, Public Health, Health Literacy, Information Literacy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
