Assessing Competencies Needed to Engage With Digital Health Services: Development of the eHealth Literacy Assessment Toolkit
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
BACKGROUND: To achieve full potential in user-oriented eHealth projects, we need to ensure a match between the eHealth technology and the user's eHealth literacy, described as knowledge and skills. However, there is a lack of multifaceted eHealth literacy assessment tools suitable for screening purposes. OBJECTIVE: The objective of our study was to develop and validate an eHealth literacy assessment toolkit (eHLA) that assesses individuals' health literacy and digital literacy using a mix of existing and newly developed scales. METHODS: From 2011 to 2015, scales were continuously tested and developed in an iterative process, which led to 7 tools being included in the validation study. The eHLA validation version consisted of 4 health-related tools (tool 1: "functional health literacy," tool 2: "health literacy self-assessment," tool 3: "familiarity with health and health care," and tool 4: "knowledge of health and disease") and 3 digitally-related tools (tool 5: "technology familiarity," tool 6: "technology confidence," and tool 7: "incentives for engaging with technology") that were tested in 475 respondents from a general population sample and an outpatient clinic. Statistical analyses examined floor and ceiling effects, interitem correlations, item-total correlations, and Cronbach coefficient alpha (CCA). Rasch models (RM) examined the fit of data. Tools were reduced in items to secure robust tools fit for screening purposes. Reductions were made based on psychometrics, face validity, and content validity. RESULTS: Tool 1 was not reduced in items; it consequently consists of 10 items. The overall fit to the RM was acceptable (Anderson conditional likelihood ratio, CLR=10.8; df=9; P=.29), and CCA was .67. Tool 2 was reduced from 20 to 9 items. The overall fit to a log-linear RM was acceptable (Anderson CLR=78.4, df=45, P=.002), and CCA was .85. Tool 3 was reduced from 23 to 5 items. The final version showed excellent fit to a log-linear RM (Anderson CLR=47.7, df=40, P=.19), and CCA was .90. Tool 4 was reduced from 12 to 6 items. The fit to a log-linear RM was acceptable (Anderson CLR=42.1, df=18, P=.001), and CCA was .59. Tool 5 was reduced from 20 to 6 items. The fit to the RM was acceptable (Anderson CLR=30.3, df=17, P=.02), and CCA was .94. Tool 6 was reduced from 5 to 4 items. The fit to a log-linear RM taking local dependency (LD) into account was acceptable (Anderson CLR=26.1, df=21, P=.20), and CCA was .91. Tool 7 was reduced from 6 to 4 items. The fit to a log-linear RM taking LD and differential item functioning into account was acceptable (Anderson CLR=23.0, df=29, P=.78), and CCA was .90. CONCLUSIONS: The eHLA consists of 7 short, robust scales that assess individual's knowledge and skills related to digital literacy and health literacy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
eHEALS: The eHealth Literacy Scale
Norman, C. D., & Skinner, H. A. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.4.e27 · Cited by 2,680 · cited 3×
“eHEALS [9] was included in the initial development phase as this, at the time, was the only available tool for measuring eHealth literacy.”
eHealth Literacy: Essential Skills for Consumer Health in a Networked World.
Norman, C. D., & Skinner, H. A. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.2.e9 · Cited by 2,643 · cited 9×
“These 3 elements are central subliteracies in the lily model’s understanding of eHealth literacy, as well as included in the eHLF’s dimensions that describe the individual and the system interaction [8,14].”
InfluentialDifferential Item Functioning.
Gudgeon, A. C., Holland, P. W., & Wainer, H. (1994) · The Statistician · doi:10.2307/2348590 · Cited by 2,091
Cognitive interviewing: verbal data in the design and pretesting of questionnaires
Drennan, J. (2003) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.2003.02579.x · Cited by 769
van der Vaart, R., & Drossaert, C. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6709 · Cited by 687
van der Vaart, R., van Deursen, A. J., Drossaert, C. H., Taal, E., van Dijk, J. A., & van de Laar, M. A. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1840 · Cited by 404 · cited 2×
“A study by van der Vaart et al showed that eHEALS scores were poorly correlated with actual Internet use [39].”
The Concept of eHealth Literacy and Its Measurement: German Translation of the eHEALS
Soellner, R., Huber, S., & Reder, M. (2014) · Journal of Media Psychology · doi:10.1027/1864-1105/a000104 · Cited by 184
Cited by
Digital literacy as a new determinant of health: A scoping review
Arias López, M. D. P., Ong, B. A., Borrat Frigola, X., Fernández, A. L., Hicklent, R. S., Obeles, A. J. T., Rocimo, A. M., & Celi, L. A. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000279 · Cited by 422
Sociodemographic determinants of digital health literacy: A systematic review and meta-analysis
Estrela, M., Semedo, G., Roque, F., Ferreira, P. L., & Herdeiro, M. T. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105124 · Cited by 414
COVID-19 Misinformation Online and Health Literacy: A Brief Overview
Bin Naeem, S., & Kamel Boulos, M. N. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18158091 · Cited by 293
Perceptions of Digital Health Education Among European Medical Students: Mixed Methods Survey
Machleid, F., Kaczmarczyk, R., Johann, D., Balčiūnas, J., Atienza-Carbonell, B., von Maltzahn, F., & Mosch, L. (2020) · Journal of Medical Internet Research · doi:10.2196/19827 · Cited by 190
“Further research on the eHealth literacy of medical students using validated assessment tools [59,60] would be necessary in order to obtain a clearer and more objective picture.”
BackgroundDigital Health Literacy: Bibliometric Analysis
Yang, K., Hu, Y., & Qi, H. (2022) · Journal of Medical Internet Research · doi:10.2196/35816 · Cited by 188
Metadata
- Authors
- Karnoe, A., Furstrand, D., Christensen, K. B., Norgaard, O., & Kayser, L.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2018-05-10
- Volume
- 20
- Issue
- 5
- Pages
- e178
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W2802585180
- Cited by
- 151
- References
- 7
- FWCI
- 13.22
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Assessing Competencies Needed to Engage With Digital Health Services: Development of the eHealth Literacy Assessment Toolkit. https://app.cochat.ai/papers/10.2196/jmir.8347
- The paper
- Karnoe, A., Furstrand, D., Christensen, K. B., Norgaard, O., & Kayser, L. (2018). Assessing Competencies Needed to Engage With Digital Health Services: Development of the eHealth Literacy Assessment Toolkit. Journal of Medical Internet Research. https://doi.org/10.2196/jmir.8347
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.