Performance-Based Measurement of eHealth Literacy: Systematic Scoping Review
Open accessPublished version of a preprint
The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Abstract
Background: eHealth literacy describes the ability to locate, comprehend, evaluate, and apply web-based health information to a health problem. In studies of eHealth literacy, researchers have primarily assessed participants’ perceived eHealth literacy using a short self-report instrument, for which ample research has shown little to no association with actual performed eHealth-related skills. Performance-based measures of eHealth literacy may be more effective at assessing actual eHealth skills, yet such measures seem to be scarcer in the literature. Objective: The primary purpose of this study was to identify tools that currently exist to measure eHealth literacy based on objective performance. A secondary purpose of this study was to characterize the prevalence of performance-based measurement of eHealth literacy in the literature compared with subjective measurement. Methods: We conducted a systematic scoping review of the literature, aligning with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist, in 3 stages: conducting the search, screening articles, and extracting data into a summary table. The summary table includes terminology for eHealth literacy, description of participants, instrument design, health topics used, and a brief note on the evidence of validity for each performance-based measurement tool. A total of 1444 unique articles retrieved from 6 relevant databases (MEDLINE; PsycINFO; CINAHL; Library and Information Science Abstracts [LISA]; Library, Information Science & Technology Abstracts [LISTA]; and Education Resources Information Center [ERIC]) were considered for inclusion, of which 313 (21.68%) included a measure of eHealth literacy. Results: Among the 313 articles that included a measure of eHealth literacy, we identified 33 (10.5%) that reported on 29 unique performance-based eHealth literacy measurement tools. The types of tools ranged from having participants answer health-related questions using the internet, having participants engage in simulated internet tasks, and having participants evaluate website quality to quizzing participants on their knowledge of health and the web-based health information–seeking process. In addition, among the 313 articles, we identified 280 (89.5%) that measured eHealth literacy using only a self-rating tool. Conclusions: This study is the first research synthesis looking specifically at performance-based measures of eHealth literacy and may direct researchers toward existing performance-based measurement tools to be applied in future projects. We discuss some of the key benefits and drawbacks of different approaches to performance-based measurement of eHealth literacy. Researchers with an interest in gauging participants’ actual eHealth literacy (as opposed to perceived eHealth literacy) should make efforts to incorporate tools such as those identified in this systematic scoping review.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The discrepancy between perceived and actual proficiency in health information evaluation is a well-documented phenomenon in the health literacy literature, whereby individuals systematically overestimate their critical appraisal abilities when evaluated against standardized measures (50, 51).”
From Health misinformation, digital health literacy, and attitudes toward science among university students: an exploratory cross-sectional survey (2026) · cites this paper as [50]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0850 · Cited by 43,565
Munn, Z., Peters, M. D. J., Stern, C., Tufanaru, C., McArthur, A., & Aromataris, E. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0611-x · Cited by 11,385
Predicting and Changing Behavior
Fishbein, M., & Ajzen, I. (2011) · doi:10.4324/9780203838020 · Cited by 2,807
eHEALS: The eHealth Literacy Scale
Norman, C. D., & Skinner, H. A. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.4.e27 · Cited by 2,680 · cited 2×
“Concurrent with their publication coining eHealth literacy, Norman and Skinner [9] published the eHealth Literacy Scale (eHEALS), an 8-item questionnaire wherein each item is rated using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree), with higher scores indicating…”
eHealth Literacy: Essential Skills for Consumer Health in a Networked World.
Norman, C. D., & Skinner, H. A. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.2.e9 · Cited by 2,643 · cited 7×
“Although evaluating the credibility of a web-based source is undoubtedly a pivotal component of eHealth literacy, it does not encompass all components of eHealth literacy as defined by Norman and Skinner [7].”
InfluentialMoorhead, S. A., Hazlett, D. E., Harrison, L., Carroll, J. K., Irwin, A., & Hoving, C. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.1933 · Cited by 2,517 · cited 1×
“The validity and trustworthiness of information quality on these platforms has been flagged as a major concern [65]; however, public health organizations have also used these engaging platforms to distribute high-quality and up-to-date information [66,67].”
Flawed Self-Assessment: Implications for Health, Education, and the Workplace
Dunning, D., Heath, C., & Suls, J. M. (2004) · Psychological Science in the Public Interest · doi:10.1111/j.1529-1006.2004.00018.x · Cited by 1,865
Kata, A. (2012) · Vaccine · doi:10.1016/j.vaccine.2011.11.112 · Cited by 1,094 · cited 1×
“Many avid followers of the antivaccine movement have spent several hours surfing the internet for health-related information [15,16].”
Dietary Guidelines for Americans
DeSalvo, K. B., Olson, R., & Casavale, K. O. (2016) · JAMA · doi:10.1001/jama.2015.18396 · Cited by 1,059
Validity of self-evaluation of ability: A review and meta-analysis.
Mabe, P. A., & West, S. G. (1982) · Journal of Applied Psychology · doi:10.1037//0021-9010.67.3.280 · Cited by 973
A global review of food-based dietary guidelines
Herforth, A., Arimond, M., Álvarez-Sánchez, C., Coates, J., Christianson, K., & Muehlhoff, E. (2019) · Advances in Nutrition · doi:10.1093/advances/nmy130 · Cited by 848
van der Vaart, R., & Drossaert, C. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6709 · Cited by 687
Consumer health information seeking in social media: a literature review
Zhao, Y., & Zhang, J. (2017) · Health Information & Libraries Journal · doi:10.1111/hir.12192 · Cited by 607
Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature
Haustein, S., Peters, I., Sugimoto, C. R., Thelwall, M., & Larivière, V. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23101 · Cited by 434
The Anti-vaccination Movement: A Regression in Modern Medicine
Hussain, A., Ali, S., Ahmed, M., & Hussain, S. (2018) · Cureus · doi:10.7759/cureus.2919 · Cited by 416
van der Vaart, R., van Deursen, A. J., Drossaert, C. H., Taal, E., van Dijk, J. A., & van de Laar, M. A. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1840 · Cited by 404 · cited 5×
“In the studies conducted by Chan and Kaufman [44] as well as van der Vaart et al [26], researchers factored in the correctness of responses in addition to researcher-observed performance to gauge participants’eHealth literacy, combining task simulation with the Health-related questions using the…”
InfluentialSelf-Insight: Roadblocks and Detours on the Path to Knowing Thyself
Dunning, D. A. (2005) · Medical Entomology and Zoology · Cited by 320
eHealth Literacy 2.0: Problems and Opportunities With an Evolving Concept
Norman, C. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.2035 · Cited by 297
Ivanitskaya, L., O’Boyle, I., & Casey, A. M. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.2.e6 · Cited by 270
eHealth Literacy Instruments: Systematic Review of Measurement Properties
Lee, J., Lee, E.-H., & Chae, D. (2021) · Journal of Medical Internet Research · doi:10.2196/30644 · Cited by 226
Sharit, J., Hernández, M. A., Czaja, S. J., & Pirolli, P. (2008) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1352782.1352785 · Cited by 217 · cited 1×
“In addition to the correctness of participants’ responses, some researchers factored completion time into participant scores such that faster completion time indicated greater proficiency [39,40].”
(2015) · Knowledge Management & E-Learning: An International Journal · doi:10.34105/j.kmel.2015.07.035 · Cited by 215
Mahmood, K., & University of the Punjab (2016) · Comminfolit · doi:10.15760/comminfolit.2016.10.2.24 · Cited by 212 · cited 1×
“Relating to web-based information seeking, ample literature indicates that people have a tendency to overestimate their ability with computers [17,18] as well as their ability to locate and understand information on the web [19,20].”
A Framework for Characterizing eHealth Literacy Demands and Barriers
Chan, C. V., & Kaufman, D. R. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1750 · Cited by 190
Social media for patients: benefits and drawbacks
Martino, I. D., D’Apolito, R., McLawhorn, A. S., Fehring, K. A., Sculco, P. K., & Gasparini, G. (2017) · Current Reviews in Musculoskeletal Medicine · doi:10.1007/s12178-017-9394-7 · Cited by 168
Internet Skills Performance Tests: Are People Ready for eHealth?
van Deursen, A. J. A. M., & van Dijk, J. A. G. M. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1581 · Cited by 163
eHealth literacy research—Quo vadis?
Griebel, L., Enwald, H., Gilstad, H., Pohl, A.-L., Moreland, J., & Sedlmayr, M. (2018) · Informatics for Health and Social Care · doi:10.1080/17538157.2017.1364247 · Cited by 163
Karnoe, A., Furstrand, D., Christensen, K. B., Norgaard, O., & Kayser, L. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.8347 · Cited by 151
Quality of health-related online search results
Kitchens, B., Harle, C. A., & Li, S. (2012) · Decision Support Systems · doi:10.1016/j.dss.2012.10.050 · Cited by 134
Quinn, S., Bond, R., & Nugent, C. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.12.032 · Cited by 132
van der Vaart, R., Drossaert, C. H., de Heus, M., Taal, E., & van de Laar, M. A. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2428 · Cited by 131
How is eHealth literacy measured and what do the measurements tell us? A systematic review
Karnoe, A., & Kayser, L. (2015) · Knowledge Management & E-Learning An International Journal · doi:10.34105/j.kmel.2015.07.038 · Cited by 131
Public Health Messaging in an Era of Social Media
Merchant, R. M., South, E. C., & Lurie, N. (2021) · JAMA · doi:10.1001/jama.2020.24514 · Cited by 126
Internet skill-related problems in accessing online health information
van Deursen, A. J. A. M. (2011) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2011.10.005 · Cited by 120
Holt, K. A., Overgaard, D., Engel, L. V., & Kayser, L. (2020) · BMC Nursing · doi:10.1186/s12912-020-00418-w · Cited by 109
Internet-based health information consumer skills intervention for people living with HIV/AIDS.
Kalichman, S. C., Cherry, C., Cain, D., Pope, H., Kalichman, M., Eaton, L., Weinhardt, L., & Benotsch, E. G. (2006) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.74.3.545 · Cited by 98
Perceived and Performed eHealth Literacy: Survey and Simulated Performance Test
Neter, E., & Brainin, E. (2017) · JMIR Human Factors · doi:10.2196/humanfactors.6523 · Cited by 96
Xie, B. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21575 · Cited by 95
Maitz, E., Maitz, K., Sendlhofer, G., Wolfsberger, C. H., Mautner, S., Kamolz, L., & Gasteiger‐Klicpera, B. (2020) · Journal of Medical Internet Research · doi:10.2196/16281 · Cited by 90
Schulz, P. J., Pessina, A., Hartung, U., & Petrocchi, S. (2021) · Journal of Medical Internet Research · doi:10.2196/20457 · Cited by 84 · cited 1×
“In the studies conducted by Kalichman et al [47], Schulz et al [49], and Trettin et al [50], participants rated 2 different websites on 5, 7, and 6 dimensions related to their credibility, respectively.”
The doctor as the second opinion and the internet as the first
Gualtieri, L. N. (2009) · CHI '09 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1520340.1520352 · Cited by 81 · cited 1×
“Research indicates that the internet is the first source people turn to for health information [2] and in some cases is deemed more credible than physician diagnoses [3].”
Agree, E. M., King, A. C., Castro, C. M., Wiley, A., & Borzekowski, D. L. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3352 · Cited by 76
Ivanitskaya, L., Brookins-Fisher, J., O´Boyle, I., Vibbert, D., Erofeev, D., & Fulton, L. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1520 · Cited by 75
AN INVESTIGATION OF SELF-REPORTED COMPUTER LITERACY: IS IT RELIABLE?
(2005) · Issues In Information Systems · doi:10.48009/1_iis_2005_289-295 · Cited by 75
Online Information Search Performance and Search Strategies in a Health Problem-Solving Scenario
Sharit, J., Taha, J., Berkowsky, R. W., Profita, H. P., & Czaja, S. J. (2015) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343415583747 · Cited by 74
Holt, K. A., Karnoe, A., Overgaard, D., Nielsen, S. E., Kayser, L., Røder, M. E., & From, G. (2019) · Interactive Journal of Medical Research · doi:10.2196/ijmr.8423 · Cited by 72
ICT skills measurement in social surveys: Can we trust self-reports?
Palczyńska, M., & Rynko, M. (2020) · Quality & Quantity · doi:10.1007/s11135-020-01031-4 · Cited by 66
COVID-19 and ‘immune boosting’ on the internet: a content analysis of Google search results
Rachul, C., Marcon, A. R., Collins, B., & Caulfield, T. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-040989 · Cited by 56
Research Readiness Self-Assessment: Assessing Students' Research Skills and Attitudes
Ivanitskaya, L. V., Laus, R., & Casey, A. M. (2004) · Journal of Library Administration · doi:10.1300/j111v41n01_13 · Cited by 54
Liu, P., Yeh, L.-L., Wang, J.-Y., & Lee, S.-T. (2020) · Journal of Medical Internet Research · doi:10.2196/19767 · Cited by 53 · cited 3×
“Liu et al [60] also created a performance-based eHealth literacy measurement tool that assesses participants’health knowledge.”
InfluentialHealthcare students' e-literacy skills.
Brown, C. A., & Dickson, R. (2010) · PubMed · Cited by 50
Mitsuhashi, T. (2018) · PeerJ · doi:10.7717/peerj.5251 · Cited by 48
Stellefson, M., Hanik, B., Chaney, J. D., & Tennant, B. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1969 · Cited by 44
Internet health information education for older adults: A pilot study
Chang, S. J., Yang, E., Lee, K., & Ryu, H. (2020) · Geriatric Nursing · doi:10.1016/j.gerinurse.2020.10.002 · Cited by 41
Jean, B. S., Taylor, N. G., Kodama, C. M., & Subramaniam, M. M. (2017) · Journal of Consumer Health on the Internet · doi:10.1080/15398285.2017.1279894 · Cited by 36
Freund, O., Reychav, I., McHaney, R., Goland, E., & Azuri, J. (2017) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.02.012 · Cited by 31
Hanna, K. M. B., Sambrook, P. J., Armfield, J. M., & Brennan, D. S. (2017) · Australian Dental Journal · doi:10.1111/adj.12509 · Cited by 29
Kordovski, V. M., Babicz, M. A., Ulrich, N., & Woods, S. P. (2020) · Perceptual and Motor Skills · doi:10.1177/0031512520938178 · Cited by 27
Witry, M. J., Comellas, A. P., Simmering, J. E., & Polgreen, P. M. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9382 · Cited by 27
The Influence of Offline and Online Intrinsic Motivations on Online Health Information Seeking
Lee, S. T., & Lin, J. (2019) · Health Communication · doi:10.1080/10410236.2019.1620088 · Cited by 27
Blakemore, L., Meek, S. E. M., & Marks, L. (2019) · Journal of Medical Internet Research · doi:10.2196/15177 · Cited by 27
Paging Dr. Google: The Effect of Online Health Information on Trust in Pediatricians’ Diagnoses
Sood, N., Jimenez, D. E., Pham, T. B., Cordrey, K., Awadalla, N. R., & Milanaik, R. L. (2019) · Clinical Pediatrics · doi:10.1177/0009922819845163 · Cited by 23
Loda, T., Erschens, R. S., Junne, F., Stengel, A., Zipfel, S., & Herrmann‐Werner, A. (2020) · JMIR Medical Informatics · doi:10.2196/16279 · Cited by 23
eHealth Literacy of High School Students in the Philippines
Camiling, M. K. S. (2019) · IAFOR Journal of Education · doi:10.22492/ije.7.2.04 · Cited by 19
Trettin, L. D., May, J. C., & McKeehan, N. C. (2008) · Journal of the Medical Library Association JMLA · doi:10.3163/1536-5050.96.4.013 · Cited by 7
Loda, T., Erschens, R. S., Junne, F., Stengel, A., Zipfel, S., & Herrmann‐Werner, A. (2020) · JMIR Medical Informatics · doi:10.2196/23253 · Cited by 4
Research works printed but not linked: 4
- 10.1037/0021-9010.67.3.280 · DOI · Paper page
- Brown, CA, 2010, J Allied Health, 39, 179
- 2010, Encyclopedia of Research Design
- Hanik, B, 2011, Int Electron J Health Educ, 14, 46
Cited by
Petrič, G., & Atanasova, S. (2024) · BMC Public Health · doi:10.1186/s12889-024-19431-8 · Cited by 31
Choosing the Best Digital Health Literacy Measure for Research: Mixed Methods Study
Thorup, C. B., Uitto, M., Butler-Henderson, K. A., Andersson, S. W., Hoffrén‐Mikkola, M., Thoft, D. S., Hæsum, L. K. E., Irrazábal, G., Pruneda, L., & Valkama, K. (2025) · Journal of Medical Internet Research · doi:10.2196/59807 · Cited by 29
eHealth literacy in the general population: a cross-sectional study in China
Sun, C., Meijer, E., Chavannes, N. H., Dai, H., Li, X., Wang, Y., Wu, L., Zhang, Q., & Kasteleyn, M. J. (2025) · BMC Public Health · doi:10.1186/s12889-025-21389-0 · Cited by 25
Li, J., Wu, X., Wu, Y., Fong, D., Song, Y., Xu, S., Kim, C., Lin, X., & Pandian, V. (2025) · Journal of Medical Internet Research · doi:10.2196/70822 · Cited by 11
Naeem, S. B., Faiola, A. J., Ur-Rehman, A., & Boulos, M. N. K. (2024) · Information · doi:10.3390/info15100636 · Cited by 10
Metadata
- Authors
- Crocker, B., Feng, O., & Duncan, L. R.
- Affiliations
- McGill University (CA)
- Corresponding author
- Crocker, Bradley
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2023-06-02
- Volume
- 25
- Pages
- e44602
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC10276324
- OpenAlex
- W4322750995
- Cited by
- 28
- References
- 67
- Citations by year
- 2026: 13 · 2025: 8 · 2024: 5 · 2023: 2
- FWCI
- 1.42
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- other-oa
- Locations
- Journal of Medical Internet Research (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Social Sciences › Library and Information Sciences › Library Science and Information Literacy
- Physical Sciences › Computer Science › Information Systems › ICT in Developing Communities
- Keywords
- digital health literacy, self-report measures, online health information, health information seeking, website quality evaluation, digital literacy, information literacy, self-efficacy, self-regulation, self-monitoring
- MeSH terms
- Humans, Surveys and Questionnaires, Telemedicine, Checklist, Health Literacy, Self Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
