Cited by 55Digital Mental Health InterventionsHealth Literacy and Information AccessibilityTechnology Use by Older Adults

The Role of Age, Education, and Digital Health Literacy in the Usability of Internet-Based Cognitive Behavioral Therapy for Chronic Pain: Mixed Methods Study

Open access
van der Vaart, Rosalie, van Driel, Dorine, Pronk, Kristel, Paulussen, Suzan, te Boekhorst, Selma, Rosmalen, Judith G M, Evers, Andrea W M·2019·JMIR Formative Research

Published version of a preprint

Abstract

Background: Internet-based cognitive behavior therapy (iCBT) can be effective in mental and somatic health care. Research on the feasibility of internet interventions in clinical practice is, however, still scarce. Studies with a focus on the patient regarding usability of interventions and digital health literacy skills are especially lacking. Objective: The goal of this study was to assess the usability of an iCBT for chronic pain, Master Your Pain, and the relationship between its usability outcomes and the factors age, educational level, and digital health literacy skills. The aims were to determine what changes were needed in the program for sufficient usability and which individual characteristics were related to the usability of the program. Methods: Patients were recruited from two mental health care practices. A mixed methods approach was used in this study. A qualitative observational study comprising performance tasks in the iCBT program was used to test usability. A quantitative questionnaire was used to measure possible related constructs. Usability was operationalized as the number of tasks that could be completed and the type and number of problems that occurred while doing so. Performance tasks were set up to measure 6 digital skills: (1) operating the computer and internet browser, (2) navigation and orientation, (3) using search strategies, (4) evaluating relevance of content, (5) adding personal content, and (6) protecting and respecting privacy. Participants were asked to think aloud while performing the tasks, and screen activities and webcam recordings were captured. The qualitative observational data was coded using inductive analysis by two independent researchers. Correlational analyses were performed to test how usability relates to sociodemographics and digital health literacy. Results: A total of 32 patients participated, with a mean age of 49.9 years and 84% (27/32) being female. All performance tasks except one (fill in a diary registration) could be completed independently by more than 50% of the participants. On operational, navigation, and search levels, participants struggled most with logging in, logging out, and finding specific parts of the intervention. Half of the sample experienced problems evaluating the relevance and adding content to the program to some extent. Usability correlated moderately negatively with age and moderately positively with digital health literacy skills but not with educational level. Conclusions: The results provide insight into what is essential for proper usability regarding the design of an iCBT program considering variations in age, educational level, and digital health literacy. Furthermore, the results provide insight into what type of support is needed by patients to properly use the intervention. Tailoring support among the needs of certain age groups or skill levels could be beneficial and could range from no extra support (only online feedback, as intended) to practical support (an additional usability introduction session) to blended care (combined face-to-face sessions throughout the therapy).

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/12883

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Digital health literacy is an important element of technology adoption.

    “First and foremost, the study adds to the evidence that digital health literacy is an important element of technology adoption (van der Vaart et al., 2019).”

    From Sociodemographics and Digital Health Literacy in Using Wearables for Health Promotion and Disease Prevention: Cross-Sectional Nationwide Survey in Germany (2025) · cites this paper as van der Vaart et al., 2019

    The Introduction (page unavailable) describes digital health literacy as skills needed for effective use of internet interventions. The Discussion, Principal Findings (page unavailable), reports that literacy scores correlated with completed tasks in the study’s iCBT platform. This supports its importance for using that intervention, but the paper does not establish its role in technology adoption more generally. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [24]Nielsen J Usability Engineering. 2nd Edition 1993 Burlington Morgan Kaufmann Publishers

Cited by

Sorted by citations

Authors
van der Vaart, R., van Driel, D., Pronk, K., Paulussen, S., te Boekhorst, S., Rosmalen, J. G. M., & Evers, A. W. M.
Affiliations
Leiden University (NL)
Venue
JMIR Formative Research
Published
2019-11-21
Volume
3
Issue
4
Pages
e12883
ISSN
2561-326X (ISSN-L)
OpenAlex
W2969797279
Cited by
55
References
23
Citations by year
2026: 5 · 2025: 2 · 2024: 11 · 2023: 14 · 2022: 11 · 2021: 5 · 2020: 5 · 2019: 1
FWCI
6.21
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Formative Research (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • University of Groningen research database (University of Groningen / Centre for Information Technology) (publishedVersion, cc-by)
  • Leiden Repository (Leiden University) (submittedVersion, cc-by)
  • Radboud Repository (Radboud University) (submittedVersion)
  • Radboud Repository (Radboud University) (submittedVersion, other-oa)
  • University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, cc-by)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
  • Social Sciences › Demography › Technology Use by Older Adults
Keywords
internet-based cognitive behavioral therapy, chronic pain, digital health literacy, usability, age differences, educational attainment, performance tasks, think-aloud, web navigation, online privacy, digital skills, mental health services, feasibility, intervention design, sociodemographic factors, user-centered design, user satisfaction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review