Cited by 331Mobile Health and mHealth ApplicationsDigital Mental Health InterventionsTelemedicine and Telehealth Implementation

The DOI matches its Crossref record; 83 of the 84 references it cites were found in a registry or an index; none of the 34 sources its claims cite is checked for support yet.

Clarifying the Concept of Adherence to eHealth Technology: Systematic Review on When Usage Becomes Adherence

Open access
Sieverink, Floor, Kelders, Saskia M, van Gemert-Pijnen, Julia EWC·2017·Journal of Medical Internet Research

Abstract

BACKGROUND: In electronic health (eHealth) evaluations, there is increasing attention for studying the actual usage of a technology in relation to the outcomes found, often by studying the adherence to the technology. On the basis of the definition of adherence, we suggest that the following three elements are necessary to determine adherence to eHealth technology: (1) the ability to measure the usage behavior of individuals; (2) an operationalization of intended use; and (3) an empirical, theoretical, or rational justification of the intended use. However, to date, little is known on how to operationalize the intended usage of and the adherence to different types of eHealth technology. OBJECTIVE: The study aimed to improve eHealth evaluations by gaining insight into when, how, and by whom the concept of adherence has been used in previous eHealth evaluations and finding a concise way to operationalize adherence to and intended use of different eHealth technologies. METHODS: A systematic review of eHealth evaluations was conducted to gain insight into how the use of the technology was measured, how adherence to different types of technologies was operationalized, and if and how the intended use of the technology was justified. Differences in variables between the use of the technology and the operationalization of adherence were calculated using a chi-square test of independence. RESULTS: In total, 62 studies were included in this review. In 34 studies, adherence was operationalized as "the more use, the better," whereas 28 studies described a threshold for intended use of the technology as well. Out of these 28, only 6 reported a justification for the intended use. The proportion of evaluations of mental health technologies reporting a justified operationalization of intended use is lagging behind compared with evaluations of lifestyle and chronic care technologies. The results indicated that a justification of intended use does not require extra measurements to determine adherence to the technology. CONCLUSIONS: The results of this review showed that to date, justifications for intended use are often missing in evaluations of adherence. Evidently, it is not always possible to estimate the intended use of a technology. However, such measures do not meet the definition of adherence and should therefore be referred to as the actual usage of the technology. Therefore, it can be concluded that adherence to eHealth technology is an underdeveloped and often improperly used concept in the existing body of literature. When defining the intended use of a technology and selecting valid measures for adherence, the goal or the assumed working mechanisms should be leading. Adherence can then be standardized, which will improve the comparison of adherence rates to different technologies with the same goal and will provide insight into how adherence to different elements contributed to the outcomes.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic review of 62 eHealth evaluations finds that adherence is often operationalized as “the more use, the better” without an intended-use threshold, and even when a threshold is given, justification is usually missing. The authors argue adherence should be defined against a justified intended use tied to the technology’s goal or assumed working mechanisms.

Key findings

  • Of 62 included studies, 34 used Category A (“more use, the better”), 23 provided an intended-use threshold without justification (Category B), and 5 provided a justified threshold (Category C); the abstract reports 6 justified among 28 threshold studies.
  • Mental health technologies lagged: 20/31 had Category A operationalizations, while Category C was more equally distributed across mental health, lifestyle, and chronic care.
  • Most operationalizations used ≤2 measures (49/62); 10/13 with ≥3 measures were Category A. Adherence to unstructured technologies more often used logins/sessions (P=.03), features accessed/used (P<.001), and time spent (P<.001); structured technologies used modules/lessons completed (P<.001).
  • Technologies were mostly Web-based (51/62); 29 were structured, 18 unstructured, and 15 hybrid, with half targeting mental health.

Methods

  • Systematic review; searched Scopus, Web of Science, ScienceDirect, and PsycINFO; 7005 records identified, 62 full texts included after title, abstract, and full-text screening.
  • Included health-related technologies intended for repeated use, with objective quantifiable usage measures and an adherence operationalization; excluded offline treatment adherence, protocol adherence, simple information-exchange tools, conference abstracts, and unavailable full texts.
  • Data were extracted and adherence operationalizations categorized as A/B/C; differences were tested with chi-square test of independence, using Monte Carlo correction when expected counts were low, at alpha .05.

Limitations

  • Stated: only studies using adherence as an outcome or explicitly stating a proxy were included, so studies using related terms (nonusage attrition, engagement, drop-out, [non]usage) may have been missed.
  • Stated: adherence operationalizations were categorized per study, not per technology, though the same technology could have different operationalizations across studies.
  • Evident from the text: the abstract and results differ on the number of justified operationalizations (6 vs 5), and the review is descriptive/categorical without meta-analytic synthesis.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/jmir.8578

Cites

83 of 83 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 of 34 cited sources checked · 0 supported · 0 flagged

Show all 57 claims

  • Probably not a substantial claim. 28
  • It may state two separate claims as one. 20
  • The quote only partly states this claim. 2
40 of 40

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Abstract reports six justified studies, but results report five

    Minor

    The abstract says 6 studies justified their intended-use threshold, whereas the results identify 5 Category C studies. This changes the reported number meeting the paper's justification criterion.

  • Category A and B denominator is one too low

    Minor

    There are 34 Category A and 23 Category B operationalizations, totaling 57 rather than the stated 56.

    Reported 56 · recomputed 57

  • Chronic-care percentage in Category C is incorrect

    Minor

    Table 4 gives one chronic-care study among five Category C studies, which is 20%, not 40%.

    Reported 40 · recomputed 20

  • Claim of no measure differences conflicts with Table 4

    Minor

    The discussion says no significant differences were found between the kinds of measures used across the three categories, but Table 4 marks the days/weeks/months measure as significant (aP<.05).

  • Time-spent percentage in Category C is incorrect

    Minor

    Table 4 gives one study using time spent among five Category C studies, which is 20%, not 17%.

    Reported 17 · recomputed 20

  • Smartphone-app count differs between prose and Table 3

    Minor

    The prose reports five smartphone apps, but Table 3 reports six, distributed as 3 structured, 2 unstructured, and 1 hybrid.

6 of 6

Citations

Cites

Flagged first, then by citations

1 of the 84 references couldn’t be matched to a known paper.

Research works printed but not linked: 4

  1. SmithWPlodererBWadleyGWebberSBorlandRTrajectories of Engagement and Disengagement with a Story-Based Smoking Cessation App20172017 CHI Conference on Human Factors in Computing SystemsMay 6-11, 2017Denver, ColoradoACM304556
  2. Batterham, PJ, 2008, Patient Prefer Adherence, 2, 97
  3. An, LC, 2006, Nicotine Tob Res, 8, S7
  4. Wangberg, SC, 2008, Patient Prefer Adherence, 2, 57

Cited by

Sorted by citations

See all 331

Metadata

Authors
Sieverink, F., Kelders, S. M., & van Gemert-Pijnen, J. E.
Affiliations
  • University of Twente (NL)
  • North-West University (ZA)
Corresponding author
Sieverink, Floor
Venue
Journal of Medical Internet Research
Published
2017-12-06
Volume
19
Issue
12
Pages
e402
ISSN
1438-8871 (ISSN-L)
OpenAlex
W2774320409
Cited by
331
References
84
Citations by year
2026: 28 · 2025: 33 · 2024: 35 · 2023: 38 · 2022: 45 · 2021: 51 · 2020: 48 · 2019: 31 · 2018: 21 · 2017: 1
FWCI
4.86
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • University of Twente Research Information (publishedVersion, cc-by)
  • University of Twente Research Information (submittedVersion, cc-by)
Topics
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
Keywords
eHealth technologies, technology adherence, intended use, mental health technology, eHealth evaluation, operating mechanism
MeSH terms
Humans, Patient Compliance, Telemedicine, Biomedical Technology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review