CoChat
Sign Up
Cited by 331Mobile Health and mHealth ApplicationsDigital Mental Health InterventionsTelemedicine and Telehealth Implementation

Clarifying the Concept of Adherence to eHealth Technology: Systematic Review on When Usage Becomes Adherence

Open access
Sieverink, Floor, Kelders, Saskia M, van Gemert-Pijnen, Julia EWC·2017·Journal of Medical Internet Research
DOI 10.2196/jmir.8578PMC PMC5738543
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: In electronic health (eHealth) evaluations, there is increasing attention for studying the actual usage of a technology in relation to the outcomes found, often by studying the adherence to the technology. On the basis of the definition of adherence, we suggest that the following three elements are necessary to determine adherence to eHealth technology: (1) the ability to measure the usage behavior of individuals; (2) an operationalization of intended use; and (3) an empirical, theoretical, or rational justification of the intended use. However, to date, little is known on how to operationalize the intended usage of and the adherence to different types of eHealth technology. OBJECTIVE: The study aimed to improve eHealth evaluations by gaining insight into when, how, and by whom the concept of adherence has been used in previous eHealth evaluations and finding a concise way to operationalize adherence to and intended use of different eHealth technologies. METHODS: A systematic review of eHealth evaluations was conducted to gain insight into how the use of the technology was measured, how adherence to different types of technologies was operationalized, and if and how the intended use of the technology was justified. Differences in variables between the use of the technology and the operationalization of adherence were calculated using a chi-square test of independence. RESULTS: In total, 62 studies were included in this review. In 34 studies, adherence was operationalized as "the more use, the better," whereas 28 studies described a threshold for intended use of the technology as well. Out of these 28, only 6 reported a justification for the intended use. The proportion of evaluations of mental health technologies reporting a justified operationalization of intended use is lagging behind compared with evaluations of lifestyle and chronic care technologies. The results indicated that a justification of intended use does not require extra measurements to determine adherence to the technology. CONCLUSIONS: The results of this review showed that to date, justifications for intended use are often missing in evaluations of adherence. Evidently, it is not always possible to estimate the intended use of a technology. However, such measures do not meet the definition of adherence and should therefore be referred to as the actual usage of the technology. Therefore, it can be concluded that adherence to eHealth technology is an underdeveloped and often improperly used concept in the existing body of literature. When defining the intended use of a technology and selecting valid measures for adherence, the goal or the assumed working mechanisms should be leading. Adherence can then be standardized, which will improve the comparison of adherence rates to different technologies with the same goal and will provide insight into how adherence to different elements contributed to the outcomes.

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic review of 62 eHealth evaluations finds that adherence is often operationalized as “the more use, the better” without an intended-use threshold, and even when a threshold is given, justification is usually missing. The authors argue adherence should be defined against a justified intended use tied to the technology’s goal or assumed working mechanisms.

Key findings

  • Of 62 included studies, 34 used Category A (“more use, the better”), 23 provided an intended-use threshold without justification (Category B), and 5 provided a justified threshold (Category C); the abstract reports 6 justified among 28 threshold studies.
  • Mental health technologies lagged: 20/31 had Category A operationalizations, while Category C was more equally distributed across mental health, lifestyle, and chronic care.
  • Most operationalizations used ≤2 measures (49/62); 10/13 with ≥3 measures were Category A. Adherence to unstructured technologies more often used logins/sessions (P=.03), features accessed/used (P<.001), and time spent (P<.001); structured technologies used modules/lessons completed (P<.001).
  • Technologies were mostly Web-based (51/62); 29 were structured, 18 unstructured, and 15 hybrid, with half targeting mental health.

Methods

  • Systematic review; searched Scopus, Web of Science, ScienceDirect, and PsycINFO; 7005 records identified, 62 full texts included after title, abstract, and full-text screening.
  • Included health-related technologies intended for repeated use, with objective quantifiable usage measures and an adherence operationalization; excluded offline treatment adherence, protocol adherence, simple information-exchange tools, conference abstracts, and unavailable full texts.
  • Data were extracted and adherence operationalizations categorized as A/B/C; differences were tested with chi-square test of independence, using Monte Carlo correction when expected counts were low, at alpha .05.

Limitations

  • Stated: only studies using adherence as an outcome or explicitly stating a proxy were included, so studies using related terms (nonusage attrition, engagement, drop-out, [non]usage) may have been missed.
  • Stated: adherence operationalizations were categorized per study, not per technology, though the same technology could have different operationalizations across studies.
  • Evident from the text: the abstract and results differ on the number of justified operationalizations (6 vs 5), and the review is descriptive/categorical without meta-analytic synthesis.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/jmir.8578

Cites

83 of 83 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 34 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Most Category A operationalizations contain a measure for the number of modules that the users accessed or completed (19/34).

    “Most Category A operationalizations contain a measure for the number of modules that the users accessed or completed (19/34)”
    Quote in the paperNot a substantial claim
  • Findingown data

    Category C definitions are mostly based on the number of accessed or completed modules (3/5 operationalizations).

    “Category C definitions are mostly based on the number of accessed or completed modules (3/5 operationalizations).”
    Quote in the paperNot a substantial claim
  • Findingown data

    Justifications of intended use are often based on the goal of the technology and/or the assumed working mechanisms, leading to more focused operationalizations that do not require additional measurements to evaluate adherence to the technology.

    “In contrast, the results indicated that justifications of intended use are often based on the goal of the technology and/or the assumed working mechanisms, leading to more focused operationalizations that do not require additional measurements to evaluate adherence to the technology.”
    Quote in the paperSubstantial claim
  • Findingown data

    In 34 studies, adherence was operationalized as "the more use, the better" (Category A).

    “In 34 studies, adherence was operationalized as “the more use, the better,””
    • Also stated in: § Operationalization of the Adherence Definition · plain
      “Out of the 62 included articles, 34 reported adherence only in terms of how often the technology was used (Category A operationalization)”
    Quote in the paperSubstantial claim
  • Findingown data

    Out of the 28 studies that described a threshold for intended use, only 6 reported a justification for the intended use.

    “Out of these 28, only 6 reported a justification for the intended use.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 28
  • May combine two claims. 20
  • The quote only partly states this claim. 2

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Almost half of the technologies (29/62) were structured technologies, 18 were unstructured technologies, and 15 had a hybrid nature.

    “Almost half of the technologies (29/62) were structured technologies, 18 were unstructured technologies, and 15 had a hybrid nature.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Most of the structured technologies (17/29) and almost all hybrid technologies (12/15) were aimed at improving mental health (P<.001).

    “Most of the structured technologies (17/29) and almost all hybrid technologies (12/15) were aimed at improving mental health (P<.001).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The technologies described in most of the articles are Web-based (51/62).

    “The technologies described in most of the articles are Web-based (51/62).”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Adherence to unstructured technologies was mostly operationalized using the time spent using the technology (P<.001).

    “Adherence to unstructured technologies was mostly operationalized using the number of logins or sessions (P=.03), the number of features accessed or used (P<.001), and the time spent using the technology (P<.001).”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Most Category B operationalizations contain the number of accessed or completed modules (8/23).

    “the number of accessed or completed modules (8/23)”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In total, 62 studies were included in this review.

    “In total, 62 studies were included in this review.”
    • Also stated in: § Study Selection · plain
      “After screening of the titles, abstracts, and full texts, 62 full texts were included in this review.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Half of all included articles reported adherence to mental health technologies.

    “Half of all included articles reported adherence to mental health technologies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

    Findingcitation

    Carolan et al describe a protocol for a study to be conducted to understand the optimum adherence to the technology in relation to the outcomes.

    “Carolan et al describe a protocol for a study to be conducted to understand the optimum adherence to the technology in relation to the outcomes [78].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [78] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    The included Category C operationalizations provided justification in various ways.

    “The included Category C operationalizations provided justification in various ways.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    Beatty et al considered a user to be highly adherent when a therapeutic dose of 66% of the intervention was received, and this threshold was obtained from previous studies.

    “Beatty et al considered a user to be highly adherent when a therapeutic dose of 66% of the intervention was received [76]. This threshold for a therapeutic dose was obtained from previous studies.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [76] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    In the study by Mertens et al, users were seen as adherent when the technology was used in accordance with the recommendations for medication use.

    “In the study by Mertens et al, technology use was represented in relation to medication use, and users were therefore seen as adherent when the technology was used in accordance with the recommendations for medication use [77].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [77] Mertens, A, 2017, Advances in Human Factors and Ergonomics in Healthcare source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    In the study by Zeng et al, the technology consisted of different elements that were all evaluated as effective in other studies, and users were considered adherent if they used all elements.

    “In the study by Zeng et al, the technology consisted of different elements that were all evaluated as effective in other studies [75]. Users were considered adherent if they used all elements.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [75] source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    The majority of the technologies described in the included studies were structured or hybrid Web-based interventions targeting mental health (mostly CBT interventions) or unstructured technologies for lifestyle support.

    “The majority of the technologies described in these studies were structured or hybrid Web-based interventions targeting mental health (mostly cognitive behavioral therapy [CBT] interventions) or unstructured technologies for lifestyle support.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Half of all operationalizations are based on the assumption of "the more use, the better" and do not include a threshold for intended use.

    “half of all operationalizations are based on the assumption of “the more use, the better” and do not include a threshold for intended use.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    eHealth evaluations often fail to demonstrate the dose-response relationship or simply define it as "the more use, the better."

    “eHealth evaluations often fail to demonstrate the dose-response relationship (the usage that is minimally needed to experience certain effects) or simply define it as “the more use, the better.””
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The operationalization of intended use is mostly based on the assumption of "the more use, the better" and when a threshold of intended use is provided, justification is often missing.

    “the results of this review show that the operationalization of intended use is mostly based on the assumption of “the more use, the better” and that when a threshold of intended use is provided, justification is often missing.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationcitation

    There should be a threshold for intended use of a CBT technology.

    “and that there should be a threshold for intended use.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Adherence to eHealth technology is an underdeveloped and often improperly used concept in the existing body of literature.

    “Therefore, it can be concluded that adherence to eHealth technology is an underdeveloped and often improperly used concept in the existing body of literature.”
    • Also stated in: § Conclusions · plain
      “it can be concluded that adherence to eHealth technology is an underdeveloped and often improperly used concept in the existing body of literature.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The majority of mental health technologies is based on (principles of) cognitive behavioral therapy (CBT), which is the most studied treatment for depression and has proven effective in many studies.

    “This is unexpected because the majority of mental health technologies is based on (principles of) CBT, which is the most studied treatment for depression and has proven effective in many studies [79].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [79] Cuijpers, P, 2013, Can J Psychiatry source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Measures that do not include an operationalization of intended use do not meet the definition of adherence and should be referred to as the actual usage of the technology.

    “However, such measures do not meet the definition of adherence and should therefore be referred to as the actual usage of the technology.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Three elements are necessary to determine adherence to eHealth technology: (1) the ability to measure the usage behavior of individuals; (2) an operationalization of intended use; and (3) an empirical, theoretical, or rational justification of the intended use.

    “On the basis of the definition of adherence, we suggest that the following three elements are necessary to determine adherence to eHealth technology: (1) the ability to measure the usage behavior of individuals; (2) an operationalization of intended use; and (3) an empirical, theoretical, or rational justification of the intended use.”
    • Also stated in: § Determining Adherence · hedged
      “To summarize, based on the definition of adherence, we suggest that the following three elements are necessary to determine adherence to eHealth technology: (1) the ability to measure the usage behavior of individuals; (2) an operationalization of intended use; and (3) an empirical, theoretical, or rational justification of the intended use.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [10] Sabaté, E, 2003, Adherence to long-term therapies: evidence for action

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    According to the definitions, the intended use is the minimum use to establish adherence.

    “According to these definitions, the intended use is thus the minimum use to establish adherence.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    An important reason for the lack of justifications for the intended use of eHealth technologies might be a lack of knowledge regarding the working mechanisms of technology-based applications.

    “An important reason for the lack of justifications for the intended use of eHealth technologies might be that there is a lack of knowledge regarding the working mechanisms of technology-based applications [8].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [8] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Adherence is mostly based on measures for usage behavior, whereas definitions of engagement usually incorporate subjective attributes of challenge, positive affect, endurability, and aesthetic and sensory appeals.

    “Moreover, definitions of engagement usually incorporate the more subjective attributes of challenge, positive affect, endurability, and aesthetic and sensory appeals [12], whereas adherence is mostly based on measures for usage behavior.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [12] SmithWPlodererBWadleyGWebberSBorlandRTrajectories of Engagement and Disengagement with a Story-Based Smoking Cessation App20172017 CHI Conference on Human Factors in Computing SystemsMay 6-11, 2017Denver, ColoradoACM304556

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    Users do not always have to experience all of the available elements of a technology or have to use the same elements because usage goals may differ across users.

    “This implies that users do not always have to experience all of the available elements of a technology or have to use the same elements because usage goals may differ across users as well.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    A technology can be designed for multiple target groups and, depending on the individual user goals and the desired outcomes, technology can be used in many different ways in terms of the features that are used, as well as the frequency, time, and place of use.

    “However, a technology can be designed for multiple target groups and, depending on the individual user goals and the desired outcomes, technology can be used in many different ways in terms of the features that are used, as well as the frequency, time, and place of use [13,14].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [13] source

      Not checked in the free run.

    • cites [14] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Exercising for at least four times per week for 6 weeks is a minimal requirement for establishing exercise habits.

    “Kaushal and Rhodes found that exercising for at least four times per week for 6 weeks is a minimal requirement for establishing exercise habits [81].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [81] source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    The intended use or therapeutic dose can be justified using existing guidelines for healthy living and medication use or using previous research regarding other technologies.

    “the intended use or the “therapeutic dose” can be justified just as well using existing guidelines for healthy living and medication use [74,77] or using previous research regarding other technologies [75,76].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [74] source

      Not checked in the free run.

    • cites [77] Mertens, A, 2017, Advances in Human Factors and Ergonomics in Healthcare source

      Not checked in the free run.

    • cites [75] source

      Not checked in the free run.

    • cites [76] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    A justified threshold for intended use in accordance with the goal of the technology provides information for a concise evaluation of adherence and the working mechanisms of a technology.

    “A justified threshold for intended use in accordance with the goal of the technology provides information for a concise evaluation of adherence and the working mechanisms of a technology.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Justified operationalization can be used to standardize adherence to different eHealth technologies, making it easier to compare the adherence rates of different technologies.

    “Subsequently, justified operationalization (comprising multidimensional measures or not) can be used to standardize adherence to different eHealth technologies, making it easier to compare the adherence rates of different technologies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given

    Previous research has shown that users do not always have to experience all of the elements of a technology and that effective usage patterns might differ across users.

    “Previous research has shown that users do not always have to experience all of the elements of a technology and that effective usage patterns might differ across users.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    This approach simplifies the comparison of adherence across technologies with the same goal but different technology characteristics and a different operationalization of intended use.

    “This approach simplifies the comparison of adherence across technologies with the same goal (eg, improving physical activity) but different technology characteristics (eg, features of the technology, persuasiveness, and user-friendliness) and a different operationalization of intended use.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Donkin et al's statement that it is difficult to compare adherence with different technologies when measures vary is based on Category A operationalizations of adherence where more use is better.

    “this statement is based on Category A operationalizations of adherence where more use is better.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Following the critical lesson can be seen as an important precondition for gaining effects from using the intervention.

    “following this lesson can also be seen as an important precondition for gaining effects from using the intervention.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationnone given

    Adherence and intended use can be defined in many different ways, and it is very possible for different operationalizations to be used for different studies regarding the same technology.

    “adherence and intended use can be defined in many different ways, and it is very possible for different operationalizations to be used for different studies regarding the same technology.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    This approach also simplifies the comparison of adherence of different users of a specific technology.

    “At the same time, this approach also simplifies the comparison of adherence of different users of a specific technology.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Using Category B or C operationalizations allows calculation of the percentage of people who adhered to the intervention, making adherence a more objective and standardized concept.

    “In turn, the percentage of people who adhered to the intervention can be calculated, making adherence a more objective and standardized concept [1].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [1] source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    We only included studies that used adherence as an outcome measure or studies that explicitly stated how other outcome measures are used as a proxy for adherence.

    “we only included studies that used adherence as an outcome measure or studies that explicitly stated how other outcome measures are used as a proxy for adherence.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    We may have missed relevant studies that formulated the intended use of a technology but used other related terms for adherence.

    “we may have missed relevant studies that formulated the intended use of a technology, but used other related terms for adherence (eg, nonusage attrition, engagement, drop-out, or [non]usage).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    We defined the categories of adherence operationalizations for every study, instead of every type of technology.

    “we defined the categories of adherence operationalizations for every study, instead of every type of technology.”
    Quote in the paperNot a substantial claim
40 of 40

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Abstract reports six justified studies, but results report five

    Minor

    The abstract says 6 studies justified their intended-use threshold, whereas the results identify 5 Category C studies. This changes the reported number meeting the paper's justification criterion.

  • Category A and B denominator is one too low

    Minor

    There are 34 Category A and 23 Category B operationalizations, totaling 57 rather than the stated 56.

    Reported 56 · recomputed 57

  • Chronic-care percentage in Category C is incorrect

    Minor

    Table 4 gives one chronic-care study among five Category C studies, which is 20%, not 40%.

    Reported 40 · recomputed 20

  • Claim of no measure differences conflicts with Table 4

    Minor

    The discussion says no significant differences were found between the kinds of measures used across the three categories, but Table 4 marks the days/weeks/months measure as significant (aP<.05).

  • Time-spent percentage in Category C is incorrect

    Minor

    Table 4 gives one study using time spent among five Category C studies, which is 20%, not 17%.

    Reported 17 · recomputed 20

  • Smartphone-app count differs between prose and Table 3

    Minor

    The prose reports five smartphone apps, but Table 3 reports six, distributed as 3 structured, 2 unstructured, and 1 hybrid.

6 of 6

Cites

Flagged first, then by citations

1 of the 84 references couldn’t be matched to a known paper.

Research works printed but not linked: 4

  1. SmithWPlodererBWadleyGWebberSBorlandRTrajectories of Engagement and Disengagement with a Story-Based Smoking Cessation App20172017 CHI Conference on Human Factors in Computing SystemsMay 6-11, 2017Denver, ColoradoACM304556
  2. Batterham, PJ, 2008, Patient Prefer Adherence, 2, 97
  3. An, LC, 2006, Nicotine Tob Res, 8, S7
  4. Wangberg, SC, 2008, Patient Prefer Adherence, 2, 57

Cited by

Sorted by citations

Related

Authors
Sieverink, F., Kelders, S. M., & van Gemert-Pijnen, J. E.
Affiliations
  • University of Twente (NL)
  • North-West University (ZA)
Corresponding author
Sieverink, Floor
ORCID
Venue
Journal of Medical Internet Research
Publisher
JMIR Publications Inc.
Published
2017-12-06
Volume
19
Issue
12
Pages
e402
ISSN
1438-8871 (ISSN-L)
DOI
10.2196/jmir.8578
PMC
PMC5738543
OpenAlex
W2774320409
Semantic Scholar
a1db2bb515ffdc128b4290c1c2d3f04a5b280310
Cited by
331
References
84
Citations by year
2026: 28 · 2025: 33 · 2024: 35 · 2023: 38 · 2022: 45 · 2021: 51 · 2020: 48 · 2019: 31 · 2018: 21 · 2017: 1
FWCI
4.86
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • University of Twente Research Information (publishedVersion, cc-by)
  • University of Twente Research Information (submittedVersion, cc-by)
Topics
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
Keywords
eHealth technologies, technology adherence, intended use, mental health technology, eHealth evaluation, operating mechanism
MeSH terms
Humans, Patient Compliance, Telemedicine, Biomedical Technology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review