CoChat
Sign Up
Cited by 50Digital Mental Health InterventionsHealth Policy Implementation ScienceTelemedicine and Telehealth Implementation

eMental Healthcare Technologies for Anxiety and Depression in Childhood and Adolescence: Systematic Review of Studies Reporting Implementation Outcomes

Open access
Wozney, Lori, McGrath, Patrick J, Gehring, Nicole D, Bennett, Kathryn, Huguet, Anna, Hartling, Lisa, Dyson, Michele P, Soleimani, Amir +1 more·2018·JMIR Mental Health
DOI 10.2196/mental.9655PMC PMC6039769

Published version of a preprint

Open this paper in CoChat — free account

Abstract

Background: Anxiety disorders and depression are frequent conditions in childhood and adolescence. eMental healthcare technologies may improve access to services, but their uptake within health systems is limited. Objective: The objective of this review was to examine and describe how the implementation of eMental healthcare technologies for anxiety disorders and depression in children and adolescents has been studied. Methods: We conducted a search of 5 electronic databases and gray literature. Eligible studies were those that assessed an eMental healthcare technology for treating or preventing anxiety or depression, included children or adolescents (<18 years), or their parents or healthcare providers and reported findings on technology implementation. The methodological quality of studies was evaluated using the Mixed Methods Appraisal Tool. Outcomes of interest were based on 8 implementation outcomes: acceptability (satisfaction with a technology), adoption (technology uptake and utilization), appropriateness (“fitness for purpose”), cost (financial impact of technology implementation), feasibility (extent to which a technology was successfully used), fidelity (implementation as intended), penetration (“spread” or “reach” of the technology), and sustainability (maintenance or integration of a technology within a healthcare service). For extracted implementation outcome data, we coded favorable ratings on measurement scales as “positive results” and unfavorable ratings on measurement scales as “negative results.” Those studies that reported both positive and negative findings were coded as having “mixed results.” Results: A total of 46 studies met the inclusion criteria, the majority of which were rated as very good to excellent in methodological quality. These studies investigated eMental healthcare technologies for anxiety (n=23), depression (n=18), or both anxiety and depression (n=5). Studies of technologies for anxiety evaluated the following: (1) acceptability (78%) reported high levels of satisfaction, (2) adoption (43%) commonly reported positive results, and (3) feasibility (43%) reported mixed results. Studies of technologies for depression evaluated the following: (1) appropriateness (56%) reported moderate helpfulness and (2) acceptability (50%) described a mix of both positive and negative findings. Studies of technologies designed to aid anxiety and depression commonly reported mixed experiences with acceptability and adoption and positive findings for appropriateness of the technologies for treatment. Across all studies, cost, fidelity, and penetration and sustainability were the least measured implementation outcomes. Conclusions: Acceptability of eMental healthcare technology is high among users and is the most commonly investigated implementation outcome. Perceptions of the appropriateness and adoption of eMental healthcare technology were varied. Implementation research that identifies, evaluates, and reports on costs, sustainability, and fidelity to clinical guidelines is crucial for making high-quality eMental healthcare available to children and adolescents.

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic review examined how implementation of eMental healthcare technologies for child and adolescent anxiety and depression has been studied, finding that acceptability was the most commonly investigated outcome and generally high among users. Adoption and appropriateness findings were mixed, while cost, fidelity, penetration, and sustainability were rarely or never measured.

Key findings

  • Across 46 included studies, technologies targeted anxiety (n=23), depression (n=18), or both (n=5); most studies were rated very good to excellent in methodological quality.
  • Anxiety technology studies most often evaluated acceptability (78%), adoption (43%), and feasibility (43%); depression technology studies most often evaluated appropriateness (56%) and acceptability (50%); both-condition studies evaluated acceptability (100%), adoption (40%), and appropriateness (40%).
  • Positive findings were reported 60% or more of the time for acceptability and cost, while mixed findings were reported more than 50% of the time for adoption, feasibility, and fidelity.
  • Cost, fidelity, and penetration were the least measured implementation outcomes, and no studies evaluated sustainability in the healthcare service/system.

Methods

  • Systematic review registered with PROSPERO (CRD42016049884), searching 5 databases plus gray literature, clinical trial registries, conference proceedings, and reference lists, with literature published until December 5, 2016.
  • Included studies assessed an eMental healthcare technology for treating or preventing anxiety or depression in children/adolescents (<18 years), or their parents or healthcare providers, within primary or secondary healthcare, and reported an implementation outcome.
  • Eight implementation outcomes from Proctor et al. were used: acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability; findings were coded as positive, negative, or mixed, and methodological quality was assessed with the Mixed Methods Appraisal Tool.

Limitations

  • The authors state that technologies deployed without scientific investigation or reported implementation data were unavailable for review, that databases such as NIH Reporter were not searched, and that the rapidly changing eMental health field means published accounts may be slightly outdated.
  • The authors also state that including multiple study designs challenged summarization and generalization, and that inconsistent eHealth terminology required judgment calls that could have misclassified factors within outcome categories.
  • The paper notes that included studies were limited by small sample sizes and mostly measured clinician attitudes rather than actual clinician behaviors; it also reports that more than half of studies used nonvalidated acceptability measures, raising the possibility of overestimated satisfaction ratings.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/mental.9655

Cites

103 of 103 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 95 cited sources checked · 0 supported · 0 flagged

  • Findingcitation

    The remainder (33%) reported mixed acceptability.

    “The remainder (33%) reported mixed acceptability [60,70,81].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [60] source

      Not checked in the free run.

    • cites [70] source

      Not checked in the free run.

    • cites [81] Carrasco, A, 2016, Research in Psychotherapy: Psychopathology, Process and Outcome

      Not checked in the free run.

  • Findingown data

    Five studies assessed a technology that targeted both anxiety and depression.

    “Five studies assessed a technology that targeted both anxiety and depression.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Studies examining feasibility of anxiety technologies described more mixed (70%) than positive (30%) results.

    “Studies that examined the feasibility of anxiety technologies described more mixed (70%) [38-40,44,52,53,55] than positive (30%) [36,45,80] results, including barriers to participation such as finding time to complete tasks and ease of use.”
    Quote in the paperNot a substantial claim
  • Findingcitation

    Positive acceptability results for anxiety technologies included high satisfaction and positive technology recommendations, reported by parents, children, and healthcare providers.

    “Positive results included high satisfaction and positive technology recommendations, with acceptability reported by parents [36,39,41,43,50], children [38,39,41,43,45,50,54,55,78,80], and healthcare providers [55].”
    Quote in the paperNot a substantial claim
  • Findingown data

    The common factor impacting quality scores for mixed-method studies was lack of consideration of data triangulation.

    “The common factor impacting quality scores for mixed-method studies was the lack of consideration of data triangulation (ie, see MMAT item 5.3 in Multimedia Appendix 2).”
    Quote in the paperNot a substantial claim
  • May combine two claims. 23
  • Classified as probably not a substantial claim. 38
  • The quote only partly states this claim. 2

  • May combine two claims.

    Findingown data

    Among included studies on depression technologies, 10 were excellent, 2 very good, 4 moderate, and 2 low quality (scores 25 and 0).

    “Studies on technologies for depression also varied in quality: 10 studies were of excellent quality [56,57,59,63-67,71,81], 2 were of very good quality [60,68], 4 were of moderate quality [58,61,69,72], and 2 were of low quality and received a score of 25 [62] and 0 [70].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The implementation of eMental healthcare technologies for anxiety and depressive disorders in childhood or adolescence was assessed in 23 and 18 studies, respectively.

    “The implementation of eMental healthcare technologies for anxiety and depressive disorders in childhood or adolescence was assessed in 23 and 18 studies, respectively.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Studies of technologies designed to aid anxiety and depression commonly reported mixed experiences with acceptability and adoption and positive findings for appropriateness of the technologies for treatment.

    “Studies of technologies designed to aid anxiety and depression commonly reported mixed experiences with acceptability and adoption and positive findings for appropriateness of the technologies for treatment.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The location of included studies was restricted to economically developed countries, with the United States (20 studies) and Australia (13 studies) being the most common locations.

    “The location of studies was restricted to economically developed countries with the United States (20 studies) and Australia (13 studies) being the most common locations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Of 6 studies examining appropriateness of anxiety technologies, 4 described positive results (67%) and 2 reported mixed results (33%).

    “Of the 6 studies that examined appropriateness, 4 described positive results (67%) [39,50,51,78] such as positive attitudes and perceived helpfulness of the technology among healthcare providers [39,51], while 2 studies [53,55] reported mixed results (33%) including moderate usefulness and helpfulness of the program for the youth [53].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Studies of technologies for depression evaluated appropriateness (56%) reported moderate helpfulness and acceptability (50%) described a mix of both positive and negative findings.

    “Studies of technologies for depression evaluated the following: (1) appropriateness (56%) reported moderate helpfulness and (2) acceptability (50%) described a mix of both positive and negative findings.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Studies testing technologies relevant to both anxiety and depression tended to evaluate acceptability (100%), adoption (40%), and appropriateness (40%).

    “Studies testing technologies relevant to both anxiety and depression tended to evaluate acceptability (100%), adoption (40%), and appropriateness (40%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Only one study investigated technology penetration for anxiety technologies, reporting positive penetration with technology purchased by 56 child psychiatric institutions or practitioners within 1 year.

    “Only one study investigated technology penetration, reporting positive penetration with technology purchased by 56 child psychiatric institutions or practitioners within 1 year [51].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Studies evaluating technologies applicable to both anxiety and depression were of excellent, very good, and moderate quality.

    “Studies evaluating technologies applicable to both anxiety and depression were of excellent [75,77], very good [74], and moderate [73,76] quality.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Among included studies on anxiety technologies, 11 were excellent quality (score 100), 6 very good (75), 4 moderate (50), and 2 poor (25).

    “In total, 11 studies on eMental healthcare technologies for anxiety were of excellent quality with a score of 100 [37,46-50,52,55,78-80], 6 were of very good quality with a score of 75 [36,38,39,44,51,53], 4 were of moderate quality with a score of 50 [42,43,45,54], and 2 were of poor quality with a MMAT score of 25 [40,41].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Technology adoption for anxiety technologies was examined by 10 studies, with positive (60%) and mixed (40%) findings for technology compliance and adherence.

    “Technology adoption was examined by 10 studies with studies reporting positive (60%) [42-44,47,50,53] and mixed (40%) [45,46,55,79] findings for technology compliance and adherence.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Most studies reported the technologies as acceptable (67%) with high satisfaction.

    “Most studies reported the technologies as acceptable (67%) with high satisfaction [56,61,63,72]”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [56] source

      Not checked in the free run.

    • cites [61] source

      Not checked in the free run.

    • cites [63] Van Voorhees, BW, 2009, J Cogn Behav Psychother

      Not checked in the free run.

    • cites [72] source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    A total of 32 studies examined internet-based technologies, 11 examined computer-based technologies, and 3 examined smartphone-based technologies as part of treatment.

    “A total of 32 studies examined internet-based technologies, 11 examined computer-based technologies, and 3 examined smartphone-based (app/short message service, SMS, text message) technologies as part of treatment.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Four studies examining fidelity reported mixed results, particularly healthcare provider adherence to the program.

    “Four studies examining fidelity reported mixed results [62-65], particularly healthcare provider adherence to the program.”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [62] source

      Not checked in the free run.

    • cites [63] Van Voorhees, BW, 2009, J Cogn Behav Psychother

      Not checked in the free run.

    • cites [64] source

      Not checked in the free run.

    • cites [65] Van Voorhees, BW, 2008, J Can Acad Child Adolesc Psychiatry

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Penetration was successful, implementing the technology successfully in 12 practices.

    “Although penetration was successful, implementing the technology successfully in 12 practices”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [66] source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Two studies examined adoption and reported high website usage rates.

    “and high website usage rates [77]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [77] source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Two studies examined cost outcomes and described intervention implementation as economically viable.

    “Two studies examined cost outcomes [66,67] and described intervention implementation as economically viable.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [66] source

      Not checked in the free run.

    • cites [67] Ruby, A, 2013, J Ment Health Policy Econ

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Both found positive attitudes and perceived helpfulness from healthcare providers.

    “and healthcare providers [74]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [74] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    We did not search databases such as the NIH Reporter.

    “We did not search databases such as the NIH Reporter”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    The inconsistent use of eHealth terminology across the literature required us to make judgment calls regarding how to group implementation data across the 8 outcome categories.

    “Finally, the inconsistent use of eHealth terminology [105-107] across the literature required us to make judgment calls regarding how to group implementation data across the 8 outcome categories.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [105] Dixon, B, 2007, Studies in Health Technology and Informatics

      Not checked in the free run.

    • cites [106] Oh, H, 2005, World Hosp Health Serv

      Not checked in the free run.

    • cites [107] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The inclusion of multiple study designs created a challenge for summarizing study features and generalizing study findings.

    “The inclusion of multiple study designs created a challenge for summarizing study features and generalizing study findings.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    Technologies being deployed in healthcare systems that have not been scientifically investigated and without reported implementation data were not available for our review.

    “Technologies being deployed in healthcare systems that have not been scientifically investigated and without reported implementation data were not available for our review.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Anxiety and depression in children and adolescents cause significant interference with relationships, academic performance, school attendance, and daily functioning.

    “The long-term impact of anxiety and depression on children and adolescents includes significant interference with relationships, academic performance, school attendance, and daily functioning, making early intervention vital [3-8].”
    Quote in the paperSubstantial claim0 of 6 sources checked
    • cites [3] source

      Not checked in the free run.

    • cites [4] Fergusson, DM, 2002, Arch Gen Psychiatry source

      Not checked in the free run.

    • cites [5] source

      Not checked in the free run.

    • cites [6] source

      Not checked in the free run.

    • cites [7] source

      Not checked in the free run.

    • cites [8] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    At least 6.5% of children and adolescents worldwide meet criteria for anxiety disorders and 2.6% for depressive disorders.

    “Worldwide, at least 6.5% and 2.6% of children and adolescents meet the criteria for anxiety and depressive disorders, respectively [1].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The search strategy was peer reviewed prior to implementation.

    “The strategy was peer reviewed prior to implementation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The burden associated with anxiety and depression rises sharply in childhood and peaks in adolescence and young adulthood (ages 15-24 years).

    “The burden associated with these disorders rises sharply in childhood and peaks in adolescence and young adulthood (ages, 15-24 years) [2].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [2] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Interrater agreement for screening was calculated with the kappa statistic for every 100 articles screened.

    “calculated the interrater agreement with the kappa statistic for every 100 articles screened [32].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [32] Altman, D, 1991, Practical statistics for medical research

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In the past 5 years, a number of literature reviews have highlighted an increase in research and development activities for eMental healthcare technologies for children and adolescents.

    “In the past 5 years, a number of literature reviews have highlighted the increase in research and development activities for eMental healthcare technologies for children and adolescents [18-22].”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [18] source

      Not checked in the free run.

    • cites [19] source

      Not checked in the free run.

    • cites [20] source

      Not checked in the free run.

    • cites [21] source

      Not checked in the free run.

    • cites [22] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Three reviewers independently screened titles and abstracts.

    “Three reviewers (AS, NDG, and MO) independently screened the titles and abstracts in the EndNote library”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    A sufficiently high kappa (≥0.80) was reached before remaining references were divided.

    “Once a sufficiently high kappa was reached (≥0.80), the remaining references in the library were divided into 3 equally sized groups.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Reporting of the review adheres to the PRISMA statement checklist.

    “Reporting of the review adheres to the Preferred Reporting Items of Systematic Reviews and Meta-Analyses statement checklist [30].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Underdiagnosis and undertreatment of childhood and adolescent depression and anxiety are well-documented concerns.

    “Underdiagnosis and undertreatment of childhood and adolescent depression and anxiety are well-documented concerns [9,10].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [9] source

      Not checked in the free run.

    • cites [10] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    A protocol for the review was developed and registered with PROSPERO.

    “A protocol for the review was developed and registered with PROSPERO (Registration #CRD42016049884).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    No studies were excluded on the basis of low-quality assessment scores.

    “We did not exclude any studies on the basis of low-quality assessment scores.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Three reviewers independently reviewed the full-text of potentially eligible studies.

    “Three reviewers (AS, NDG, MO) independently reviewed the full-text of studies that were identified as potentially eligible using the review’s inclusion and exclusion criteria.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Each article was assessed by 2 reviewers.

    “allowing each article to be assessed by 2 reviewers”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Interpretationown data

    Acceptability is the most commonly investigated implementation outcome.

    “and is the most commonly investigated implementation outcome.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Study quality was assessed using the Mixed Methods Appraisal Tool (MMAT).

    “The quality of studies was assessed using the Mixed Methods Appraisal Tool (MMAT) [33].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [33] PluyePRobertECargoMBartlettGO'CathainAGriffithsFMcGill University20112018-06-05Proposal: a mixed methods appraisal tool for systematic mixed studies reviews http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/84371689/MMAT%202011%20criteria%20and%20tutorial%202011-06-29updated2014.08.21.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Four team members reviewed assignments of study outcome data to implementation categories and finalized them after confidence that categorization was accurate.

    “Four team members (NDG, MO, AS, and ASN) reviewed the assignments of the study outcome data to the implementation categories, and assignments were finalized after all team members were confident that the data were categorized accurately.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Methodological quality was assessed independently by 2 of 3 assessors.

    “Methodological quality was assessed independently by 2 of the 3 assessors (AS, NDG, and MO).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The MMAT scoring scale was pilot tested for reliability.

    “The scoring scale ranges from 0 (low quality) to 100 (high quality) and was pilot tested for reliability [34].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [34] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Data were extracted by one reviewer and reviewed for accuracy and completeness by another.

    “Data were extracted by one reviewer (AS, NDG, or MO), and reviewed for accuracy and completeness by another.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Young people's internet self-efficacy, self-regulatory skills, and perceived quality of online learner-instructor interaction are important predictors of online engagement.

    “research has shown that young people’s internet self-efficacy, self-regulatory skills, and perceived quality of online learner-instructor interaction are important predictors of online engagement [90]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [90] Kuo, Y, 2013, Int Rev of Res in Open and Dis Learn

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    All of these factors play a central role in behavior change associated with effective mental health treatments.

    “All of these factors play a central role in behavior change associated with effective mental health treatments [87,88,89]”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [87] John, J, 1992, J Health Care Mark

      Not checked in the free run.

    • cites [88] source

      Not checked in the free run.

    • cites [89] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Treatment ideologies held by clinicians, parents, and children/youth lead them to greater skepticism about whether eMental healthcare can deliver the same quality of care as face-to-face services for children and adolescents.

    “It may because the treatment ideologies (ie, beliefs about the etiology of illness, the roles of the provider/patient, and the efficacy of various treatments) [92] held by clinicians, parents, and children/youth lead them to greater skepticism about whether eMental healthcare can deliver the same quality of care [93] as face-to-face services for children and adolescents.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [92] source

      Not checked in the free run.

    • cites [93] source

      Not checked in the free run.

    • Rests on an assumption: Investments in eMental healthcare are rarely withdrawn because of purported safety risks or over concerns about the quality of care.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Administrators, providers, and the general public feel that eMental healthcare is an appropriate treatment modality.

    “This suggests that administrators, providers, and the general public feel that eMental healthcare is an appropriate treatment modality”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Investments in eMental healthcare are rarely withdrawn because of purported safety risks or over concerns about the quality of care.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). The quote only partly states this claim.

    Interpretationreasoning

    Higher acceptability rates of anxiety-focused interventions than of depression-focused ones could be explained by public opinion and clinician beliefs.

    “This could be one explanation for the higher acceptability rates of anxiety-focused interventions than of depression-focused ones.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    These implementation facets are important macro-level determinants of policies and strategies for technology integration.

    “These implementation facets are important macro-level determinants of policies and strategies for technology integration [100,101]”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [100] source

      Not checked in the free run.

    • cites [101] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The implementation science of eMental health for adolescent anxiety and depression needs to mature.

    “These findings suggest that the implementation science of eMental health for adolescent anxiety and depression needs to mature.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Published accounts of the state of this field will likely always be slightly outdated.

    “published accounts of the state of this field will likely always be slightly outdated”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: eMental healthcare technologies are constantly appearing and disappearing from the behavioral health service landscape.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    This approach allowed for the comparison of different kinds of evidences that shape real-world policy and service delivery.

    “Nonetheless, this approach allowed for the comparison of different kinds of evidences that shape real-world policy and service delivery.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    Closing these knowledge gaps has the potential to make treatments more accessible and reduce the burden of anxiety and depression on affected children and adolescents.

    “Closing these knowledge gaps has the potential to make treatments more accessible and reduce the burden of anxiety and depression on affected children and adolescents.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    We reported on quality via a validated appraisal tool.

    “By not limiting our search based on study design, but rather reporting on quality via a validated appraisal tool”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    This could have resulted in the misclassification of some factors within the wrong outcome category.

    “This could have resulted in the misclassification of some factors within the wrong outcome category [31].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    There are gaps in knowledge about the fidelity of eMental health interventions over time, how eMental health technologies spread through the healthcare system, direct and indirect costs, and sustainability models.

    “Future studies should help bridge gaps in knowledge about the fidelity of eMental health interventions over time and how eMental health technologies spread through the healthcare system, direct and indirect costs, and sustainability models.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationnone given

    These factors often require longer-term follow-up to adequately assess.

    “because these factors often require longer-term follow-up to adequately assess”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: There is concern that the current methodological approaches for eMental healthcare technology have a long lag-time from initiation, to publication of outcomes or implementation.
56 of 56

  • The paper relies on this without support.

    The 46 included studies are representative of the available evidence on implementation of eMental healthcare technologies for anxiety and depression in children and adolescents.

    “After full-text review, 46 studies (plus one erratum) articles met the inclusion criteria.”

  • The paper relies on this without support.

    Almost all validated psychiatric patient satisfaction measures are validated for adults (not children and adolescents).

    “Given that almost all validated psychiatric patient satisfaction measures are validated for adults (not children and adolescents)”

  • The paper relies on this without support.

    Investments in eMental healthcare are rarely withdrawn because of purported safety risks or over concerns about the quality of care.

    “yet, investments in eMental healthcare are rarely withdrawn because of purported safety risks or over concerns about the quality of care”

  • The paper relies on this without support.

    Adoption continues to be conceptualized in the literature primarily as a posthoc measure of “adherence”.

    “As adoption continues to be conceptualized in the literature primarily as a posthoc measure of “adherence,””

  • The paper relies on this without support.

    There is concern that the current methodological approaches for eMental healthcare technology have a long lag-time from initiation, to publication of outcomes or implementation.

    “and there is concern that the current methodological approaches for eMental healthcare technology have a long lag-time from initiation, to publication of outcomes or implementation”

  • The paper relies on this without support.

    The promise of scalable, more cost-effective treatments is widely argued in eMental health planning.

    “While the promise of scalable, more cost-effective treatments is widely argued in eMental health planning”

  • The paper relies on this without support.

    eMental healthcare technologies are constantly appearing and disappearing from the behavioral health service landscape.

    “Given that eMental healthcare technologies are constantly appearing and disappearing from the behavioral health service landscape”
7 of 7

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Depression appropriateness is described as both below and above half

    Minor

    The discussion says a little less than half of depression studies measured appropriateness, whereas the results report 56%, which is more than half.

  • Study [79] has conflicting acceptability findings

    Minor

    The results text classifies study [79] as having mixed acceptability findings, but Table 2 marks its acceptability finding as positive.

2 of 2

Cites

Flagged first, then by citations

1 of the 104 references couldn’t be matched to a known paper.

Research works printed but not linked: 15

  1. Boydell, KM, 2014, J Can Acad Child Adolesc Psychiatry, 23, 87
  2. Christensen, H, 2010, Med J Aust, 192, S2
  3. Christensen, H, 2010, Med J Aust, 192, S53
  4. Crabtree, B, 1992, Research methods for primary care, 93
  5. Cunningham, M, 2006, Aust E J Adv Ment Health, 5, 1
  6. Van Voorhees, BW, 2009, J Cogn Behav Psychother, 9, 1
  7. Van Voorhees, BW, 2008, J Can Acad Child Adolesc Psychiatry, 17, 184
  8. Ruby, A, 2013, J Ment Health Policy Econ, 16, 121
  9. Carrasco, A, 2016, Research in Psychotherapy: Psychopathology, Process and Outcome, 19, 10
  10. John, J, 1992, J Health Care Mark, 12, 56
  11. Kuo, Y, 2013, Int Rev of Res in Open and Dis Learn, 14, 16
  12. Flewett, T, 2010, Clinical Risk Management: An introductory text for mental health clinicians
  13. Sarvet, B, 2016, Child and Adolescent Psychiatric Clinics of North America, 26
  14. Dixon, B, 2007, Studies in Health Technology and Informatics, 129, 616
  15. Oh, H, 2005, World Hosp Health Serv, 41, 32

Cited by

Sorted by citations

Related

Authors
Wozney, L., McGrath, P. J., Gehring, N. D., Bennett, K., Huguet, A., Hartling, L., Dyson, M. P., Soleimani, A., & Newton, A. S.
Affiliations
  • Izaak Walton Killam Health Centre (CA)
  • Dalhousie University (CA)
  • University of Alberta (CA)
  • McMaster University (CA)
Corresponding author
Newton, Amanda S
ORCID
Venue
JMIR Mental Health
Publisher
JMIR Publications Inc.
Published
2018-06-26
Volume
5
Issue
2
Pages
e48
ISSN
2368-7959 (ISSN-L)
DOI
10.2196/mental.9655
PMC
PMC6039769
OpenAlex
W2803107171
Cited by
50
References
104
Citations by year
2026: 5 · 2025: 5 · 2024: 4 · 2023: 6 · 2022: 7 · 2021: 6 · 2020: 11 · 2019: 5 · 2018: 1
FWCI
0.78
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Mental Health (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
Keywords
digital mental health interventions, childhood anxiety, adolescent depression, implementation outcomes, acceptability, technology adoption, feasibility, appropriateness, cost, fidelity, penetration, sustainability, children and adolescents, access to mental health care, implementation sustainability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review