CoChat
Sign Up
Cited by 693Digital Mental Health InterventionsMobile Health and mHealth ApplicationsTelemedicine and Telehealth Implementation

Behavioral Intervention Technologies: Evidence review and recommendations for future research in mental health

Open access
Mohr, David C., Burns, Michelle Nicole, Schueller, Stephen M., Clarke, Gregory, Klinkman, Michael·2013·General Hospital Psychiatry
DOI 10.1016/j.genhosppsych.2013.03.008PMC PMC3719158
Open this paper in CoChat — free accountOpen PDF

Abstract

OBJECTIVE: A technical expert panel convened by the Agency for Healthcare Research and Quality and the National Institute of Mental Health was charged with reviewing the state of research on behavioral intervention technologies (BITs) in mental health and identifying the top research priorities. BITs refers to behavioral and psychological interventions that use information and communication technology features to address behavioral and mental health outcomes. METHOD: This study on the findings of the technical expert panel. RESULTS: Videoconferencing and standard telephone technologies to deliver psychotherapy have been well validated. Web-based interventions have shown efficacy across a broad range of mental health outcomes. Social media such as online support groups have produced disappointing outcomes when used alone. Mobile technologies have received limited attention for mental health outcomes. Virtual reality has shown good efficacy for anxiety and pediatric disorders. Serious gaming has received little work in mental health. CONCLUSION: Research focused on understanding reach, adherence, barriers and cost is recommended. Improvements in the collection, storage, analysis and visualization of big data will be required. New theoretical models and evaluation strategies will be required. Finally, for BITs to have a public health impact, research on implementation and application to prevention is required.

Integrity signals

Registry

Crossref · 10.1016/j.genhosppsych.2013.03.008

Cites

86 of 86 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 98 cited sources checked · 0 supported · 0 flagged

  • Premisecitation

    Adherence to web-based interventions is widely acknowledged to be a problem.

    “Adherence to web-based interventions has been widely acknowledged to be a problem (25, 28) and should constitute a standard outcome measure and a focus of new research.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [25] source

      Not checked in the free run.

    • cites [28] source

      Not checked in the free run.

  • Interpretationnone given

    Little is known about how to optimize the design and implementation of web-based interventions.

    “While the potential for web-based interventions to be effective has been established, little is known about how to optimize their design and implementation.”
    Quote in the paperSubstantial claim
  • Premisecitation

    Personalization and tailoring remain largely unexamined in mental health web-based interventions.

    “Personalization and tailoring, which have received some attention in web-based interventions outside of mental health (26, 27), remain largely unexamined in mental health.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [26] source

      Not checked in the free run.

    • cites [27] source

      Not checked in the free run.

  • Premisecitation

    Coach- or therapist-supported internet treatments produce moderate to large outcomes for depression.

    “Outcomes of coach or therapist-supported internet treatments are moderate to large for depression, and have found to be equivalent to face-to-face group intervention for some anxiety disorders (16, 17).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [16] source

      Not checked in the free run.

    • cites [17] source

      Not checked in the free run.

  • Interpretationreasoning

    The potential for web-based interventions to be effective has been established.

    “While the potential for web-based interventions to be effective has been established, little is known about how to optimize their design and implementation.”
    Quote in the paperSubstantial claim
  • May combine two claims. 32
  • The quote only partly states this claim. 2
  • Classified as probably not a substantial claim. 41

  • May combine two claims. The quote only partly states this claim.

    Interpretationreasoning

    There is no evidence that videoconferencing, telephone, instant messaging, or email delivery media increase risk.

    “There is no evidence that these delivery media increase risk, as emergent mental health situations for outpatients are often managed at a distance, even in the context of traditional services.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Human support via periodic phone or email improves adherence and outcomes in internet-based treatments, though the effect may vary by patient population, disorder, or program quality, usability, and utility.

    “In general, human support via periodic phone or e-mail contact improves adherence and outcomes when using internet-based treatments, although this may vary depending on the patient population, disorder, or the quality, usability, and utility of the program.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Attrition and non-adherence in standalone web-based interventions range widely, from over 95% for standalone depression sites, modest for many anxiety sites, to minimal for insomnia, depending on disorder and sampling/screening.

    “Attrition and non-adherence can also range widely, from extremely high (>95% for standalone depression sites), modest (many anxiety sites), to minimal (insomnia), depending on the disorder and how participants were sampled or screened.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [14] source

      Not checked in the free run.

    • cites [15] source

      Not checked in the free run.

    • cites [18] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Standalone web-based interventions produce a wide range of outcomes: small or nonsignificant for many depression and substance abuse sites, moderate for some anxiety disorders, and fairly large for insomnia.

    “Standalone interventions have produced a wide range of outcomes, from small or nonsignificant (e.g. many depression sites, substance abuse) to moderate (some anxiety disorders) to fairly large (insomnia) (14, 15, 18, 25).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [14] source

      Not checked in the free run.

    • cites [15] source

      Not checked in the free run.

    • cites [18] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Videoconferencing-delivered psychotherapy is effective and acceptable to patients.

    “These studies have generally found that videoconferencing is effective and acceptable to patients (7).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Premisecitation

    Mobile BITs have not been effective at treating depression.

    “Trials of mobile BITs have found some positive short-term benefits for anxiety disorders (39), eating disorders (40), bipolar disorder (41), and schizophrenia (42), although others have not been effective at treating depression (43).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [43] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Web-based interventions have demonstrated efficacy for depression, anxiety disorders, alcohol and substance abuse, and insomnia.

    “The efficacy of web-based intervention has been demonstrated by a large number of trials for a number of mental health problems, including depression, anxiety disorders, alcohol and substance abuse, and insomnia (14–18).”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [14] source

      Not checked in the free run.

    • cites [15] source

      Not checked in the free run.

    • cites [16] source

      Not checked in the free run.

    • cites [17] source

      Not checked in the free run.

    • cites [18] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Psychotherapy delivered via instant messaging is effective and yields reasonable effect sizes.

    “Psychotherapy has also been provided effectively and with reasonable effect sizes via instant messaging (12).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Games have been examined as treatments for attention deficit hyperactivity disorder, autism spectrum disorders, reduction of aggression in children, and improving behavioral outcomes related to diabetes and cancer.

    “Games have been examined as treatments for attention deficit hyperactivity disorder, autism spectrum disorders, and reduction of aggression in children and for improving behavioral outcomes related to medical issues such as diabetes and cancer. Much of this work remains in early stages (61).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [61] source

      Not checked in the free run.

  • May combine two claims.

    Premisenone given

    Virtual reality has demonstrated good efficacy for anxiety disorders and pediatric disorders.

    “Virtual reality has shown good efficacy for anxiety and pediatric disorders.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    The use of internet support groups has not been rigorously investigated.

    “The use of ISGs has not been rigorously investigated.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Although evidence-based psychotherapies are numerous, providing them to all who need them remains a persistent problem because trained providers are scarce and unevenly distributed.

    “Despite the proliferation of evidence-based psychotherapies for numerous mental disorders, the delivery of those treatments to those who require them is a perennial challenge, given the shortage and uneven geographic distribution of trained providers (3).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    Most games have been developed for children and adolescents, and few trials have evaluated the efficacy of these games for mental health outcomes.

    “Most games have been developed for children and adolescents and few trials have evaluated the efficacy of these games for mental health outcomes.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisenone given

    Videoconferencing removes physical co-presence and certain cues.

    “Videoconferencing eliminates immediate presence and some cues, the telephone eliminates visual cues, instant messaging eliminates voice quality, and email eliminates synchronous communication.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Games might offer more engaging ways to teach.

    “Gaming may provide teaching methods that are more engaging.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    The use of conversational agents is just beginning and still requires initial developmental work and evaluation.

    “The use of conversational agents is just beginning and still requires initial developmental work and evaluation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    Serious gaming has attracted relatively little mental health research.

    “Serious gaming has received relatively little work in mental health.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Virtual reality builds simulated settings that allow substantial control when designing therapeutic experiences.

    “Virtual reality creates simulated environments that afford a high degree of control in engineering the provision of therapeutic experiences.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    In urban settings, 75% of primary care patients with depression report at least one structural or psychological obstacle that greatly blocks or prevents behavioral treatment access; rural rates are much higher.

    “Even in urban areas, 75% of primary care patients with depression can identify one or more structural or psychological barriers that substantially interfere with or prevent access to behavioral treatment; these rates are far higher in rural areas (2).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [2] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Web-based interventions are pull technologies, requiring patients to take the initiative to use them.

    “One disadvantage of web-based interventions, however, is their status as “pull” technologies that rely on the patient’s initiative to access the intervention.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning

    These resources require professionals to conduct psychotherapy but overcome access barriers such as limited transportation, medical disabilities, or absence of available local services.

    “These resources require professionals to conduct the psychotherapy but overcome access barriers such as limited transportation, medical disabilities, or absence of available local services.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Web-based interventions have used number of logins, time on site, and number of modules completed as adherence metrics.

    “For example, webbased interventions have used number of logins, time on site, and number of modules completed to name a few.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Machine learning models for passive data can be automatically generated and personalized to each individual patient.

    “The advantage to this second approach is that the models are automatically generated and personalized to each individual patient.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [49]

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    Games for mental health that embody therapeutic principles have been developed but lack formal evaluations of their efficacy.

    “Games for mental health that embody therapeutic principles have been developed but lack formal evaluations of their efficacy.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Unmoderated support groups tend to show either no benefits relative to control arms or some worsening of symptoms.

    “However, findings of unmoderated support groups have tended to show either no benefits relative to control arms (32, 33) or some worsening of symptoms (34, 35).”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [32] source

      Not checked in the free run.

    • cites [33] source

      Not checked in the free run.

    • cites [34] source

      Not checked in the free run.

    • cites [35] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Email removes synchronous communication.

    “Videoconferencing eliminates immediate presence and some cues, the telephone eliminates visual cues, instant messaging eliminates voice quality, and email eliminates synchronous communication.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    Many developers add push features, such as automated email reminders, to prompt patients to revisit the site.

    “Thus, many developers include features that “push” information to the user, such as automated email reminders to encourage patients to return to the website.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Instant messaging removes voice quality.

    “Videoconferencing eliminates immediate presence and some cues, the telephone eliminates visual cues, instant messaging eliminates voice quality, and email eliminates synchronous communication.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisenone given

    A growing number of studies have examined videoconferencing used to extend specialized care to remote clinics, while newer studies evaluate videoconferencing with patients in their own homes.

    “A growing number of studies have examined videoconferencing used to extend specialized care to remote clinics, while newer studies are evaluating the use of videoconferencing with patients in their own homes.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Re-Mission increased adherence to prescribed oral chemotherapy medication and enhanced psychological resilience such as cancer self-efficacy.

    “This game increased adherence to prescribed oral chemotherapy medication and enhanced psychological resilience such as cancer self-efficacy.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [62]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Such treatments usually keep psychotherapy's standard structure, but the communication channel lowers the amount of information exchanged between therapist and patient.

    “These interventions generally maintain the standard structure of psychotherapy, but the delivery media reduce the “band-width” of information communicated between the therapist and patient (4, 5).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [4] source

      Not checked in the free run.

    • cites [5]

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    Investigations indicate that gaming may do more than just teach people what to do, and the interactivity aspect of playing games might enhance the uptake of interventions.

    “Investigations indicate that gaming may do more than just teach people what to do; the interactivity aspect of playing games might enhance the uptake of interventions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    BITs may help overcome many access barriers, but objective evidence for this remains largely absent.

    “BITs offer the potential to overcome many of those barriers to access, although the objective evidence for this is still largely unavailable (see Clarke et al., this issue).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Clarke et al., this issue

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    A recent shift is the creation of entertaining games with nonentertainment-related goals, encompassed by the serious gaming movement.

    “A recent shift is the creation of entertaining games with nonentertainment related goals (such as promoting mental health) encompassed by the serious gaming movement.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Greater use of modules may produce a dose-related response, or may simply reflect user characteristics associated with improvement such as conscientiousness.

    “Login rate may reflect self-titration of intervention use based on the patients recognition of their own need (68) while greater use of modules may produce a dose-related response, or may simply reflect user characteristics associated with improvement such as conscientiousness.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Researchers have acknowledged that BITs may create barriers.

    “However, researchers have also acknowledged that BITs may create barriers.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given

    Early BIT development often results in multiple features that may or may not contribute to efficacy.

    “Early BIT development often results in multiple features that may or may not contribute to efficacy.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    Harnessing big data to allow BITs systems to continuously learn from users has the potential to substantially improve the effectiveness of these interventions.

    “Harnessing big data to allow BITs systems to continuously learn from users has the potential to substantially improve the effectiveness of these interventions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    To enhance the design of BITs, theoretical models are necessary that can explain how users interact with the technologies and how such interactions can lead to behavior change and improved clinical outcomes.

    “To enhance the design of BITs, theoretical models are necessary that can explain how users interact with the technologies, and how such interactions can lead to behavior change and improved clinical outcomes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Behavioral and psychological theory may continue to be relevant at the larger level of how behavioral, affective, environmental, and cognitive factors interact to effect improvements in mental health.

    “Behavioral and psychological theory may continue to be relevant at the larger level of how behavioral, affective, environmental, and cognitive factors interact to effect improvements in mental health.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    BIT research requires new models for development, optimization, and evaluation.

    “BIT research requires new models for development, optimization, and evaluation.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    BITs provide promising avenues for creating new streams to deliver resources for the prevention of mental health problems.

    “Beyond integration into existing delivery systems, BITs provide promising avenues for creating new streams to deliver resources for the prevention of mental health problems.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    Some patients may stop using a BIT because they lose interest, others because they have received the assistance they sought, because the BIT does not meet their needs, or for other reasons.

    “Some patients may stop using a BIT because they lose interest, but others may stop using because they have received the assistance they sought, because the BIT does not meet their needs, or a variety of other reasons.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    These datasets create difficulties in data capture, storage, cleaning, analytics, visualization and sharing.

    “However, these datasets, sometimes referred to as “big data,” create difficulties in data capture, storage, cleaning, analytics, visualization and sharing.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationnone given

    Developing early BIT prototypes requires a clear understanding of how patients interact with BITs to guide decisions about structure and interface.

    “Developing early BIT prototypes requires a clear understanding of how patients interact with BITs, that can guide the numerous decisions regarding the structure and interface of the technology.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    As a new and evolving field, BITs research is encountering challenges that no single field or specialty can address alone.

    “As a new and evolving field, BITs research is encountering challenges that no single field or specialty can address alone.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The continuously evolving nature of technologies and the BITs that use those technologies will require strategies that can continuously evaluate efficacy as new versions and changes are introduced.

    “Furthermore, the continuously evolving nature of technologies and the BITs that use those technologies will require strategies that can continuously evaluate efficacy as new versions and changes are introduced.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    BITs can improve both screening and early delivery of interventions to reduce risk factors and build psychological resources.

    “BITs can improve both the screening and early delivery of interventions to reduce risk factors and build psychological resources.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given

    The literature on practice adoption of behavioral treatments in general is limited, and this is particularly true for BITs.

    “The literature on practice adoption of behavioral treatments in general is limited, and this is particularly true for BITs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationnone given

    These intervention areas are currently in widely varying stages of evaluation and maturity, requiring research that ranges from basic development and evaluation to implementation studies.

    “These intervention areas are currently in widely varying stages of evaluation and maturity, requiring research that ranges from basic development and evaluation to implementation studies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Integrating knowledge from control systems engineering into behavioral science could potentially help explain how people use and can benefit from BITs.

    “Integrating knowledge from this field into behavioral science could potentially help explain how people use and can benefit from BITs (78).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [78] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning

    These strategies involve systematic problem analysis, design, and an iterative process of evaluation to ensure that the BIT is understandable, attractive, and usable for the targeted patient population.

    “These strategies involve systematic problem analysis, design, and an iterative process of evaluation to ensure that the BIT is understandable, attractive, and usable for the targeted patient population.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Because BIT research lies at the intersection of behavioral science, engineering, and clinical science, existing developmental models from engineering can be borrowed and adapted.

    “Because this emerging area of research lies at the intersection of behavioral science, engineering, and clinical science, existing developmental models from engineering can be borrowed and adapted.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    BITs research will require integration of theoretical and research paradigms from many different disciplines.

    “BITs research will require integration of theoretical and research paradigms from many different disciplines, including behavioral and psychological sciences, engineering, computer science, human-computer interaction, communications, and design to name a few.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    If BITs are integrated into electronic medical records and patient portals, then referrals, treatment monitoring, appropriate follow-up, and fuller integration into the patient's overall treatment are more likely to occur.

    “If BITs are integrated into electronic medical records and patient portals, then referrals, treatment monitoring, appropriate follow-up, and fuller integration into the patient’s overall treatment are more likely to occur”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Advances in computational approaches for measuring affect through video data can support the creation of wearable technologies to improve recognition of affective cues.

    “For example, advances in computational approaches for measuring affect through video data can support the creation of wearable technologies to improve recognition of affective cues.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    User centered design and usability testing are common methods used in product development.

    “For example, user centered design (79) and usability testing (80) are common methods used in product development.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [79]

      Not checked in the free run.

    • cites [80]

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    Such strategies have yielded benefits, but the field is encountering the limitations of these paradigms.

    “While such strategies have yielded benefits, the field is also encountering the limitations of these paradigms.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    Control systems engineering is one possible example that operationalizes inputs, decision rules, and outputs.

    “One possible example is control systems engineering, which operationalize inputs, decision rules, and outputs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    These data offer the opportunity to improve the quality of interventions and develop deeper tailoring and personalization.

    “These data offer the opportunity to improve the quality of interventions and develop deeper tailoring and personalization.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    BITs can only begin to reach their public health potential if they are integrated into existing healthcare delivery systems.

    “BITs can only begin to reach their public health potential if they are integrated into existing healthcare delivery systems.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation

    This could improve social interactions as quality social relationships are an important protective factor against mental disorders.

    “This could improve social interactions as quality social relationships are an important protective factor against mental disorders (76).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [76] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    A growing number of investigators have recognized that current theoretical models are inadequate to inform the development of BITs that can increasingly interact dynamically and adaptively with users.

    “However, a growing number of investigators have recognized that current theoretical models are inadequate to inform the development of BITs that can increasingly interact dynamically and adaptively with users (77).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [77] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    Much of the development of BITs to date has either attempted to adapt accepted intervention paradigms such as cognitive behavioral therapy through new media or adopted new technologies such as SMS based on common sense approaches.

    “Much of the development of BITs to date has either attempted to adapt accepted intervention paradigms such as cognitive behavioral therapy (CBT) through new media, such as web-based CBT, or has adopted new technologies such as SMS to engage patients based on common sense approaches.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    Usability and attractiveness are necessary for engagement but not necessarily sufficient for efficacy.

    “The properties of usability and attractiveness, while necessary for engagement, are not necessarily sufficient for efficacy.”
    Quote in the paperSubstantial claim
65 of 65

  • The paper relies on this without support.

    Companies have started creating and selling BITs for many mental health conditions.

    “Industry has begun developing and marketing BITs for a wide variety of mental health problems.”

  • The paper relies on this without support.

    These products often reach the market with little or no evidence about whether they work or are useful.

    “These are frequently brought to market with little or no evaluation of their efficacy or utility.”

  • The paper relies on this without support.

    Sensors in passive data collection gather information that can be used to infer a patient's state.

    “In passive data collection, sensors automatically collect data that can help to infer the patient state.”

  • The paper relies on this without support.

    The cost of immersive 3-D systems has been a barrier to widespread implementation.

    “However, the cost of immersive 3-D systems has been a barrier to widespread implementation.”

  • The paper relies on this without support.

    Many BITs, such as web-based interventions and some virtual reality interventions, have high dropout and low utilization rates.

    “Many BITs, such as web-based interventions and in some instances virtual reality interventions, have high dropout and low utilization rates.”

  • The paper relies on this without support.

    Greater adherence and engagement is often associated with better outcomes.

    “as there is evidence that greater adherence and engagement is often associated with better outcomes”

  • The paper relies on this without support.

    Many patients may be uncomfortable with, or unable to use or access, the technologies employed by BITs.

    “For example, many patients may be uncomfortable with, or unable to use or access, the technologies employed by the interventions.”

  • The paper relies on this without support.

    BITs may increase costs by delaying traditional treatments with greater evidence of efficacy or by increasing the numbers of patients receiving services.

    “It is also possible that BITs may increase costs by delaying the provision of traditional treatments with greater evidence of efficacy, or by increasing the numbers of patients receiving services.”

  • The paper relies on this without support.

    Traditional intervention evaluation models typically require years and can be costly.

    “Traditional intervention evaluation models typically require years and can be costly.”
9 of 9

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

7 of the 93 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. Burns MN, Mohr DC. eHealth, Telehealth, and Telemedicine, in Encyclopedia of Behavioral Medicine. In: Gellman MD, Turner JR, Editors. New York: In Press, Springer.
  2. Miluzzo, E., et al. Research in the App Store Era: Experiences from the CenceMe App Deployment on the iPhone. in Proceedings of The First International Workshop Research in the Large: Using App Stores, Markets, and other Wide Distribution Channels in UbiComp Research. 2010. Copenhagen, Denmark.: ACM

Cited by

Sorted by citations

Related

Authors
Mohr, D. C., Burns, M. N., Schueller, S. M., Clarke, G., & Klinkman, M.
Affiliations
  • Northwestern University (US)
  • Kaiser Permanente Center for Health Research (US)
  • University of Michigan (US)
Corresponding author
Mohr, David C.
ORCID
Venue
General Hospital Psychiatry
Publisher
Elsevier BV
Published
2013-07
Volume
35
Issue
4
Pages
332-338
ISSN
0163-8343 (ISSN-L)
DOI
10.1016/j.genhosppsych.2013.03.008
PMC
PMC3719158
OpenAlex
W2000217803
Semantic Scholar
210c8e2b63163486e8adcb2e6aa43fe1fa3894d8
Cited by
693
References
93
Citations by year
2026: 35 · 2025: 60 · 2024: 48 · 2023: 56 · 2022: 48 · 2021: 64 · 2020: 65 · 2019: 60 · 2018: 49 · 2017: 68 · 2016: 65 · 2015: 37 · 2014: 31 · 2013: 5
FWCI
5.62
Type
journal article
Language
en
Open access
Licence
Locations
  • General Hospital Psychiatry (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • National Institutes of Health (UL1TR000150)
  • National Institute of Mental Health (P20MH090318, R01MH059708)
  • National Center for Advancing Translational Sciences
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
Keywords
mental health, telemedicine, videoconferencing, online intervention, online support groups, social media, virtual reality, serious games, anxiety, childhood diseases, research priorities, implementation, mental health prevention, big data
MeSH terms
Mental Disorders, Health Services Accessibility, Humans, National Institute of Mental Health (U.S.), Patient Compliance, Psychiatry, Psychotherapy, Self-Help Groups, United States, United States Agency for Healthcare Research and Quality, Treatment Outcome, Telemedicine, Social Networking, Social Media, Virtual Reality Exposure Therapy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read under definition version 3 (extract-v3) on 2026-09-28.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review