Banbury Forum Consensus Statement on the Path Forward for Digital Mental Health Treatment
Open accessThe DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index; 35 of the 85 sources its claims cite were checked for support.
Abstract
A major obstacle to mental health treatment for many Americans is accessibility: the United States faces a shortage of mental health providers, resulting in federally designated shortage areas. Although digital mental health treatments (DMHTs) are effective interventions for common mental disorders, they have not been widely adopted by the U.S. health care system. National and international expert stakeholders representing health care organizations, insurance companies and payers, employers, patients, researchers, policy makers, health economists, and DMHT companies and the investment community attended two Banbury Forum meetings. The Banbury Forum reviewed the evidence for DMHTs, identified the challenges to successful and sustainable implementation, investigated the factors that contributed to more successful implementation internationally, and developed the following recommendations: guided DMHTs should be offered to all patients experiencing common mental disorders, DMHT products and services should be reimbursable to support integration into the U.S. health care landscape, and an evidence standards framework should be developed to support decision makers in evaluating DMHTs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
22 Valid · 1 Not valid · 34 Not checked · 5 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Although digital mental health treatments (DMHTs) are effective interventions for common mental disorders, they have not been widely adopted by the U.S. health care system.”
§ Banbury Forum Consensus Statement on the Path Forward for Digital Mental Health Treatment > Abstract
What the sources say
- Supportedcites [13] Andrews G, Basu A, Cuijpers P,: Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord 2018; 55: 70– 78 29422409 source
The Results (section 3.2; page unavailable) report that internet-delivered cognitive behavioural therapy outperformed controls across major depression and three anxiety disorders. This supports the cited claim that digital treatments can be effective for common mental disorders; the paper's evidence is specifically for computer-delivered therapy for anxiety and depression. (LLM, conf=0.90)
- Not checked yetcites [14] Firth J, Torous J, Nicholas J,: The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017; 16: 287– 298 28941113 PMC5608852 source
- Partly supportedcites [15] Firth J, Torous J, Nicholas J,: Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord 2017; 218: 15– 22 28456072 sourceExperimental — accuracy not yet measured
Section 3.3 reports that smartphone interventions reduced anxiety symptoms more than control conditions. That supports effectiveness for anxiety, but the paper’s analysis does not establish that digital mental health treatments generally are effective across common mental disorders. The U.S. adoption clause is not part of the claim attributed to this source. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“DMHTs for depression and anxiety are effective for mild, moderate, and severe symptoms (25, 26); moreover, they can be effective across the life span, with a growing number of RCTs indicating effectiveness among children and adolescents (27) as well as older adults (28, 29).”
§ Banbury Forum Consensus Statement on the Path Forward for Digital Mental Health Treatment > RESULTS > Opportunities
What the sources say
- Supportedcites [27] Ebert DD, Zarski AC, Christensen H,: Internet and computer-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of randomized controlled outcome trials. PLoS One 2015; 10: e0119895 25786025 PMC4364968 source
The Abstract (p. 1) and Discussion report that this meta-analysis pooled 13 randomized controlled trials (796 children and adolescents) and found cCBT superior to controls on anxiety/depression symptoms in youth (g = 0.72, 95% CI 0.55–0.90, NNT = 2.56), with significant effects for anxiety-targeting (g = 0.68), depression-targeting (g = 0.76) and transdiagnostic (g = 0.94) interventions. The Discussion states plainly that 'Findings from the present study provide evidence for the efficacy of cCBT in the treatment of anxiety and depressive symptoms in youth.' That covers exactly the part of the citing sentence this citation is offered for — effectiveness among children and adolescents; the 'across the life span' framing and the older-adult evidence are carried by other citations (28, 29) and are not attributed here. (LLM, conf=0.92)
- Not checked yetcites [28] Titov N, Dear BF, Ali S,: Clinical and cost-effectiveness of therapist-guided Internet-delivered cognitive behavior therapy for older adults with symptoms of depression: a randomized controlled trial. Behav Ther 2015; 46: 193– 205 25645168 source
- Not checked yetcites [29] Silfvernagel K, Westlinder A, Andersson S,: Individually tailored Internet-based cognitive behaviour therapy for older adults with anxiety and depression: a randomised controlled trial. Cogn Behav Ther 2018; 47: 286– 300 29068266 source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Emerging evidence supports the cost-effectiveness of guided DMHTs (20, 41).”
§ Banbury Forum Consensus Statement on the Path Forward for Digital Mental Health Treatment > RESULTS > Opportunities
What the sources say
- Supportedcites [20] Richards D, Enrique A, Eilert N,: A pragmatic randomized waitlist-controlled effectiveness and cost-effectiveness trial of digital interventions for depression and anxiety. NPJ Digit Med 2020; 3: 85 32566763 PMC7295750 source
Richards et al. (2020) is itself a cost-effectiveness trial of supported (guided) iCBT: the Method > Procedures section describes participants being supported by trained wellbeing practitioners who delivered six scheduled online reviews over the 8-week programme, and the Results > Cost-effectiveness section reports ICERs below NICE thresholds with probabilities of cost-effectiveness rising to 91.2-92.0% at £30,000/QALY over 12 months. The Discussion (p. 1 of that section) states the QALY-based health improvements were 'considered cost-effective by criteria typically used by NICE (albeit against a waiting-list control) when assessed beyond the 8-week treatment period'. This is an exemplifying identification citation for the claim: the caveat that cost-effectiveness was clearest beyond the 8-week horizon (base-case 8-week probabilities were 46.6-65.5%) is a specificity difference rather than a contradiction, and the citing sentence is already hedged ('emerging evidence supports'). (LLM, conf=0.85)
- Not checked yetcites [41] Paganini S, Teigelkötter W, Buntrock C,: Economic evaluations of Internet- and mobile-based interventions for the treatment and prevention of depression: a systematic review. J Affect Disord 2018; 225: 733– 755 28922737 source
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Results from meta-analyses of at least 66 well-designed RCTs have indicated that guided DMHT has treatment efficiency for common mental health conditions such as depression and anxiety disorders that is comparable to the efficiencies of standard face-to-face therapies (13, 22, 23).”
§ Banbury Forum Consensus Statement on the Path Forward for Digital Mental Health Treatment > RESULTS > Opportunities
What the sources say
- Supportedcites [13] Andrews G, Basu A, Cuijpers P,: Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. J Anxiety Disord 2018; 55: 70– 78 29422409 source
Section 3.3.5 (Face to face CBT vs iCBT) reports nine head-to-head trials (568 subjects, MDD/PD/SP) with a non-significant difference of g = 0.14 favouring face-to-face CBT (95% CI -0.04 to 0.32), and the Abstract concludes 'All three modes of treatment delivery appeared equally beneficial' for major depression, panic, social anxiety and generalised anxiety disorder. The Introduction likewise states iCBT 'has been shown to be equally effective as face-to-face CBT', so the paraphrase's comparability claim for depression and anxiety is in our copy; the pooled head-to-head trials were therapist-supported/guided programmes, and the paper also credits Andersson et al. (2014) and the Olthius et al. (2015) Cochrane review for equal efficacy of guided iCBT versus face-to-face. Caveats not attributable to this source alone: the paper analysed 64 efficacy trials drawn from 53 RCTs (risk of bias low in 50, unclear in 14) rather than '66 well-designed RCTs', a figure pooled across the three citations in the sentence. (LLM, conf=0.85)
- Partly supportedcites [22] Linardon J, Cuijpers P, Carlbring P,: The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry 2019; 18: 325– 336 31496095 PMC6732686 sourceExperimental — accuracy not yet measured
Linardon et al. (2019) is indeed a meta-analysis of 66 RCTs, and its Abstract and 'Smartphone interventions vs. active comparisons' section support the core point: smartphone interventions 'did not differ significantly from active interventions (face-to-face, computerized treatment)' (pooled g=0.13, 95% CI -0.07 to 0.34 across 12 comparisons for depressive symptoms; g=0.09, 95% CI -0.21 to 0.39 for generalized anxiety). However, the claim's qualifiers are not the source's: the head-to-head analyses were not restricted to *guided* interventions (guidance was only a moderator, and guided vs unguided comparisons were against control conditions, not face-to-face therapy), the interventions are app-supported smartphone apps rather than DMHT broadly, and the 66 RCTs are not characterised as 'well-designed' (only 17 trials met all four risk-of-bias criteria; low-risk-of-bias active-comparison effects were g=-0.00). The source also hedges explicitly — 'although the number of studies was low (n≤13)', 'these findings, although preliminary', 'these analyses may have been underpowered', and apps 'are not intended to replace professional clinical services' (Discussion) — whereas the citing sentence states equivalence to face-to-face therapy as an established result. (LLM, conf=0.80)
- Partly supportedcites [23] Andersson G, Cuijpers P, Carlbring P,: Guided Internet-based vs face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry 2014; 13: 288– 295 25273302 PMC4219070 sourceExperimental — accuracy not yet measured
Our copy does support the core comparative claim: the RESULTS report a pooled between-group Hedges' g of −0.01 (95% CI −0.13 to 0.12) across 13 direct-comparison RCTs (N=1053), with equivalent effects for depressive symptoms (g=0.05), social anxiety (g=−0.16) and panic disorder (g=0.05), and the DISCUSSION concludes the formats are 'equally effective'. However, the 'at least 66 well-designed RCTs' figure is not this paper's — it synthesized 13 studies, and no passage gives a 66-trial count; the source is also narrower than 'guided DMHT' (guided Internet-delivered CBT only) and more hedged than the citing sentence ('more research is needed to establish equivalence'; 'we cannot at this stage ... conclude that guided ICBT and face-to-face therapy are equally effective on all outcomes'). Its own limitations note that in 7 of 13 studies the face-to-face comparator was group-format CBT, which it calls 'a suboptimal comparison', so 'standard face-to-face therapies' overstates it. (LLM, conf=0.78)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Several RCTs of guided digital treatments for alcohol and substance abuse have shown significant but more modest benefits of DMHTs (31, 32).”
§ Banbury Forum Consensus Statement on the Path Forward for Digital Mental Health Treatment > RESULTS > Opportunities
What the sources say
- Partly supportedcites [31] Riper H, Blankers M, Hadiwijaya H,: Effectiveness of guided and unguided low-intensity Internet interventions for adult alcohol misuse: a meta-analysis. PLoS One 2014; 9: e99912 24937483 PMC4061051 sourceExperimental — accuracy not yet measured
Riper et al. (2014) is a meta-analysis of 16 RCTs (23 comparisons) of low-intensity Internet interventions for adult ALCOHOL misuse, and it does support the 'significant but modest' part: the Abstract/Findings report a small but significant overall effect (g = 0.20, 95% CI 0.13–0.27, p<.001), and Table 2 / Discussion (Subgroup Analyses) give guided interventions g = 0.23 (95% CI 0.05–0.41, p<.01) across 5 comparisons, with the effect described as lower than that of face-to-face brief interventions. What our copy does not support is the extension to 'substance abuse': the paper's scope, inclusion criteria and outcomes are alcohol only (Introduction; Table 1 study characteristics), and drug/substance-use misuse appears only in background burden-of-disease references, not as an analysed outcome. Also, the source is itself the pooled analysis rather than a report of 'several RCTs', and it explicitly found no significant guided-vs-unguided difference (g = 0.23 vs 0.20). (LLM, conf=0.78)
- Partly supportedcites [32] Boumparis N, Karyotaki E, Schaub MP,: Internet interventions for adult illicit substance users: a meta-analysis. Addiction 2017; 112: 1521– 1532 28295758 PMC5573910 sourceExperimental — accuracy not yet measured
The Abstract Conclusions and the overall results (Results, 'The overall effect of internet interventions on illicit substance users': g = 0.30 at post-treatment, g = 0.22 at follow-up) support the 'significant but modest' part of the claim, and the add-on/guided subgroup (Table 3, g = 0.41 vs unguided standalone g = 0.17) is consistent with guided treatments yielding small effects. However, the population is not as the citing sentence states: the Methods ('Eligibility criteria') restrict inclusion to adult current users of illicit substances, with alcohol entering only because transdiagnostic trials screen for common substances 'including opioids, cocaine, stimulants, cannabis and alcohol' — the meta-analysis reports no separate alcohol outcome. Nor is the source limited to guided treatments: it pools both add-on (guided) and unguided standalone internet/computer-delivered interventions across 17 RCTs/18 comparisons, so 'several RCTs of guided digital treatments for alcohol and substance abuse' overstates its scope. (LLM, conf=0.78)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
38 more claims · 18 with their cited sources checked
Create a free account to see all 62 claims and to verify the 39 references still open.
- It may state two separate claims as one. 14
- Probably not a substantial claim. 7
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“At the same time, the relatively novel field of DTx is still in search of common standards for how to generate evidence necessary to support medical claims and regulatory clearance (12).”
From Appropriate controls for digital therapeutic clinical trials: A narrative review of control conditions in clinical trials of digital therapeutics (DTx) deploying psychosocial, cognitive, or behavioral content (2022) · cites this paper as [12]
Experimental — accuracy not yet measuredIn the citing paper
“In addition, relatively more novel interventions may require more stringent controls compared to digital translations of known psychosocial behavioral interventions, such as CBT (12).”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Banbury Forum Consensus Statement on the Path Forward for Digital Mental Health Treatment > RESULTS > Challenges
“Recently, the new CPT codes 98970–98972 were opened for online digital evaluation and management services (50).”
§ Banbury Forum Consensus Statement on the Path Forward for Digital Mental Health Treatment > References
“50. Nicoletti B: CPT Codes (99421–99423)—and Payment for—Online Digital Evaluation and Management (E/M) Services.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Banbury Forum Consensus Statement on the Path Forward for Digital Mental Health Treatment > RESULTS > Challenges
“because there is a strong evidence base for guided iCBT (37)”
§ Banbury Forum Consensus Statement on the Path Forward for Digital Mental Health Treatment > References
“37. Karyotaki E, Riper H, Twisk J,: Efficacy of self-guided Internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data.”
Citations
Cites
Nielsen, J. (1994) · Conference companion on Human factors in computing systems - CHI '94 · doi:10.1145/259963.260531 · Cited by 2,759
Nielsen, J. (1995) · Conference companion on Human factors in computing systems - CHI '95 · doi:10.1145/223355.223730 · Cited by 2,620 · cited 1×
“Because provider engagement from RCTs may be difficult to generalize, as providers often work closely with research staff in trials, an evaluation of the user experience, including how useful it is, ease of use, efficiency, and satisfaction, may be conducted with representative providers (63).”
Risk factors for suicide in individuals with depression: a systematic review
Hawton, K., Casañas i Comabella, C., Haw, C., & Saunders, K. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.01.004 · Cited by 1,469 · cited 1×
“Mental health conditions also can increase risks, most notably for suicide (69).”
Computer-based psychological treatments for depression: A systematic review and meta-analysis
Richards, D., & Richardson, T. (2012) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.02.004 · Cited by 1,442
Self-citationRebooting psychotherapy research and practice to reduce the burden of mental illness
Kazdin, A. E., & Blase, S. L. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393527 · Cited by 1,254 · cited 1×
“The United States has a shortage of mental health care specialists, with nearly 120 million Americans living in federally designated areas experiencing shortages of mental health providers (6, 7).”
Undertreatment of people with major depressive disorder in 21 countries
Thornicroft, G., Chatterji, S., Evans-Lacko, S., Gruber, M., Sampson, N., Aguilar-Gaxiola, S., Al-Hamzawi, A., Alonso, J., Andrade, L., Borges, G., et al. (2017) · British Journal of Psychiatry · doi:10.1192/bjp.bp.116.188078 · Cited by 1,121
Mohr, D. C., Cuijpers, P., & Lehman, K. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1602 · Cited by 1,097 · cited 1×
“It is also useful for providers to be able to see usage of the technology so that patient engagement is supported (35, 65).”
Self-citationFirth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017) · World Psychiatry · doi:10.1002/wps.20472 · Cited by 1,097
Self-citationAndersson, G., Cuijpers, P., Carlbring, P., Riper, H., & Hedman, E. (2014) · World Psychiatry · doi:10.1002/wps.20151 · Cited by 1,077 · cited 1×
“Results from meta-analyses of at least 66 welldesigned RCTs have indicated that guided DMHT has treatment efficiency for common mental health conditions such as depression and anxiety disorders that is comparable to the efficiencies of standard face-to-face therapies (13, 22, 23).”
Andrews, G., Basu, A., Cuijpers, P., Craske, M. G., McEvoy, P., English, C. L., & Newby, J. M. (2018) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.01.001 · Cited by 922 · cited 1×
“Results from meta-analyses of at least 66 welldesigned RCTs have indicated that guided DMHT has treatment efficiency for common mental health conditions such as depression and anxiety disorders that is comparable to the efficiencies of standard face-to-face therapies (13, 22, 23).”
Linardon, J., Cuijpers, P., Carlbring, P., Messer, M., & Fuller‐Tyszkiewicz, M. (2019) · World Psychiatry · doi:10.1002/wps.20673 · Cited by 860
Alonso, J., Liu, Z., Evans-Lacko, S., Sadikova, E., Sampson, N., Chatterji, S., Abdulmalik, J., Aguilar-Gaxiola, S., Al-Hamzawi, A., Andrade, L. H., et al. (2018) · Depression and Anxiety · doi:10.1002/da.22711 · Cited by 716
Zachariae, R., Lyby, M. S., Ritterband, L. M., & O'Toole, M. S. (2016) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2015.10.004 · Cited by 704
Karyotaki, E., Riper, H., Twisk, J., Hoogendoorn, A., Kleiboer, A., Mira, A., Mackinnon, A., Meyer, B., Botella, C., Littlewood, E., et al. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.0044 · Cited by 691 · cited 2×
“People can benefit from fully automated, unguided DMHTs (36); however, at a population level, they show small benefits, likely because dropout rates are higher in unguided than guided interventions (37, 38).”
Firth, J., Torous, J., Nicholas, J., Carney, R., Rosenbaum, S., & Sarris, J. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.04.046 · Cited by 675
Self-citationEbert, D. D., Zarski, A.-C., Christensen, H., Stikkelbroek, Y., Cuijpers, P., Berking, M., & Riper, H. (2015) · PLOS ONE · doi:10.1371/journal.pone.0119895 · Cited by 533 · cited 1×
“DMHTs for depression and anxiety are effective for mild, moderate, and severe symptoms (25, 26); moreover, they can be effective across the life span, with a growing number of RCTs indicating effectiveness among children and adolescents (27) as well as older adults (28, 29).”
Why digital medicine depends on interoperability
Lehne, M., Sass, J., Essenwanger, A., Schepers, J., & Thun, S. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0158-1 · Cited by 501 · cited 1×
“Thus, DMHTs need requirements for data interoperability that support their intended aims and align with the larger movement in health care (66).”
Standalone smartphone apps for mental health—a systematic review and meta-analysis
Weisel, K. K., Fuhrmann, L. M., Berking, M., Baumeister, H., Cuijpers, P., & Ebert, D. D. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0188-8 · Cited by 447 · cited 1×
“People can benefit from fully automated, unguided DMHTs (36); however, at a population level, they show small benefits, likely because dropout rates are higher in unguided than guided interventions (37, 38).”
Mobile Apps for Bipolar Disorder: A Systematic Review of Features and Content Quality
Nicholas, J., Larsen, M. E., Proudfoot, J., & Christensen, H. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4581 · Cited by 438 · cited 1×
“Although there is no evidence from trials that DMHTs themselves are harmful or pose a risk (67), some DMHTs have been be found to provide advice that is potentially harmful (68).”
Gilbody, S., Littlewood, E., Hewitt, C., Brierley, G., Tharmanathan, P., Araya, R., Barkham, M., Bower, P., Cooper, C., Gask, L., et al. (2015) · BMJ · doi:10.1136/bmj.h5627 · Cited by 409
Self-citationTitov, N., Andrews, G., Davies, M., McIntyre, K., Robinson, E., & Solley, K. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010939 · Cited by 406 · cited 1×
“Moreover, outcomes for DMHTs supported by non–mental health specialists do not differ from outcomes for DMHTs supported by mental health specialists (43), which allows for task-shifting options to lower levels of licensure, education requirements, and cost.”
Self-citationPerceived barriers to psychological treatments and their relationship to depression
Mohr, D. C., Ho, J., Duffecy, J., Baron, K. G., Lehman, K. A., Jin, L., & Reifler, D. (2010) · Journal of Clinical Psychology · doi:10.1002/jclp.20659 · Cited by 347
Self-citationBower, P., Kontopantelis, E., Sutton, A., Kendrick, T., Richards, D. A., Gilbody, S., Knowles, S., Cuijpers, P., Andersson, G., Christensen, H., et al. (2013) · BMJ · doi:10.1136/bmj.f540 · Cited by 341 · cited 1×
“DMHTs for depression and anxiety are effective for mild, moderate, and severe symptoms (25, 26); moreover, they can be effective across the life span, with a growing number of RCTs indicating effectiveness among children and adolescents (27) as well as older adults (28, 29).”
Self-citationMental distress among U.S. adults during the COVID-19 pandemic
Twenge, J. M., & Joiner, T. E. (2020) · Journal of Clinical Psychology · doi:10.1002/jclp.23064 · Cited by 330
Mental distress among U.S. adults during the COVID-19 pandemic
Twenge, J., & Joiner, T. (2020) · doi:10.31234/osf.io/wc8ud · Cited by 330 · cited 1×
“The recent COVID-19 pandemic, which has increased the incidence of mental health problems, has further highlighted the challenges of mental health care access (8).”
A Systematic Review of Cognitive Behavioral Therapy and Behavioral Activation Apps for Depression
Huguet, A., Rao, S., McGrath, P. J., Wozney, L., Wheaton, M., Conrod, J., & Rozario, S. (2016) · PLOS ONE · doi:10.1371/journal.pone.0154248 · Cited by 328 · cited 1×
“For iCBT, very few of the products that claim to be based on cognitive-behavioral therapy (CBT) actually contain the CBT core elements (56).”
Beyond validation: getting health apps into clinical practice
Gordon, W. J., Landman, A., Zhang, H., & Bates, D. W. (2020) · npj Digital Medicine · doi:10.1038/s41746-019-0212-z · Cited by 326 · cited 1×
“An evidence standards framework is needed to support “digital formularies,” which allow provider organizations and payers to identify preferred products (51).”
Wright, J. H., Wright, A. S., Albano, A. M., Basco, M. R., Goldsmith, L. J., Raffield, T., & Otto, M. W. (2005) · American Journal of Psychiatry · doi:10.1176/appi.ajp.162.6.1158 · Cited by 319 · cited 1×
“Although these studies have found that patients improve in treatments that blend DMHTwith face-toface psychotherapy, it is unclear whether the addition of DMHT tools results in any greater improvement or reduction in the amount of time required in psychotherapy (39, 40).”
Addressing Equity in Telemedicine for Chronic Disease Management During the Covid-19 Pandemic
NouriSarah, KhoongElaine, C., LylesCourtney, R., & KarlinerLeah (2020) · NEJM Catalyst · Cited by 316
Internet interventions: Past, present and future
Andersson, G. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.03.008 · Cited by 315
Huckvale, K., Torous, J., & Larsen, M. E. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.2542 · Cited by 312 · cited 1×
“Although some companies have not been transparent about the use, sharing, or sale of data (70), vendors also may be interested in using the data for a variety of reasonable purposes, including continuous improvement of the product.”
Self-citationBarriers to psychotherapy among depressed and nondepressed primary care patients
Mohr, D. C., Hart, S. L., Howard, I., Julian, L., Vella, L., Catledge, C., & Feldman, M. D. (2006) · Annals of Behavioral Medicine · doi:10.1207/s15324796abm3203_12 · Cited by 276
Self-citationSmartphone Apps for Schizophrenia: A Systematic Review
Firth, J., & Torous, J. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.4930 · Cited by 268 · cited 1×
“Research on DMHTs for severe mental illness, such as bipolar and psychotic disorders, has shown feasibility, although efficacy data remain more limited (33, 34).”
Self-citationRiper, H., Blankers, M., Hadiwijaya, H., Cunningham, J., Clarke, S., Wiers, R., Ebert, D., & Cuijpers, P. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099912 · Cited by 265
Titov, N., Dear, B. F., Ali, S., Zou, J. B., Lorian, C. N., Johnston, L., Terides, M. D., Kayrouz, R., Klein, B., Gandy, M., et al. (2015) · Behavior Therapy · doi:10.1016/j.beth.2014.09.008 · Cited by 210 · cited 1×
“DMHTs for depression and anxiety are effective for mild, moderate, and severe symptoms (25, 26); moreover, they can be effective across the life span, with a growing number of RCTs indicating effectiveness among children and adolescents (27) as well as older adults (28, 29).”
Self-citationTorous, J. B., Chan, S. R., Gipson, S. Y.-M. T., Kim, J. W., Nguyen, T.-Q., Luo, J., & Wang, P. (2018) · Psychiatric Services · doi:10.1176/appi.ps.201700423 · Cited by 184 · cited 1×
“Several exemplar frameworks exist that cover core principles for DMHT standards (52, 54, 55), which we elaborate on next.”
Self-citationTitov, N., Dear, B. F., Staples, L. G., Bennett-Levy, J., Klein, B., Rapee, R. M., Andersson, G., Purtell, C., Bezuidenhout, G., & Nielssen, O. B. (2017) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867416671598 · Cited by 183
Self-citationMeasurement-Based Care in Psychiatric Practice: A Policy Framework for Implementation
Harding, K. J. K., Rush, A. J., Arbuckle, M., Trivedi, M. H., & Pincus, H. A. (2011) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.10r06282whi · Cited by 175 · cited 1×
“This component is critical for measurement-based care, allowing the provider to monitor improvement and to intervene and triage to a higher level of care (64).”
By the numbers: ratings and utilization of behavioral health mobile applications
Carlo, A. D., Hosseini Ghomi, R., Renn, B. N., & Areán, P. A. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0129-6 · Cited by 164 · cited 1×
“Several exemplar frameworks exist that cover core principles for DMHT standards (52, 54, 55), which we elaborate on next.”
Self-citationGraham, A. K., Greene, C. J., Kwasny, M. J., Kaiser, S. M., Lieponis, P., Powell, T., & Mohr, D. C. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.1011 · Cited by 162 · cited 1×
“patient engagement in health care settings (59), others have had high dropout rates (60, 61).”
Self-citationMehrotra, A., Huskamp, H. A., Souza, J., Uscher-Pines, L., Rose, S., Landon, B. E., Jena, A. B., & Busch, A. B. (2017) · Health Affairs · doi:10.1377/hlthaff.2016.1461 · Cited by 158 · cited 1×
“Tele–mental health has been expanding for more than a decade to overcome regional disparities (9), and it has now been expanded more broadly under COVID-19 (10), with calls to make this expansion permanent (11).”
Ebert, D. D., Donkin, L., Andersson, G., Andrews, G., Berger, T., Carlbring, P., Rozenthal, A., Choi, I., Laferton, J. A. C., Johansson, R., et al. (2016) · Psychological Medicine · doi:10.1017/s0033291716001562 · Cited by 154 · cited 1×
“Although there is no evidence from trials that DMHTs themselves are harmful or pose a risk (67), some DMHTs have been be found to provide advice that is potentially harmful (68).”
Self-citationRichards, D., Enrique, A., Eilert, N., Franklin, M., Palacios, J., Duffy, D., Earley, C., Chapman, J., Jell, G., Sollesse, S., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0293-8 · Cited by 154 · cited 1×
“Emerging evidence supports the costeffectiveness of guidedDMHTs (20, 41).”
Self-citationHermes, E. D., Lyon, A. R., Schueller, S. M., & Glass, J. E. (2019) · Journal of Medical Internet Research · doi:10.2196/11752 · Cited by 149 · cited 1×
“, to support psychotherapy), to “guided” (key aspects of care are delivered by the technology and supported by a clinician, coach, or peer, who provides low-intensity support), to “fully automated” (used without human support, such as apps available from app stores) (21).”
Self-citationPaganini, S., Teigelkötter, W., Buntrock, C., & Baumeister, H. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.07.018 · Cited by 144 · cited 2×
“Although some guided DMHTs met cost-effectiveness criteria, automated DMHTs typically have not, in part because their effectiveness is lower (41).”
Gilbody, S., Brabyn, S., Lovell, K., Kessler, D., Devlin, T., Smith, L., Araya, R., Barkham, M., Bower, P., Cooper, C., et al. (2017) · British Journal of Psychiatry · doi:10.1192/bjp.bp.116.192435 · Cited by 116 · cited 1×
“patient engagement in health care settings (59), others have had high dropout rates (60, 61).”
Self-citationMohr, D. C., Schueller, S. M., Tomasino, K. N., Kaiser, S. M., Alam, N., Karr, C., Vergara, J. L., Gray, E. L., Kwasny, M. J., & Lattie, E. G. (2019) · Journal of Medical Internet Research · doi:10.2196/13609 · Cited by 116 · cited 1×
“People can benefit from fully automated, unguided DMHTs (36); however, at a population level, they show small benefits, likely because dropout rates are higher in unguided than guided interventions (37, 38).”
Self-citationKenter, R. M. F., van de Ven, P. M., Cuijpers, P., Koole, G., Niamat, S., Gerrits, R. S., Willems, M., & van Straten, A. (2015) · Internet Interventions · doi:10.1016/j.invent.2015.01.001 · Cited by 105 · cited 1×
“Although these studies have found that patients improve in treatments that blend DMHTwith face-toface psychotherapy, it is unclear whether the addition of DMHT tools results in any greater improvement or reduction in the amount of time required in psychotherapy (39, 40).”
Internet interventions for adult illicit substance users: a meta‐analysis
Boumparis, N., Karyotaki, E., Schaub, M. P., Cuijpers, P., & Riper, H. (2017) · Addiction · doi:10.1111/add.13819 · Cited by 104 · cited 1×
“Several RCTs of guided digital treatments for alcohol and substance abuse have shown significant but more modest benefits of DMHTs (31, 32).”
Nebeker, C., Bartlett Ellis, R. J., & Torous, J. (2020) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz074 · Cited by 99 · cited 1×
“An evidence standards framework should integrate core ethical principles of respect for persons, beneficence, and justice (53); be specific enough to protect stakeholders, including patients, families, providers, and payers; and be flexible enough to be applied to new forms of DMHTs that harness new knowledge, design,…”
Self-citationComputer-Assisted Cognitive-Behavior Therapy and Mobile Apps for Depression and Anxiety
Wright, J. H., Mishkind, M., Eells, T. D., & Chan, S. R. (2019) · Current Psychiatry Reports · doi:10.1007/s11920-019-1031-2 · Cited by 93 · cited 1×
“DMHTs for depression and anxiety are effective for mild, moderate, and severe symptoms (25, 26); moreover, they can be effective across the life span, with a growing number of RCTs indicating effectiveness among children and adolescents (27) as well as older adults (28, 29).”
Online and mobile technologies for self-management in bipolar disorder: A systematic review.
Gliddon, E., Barnes, S. J., Murray, G., & Michalak, E. E. (2017) · Psychiatric Rehabilitation Journal · doi:10.1037/prj0000270 · Cited by 84 · cited 1×
“Research on DMHTs for severe mental illness, such as bipolar and psychotic disorders, has shown feasibility, although efficacy data remain more limited (33, 34).”
Schueller, S. M., Tomasino, K. N., & Mohr, D. C. (2017) · Clinical Psychology: Science and Practice · doi:10.1037/h0101740 · Cited by 70 · cited 2×
“It is also useful for providers to be able to see usage of the technology so that patient engagement is supported (35, 65).”
Self-citationSilfvernagel, K., Westlinder, A., Andersson, S., Bergman, K., Diaz Hernandez, R., Fallhagen, L., Lundqvist, I., Masri, N., Viberg, L., Forsberg, M.-L., et al. (2018) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2017.1388276 · Cited by 65 · cited 1×
“DMHTs for depression and anxiety are effective for mild, moderate, and severe symptoms (25, 26); moreover, they can be effective across the life span, with a growing number of RCTs indicating effectiveness among children and adolescents (27) as well as older adults (28, 29).”
Distance-delivered interventions for PTSD: A systematic review and meta-analysis
Olthuis, J. V., Wozney, L., Asmundson, G. J. G., Cramm, H., Lingley-Pottie, P., & McGrath, P. J. (2016) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2016.09.010 · Cited by 64 · cited 1×
“A meta-analysis of eight RCTs has shown effectiveness of DMHTs for posttraumatic stress disorder (24).”
Mohr, D. C., Lattie, E. G., Tomasino, K. N., Kwasny, M. J., Kaiser, S. M., Gray, E. L., Alam, N., Jordan, N., & Schueller, S. M. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103485 · Cited by 57 · cited 2×
“Integration of DMHTs into care pathways would improve the efficiency of mental health services, for example, through stepped care models in which patients initiate treatment with lower-cost DMHTs, preserving mental health specialist time for those who do not show sufficient responses to DMHT (42).”
Self-citationDuffy, D., Enrique, A., Connell, S., Connolly, C., & Richards, D. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00902 · Cited by 49
Self-citationCarlo, A. D., Drake, L., Ratzliff, A., Chang, D., & Unützer, J. (2020) · Psychiatric Services · doi:10.1176/appi.ps.201900581 · Cited by 36 · cited 1×
“To use these codes, health care organizations will require substantial procedural and billing workflow adjustments, which can be difficult to implement (75).”
Reimbursement of Apps for Mental Health: Findings From Interviews
Powell, A. C., Bowman, M. B., & Harbin, H. T. (2019) · JMIR Mental Health · doi:10.2196/14724 · Cited by 35
Self-citationLee, Y. C., Gao, L., Dear, B. F., Titov, N., & Mihalopoulos, C. (2017) · PubMed · Cited by 26
Self-citationAbhulimen, S., & Hirsch, A. (2018) · Journal of Medical Economics · doi:10.1080/13696998.2018.1510834 · Cited by 6 · cited 1×
“Indeed, even though automated DMHTs are less effective than guided DMHTs, automated DMHTs have been shown to cost less overall (44).”
Research works printed but not linked: 2
- The Treatment and Management of Depression in Adults. London, National Institute for Health and Care Excellence, 2009 35977056
- Digital Medicine Payment Advisory Group Initiative. Chicago, American Medical Association, 2020. https://www.ama-assn.org/practice-management/digital/digital-medicine-payment-advisory-group. Accessed June 24, 2020
Cited by
An overview of and recommendations for more accessible digital mental health services
Lattie, E. G., Stiles-Shields, C., & Graham, A. K. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-021-00003-1 · Cited by 446
Problems and Barriers Related to the Use of Digital Health Applications: Scoping Review
Giebel, G. D., Speckemeier, C., Abels, C., Plescher, F., Börchers, K., Wasem, J., Blase, N., & Neusser, S. (2023) · Journal of Medical Internet Research · doi:10.2196/43808 · Cited by 229
Berardi, C., Antonini, M., Jordan, Z., Wechtler, H., Paolucci, F., & Hinwood, M. (2024) · BMC Health Services Research · doi:10.1186/s12913-023-10536-1 · Cited by 190
Leung, C., Pei, J., Hudec, K., Shams, F., Munthali, R., & Vigo, D. (2022) · Journal of Medical Internet Research · doi:10.2196/36004 · Cited by 94
Lutz, J., Offidani, E., Taraboanta, L., Lakhan, S. E., & Campellone, T. R. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.823977 · Cited by 89
“TAU was the main comparator arm in a 3-arm RCT of iCBT for Irritable Bowel Syndrome (IBS), the data from which was used to support the FDA clearance of Parallel (10).”
Influential
Metadata
- Authors
- Mohr, D. C., Azocar, F., Bertagnolli, A., Choudhury, T., Chrisp, P., Frank, R., Harbin, H., Histon, T., Kaysen, D., Nebeker, C., Richards, D., Schueller, S. M., Titov, N., Torous, J., Areán, P. A., & Health, O. B. O. T. B. F. O. D. M.
- Affiliations
- Northwestern University (US)
- ORCID
- Venue
- Psychiatric Services
- Published
- 2021-06
- Volume
- 72
- Issue
- 6
- Pages
- 677-683
- ISSN
- 0022-1597 (ISSN-L)
- 1075-2730
- 1557-9700
- PMC
- PMC8822332
- OpenAlex
- W3122329668
- Semantic Scholar
- 29dbdb44d2d4e524f2cbae994c9a14955eed8e53
- Cited by
- 115
- References
- 61
- Citations by year
- 2026: 11 · 2025: 23 · 2024: 25 · 2023: 21 · 2022: 26 · 2021: 8
- FWCI
- 17.96
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Psychiatric Services (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Arrow@dit (Dublin Institute of Technology) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Trinity's Access to Research Output (TARA) (Trinity College Dublin) (submittedVersion)
- Funders
- Microsoft
- National Institute of Mental Health (P50MH119029)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
- Keywords
- access to mental health care, mental health workforce shortage, implementation barriers, health care reimbursement, insurance coverage, common mental disorders, telemedicine, health policy recommendations
- MeSH terms
- Administrative Personnel, Delivery of Health Care, Humans, Mental Health, United States, Consensus
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.