Three Problems With Current Digital Mental Health Research . . . and Three Things We Can Do About Them
Open accessAbstract
An increasingly large body of randomized controlled trials has demonstrated the efficacy of mental health technologies, such as Web-based and mobile interventions, to prevent and treat mental disorders and increase psychological well-being. However, there is little evidence that these tools can be successfully implemented in clinical settings. The authors highlight three widely held misconceptions that they believe are holding back the field, and they reconceptualize the issues to strengthen the path toward implementation and accelerate innovation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 15 cited sources checked · 0 supported · 0 flagged
- Interpretationreasoning
Treating mental health technologies as products has led to developing tools without understanding how they fit into mental health services.
“Treating mental health technologies as products has meant that we have largely developed the tools without understanding how they can fit in the context of mental health services.”
Quote in the paperSubstantial claim - Premisecitation
Efficacy trials using randomization have repeatedly found that Web-based and mobile programs can treat various psychiatric disorders, especially with low-intensity support by brief calls or messaging.
“Randomized controlled efficacy trials have consistently demonstrated that Web-based and mobile interventions can treat a range of psychiatric disorders, particularly when coupled with low-intensity support provided via brief phone calls or messaging (1,2).”
Quote in the paperSubstantial claim0 of 2 sources checked - Premisecitation
Greater mental health benefits are more consistently obtained when human support is included.
“However, the relatively consistent finding has been that substantive mental health benefits are more consistently achieved in the context of human support (1), suggesting that mental health technologies are not products–rather, they are technology-enabled services (TESs).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [1] source
Not checked in the free run.
- Premisecitation
There is little evidence that mental health technologies can be successfully and sustainably implemented in typical clinical or health care settings.
“However, there is little evidence that these tools can be successfully implemented in clinical settings.”
- Also stated in: § Abstract · plain
“However, despite this overwhelming support, there is little evidence that technology-based mental health interventions can be sustainably and effectively implemented in typical health care settings (3).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [3] source
Not checked in the free run.
- Also stated in: § Abstract · plain
- Premisecitation
A number of today's digital mental health tools need support from a coach or therapist to keep users engaged and produce reliable, substantive outcomes.
“Although improved design and technology may make mental health technologies easier and more engaging to use in the future, many of today’s mental health technologies require some human support from a coach or therapist to sustain engagement and obtain substantive, reliable outcomes (1).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [1] source
Not checked in the free run.
- Interpretationreasoning
This expansion has the unintended consequence of searching until people willing to use the tools are found, thereby damaging the generalizability of the findings.
“This has had the unintended consequence of searching until we find people who are willing to use the tools, thereby damaging the generalizability of the findings.”
Quote in the paperSubstantial claim- Rests on an assumption: When clinical research faces recruitment difficulties, a common response is to broaden the pool of potential participants—adding venues, adjusting social media tactics, and similar steps.
- Interpretationcitation
Framing digital mental health tools as copies of established evidence-based therapies has turned into a skeuomorph, constraining what we imagine is possible by keeping an old conceptual frame.
“The notion that mental health technologies should mimic existing evidence-based treatments has become a skeuomorph, limiting our vision of what is possible by maintaining a frame based on past conceptualizations (11).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [11] source
Not checked in the free run.
- Interpretationreasoning
The general population does not find these products and services appealing.
“Thus, we should not be surprised that these products and services are not appealing to the general population.”
Quote in the paperSubstantial claim - Interpretationreasoning
Researchers have generally failed to obtain patient input on goals, needs, and preferences.
“We have typically not done a good job of getting input from patients about their goals, needs, or preferences.”
Quote in the paperSubstantial claim - Premisecitation
Emerging approaches like artificial intelligence, natural language processing, and virtual reality create entirely new ways to intervene.
“New technologies and methods, such as artificial intelligence (12), natural language processing (13), and virtual reality (14), open fundamentally new intervention paradigms.”
Quote in the paperSubstantial claim0 of 3 sources checked - Interpretationnone given
In the research literature, digital mental health tools are usually described and assessed as though they were products.
“The research literature typically describes and evaluates mental health technologies as if they were products.”
Quote in the paperSubstantial claim- Rests on an assumption: In the literature, the technology receives thorough description and evaluation, while its surrounding ecosystem—human support and organizational factors—receives little.
- Interpretationreasoning
Clinical researchers have designed tools to get people to do what the researchers want and then searched for and found people willing to use those tools in trials.
“Essentially, clinical researchers have designed tools to try to get people to do what we want them to do and how we want them to do it–and then searched for and found people who were interested in or willing to use these tools in our trials.”
Quote in the paperSubstantial claim - Premisecitation
In a recent large pragmatic trial, Gilbody and colleagues found no significant effect for two commonly used coached digital depression treatments.
“Gilbody and colleagues (3) found no significant effect in recent large pragmatic trial of two widely used coached mental health technologies for depression.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [3] source
Not checked in the free run.
- Interpretationreasoning
Trial recruitment tends to select individuals who want to use digital mental health tools and are likely to stick with them, a tiny fraction of the broader care population.
“Thus, recruitment favors people who are interested in using and likely to adhere to mental health technologies, which is likely an infinitesimal portion of the larger population served by care systems.”
Quote in the paperSubstantial claim- Rests on an assumption: When clinical research faces recruitment difficulties, a common response is to broaden the pool of potential participants—adding venues, adjusting social media tactics, and similar steps.
- Premisecitation
Such interventions have been shown to be feasible at large scale.
“Although the large-scale feasibility of such interventions has been demonstrated (5), emerging evidence suggests that the benefits seen in these trials are not realized in practice settings.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [5] source
Not checked in the free run.
- Interpretationcitation
Benefits observed in efficacy trials are not realized in practice settings.
“Although the large-scale feasibility of such interventions has been demonstrated (5), emerging evidence suggests that the benefits seen in these trials are not realized in practice settings.”
Quote in the paperSubstantial claim- Rests on an assumption: Anecdotal accounts from U.S. health care organizations that attempted these interventions align with the trial results.
- Interpretationnone given
Digital mental health tools have mostly been designed in a top-down manner.
“The design of mental health technologies has been largely top down.”
Quote in the paperNot a substantial claim - Premisecitation
Because technology advances much faster than research methods, digital mental health tools are often obsolete by the time they are validated.
“The rapid pace of technology development relative to the slow pace of research methods often results in mental health technologies that are outdated and obsolete by the time they are validated (6).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [6] source
Not checked in the free run.
- Interpretationnone given
The majority of online intervention programs offer mainly educational content plus a few interactive features, amounting to upgraded digital self-help manuals.
“Most Web-based intervention technologies provide primarily didactic information and some interactive tools–essentially more sophisticated, digital versions of self-help manuals.”
Quote in the paperNot a substantial claim- Rests on an assumption: Digital mental health tools are largely educational or informational, which may be poorly suited to sustaining engagement and behavior change for many individuals.
- Interpretationreasoning
The research-to-practice gap suggests there is much to learn about implementing these interventions.
“This research-to-practice gap suggests that there is much to learn about how to implement these interventions.”
Quote in the paperNot a substantial claim
- Classified as probably not a substantial claim. 3
- May combine two claims. 3
Probably not a substantial claim: may not pass S2 (checkable).
InterpretationcitationMental health technologies are not products but technology-enabled services (TESs).
“However, the relatively consistent finding has been that substantive mental health benefits are more consistently achieved in the context of human support (1), suggesting that mental health technologies are not products–rather, they are technology-enabled services (TESs).”
Quote in the paperNot a substantial claim- Rests on an assumption: Getting better from mental health conditions typically demands behavior change maintained across weeks or months.
- Rests on an assumption: Today’s digital mental health tools demand time, attention, and motivation from patients—resources that are scarce when psychiatric illness reduces motivation and brings hopelessness and helplessness.
May combine two claims.
PremisecitationRandomized controlled trials have shown that mental health technologies, including Web-based and mobile interventions, can prevent and treat mental disorders and improve psychological well-being.
“An increasingly large body of randomized controlled trials has demonstrated the efficacy of mental health technologies, such as Web-based and mobile interventions, to prevent and treat mental disorders and increase psychological well-being.”
- Also stated in: § Abstract · boosted
“More than 15 years of research and a large number of randomized controlled trials have repeatedly demonstrated the efficacy of these interventions across a range of psychiatric disorders (1,2).”
Quote in the paperSubstantial claim0 of 2 sources checked- Also stated in: § Abstract · boosted
Classified as probably not a substantial claim.
InterpretationreasoningThis approach takes the technology itself to be the main driver of change.
“Doing so assumes that the technology is the primary agent of change.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
InterpretationreasoningClutching old models prevents a genuine paradigm shift; progress needs fresh behavior-change models that depart from traditional psychotherapy and include finer-grained knowledge of patients' lives.
“A true paradigm shift cannot be achieved by clinging to old models, however; innovation will require new models of behavior change that move away from traditional psychotherapy models and incorporate more granular understanding of the lives of patients and applications of these technologies.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationParticipants failed to engage with the online treatments and ceased responding to support calls.
“Patients did not engage with the Web-based treatments and stopped answering the telephone support calls.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [3] source
Not checked in the free run.
The paper relies on this without support.
Getting better from mental health conditions typically demands behavior change maintained across weeks or months.
“Improvement in mental health conditions usually requires sustained behavior change over many weeks or months.”
The paper relies on this without support.
Today’s digital mental health tools demand time, attention, and motivation from patients—resources that are scarce when psychiatric illness reduces motivation and brings hopelessness and helplessness.
“Currently available mental health technologies require a patient’s time, attention, and motivation–all things that are in short supply when psychiatric disorders sap motivation and are accompanied by hopelessness and helplessness.”
The paper relies on this without support.
Digital mental health tools are largely educational or informational, which may be poorly suited to sustaining engagement and behavior change for many individuals.
“Furthermore, mental health technologies are mainly didactic or informational, which might not be ideal for promoting sustained engagement and behavior change for many people.”
The paper relies on this without support.
In the literature, the technology receives thorough description and evaluation, while its surrounding ecosystem—human support and organizational factors—receives little.
“The technology itself is thoroughly described and evaluated, but there is little description or evaluation of the ecosystem around that technology (such as human support and organizational factors).”
The paper relies on this without support.
Clinical trials frequently look unlike actual care settings because they prioritize internal validity over practical matters like technology environment, implementation, and sustainment.
“Trials often bear little resemblance to clinical settings, having largely emphasized internal validity over real-world issues, such as the technological environment and implementation and sustainment.”
The paper relies on this without support.
When clinical research faces recruitment difficulties, a common response is to broaden the pool of potential participants—adding venues, adjusting social media tactics, and similar steps.
“Recruitment challenges, common in clinical research, are often addressed by increasing the recruitment pool (expanding the number of venues, refining social media strategies, and so on).”
The paper relies on this without support.
Anecdotal accounts from U.S. health care organizations that attempted these interventions align with the trial results.
“This is consistent with reports we have heard from health care organizations in the United States that have tried these interventions.”
The paper relies on this without support.
The use of technologies for behavior change outside mental health has tended to view these tools as new forms of interventions.
“Although the use of technologies for behavior change outside of mental health has tended to view these tools as new forms of interventions (such as activity monitoring or diet management), the perception has persisted in some mental health circles that mental health technologies are a new method of delivering evidence-based psychotherapy via the Web or mobile phone.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The time and resources required for planning and undertaking an RCT are not always appropriate for the assessment of dMH tools due to their rapid development and the accelerating time to the obsolescence of technological innovations.
“The time and resources required for planning and undertaking an RCT are not always appropriate for the assessment of dMH tools due to their rapid development and the accelerating time to the obsolescence of technological innovations [9,22,23].”
From Feasibility and Acceptability of the Aboriginal and Islander Mental Health Initiative for Youth App: Nonrandomized Pilot With First Nations Young People (2023) · cites this paper as [23]
Research officer contact guidelines for intervention and follow-up were strictly scripted.
“Research officer contact guidelines for intervention and follow-up were strictly scripted in recognition that the human-support component of digital health approaches requires definition and concurrent evaluation [23].”
From Feasibility and Acceptability of the Aboriginal and Islander Mental Health Initiative for Youth App: Nonrandomized Pilot With First Nations Young People (2023) · cites this paper as [23]
Providing clear guidelines for how, when, and why to use support might also be of benefit.
“Providing clear guidelines for how, when, and why to use support might also be of benefit [23] and highlight new research questions to be answered beyond this study.”
From Feasibility and Acceptability of the Aboriginal and Islander Mental Health Initiative for Youth App: Nonrandomized Pilot With First Nations Young People (2023) · cites this paper as [23]
Benefits of mobile apps are more consistently achieved in the context of human support.
“Nevertheless, the benefits of mobile apps are more consistently achieved in the context of human support [23].”
From Feasibility and Acceptability of the Aboriginal and Islander Mental Health Initiative for Youth App: Nonrandomized Pilot With First Nations Young People (2023) · cites this paper as [23]
For providers, the dMH tool must fit into their workflows and offer some meaningful benefit rather than just adding another task to their workdays in order to aid implementation.
“For providers (care managers, physicians, and mental health providers), the dMH tool must fit into their workflows and offer some meaningful benefit rather than just adding another task to their workdays in order to aid implementation [23].”
From Feasibility and Acceptability of the Aboriginal and Islander Mental Health Initiative for Youth App: Nonrandomized Pilot With First Nations Young People (2023) · cites this paper as [23]
Cites
Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012) · Medical Care · doi:10.1097/mlr.0b013e3182408812 · Cited by 4,058 · cited 1× · see citing works
“This can be achieved through hybrid trial designs that integrate simultaneous testing of treatment effects and implementation models (8).”
BackgroundClinical Psychology: Science and Practice
(2021) · Clinical Psychology Science and Practice · doi:10.1111/(issn)1468-2850 · Cited by 3,683 · see citing works
Advances in natural language processing
Hirschberg, J., & Manning, C. D. (2015) · Science · doi:10.1126/science.aaa8685 · Cited by 1,712 · see citing works
Computer-based psychological treatments for depression: A systematic review and meta-analysis
Richards, D., & Richardson, T. (2012) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.02.004 · Cited by 1,442 · see citing works
Mobile Health Technology Evaluation
Kumar, S., Nilsen, W. J., Abernethy, A., Atienza, A., Patrick, K., Pavel, M., Riley, W. T., Shar, A., Spring, B., Spruijt-Metz, D., et al. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.03.017 · Cited by 900 · see citing works
Self-citationSaeb, S., Zhang, M., Karr, C. J., Schueller, S. M., Corden, M. E., Kording, K. P., & Mohr, D. C. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4273 · Cited by 782 · see citing works
Self-citationGilbody, S., Littlewood, E., Hewitt, C., Brierley, G., Tharmanathan, P., Araya, R., Barkham, M., Bower, P., Cooper, C., Gask, L., et al. (2015) · BMJ · doi:10.1136/bmj.h5627 · Cited by 409 · see citing works
Glasgow, R. E. (2013) · Health Education & Behavior · doi:10.1177/1090198113486805 · Cited by 395 · see citing works
Kuester, A., Niemeyer, H., & Knaevelsrud, C. (2016) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.11.004 · Cited by 261 · see citing works
Botella, C., Serrano, B., baños, R., & García-Palacios, A. (2015) · Neuropsychiatric Disease and Treatment · doi:10.2147/ndt.s89542 · Cited by 223 · see citing works
Mohr, D. C., Schueller, S. M., Riley, W. T., Brown, C. H., Cuijpers, P., Duan, N., Kwasny, M. J., Stiles-Shields, C., & Cheung, K. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4391 · Cited by 216 · see citing works
Self-citationTitov, N., Dear, B. F., Staples, L. G., Bennett-Levy, J., Klein, B., Rapee, R. M., Shann, C., Richards, D., Andersson, G., Ritterband, L., et al. (2015) · Psychiatric Services · doi:10.1176/appi.ps.201400477 · Cited by 181 · see citing works
Realizing the Potential of Behavioral Intervention Technologies
Schueller, S. M., Muñoz, R. F., & Mohr, D. C. (2013) · Current Directions in Psychological Science · doi:10.1177/0963721413495872 · Cited by 152 · see citing works
Self-citationNew Perspectives on Human-Computer Interaction.
Moran, T. P. (1985) · Human-Computer Interaction · Cited by 63 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Objective user engagement with mental health apps: systematic search and panel-based usage analysis
Baumel, A., Muench, F., Edan, S., & Kane, J. M. (2019) · Journal of Medical Internet Research · doi:10.2196/14567 · Cited by 965 · see citing works
Torous, J., Jän Myrick, K., Rauseo-Ricupero, N., & Firth, J. (2020) · JMIR Mental Health · doi:10.2196/18848 · Cited by 946 · see citing works
The Lancet Psychiatry Commission on youth mental health
McGorry, P. D., Mei, C., Dalal, N., Alvarez-Jimenez, M., Blakemore, S.-J., Browne, V., Dooley, B., Hickie, I. B., Jones, P. B., McDaid, D., et al. (2024) · The Lancet Psychiatry · doi:10.1016/s2215-0366(24)00163-9 · Cited by 663 · see citing works
Torous, J., Lipschitz, J., Ng, M., & Firth, J. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.11.167 · Cited by 546 · see citing works
Mohr, D. C., Lyon, A. R., Lattie, E. G., Reddy, M., & Schueller, S. M. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7725 · Cited by 432 · see citing works
Self-citationEbert, D. D., Van Daele, T., Nordgreen, T., Karekla, M., Compare, A., Zarbo, C., Brugnera, A., Øverland, S., Trebbi, G., Jensen, K. L., et al. (2018) · European Psychologist · doi:10.1027/1016-9040/a000318 · Cited by 437 · see citing works
Digital interventions for the treatment of depression: a meta-analytic review
Moshe, I., Terhorst, Y., Philippi, P., Domhardt, M., Cuijpers, P., Cristea, I., Pulkki-Råback, L., Baumeister, H., & Sander, L. B. (2021) · Psychological Bulletin · doi:10.1037/bul0000334 · Cited by 390 · see citing works
Jakob, R., Harperink, S., Rudolf, A. M., Fleisch, E., Haug, S., Mair, J. L., Salamanca-Sanabria, A., & Kowatsch, T. (2022) · Journal of Medical Internet Research · doi:10.2196/35371 · Cited by 446 · see citing works
ICBT in routine care: A descriptive analysis of successful clinics in five countries
Titov, N., Dear, B., Nielssen, O., Staples, L., Hadjistavropoulos, H., Nugent, M., Adlam, K., Nordgreen, T., Bruvik, K. H., Hovland, A., et al. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.07.006 · Cited by 353 · see citing works
Towards a consensus around standards for smartphone apps and digital mental health
Torous, J., Andersson, G., Bertagnoli, A., Christensen, H., Cuijpers, P., Firth, J., Haim, A., Hsin, H., Hollis, C., Lewis, S., et al. (2019) · World Psychiatry · doi:10.1002/wps.20592 · Cited by 325 · see citing works
Self-citationStein, D. J., Shoptaw, S. J., Vigo, D. V., Lund, C., Cuijpers, P., Bantjes, J., Sartorius, N., & Maj, M. (2022) · World Psychiatry · doi:10.1002/wps.20998 · Cited by 331 · see citing works
Responsible AI—two frameworks for ethical design practice
Peters, D., Vold, K., Robinson, D., & Calvo, R. A. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.2974991 · Cited by 293 · see citing works
“Furthermore, as mentioned, researchers have expressed a need for more involvement of people living with mental health issues in technology design [43]–[45].”
BackgroundDigital Mental Health for Young People: A Scoping Review of Ethical Promises and Challenges
Wies, B., Landers, C., & Ienca, M. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.697072 · Cited by 242 · see citing works
Hollis, C., Sampson, S., Simons, L., Davies, E. B., Churchill, R., Betton, V., Butler, D., Chapman, K., Easton, K., Gronlund, T. A., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30296-7 · Cited by 186 · see citing works
By the numbers: ratings and utilization of behavioral health mobile applications
Carlo, A. D., Hosseini Ghomi, R., Renn, B. N., & Areán, P. A. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0129-6 · Cited by 164 · see citing works
Naslund, J. A., Shidhaye, R., & Patel, V. (2019) · Harvard Review of Psychiatry · doi:10.1097/hrp.0000000000000217 · Cited by 159 · see citing works
Weisz, J. R., Kuppens, S., Ng, M. Y., Vaughn-Coaxum, R. A., Ugueto, A. M., Eckshtain, D., & Corteselli, K. A. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691618805436 · Cited by 180 · see citing works
Scaling evidence-based treatments through digital mental health.
Schueller, S. M., & Torous, J. B. (2020) · American Psychologist · doi:10.1037/amp0000654 · Cited by 174 · see citing works
Self-citationLópez-López, J. A., Davies, S. R., Caldwell, D. M., Churchill, R., Peters, T. J., Tallon, D., Dawson, S., Wu, Q., Li, J., Taylor, A., et al. (2019) · Psychological Medicine · doi:10.1017/s003329171900120x · Cited by 161 · see citing works
Graham, A. K., Greene, C. J., Kwasny, M. J., Kaiser, S. M., Lieponis, P., Powell, T., & Mohr, D. C. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.1011 · Cited by 162 · see citing works
Self-citationBaumel, A., & Kane, J. M. (2018) · Journal of Medical Internet Research · doi:10.2196/11491 · Cited by 131 · see citing works
Digital navigators to implement smartphone and digital tools in care
Wisniewski, H., & Torous, J. B. (2020) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13149 · Cited by 144 · see citing works
Baumel, A., Edan, S., & Kane, J. M. (2019) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz147 · Cited by 143 · see citing works
From Research to Practice: Ten Lessons in Delivering Digital Mental Health Services
Titov, N., Hadjistavropoulos, H. D., Nielssen, O. B., Mohr, D. C., Andersson, G., & Dear, B. F. (2019) · Journal of Clinical Medicine · doi:10.3390/jcm8081239 · Cited by 117 · see citing works
Self-citationDigital apothecaries: a vision for making health care interventions accessible worldwide
Muñoz, R. F., Chavira, D. A., Himle, J. A., Koerner, K., Muroff, J. R., Reynolds, J., Rose, R. D., Ruzek, J. I., Teachman, B. A., & Schueller, S. M. (2018) · mHealth · doi:10.21037/mhealth.2018.05.04 · Cited by 117 · see citing works
Self-citationDesigning the Future of Children’s Mental Health Services
Lyon, A. R., Dopp, A. R., Brewer, S. K., Kientz, J. A., & Munson, S. A. (2020) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-020-01038-x · Cited by 117 · see citing works
Stawarz, K., Preist, C., & Coyle, D. (2019) · JMIR Mental Health · doi:10.2196/12546 · Cited by 107 · see citing works
Alvarez-Jimenez, M., Rice, S., D'Alfonso, S., Leicester, S., Bendall, S., Pryor, I., Russon, P., McEnery, C., Santesteban-Echarri, O., Da Costa, G., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/17155 · Cited by 106 · see citing works
User‐centered design for technology‐enabled services for eating disorders
Graham, A. K., Wildes, J. E., Reddy, M. C., Munson, S. A., Taylor, C. B., & Mohr, D. C. (2019) · International Journal of Eating Disorders · doi:10.1002/eat.23130 · Cited by 103 · see citing works
Self-citationNaslund, J. A., Gonsalves, P. P., Gruebner, O., Pendse, S. R., Smith, S. L., Sharma, A., & Raviola, G. J. (2019) · Current Treatment Options in Psychiatry · doi:10.1007/s40501-019-00186-8 · Cited by 108 · see citing works
The digital clinic: Implementing technology and augmenting care for mental health
Rodriguez-Villa, E., Rauseo-Ricupero, N., Camacho, E., Wisniewski, H., Keshavan, M., & Torous, J. (2020) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2020.06.009 · Cited by 97 · see citing works
Gual-Montolio, P., Jaén, I., Martínez-Borba, V., Castilla, D., & Suso-Ribera, C. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19137737 · Cited by 93 · see citing works
Designing m-Health interventions for precision mental health support
Bidargaddi, N., Schrader, G., Klasnja, P., Licinio, J., & Murphy, S. (2020) · Translational Psychiatry · doi:10.1038/s41398-020-00895-2 · Cited by 92 · see citing works
Dailah, H. G., Koriri, M. D., Sabei, A., Kriry, T., & Zakri, M. (2024) · Healthcare · doi:10.3390/healthcare12242555 · Cited by 106 · see citing works
Digital Technologies in Psychiatry: Present and Future
Hirschtritt, M. E., & Insel, T. R. (2018) · FOCUS The Journal of Lifelong Learning in Psychiatry · doi:10.1176/appi.focus.20180001 · Cited by 88 · see citing works
Designing Mental Health Technologies that Support the Social Ecosystem of College Students
Lattie, E. G., Kornfield, R., Ringland, K. E., Zhang, R., Winquist, N. W., & Reddy, M. C. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376362 · Cited by 88 · see citing works
Self-citationGamifying CBT to deliver emotional health treatment to young people on smartphones
Christie, G. I., Shepherd, M., Merry, S. N., Hopkins, S., Knightly, S., & Stasiak, K. (2019) · Internet Interventions · doi:10.1016/j.invent.2019.100286 · Cited by 81 · see citing works
Achilles, M. R., Anderson, M., Li, S., Subotic-Kerry, M., Parker, B., & O’Dea, B. (2020) · Digital Health · doi:10.1177/2055207620926064 · Cited by 77 · see citing works
Syrjala, K. L., Yi, J. C., Artherholt, S. B., Romano, J. M., Crouch, M.-L., Fiscalini, A. S., Hegel, M. T., Flowers, M. E. D., Martin, P. J., & Leisenring, W. M. (2018) · Journal of Cancer Survivorship · doi:10.1007/s11764-018-0693-9 · Cited by 76 · see citing works
Deterioration rates in Virtual Reality Therapy: An individual patient data level meta-analysis
Fernández-Álvarez, J., Rozental, A., Carlbring, P., Colombo, D., Riva, G., Anderson, P. L., Baños, R. M., Benbow, A. A., Bouchard, S., Bretón-López, J. M., et al. (2019) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.06.005 · Cited by 79 · see citing works
What Is the Current and Future Status of Digital Mental Health Interventions?
Baños, R. M., Herrero, R., & Vara, M. D. (2022) · The Spanish Journal of Psychology · doi:10.1017/sjp.2022.2 · Cited by 82 · see citing works
Digital Psychiatry: Risks and Opportunities for Public Health and Wellbeing
Burr, C., Morley, J., Taddeo, M., & Floridi, L. (2020) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2020.2977059 · Cited by 78 · see citing works
Ebert, D. D., Harrer, M., Apolinário-Hagen, J., & Baumeister, H. (2019) · Advances in Experimental Medicine and Biology · doi:10.1007/978-981-32-9721-0_29 · Cited by 76 · see citing works
Raney, L. E., Bergman, D., Torous, J. B., & Hasselberg, M. J. (2017) · Current Psychiatry Reports · doi:10.1007/s11920-017-0838-y · Cited by 76 · see citing works
Zhang, R., Ringland, K. E., Paan, M., Mohr, D. C., & Reddy, M. C. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445771 · Cited by 75 · see citing works
Self-citationPickersgill, M. D. (2018) · Sociology of Health & Illness · doi:10.1111/1467-9566.12811 · Cited by 74 · see citing works
Birk, M. V., & Mandryk, R. L. (2018) · Journal of Medical Internet Research · doi:10.2196/10133 · Cited by 70 · see citing works
Going beyond killer apps: building a better mHealth evidence base
Peiris, D., Miranda, J. J., & Mohr, D. C. (2018) · BMJ Global Health · doi:10.1136/bmjgh-2017-000676 · Cited by 70 · see citing works
Self-citationHarrer, M., Apolinário-Hagen, J., Fritsche, L., Salewski, C., Zarski, A.-C., Lehr, D., Baumeister, H., Cuijpers, P., & Ebert, D. D. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100374 · Cited by 63 · see citing works
mHealth Assessment and Intervention of Depression and Anxiety in Older Adults
Grossman, J. T., Frumkin, M., Rodebaugh, T., & Lenze, E. (2020) · Harvard Review of Psychiatry · doi:10.1097/hrp.0000000000000255 · Cited by 45 · see citing works
Ramos, G., Hernandez-Ramos, R., Taylor, M. E., & Schueller, S. M. (2024) · Behavior Therapy · doi:10.1016/j.beth.2024.04.004 · Cited by 64 · see citing works
Self-citationCohen, K. A., & Schleider, J. L. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100496 · Cited by 62 · see citing works
Wani, C., McCann, L., Lennon, M., & Radu, C. (2024) · Journal of Medical Internet Research · doi:10.2196/51376 · Cited by 75 · see citing works
Fitzsimmons‐Craft, E. E., Firebaugh, M., Graham, A. K., Eichen, D. M., Monterubio, G. E., Balantekin, K. N., Karam, A. M., Seal, A., Funk, B., Taylor, C. B., et al. (2018) · Psychological Services · doi:10.1037/ser0000264 · Cited by 60 · see citing works
Mohr, D. C., Lattie, E. G., Tomasino, K. N., Kwasny, M. J., Kaiser, S. M., Gray, E. L., Alam, N., Jordan, N., & Schueller, S. M. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103485 · Cited by 57 · see citing works
Self-citationJardine, J., Nadal, C., Robinson, S., Enrique, Á., Hanratty, M., & Doherty, G. J. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3635472 · Cited by 55 · see citing works
Kanuri, N., Arora, P. G., Talluru, S. M., Colaco, B., Dutta, R., Rawat, A., Taylor, C. B., Manjula, M., & Newman, M. G. (2019) · International Journal of Psychology · doi:10.1002/ijop.12640 · Cited by 57 · see citing works
Defining and Addressing Gaps in Care for Obsessive-Compulsive Disorder in the United States
Senter, M. S., Patel, S. R., Dixon, L. B., Myers, R. W., & Simpson, H. B. (2021) · Psychiatric Services · doi:10.1176/appi.ps.202000296 · Cited by 54 · see citing works
Mendes-Santos, C., Nunes, F., Weiderpass, E., Santana, R., & Andersson, G. (2021) · doi:10.2196/preprints.32558 · Cited by 52 · see citing works
A Smart Toy Intervention to Promote Emotion Regulation in Middle Childhood: Feasibility Study
Theofanopoulou, N., Isbister, K., Edbrooke-Childs, J. H., & Slovák, P. (2019) · JMIR Mental Health · doi:10.2196/14029 · Cited by 52 · see citing works
Hynie, M., Jaimes, A., Oda, A., Rivest‐Beauregard, M., Gonzalez, L. P., Ives, N. G., Ahmad, F., Kuo, B. C. H., Arya, N., Bokore, N., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19095001 · Cited by 51 · see citing works
Bauer, A. M., Cooper, S. H., Bechtel, J. M., & Fortney, J. C. (2018) · Journal of Medical Internet Research · doi:10.2196/10048 · Cited by 49 · see citing works
Taylor, M. E., Liu, M., Abelson, S., Eisenberg, D., Lipson, S. K., & Schueller, S. M. (2024) · Current Psychiatry Reports · doi:10.1007/s11920-024-01545-w · Cited by 53 · see citing works
Self-citationNaslund, J. A., & Aschbrenner, K. A. (2019) · Evidence Based Mental Health · doi:10.1136/ebmental-2018-300034 · Cited by 44 · see citing works
Ye, J. (2021) · JMIR Formative Research · doi:10.2196/22075 · Cited by 46 · see citing works
Recent developments in the use of smartphone interventions for mental health
Mehrotra, S., & Tripathi, R. (2018) · Current Opinion in Psychiatry · doi:10.1097/yco.0000000000000439 · Cited by 45 · see citing works
Spadaro, B., Martin-Key, N. A., & Bahn, S. (2021) · Journal of Medical Internet Research · doi:10.2196/27507 · Cited by 45 · see citing works
Jonathan, G., Carpenter-Song, E. A., Brian, R. M., & Ben-Zeev, D. (2019) · Psychiatric Rehabilitation Journal · doi:10.1037/prj0000337 · Cited by 45 · see citing works
Williams, J., & Pykett, J. (2022) · Social Science & Medicine · doi:10.1016/j.socscimed.2022.114802 · Cited by 49 · see citing works
Liu, M., & Schueller, S. M. (2023) · Current Treatment Options in Psychiatry · doi:10.1007/s40501-023-00298-2 · Cited by 47 · see citing works
Self-citationJameel, L., Valmaggia, L., Barnes, G., & Cella, M. (2022) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2021.11.033 · Cited by 43 · see citing works
Arnold, C., Williams, A., & Thomas, N. (2020) · JMIR Mental Health · doi:10.2196/16730 · Cited by 42 · see citing works
“I think people are powerful”: The Sociality of Individuals Managing Depression
Burgess, E. R., Ringland, K. E., Nicholas, J., Knapp, A. A., Eschler, J., Mohr, D., & Reddy, M. C. (2019) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3359143 · Cited by 60 · see citing works
Self-citationBaumel, A., Torous, J. B., Edan, S., & Kane, J. M. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.05.011 · Cited by 40 · see citing works
Isbister, K., Cottrell, P. S., Cecchet, A., Dagan, E., Theofanopoulou, N., Bertran, F. A., Horowitz, A. J., Mead, N., Schwartz, J. B., & Slovák, P. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3491083 · Cited by 40 · see citing works
Kip, H., Jong, N. B.-D., van Gemert‐Pijnen, J. E. W. C., & Kelders, S. M. (2024) · Journal of Medical Internet Research · doi:10.2196/59601 · Cited by 41 · see citing works
Interactive Technologies for Emotion Regulation Training: A Scoping Review
Sadka, O., & Antle, A. N. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102906 · Cited by 37 · see citing works
Li, D. H., Brown, C. H., Gallo, C. G., Morgan, E., Sullivan, P. S., Young, S. D., & Mustanski, B. S. (2019) · Current HIV/AIDS Reports · doi:10.1007/s11904-019-00455-4 · Cited by 36 · see citing works
Dear, B. F., Johnson, B., Singh, A., Wilkes, B., Brkic, T., Gupta, R., Jones, M. P., Bailey, S., Dudeney, J., Gandy, M., et al. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2019.06.044 · Cited by 36 · see citing works
Designing accessible, explainable AI (XAI) experiences
Wolf, C. T., & Ringland, K. E. (2020) · ACM SIGACCESS Accessibility and Computing · doi:10.1145/3386296.3386302 · Cited by 36 · see citing works
Graham, A. K., Greene, C. J., Powell, T., Lieponis, P., Lunsford, A., Peralta, C. D., Orr, L. C., Kaiser, S. M., Alam, N., Berhane, H., et al. (2020) · Translational Behavioral Medicine · doi:10.1093/tbm/ibz140 · Cited by 36 · see citing works
Self-citationExploring gamification design elements for mental health support
Ibrahim, E. N. M., Jamali, N., & Suhaimi, A. I. H. (2021) · International Journal of Advanced Technology and Engineering Exploration · doi:10.19101/ijatee.2020.s1762123 · Cited by 37 · see citing works
Natural Language Processing and Psychosis: On the Need for Comprehensive Psychometric Evaluation
Cohen, A. S., Rodriguez, Z. B., Warren, K. K., Cowan, T., Masucci, M. D., Granrud, O. E., Holmlund, T. B., Chandler, C., Foltz, P. W., & Strauss, G. P. (2022) · Schizophrenia Bulletin · doi:10.1093/schbul/sbac051 · Cited by 36 · see citing works
Topooco, N., Fowler, L. A., Fitzsimmons‐Craft, E. E., DePietro, B., Vázquez, M. M., Firebaugh, M., Ceglarek, P. J. D., Monterubio, G. E., Newman, M. G., Eisenberg, D., et al. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100528 · Cited by 37 · see citing works
Seegan, P. L., Miller, M. J., Heliste, J. L., Fathi, L., & Mcguire, J. F. (2023) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2023.06.019 · Cited by 37 · see citing works
Digitalising mental health care: Practical recommendations from the European Psychiatric Association
Kalman, J. L., Burkhardt, G., Samochowiec, J., Gebhard, C., Dom, G., John, M., Kılıç, Ö., Kurimay, T., Lien, L., Schouler‐Ocak, M., et al. (2023) · European Psychiatry · doi:10.1192/j.eurpsy.2023.2466 · Cited by 34 · see citing works
Implementation of e‐mental health for depression and anxiety: A critical scoping review
Ellis, L. A., Augustsson, H., Grødahl, A. I., Pomare, C., Churruca, K., Long, J. C., Ludlow, K., Zurynski, Y. A., & Braithwaite, J. (2020) · Journal of Community Psychology · doi:10.1002/jcop.22309 · Cited by 33 · see citing works
Grand Challenges in Human Factors and Digital Health
Schueller, S. M. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.635112 · Cited by 33 · see citing works
Self-citationEcological Momentary Interventions in Public Mental Health Provision
Schulte‐Strathaus, J. C. C., Rauschenberg, C., Baumeister, H., & Reininghaus, U. (2022) · Studies in neuroscience, psychology and behavioral economics · doi:10.1007/978-3-030-98546-2_25 · Cited by 29 · see citing works
Lukka, L., & Palva, J. M. (2023) · JMIR Serious Games · doi:10.2196/42173 · Cited by 32 · see citing works
Myin‐Germeys, I., Schick, A., Ganslandt, T., Hajdúk, M., Heretik, A., Hoyweghen, I. V., Kiekens, G., Koppe, G., Marelli, L., Nagyová, I., et al. (2024) · Psychological Medicine · doi:10.1017/s0033291724001454 · Cited by 33 · see citing works
Evaluating Triple P Online: A Digital Parent Training Program for Child Behavior Problems
Baumel, A., & Faber, K. (2017) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2017.10.001 · Cited by 32 · see citing works
Shire, S. Y., Worthman, L. B., Shih, W., & Kasari, C. L. (2020) · Journal of Behavioral Education · doi:10.1007/s10864-020-09376-4 · Cited by 32 · see citing works
Access to Virtual Mental Healthcare and Support for Refugee and Immigrant Groups: A Scoping Review
Hynie, M., Oda, A., Calaresu, M., Kuo, B. C. H., Ives, N. G., Jaimes, A., Bokore, N., Beukeboom, C., Ahmad, F., Arya, N., et al. (2023) · Journal of Immigrant and Minority Health · doi:10.1007/s10903-023-01521-1 · Cited by 31 · see citing works
Hom, M. A., Weiss, R. B., Millman, Z. B., Christensen, K., Lewis, E. J., Cho, S., Yoon, S., Meyer, N., Kosiba, J. D., Shavit, E., et al. (2020) · Journal of Psychotherapy Integration · doi:10.1037/int0000212 · Cited by 31 · see citing works
Areán, P. A., Pratap, A., Hsin, H., Huppert, T. K., Hendricks, K. E., Heagerty, P. J., Cohen, T., Bagge, C. L., & Comtois, K. A. (2021) · Journal of Medical Internet Research · doi:10.2196/27918 · Cited by 27 · see citing works
Álvarez‐Jiménez, M., Nicholas, J., Valentine, L., Liu, P., Mangelsdorf, S. N., Baker, S., Gilbertson, T., O'Loughlin, G., McEnery, C., McGorry, P. D., et al. (2024) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.13751 · Cited by 30 · see citing works
Manole, A., Cârciumaru, R., Brînzaș, R., & Manole, F. (2024) · Information · doi:10.3390/info16010011 · Cited by 34 · see citing works
Toscos, T., Coupe, A., Flanagan, M., Drouin, M. A., Carpenter, M., Reining, L. E., Roebuck, A. E., & Mirro, M. J. (2019) · JMIR Mental Health · doi:10.2196/13230 · Cited by 32 · see citing works
HCI Contributions in Mental Health: A Modular Framework to Guide Psychosocial Intervention Design
Slovák, P., & Munson, S. A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642624 · Cited by 30 · see citing works
- Authors
- Mohr, D. C., Weingardt, K. R., Reddy, M., & Schueller, S. M.
- Affiliations
- Northwestern University (US)
- Venue
- Psychiatric Services
- Published
- 2017-05-01
- Volume
- 68
- Issue
- 5
- Pages
- 427-429
- ISSN
- 0022-1597 (ISSN-L)
- 1075-2730
- 1557-9700
- PMC
- PMC6903906
- OpenAlex
- W2605602096
- Cited by
- 365
- References
- 14
- Citations by year
- 2026: 41 · 2025: 57 · 2024: 52 · 2023: 32 · 2022: 41 · 2021: 42 · 2020: 42 · 2019: 29 · 2018: 24 · 2017: 5
- FWCI
- 31.24
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Psychiatric Services (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- Funders
- National Institute of Mental Health (P20MH090318, R01MH111610, R01MH100482, R01MH095753, K08MH102336)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Social Psychology › Mental Health Treatment and Access
- Keywords
- digital mental health, online intervention, mobile-based interventions, mental health technology, psychological well-being
- MeSH terms
- Health Services Research, Humans, Medical Informatics Applications, Mental Health Services, Biomedical Technology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
