Smartphone psychology: New approaches towards safe and efficacious mobile mental health apps.
Abstract
Practicing psychologists are being faced with the reality that mobile mental health apps for smartphones and tablet devices are increasing in popularity. This growth area within e-mental health has been well documented in Professional Psychology: Research and Practice. This article provides an update on the issues of safety and efficacy in mental health app development, two of the biggest concerns that practicing psychologists have about these new digital tools. Governments and medical authorities are wrestling with how to regulate the health app market to avoid harm to users. At the same time, a lack of research into the efficacy and effectiveness of most mental health apps in the various app stores leaves clinicians and consumers with uncertainty. The vast majority of the limited research to date has been completed by those involved in an app’s development. Further independent research and replication are required to demonstrate legitimacy and increase the acceptance of mental health apps as valid sources of therapy. Complicating this issue is disparity about an acceptable methodology for examining the effectiveness of a mental health app. This article proposes a new approach to incorporate multiple baseline single-case designs to increase the amount of evidence and to guide larger-scale randomized controlled trials, something that could and should include practicing psychologists. This novel approach also proposes that mental health apps undergo a new “certification” process with the participation of app store marketplaces.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Government agencies have started to regulate the health app space, but this regulation has thus far focused on apps that only pose a risk of harm to users.
“While government agencies have started to regulate the health app space, this regulation has thus far focused on apps that only pose a risk of harm to users [62].”
From Apps with maps-anxiety and depression mobile apps with evidence-based frameworks: Systematic search of major app stores (2020) · cites this paper as [62]
Cites
Jacobson, N. S., & Truax, P. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.12 · Cited by 7,770 · see citing works
Ecological momentary assessment
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091415 · Cited by 6,454 · see citing works
The Use of Single-Subject Research to Identify Evidence-Based Practice in Special Education
Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005) · Exceptional Children · doi:10.1177/001440290507100203 · Cited by 3,368 · see citing works
Single Case Experimental Designs: Strategies for Studying Behavior Change
Barlow, D. H., Nock, M. K., & Hersen, M. (1976) · Cited by 3,036 · see citing works
Mobile app rating scale: a new tool for assessing the quality of health mobile apps
Stoyanov, S. R., Hides, L., Kavanagh, D. J., Zelenko, O., Tjondronegoro, D., & Mani, M. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3422 · Cited by 2,233 · see citing works
Research design in clinical psychology
Kazdin, A. E. (1952) · Journal of Clinical Psychology · doi:10.1002/1097-4679(195201)8:1 · Cited by 2,153 · see citing works
Smartphones for smarter delivery of mental health programs: A systematic review
Donker, T., Petrie, K., Proudfoot, J., Clarke, J., Birch, M.-R., & Christensen, H. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2791 · Cited by 1,136 · see citing works
Mohr, D. C., Cuijpers, P., & Lehman, K. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1602 · Cited by 1,097 · see citing works
Firth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017) · World Psychiatry · doi:10.1002/wps.20472 · Cited by 1,097 · see citing works
Mental Health Smartphone Apps: Review and Evidence-Based Recommendations for Future Developments
Bakker, D., Kazantzis, N., Rickwood, D., & Rickard, N. (2016) · JMIR Mental Health · doi:10.2196/mental.4984 · Cited by 886 · see citing works
Jacobson, N. S., Roberts, L. J., Berns, S. B., & McGlinchey, J. B. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.67.3.300 · Cited by 853 · see citing works
Jacobson, N. S., Roberts, L. J., Berns, S. B., & McGlinchey, J. B. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.67.3.300 · Cited by 853 · see citing works
Mohr, D. C., Burns, M. N., Schueller, S. M., Clarke, G., & Klinkman, M. (2013) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2013.03.008 · Cited by 693 · see citing works
Firth, J., Torous, J., Nicholas, J., Carney, R., Rosenbaum, S., & Sarris, J. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.04.046 · Cited by 675 · see citing works
Availability and quality of mobile health app privacy policies
Sunyaev, A., Dehling, T., Taylor, P. L., & Mandl, K. D. (2015) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002605 · Cited by 360 · see citing works
Flett, J. A. M., Hayne, H., Riordan, B. C., Thompson, L. M., & Conner, T. S. (2019) · Mindfulness · doi:10.1007/s12671-018-1050-9 · Cited by 338 · see citing works
Wang, K., Varma, D. S., & Prosperi, M. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2018.10.006 · Cited by 308 · see citing works
The Use of Outcome Measures by Psychologists in Clinical Practice.
Hatfield, D. R., & Ogles, B. M. (2004) · Professional Psychology: Research and Practice · doi:10.1037/0735-7028.35.5.485 · Cited by 305 · see citing works
Finding a Depression App: A Review and Content Analysis of the Depression App Marketplace
Shen, N., Levitan, M.-J., Johnson, A., Bender, J. L., Hamilton-Page, M., Jadad, A. (. R., & Wiljer, D. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3713 · Cited by 282 · see citing works
State of the Field of Mental Health Apps
Neary, M., & Schueller, S. M. (2018) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2018.01.002 · Cited by 247 · see citing works
A randomized controlled trial of a smartphone app for posttraumatic stress disorder symptoms.
Kuhn, E., Kanuri, N., Hoffman, J. E., Garvert, D. W., Ruzek, J. I., & Taylor, C. B. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000163 · Cited by 225 · see citing works
Evidence-based apps? A review of mental health mobile applications in a psychotherapy context
Lui, J. H. L., Marcus, D. K., & Barry, C. T. (2017) · Professional Psychology Research and Practice · doi:10.1037/pro0000122 · Cited by 217 · see citing works
Stawarz, K., Preist, C., Tallon, D., Wiles, N., & Coyle, D. (2018) · Journal of Medical Internet Research · doi:10.2196/10120 · Cited by 198 · see citing works
What Exactly Is Patient Safety?
Emanuel, L. L., Berwick, D., Conway, J., Combes, J., Hatlie, M. J., Leape, L. L., REASON, J. T., Schyve, P. M., Vincent, C. A., & Walton, M. (2009) · Journal of Medical Regulation · doi:10.30770/2572-1852-95.1.13 · Cited by 190 · see citing works
Continuous Evaluation of Evolving Behavioral Intervention Technologies
Mohr, D. C., Cheung, K., Schueller, S. M., Brown, C. H., & Duan, N. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.06.006 · Cited by 169 · see citing works
Evaluation of an mHealth App (DeStressify) on University Students’ Mental Health: Pilot Trial
Lee, R. A., & Jung, M. E. (2018) · JMIR Mental Health · doi:10.2196/mental.8324 · Cited by 138 · see citing works
The Digital Psychiatrist: In Search of Evidence-Based Apps for Anxiety and Depression
Marshall, J. M., Dunstan, D. A., & Bartik, W. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00831 · Cited by 107 · see citing works
Self-citationThe smart therapist: A look to the future of smartphones and mHealth technologies in psychotherapy.
Clough, B. A., & Casey, L. M. (2015) · Professional Psychology Research and Practice · doi:10.1037/pro0000011 · Cited by 106 · see citing works
Torous, J., Firth, J., Huckvale, K., Larsen, M. E., Cosco, T. D., Carney, R., Chan, S., Pratap, A., Yellowlees, P., Wykes, T., et al. (2018) · Journal of Nervous & Mental Disease · doi:10.1097/nmd.0000000000000864 · Cited by 101 · see citing works
Bentley, K. H., Kleiman, E. M., Elliott, G., Huffman, J. C., & Nock, M. K. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2018.11.017 · Cited by 88 · see citing works
Users’ Adoption of Mental Health Apps: Examining the Impact of Information Cues
Huang, H., & Bashir, M. N. (2017) · JMIR mhealth and uhealth · doi:10.2196/mhealth.6827 · Cited by 75 · see citing works
Developing Digital Intervention Games for Mental Disorders: A Review
Shah, A., Kraemer, K. R., Won, C. R., Black, S., & Hasenbein, W. (2018) · Games for Health Journal · doi:10.1089/g4h.2017.0150 · Cited by 74 · see citing works
Torous, J., Levin, M. E., Ahern, D. K., & Oser, M. L. (2017) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2016.05.007 · Cited by 73 · see citing works
Ethical guidelines for mobile app development within health and mental health fields
Jones, N., & Moffitt, M. (2016) · Professional Psychology Research and Practice · doi:10.1037/pro0000069 · Cited by 70 · see citing works
Digital Technologies in the Treatment of Anxiety: Recent Innovations and Future Directions
Firth, J., Torous, J., Carney, R., Newby, J., Cosco, T. D., Christensen, H., & Sarris, J. (2018) · Current Psychiatry Reports · doi:10.1007/s11920-018-0910-2 · Cited by 70 · see citing works
Alyami, M. M., Giri, B., Alyami, H., & Sundram, F. (2017) · Evidence-Based Mental Health · doi:10.1136/eb-2017-102664 · Cited by 69 · see citing works
Menon, V., Rajan, T. M., & Sarkar, S. (2017) · Indian Journal of Psychological Medicine · doi:10.4103/0253-7176.198956 · Cited by 63 · see citing works
An Assessment Framework for e-Mental Health Apps in Canada: Results of a Modified Delphi Process
Zelmer, J., van Hoof, K., Notarianni, M., van Mierlo, T., Schellenberg, M., & Tannenbaum, C. (2018) · JMIR mHealth and uHealth · doi:10.2196/10016 · Cited by 62 · see citing works
Mobile Mental Health: Navigating New Rules and Regulations for Digital Tools
Armontrout, J. A., Torous, J. B., Fisher, M., Drogin, E. Y., & Gutheil, T. G. (2016) · Current Psychiatry Reports · doi:10.1007/s11920-016-0726-x · Cited by 61 · see citing works
App-guided exposure and response prevention for obsessive compulsive disorder: an open pilot trial
Boisseau, C. L., Schwartzman, C. M., Lawton, J., & Mancebo, M. C. (2017) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2017.1321683 · Cited by 51 · see citing works
Regulating mobile mental health apps
Terry, N. P., & Gunter, T. D. (2018) · Behavioral Sciences & the Law · doi:10.1002/bsl.2339 · Cited by 48 · see citing works
Recent developments in the use of smartphone interventions for mental health
Mehrotra, S., & Tripathi, R. (2018) · Current Opinion in Psychiatry · doi:10.1097/yco.0000000000000439 · Cited by 45 · see citing works
Roy, M. J., Costanzo, M. E., Highland, K. B., Olsen, C., Clayborne, D., & Law, W. (2017) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2017.0221 · Cited by 43 · see citing works
Current Regulation of Mobile Mental Health Applications.
Armontrout, J. A., Torous, J. B., Cohen, M., McNiel, D. E., & Binder, R. L. (2018) · PubMed · doi:10.29158/jaapl.003748-18 · Cited by 39 · see citing works
AppWatcher: unveiling the underground market of trading mobile app reviews
Xie, Z., & Zhu, S. (2015) · Proceedings of the 8th ACM Conference on Security & Privacy in Wireless and Mobile Networks · doi:10.1145/2766498.2766510 · Cited by 38 · see citing works
Hendrikoff, L., Kambeitz‐Ilankovic, L., Pryss, R. C., Senner, F., Falkai, P., Pogarell, O. M., Hasan, A., & Peters, H. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2018.11.025 · Cited by 35 · see citing works
Clough, B. A., & Casey, L. M. (2015) · Professional Psychology Research and Practice · doi:10.1037/pro0000053 · Cited by 29 · see citing works
Miller, K. E., Kuhn, E. R., Yu, J., Owen, J. E., Jaworski, B. K., Taylor, K. L., Blonigen, D. M., & Possemato, K. (2019) · Professional Psychology Research and Practice · doi:10.1037/pro0000229 · Cited by 25 · see citing works
Armstrong, C. M., Ciulla, R., Edwards‐Stewart, A., Hoyt, T., & Bush, N. E. (2018) · Professional Psychology Research and Practice · doi:10.1037/pro0000194 · Cited by 24 · see citing works
Frank, E., Pong, J., Asher, Y., & Soares, C. N. (2018) · Current Opinion in Psychiatry · doi:10.1097/yco.0000000000000382 · Cited by 23 · see citing works
Anxiety Management on Campus: An Evaluation of a Mobile Health Intervention
Paul, A. M., & Fleming, C. J. E. (2018) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-018-0074-2 · Cited by 20 · see citing works
Special Issue on Advances in Single-Case Research Design and Analysis
Machalicek, W., & Horner, R. H. (2018) · Developmental Neurorehabilitation · doi:10.1080/17518423.2018.1468600 · Cited by 15 · see citing works
Broussard, J. D., & Teng, E. J. (2019) · Professional Psychology Research and Practice · doi:10.1037/pro0000234 · Cited by 12 · see citing works
McLaughlin, J. E. (2014) · American Psychological Association eBooks · doi:10.1037/14376-010 · Cited by 10 · see citing works
Single-case designs and large N studies: The best of both worlds.
Sheridan, S. M. (2014) · American Psychological Association eBooks · doi:10.1037/14376-011 · Cited by 8 · see citing works
Report on International Practice on Digital Apps
Nielsen, S. L. (2018) · Strathprints: The University of Strathclyde institutional repository (University of Strathclyde) · doi:10.17868/66139 · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Marshall, J. M., Dunstan, D. A., & Bartik, W. (2020) · JMIR Mental Health · doi:10.2196/16525 · Cited by 83 · see citing works
“These reviews all call for more research and ongoing evaluation of research methodology into studying apps, as the current methodologies may not be the most appropriate [62,67,68].”
BackgroundSelf-citationLitvin, S., Saunders, R., Jefferies, P., Seely, H., Pössel, P., & Lüttke, S. (2023) · JMIR Mental Health · doi:10.2196/47285 · Cited by 43 · see citing works
Connected Mental Health: Systematic Mapping Study
(2020) · doi:10.2196/19950 · Cited by 33 · see citing works
Bowie-DaBreo, D., Sas, C., Iles‐Smith, H., & Sünram‐Lea, S. I. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517498 · Cited by 36 · see citing works
Palmer, K. M., & Burrows, V. (2020) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-020-00160-9 · Cited by 21 · see citing works
Marshall, J. M., Dunstan, D. A., & Bartik, W. (2020) · JMIR Research Protocols · doi:10.2196/17159 · Cited by 24 · see citing works
Self-citationMarshall, J. M., Dunstan, D. A., & Bartik, W. J. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.775775 · Cited by 19 · see citing works
Self-citationGama, B., & Laher, S. (2023) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-023-00360-z · Cited by 18 · see citing works
Treating Psychological Trauma in the Midst of COVID-19: The Role of Smartphone Apps
Marshall, J. M., Dunstan, D. A., & Bartik, W. J. (2020) · Frontiers in Public Health · doi:10.3389/fpubh.2020.00402 · Cited by 12 · see citing works
Self-citationMobile applications in adolescent psychotherapy during the COVID-19 pandemic: a systematic review
Wüllner, S., Hermenau, K., Krutkova, M., Petras, I.-K., Hecker, T., & Siniatchkin, M. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1345808 · Cited by 10 · see citing works
Harned, M., Dhami, M., & Reger, G. M. (2023) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-023-00299-1 · Cited by 7 · see citing works
Jungnickel, T., von Jan, U., & Albrecht, U. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.785591 · Cited by 7 · see citing works
Keinert, M., Schindler‐Gmelch, L., Eskofier, B. M., & Berking, M. (2024) · International Journal of Cognitive Therapy · doi:10.1007/s41811-024-00218-z · Cited by 5 · see citing works
KPIs for Mobile Apps and Digital Data Management in Healthcare
Antolín-Prieto, R., Ruiz-Lacaci, N., & Reyes-Menéndez, A. (2021) · Advances in marketing, customer relationship management, and e-services book series · doi:10.4018/978-1-7998-7263-4.ch011 · Cited by 4 · see citing works
Çalışır, Ö. M., & Can, N. (2021) · İnönü Üniversitesi Eğitim Fakültesi Dergisi · doi:10.17679/inuefd.952666 · Cited by 4 · see citing works
Baker, J. A., Kendal, S., Bojke, C., Louch, G., Halligan, D., Shafiq, S., Sturley, C., Walker, L. E., Brown, M., Berzins, K. M., et al. (2024) · Health and Social Care Delivery Research · doi:10.3310/udbq8402 · Cited by 4 · see citing works
Enabling Mental Wellness Through Interactive Platforms
Umanandhini, D., Karthika, S., Chakradhar, V., Teja, M. S., & Teja, M. D. (2024) · International Conference on Communication and Electronics Systems (ICCES) · doi:10.1109/icces63552.2024.10859837 · Cited by 3 · see citing works
Schaefer, M. R., Iskander, J. M., Tams, S. T., & Butz, C. (2021) · Clinical Practice in Pediatric Psychology · doi:10.1037/cpp0000391 · Cited by 2 · see citing works
Boucher, E. M., Ward, H. E., Stafford, J. L., & Parks, A. C. (2021) · JMIR Research Protocols · doi:10.2196/25545 · Cited by 1 · see citing works
Estimating Mental Health Using Human-generated Big Data and Machine Learning
Fukazawa, Y. (2022) · The Brain & Neural Networks · doi:10.3902/jnns.29.78 · Cited by 1 · see citing works
Mandelbaum, M. M., Rumelt, A. M. A., Wein, S. N., & Vidair, H. B. (2024) · Journal of Child and Family Studies · doi:10.1007/s10826-024-02789-z · Cited by 1 · see citing works
Daly, K. A., Diaz-Gutierrez, K. A., Beheshtian, A., Heyman, R. E., Slep, A. M. S., & Wolff, M. S. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000690 · Cited by 1 · see citing works
Struggling to help others: A reflection on the impact of Covid-19 on newly graduated psychologists
Diggikar, T. A. (2021) · PsyPag Quarterly · doi:10.53841/bpspag.2021.1.119.36
Youth-focused Design and Regulation in eHealth Can Help Address the Mental Healthcare Crisis
Gasser, M. L., Gray, C. E., & Seldin, K. (2023) · Journal of Science Policy & Governance · doi:10.38126/jspg210304
ReNeuWell mental well-being app: protocol for a randomised controlled trial
Egan, L. A., & Gatt, J. M. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-094557
- Authors
- Marshall, J. M., Dunstan, D. A., & Bartik, W.
- Affiliations
- University of New England (AU)
- Venue
- Professional Psychology: Research and Practice
- Published
- 2020-06
- Volume
- 51
- Issue
- 3
- Pages
- 214-222
- ISSN
- 0735-7028 (ISSN-L)
- 1939-1323
- OpenAlex
- W2992218278
- Cited by
- 25
- References
- 55
- Citations by year
- 2025: 1 · 2024: 6 · 2023: 4 · 2022: 3 · 2021: 6 · 2020: 5
- FWCI
- 3.96
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Professional Psychology Research and Practice (publishedVersion)
- RUNE (Research UNE) (submittedVersion)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- mental health apps, mobile application, e-mental health, mobile application security, app efficacy, randomized controlled trials, digital mental health interventions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
