Suicide prevention and depression apps’ suicide risk assessment and management: a systematic assessment of adherence to clinical guidelines
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Background: There are an estimated 800,000 suicides per year globally, and approximately 16,000,000 suicide attempts. Mobile apps may help address the unmet needs of people at risk. We assessed adherence of suicide prevention advice in depression management and suicide prevention apps to six evidence-based clinical guideline recommendations: mood and suicidal thought tracking, safety plan development, recommendation of activities to deter suicidal thoughts, information and education, access to support networks, and access to emergency counseling. Methods: A systematic assessment of depression and suicide prevention apps available in Google Play and Apple’s App Store was conducted. Apps were identified by searching 42matters in January 2019 for apps launched or updated since January 2017 using the terms “depression,” “depressed,” “depress,” “mood disorders,” “suicide,” and “self-harm.” General characteristics of apps, adherence with six suicide prevention strategies identified in evidence-based clinical guidelines using a 50-question checklist developed by the study team, and trustworthiness of the app based on HONcode principles were appraised and reported as a narrative review, using descriptive statistics. Results: The initial search yielded 2690 potentially relevant apps. Sixty-nine apps met inclusion criteria and were systematically assessed. There were 20 depression management apps (29%), 3 (4%) depression management and suicide prevention apps, and 46 (67%) suicide prevention apps. Eight (12%) depression management apps were chatbots. Only 5/69 apps (7%) incorporated all six suicide prevention strategies. Six apps (6/69, 9%), including two apps available in both app stores and downloaded more than one million times each, provided an erroneous crisis helpline number. Most apps included emergency contact information (65/69 apps, 94%) and direct access to a crisis helpline through the app (46/69 apps, 67%). Conclusions: Non-existent or inaccurate suicide crisis helpline phone numbers were provided by mental health apps downloaded more than 2 million times. Only five out of 69 depression and suicide prevention apps offered all six evidence-based suicide prevention strategies. This demonstrates a failure of Apple and Google app stores, and the health app industry in self-governance, and quality and safety assurance. Governance levels should be stratified by the risks and benefits to users of the app, such as when suicide prevention advice is provided.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
Suicide Prevention Strategies: A Systematic Review
Mann, J. J., Apter, A., Bertolote, J., Beautrais, A., Currier, D., Haas, A., Hegerl, U., Lonnqvist, J., Malone, K., Marusic, A., et al. (2005) · JAMA · doi:10.1001/jama.294.16.2064 · Cited by 2,852
Cross-national prevalence and risk factors for suicidal ideation, plans and attempts
Nock, M. K., Borges, G., Bromet, E. J., Alonso, J., Angermeyer, M., Beautrais, A., Bruffaerts, R., Chiu, W. T., de Girolamo, G., Gluzman, S., et al. (2008) · British Journal of Psychiatry · doi:10.1192/bjp.bp.107.040113 · Cited by 2,654 · cited 1×
“Sixty percent of individuals with suicidal ideation transitioned to a first attempt within a year of onset [4], a significant figure considering that half to two thirds of suicide deaths occur in the first attempt [5, 6].”
BackgroundPignone, M. P., Gaynes, B. N., Rushton, J. L., Burchell, C. M., Orleans, C. T., Mulrow, C. D., & Lohr, K. N. (2002) · Annals of Internal Medicine · doi:10.7326/0003-4819-136-10-200205210-00013 · Cited by 1,086
Naghavi, M. (2019) · BMJ · doi:10.1136/bmj.l94 · Cited by 1,043
Suicide Attempt as a Risk Factor for Completed Suicide: Even More Lethal Than We Knew
Bostwick, J. M., Pabbati, C., Geske, J. R., & McKean, A. J. (2016) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2016.15070854 · Cited by 759 · cited 1×
“Sixty percent of individuals with suicidal ideation transitioned to a first attempt within a year of onset [4], a significant figure considering that half to two thirds of suicide deaths occur in the first attempt [5, 6].”
BackgroundTorous, J., & Roberts, L. W. (2017) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2017.0262 · Cited by 362 · cited 1×
“Over the last decade, the health app market has grown to include about 318,000 apps [19], of which more than 10,000 are mental health apps [20], making selection of an appropriate app cumbersome, particularly for lay users [21].”
BackgroundJosephine, K., Josefine, L., Philipp, D., David, E., & Harald, B. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.07.021 · Cited by 341
Treatment of suicidal people around the world
Bruffaerts, R., Demyttenaere, K., Hwang, I., Chiu, W.-T., Sampson, N., Kessler, R. C., Alonso, J., Borges, G., de Girolamo, G., de Graaf, R., et al. (2011) · British Journal of Psychiatry · doi:10.1192/bjp.bp.110.084129 · Cited by 338 · cited 1×
“Forty percent of people with suicidal thoughts or behaviors do not seek medical care [16], or may not have access to healthcare, particularly in developing countries.”
BackgroundWang, K., Varma, D. S., & Prosperi, M. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2018.10.006 · Cited by 308
Research works printed but not linked: 17
- Matthay EC. Suicide falls by a third globally. BMJ. 2019;364:l416. https://doi.org/10.1136/bmj.l416. · DOI
- Hawton K, ICC C, Haw C, Saunders K. Risk factors for suicide in individuals with depression: a systematic review. J Affect Disord. 2013;147(1–3):17–28. https://doi.org/10.1016/j.jad.2013.01.004. · DOI · Paper page
- Zalsman G, Hawton K, Wasserman D, et al. Suicide prevention strategies revisited: 10-year systematic review. Lancet Psychiatry. 2016;3(7):646–59. https://doi.org/10.1016/S2215-0366(16)30030-X. · DOI · Paper page
- Pearson A, Saini P, Da Cruz D, et al. Primary care contact prior to suicide in individuals with mental illness. Br J Gen Pract. 2009;59(568):825–32. https://doi.org/10.3399/bjgp09X472881. · DOI · Paper page
- Frey LM, Hans JD, Cerel J. Perceptions of suicide stigma: how do social networks and treatment providers compare? Crisis. 2016;37(2):95–103 https://doi.org/10.1027/0227-5910/a000358. · DOI · Paper page
- de Beurs D, Kirtley O, Kerkhof A, Portzky G, O'Connor RC. The role of mobile phone technology in understanding and preventing suicidal behavior. Crisis. 2015;36(2):79–82. https://doi.org/10.1027/0227-5910/a000316. · DOI · Paper page
- Christensen H, Batterham PJ, O'Dea B. E-health interventions for suicide prevention. Int J Environ Res Public Health. 2014;11(8):8193–212. https://doi.org/10.3390/ijerph110808193. · DOI · Paper page
- de la Torre I, Castillo G, Arambarri J, López-Coronado M, Franco MA. Mobile apps for suicide prevention: review of virtual stores and literature. JMIR Mhealth Uhealth. 2017;5(10):e130. https://doi.org/10.2196/mhealth.8036. · DOI
- Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009;6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097. · DOI · Paper page
- Stanley B, Brown GK. Safety planning intervention: a brief intervention to mitigate suicide risk. Cogn Behav Pract. 2012;19(2):256–64. https://doi.org/10.1016/j.cbpra.2011.01.001. · DOI · Paper page
- Cohen JA. Coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20(1):37–46. https://doi.org/10.1177/001316446002000104. · DOI · Paper page
- Lum E, Jimenez G, Huang Z, et al. Decision support and alerts of apps for self-management of blood glucose for type 2 diabetes. JAMA. 2019;321(15):1530–2. https://doi.org/10.1001/jama.2019.1644. · DOI
- Huckvale K, Adomaviciute S, Prieto JT, Leow MK, Car J. Smartphone apps for calculating insulin dose: a systematic assessment. BMC Med. 2015;13:106. https://doi.org/10.1186/s12916-015-0314-7. · DOI · Paper page
- Huckvale K, Car M, Morrison C, Car J. Apps for asthma self-management: a systematic assessment of content and tools. BMC Med. 2012;10(1):144. https://doi.org/10.1186/1741-7015-10-144. · DOI · Paper page
- Assing Hvidt E, Ploug T, Holm S. The impact of telephone crisis services on suicidal users: a systematic review of the past 45 years. Mental Health Review Journal. 2016;21(2):141–60. https://doi.org/10.1108/MHRJ-07-2015-0019. · DOI
- PsyberGuide - App review summary. https://psyberguide.org/about-psyberguide/. Accessed 18 Sept 2019.
- Wykes T, Schueller S. Why reviewing apps is not enough: transparency for trust (T4T) principles of responsible health app marketplaces. J Med Internet Res. 2019;21(5):e12390. https://doi.org/10.2196/12390. · DOI
Cited by
Blockchain and Artificial Intelligence Technology in E-Health
Tagde, P., Tagde, S., Bhattacharya, T., Tagde, P., Chopra, H., Akter, R., Kaushik, D., & Rahman, M. H. (2021) · Environmental Science and Pollution Research · doi:10.1007/s11356-021-16223-0 · Cited by 281
Chatbots and mental health: Insights into the safety of generative AI
De Freitas, J., Uğuralp, A. K., Oğuz‐Uğuralp, Z., & Puntoni, S. (2024) · Journal of Consumer Psychology · doi:10.1002/jcpy.1393 · Cited by 211
Evaluating evaluation frameworks: a scoping review of frameworks for assessing health apps
Lagan, S., Sandler, L., & Torous, J. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047001 · Cited by 108
Implementation of Cognitive Behavioral Therapy in e–Mental Health Apps: Literature Review
Denecke, K., Schmid, N., & Nüssli, S. (2022) · Journal of Medical Internet Research · doi:10.2196/27791 · Cited by 107
A multitask framework to detect depression, sentiment and multi-label emotion from suicide notes
Ghosh, S., Ekbal, A., & Bhattacharyya, P. (2022) · Cognitive Computation · doi:10.1007/s12559-021-09828-7 · Cited by 102
Metadata
- Authors
- Martinengo, L., Van Galen, L., Lum, E., Kowalski, M., Subramaniam, M., & Car, J.
- Venue
- BMC Medicine
- Published
- 2019-12
- Volume
- 17
- Issue
- 1
- Pages
- 231-231
- ISSN
- 1741-7015 (ISSN-L)
- OpenAlex
- W2995061552
- Cited by
- 155
- References
- 26
- FWCI
- 23.31
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0/ (tdm)
- creativecommons.org/licenses/by/4.0/ (vor, 18 d)
- Funders
- Lee Kong Chian School of Medicine, Nanyang Technological University (NA)
- NTU Research Scholarship (NA)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Suicide prevention and depression apps’ suicide risk assessment and management: a systematic assessment of adherence to clinical guidelines. https://app.cochat.ai/papers/10.1186/s12916-019-1461-z
- The paper
- Martinengo, L., Van Galen, L., Lum, E., Kowalski, M., Subramaniam, M., & Car, J. (2019). Suicide prevention and depression apps’ suicide risk assessment and management: a systematic assessment of adherence to clinical guidelines. BMC Medicine. https://doi.org/10.1186/s12916-019-1461-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.