A Systematic Review of Cognitive Behavioral Therapy and Behavioral Activation Apps for Depression
Open accessAbstract
Depression is a common mental health condition for which many mobile apps aim to provide support. This review aims to identify self-help apps available exclusively for people with depression and evaluate those that offer cognitive behavioural therapy (CBT) or behavioural activation (BA). One hundred and seventeen apps have been identified after searching both the scientific literature and the commercial market. 10.26% (n = 12) of these apps identified through our search offer support that seems to be consistent with evidence-based principles of CBT or BA. Taking into account the non existence of effectiveness/efficacy studies, and the low level of adherence to the core ingredients of the CBT/BA models, the utility of these CBT/BA apps are questionable. The usability of reviewed apps is highly variable and they rarely are accompanied by explicit privacy or safety policies. Despite the growing public demand, there is a concerning lack of appropiate CBT or BA apps, especially from a clinical and legal point of view. The application of superior scientific, technological, and legal knowledge is needed to improve the development, testing, and accessibility of apps for people with depression.
AI snapshot
Written from the paper’s full text.
TL;DR
This systematic review identified 117 depression self-help apps, but only 12 (10.26%) appeared consistent with CBT or BA principles. The authors conclude that these CBT/BA apps have no published efficacy/effectiveness evidence, low fidelity to core CBT/BA ingredients, variable usability, and rarely include privacy or safety policies.
Key findings
- Of 117 apps, 12 (10.26%) were classified as CBT/BA; no studies reported in the scientific literature determined benefits for any of these CBT/BA apps.
- Median adherence to CBT principles was 15% (range 0–75%) and to BA principles was 18.75% (range 6.25–25%); the best CBT apps were Depression CBT Self-Help Guide (75%) and eCBT Mood (55%).
- Usability heuristic adherence had a median of 83% (range 42–98%); highest scorers were MoodTools–Depression Aid (98%), Activity Diary (98%), and Depression Cure–The Free 12 Week Course (92%).
- Only eCBT Mood and Depression CBT Self-Help Guide offered a privacy policy, and 5 of 12 apps (41.66%) provided safety information during crisis; no relationship was found between CBT/BA adherence and usability or popularity indicators.
Methods
- Systematic review searching scientific databases (IEEE, ACM Digital Library, EMBASE, PubMed/Medline, PsychINFO, Web of Science) in November 2015 and the Canadian Apple App Store and Android Market/Google Play in November 2015 using “depression”; inclusion required native smartphone apps exclusively for depression, publicly available in Canada in December 2015, and in English.
- Two reviewers screened and categorized apps with Cohen’s kappa reported at 0.69, 0.85, 0.92, and 0.89 across stages; CBT/BA apps were downloaded and evaluated on usefulness (expert-rated fidelity to CBT/BA core ingredients on a 0–2 scale and literature-based effectiveness), usability (Nielsen heuristics rated 1–5), and integration/infrastructure (privacy policy and safety).
- Analysis used counts and percentages plus Spearman correlations between CBT/BA adherence, usability adherence, and popularity/acceptability indicators.
Limitations
- Stated: the review was limited to English downloadable apps in Canada and only the two most popular platforms; apps on less prevalent platforms, in other languages/countries, or developed and tested in academic settings were excluded.
- Stated: CBT/BA app evaluation was based on one expert’s opinion, and an expert panel could have increased credibility; lack of common constructs, outcome measures, definitions, and standards for other app types made cross-case comparison impossible.
- Evident from the text: the included CBT/BA apps had no published efficacy/effectiveness studies, the CBT/BA sample was only 12 apps, and usability, privacy, and safety assessments each relied on single expert evaluators.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 29 cited sources checked · 0 supported · 0 flagged
- Findingown data
The commercial marketplace search identified a total of 310 unique apps.
“Our search of commercial marketplace identified a total of 310 unique apps.”
Quote in the paperNot a substantial claim- Rests on an assumption: The search query 'depression' and the two app stores (Canadian Apple App Store and Android Market) capture the relevant available apps for depression.
- Findingown data
The 53 relevant manuscripts mentioned a total of 48 unique apps that met the inclusion/exclusion criteria.
“The 53 manuscripts [9,14,15,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48, 49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65, 66,67,68,69,70,71] identified as relevant to the review made mention of a total of 48 unique apps that met our inclusion/exclusion criteria.”
Quote in the paperNot a substantial claim- Rests on an assumption: The search query 'depression' and the two app stores (Canadian Apple App Store and Android Market) capture the relevant available apps for depression.
- Findingown data
One hundred and four commercial marketplace apps met the inclusion/exclusion criteria.
“One hundred and four of these apps identified in the commercial marketplace meet our inclusion/exclusion criteria.”
Quote in the paperSubstantial claim- Rests on an assumption: The search query 'depression' and the two app stores (Canadian Apple App Store and Android Market) capture the relevant available apps for depression.
- Findingown data
10.26% (n = 12) of the 117 identified apps offer support consistent with evidence-based principles of CBT or BA.
“10.26% (n = 12) of these apps identified through our search offer support that seems to be consistent with evidence-based principles of CBT or BA.”
- Also stated in: § App characteristics · plain
“Twelve of these 117 apps (10.26%) were classified by the reviewers as delivering CBT or BA; these CBT/BA apps were identified in the description by their developers as CBT or BA apps or they seemed to offer CBT or BA based on their general description (Table 3).”
Quote in the paperSubstantial claim- Rests on an assumption: App store descriptions and manuscript descriptions accurately represent the type of support an app offers.
- Also stated in: § App characteristics · plain
- Findingown data
Fifty-three of the 160 full-text manuscripts were relevant because all mentioned at least one native app addressed to people with depression.
“Fifty-three out of 160 were relevant for our review because all them mention at least one native app addressed to people for depression.”
Quote in the paperSubstantial claim- Rests on an assumption: The search query 'depression' and the two app stores (Canadian Apple App Store and Android Market) capture the relevant available apps for depression.
- Findingown data
One hundred and seventeen apps were identified after searching both the scientific literature and the commercial market.
“One hundred and seventeen apps have been identified after searching both the scientific literature and the commercial market.”
Quote in the paperSubstantial claim - Findingnone given
The usability of the reviewed apps is highly variable.
“The usability of reviewed apps is highly variable”
Quote in the paperNot a substantial claim- Rests on an assumption: The usability expert's ratings are valid.
- Findingnone given
The reviewed apps rarely are accompanied by explicit privacy or safety policies.
“they rarely are accompanied by explicit privacy or safety policies.”
Quote in the paperSubstantial claim - Findingown data
No relationship was found between the level of adherence of the app to the theoretical BA model and the level of adherence with the heuristics usability (rs = 0.30, p = 0.33).
“No relationship was found between the level of adherence of the app to the theoretical CBT or BA model and the level of adherence with the heuristics usability (rs = -0.45, p = 0.13 and rs = 0.30, p = 0.33, respectively).”
Quote in the paperNot a substantial claim - Findingnone given
The Depression CBT Self-Help Guide app has the highest fidelity to CBT models.
“For instance, the Depression CBT Self-Help Guide app has the highest fidely to CBT models, but the low usability score could complicate its use.”
Quote in the paperNot a substantial claim- Rests on an assumption: The evaluation of CBT/BA content by one expert is a valid measure of the apps' fidelity to CBT/BA models.
- Findingown data
There were no studies reported in the scientific literature that determined the benefits of any of the CBT/BA apps.
“Regarding the effectiveness of the apps, there were no studies reported in the scientific literature that determined the benefits of any of these CBT/BA apps.”
Quote in the paperSubstantial claim- Rests on an assumption: Cross-referencing with apps identified in the scientific literature is sufficient to determine whether any efficacy or effectiveness study exists for the included CBT/BA apps.
- Findingown data
The most frequent heuristic violations of these CBT/BA apps were visibility of the system status, and consistency and standards.
“The most frequent heuristic violations of these CBT/BA apps were: visibility of the system status, and consistency and standards.”
Quote in the paperNot a substantial claim- Rests on an assumption: The usability heuristics used are appropriate for evaluating the usability of CBT/BA apps.
- Findingown data
The rest of the BA core ingredients were never completely integrated into the apps.
“and the rest of the core ingredients were never completely integrated into the apps.”
Quote in the paperNot a substantial claim- Rests on an assumption: The expert's ratings of adherence to CBT/BA core ingredients are valid.
- Findingown data
No relationship was found between the level of adherence of the app to the theoretical CBT model and the level of adherence with the heuristics usability (rs = -0.45, p = 0.13).
“No relationship was found between the level of adherence of the app to the theoretical CBT or BA model and the level of adherence with the heuristics usability (rs = -0.45, p = 0.13 and rs = 0.30, p = 0.33, respectively).”
Quote in the paperNot a substantial claim - Findingown data
Users are very rarely provided with a privacy policy in CBT/BA apps.
“When evaluating the integration and infrastructure dimension, we have seen that safety information is not always available in apps, and very rarely are users provided with a privacy policy.”
Quote in the paperSubstantial claim - Findingown data
No relationship was found between the level of adherence of the app to the theoretical CBT model and indicators of popularity (rs = -0.02, p = 0.96).
“Also, no relationship was found between level of adherence of the app to the theoretical models and the indicators of popularity (range = rs = -0.02, p = 0.96 and rs = 0.57, p = 0.18), or between level of adherence of the app with the heuristic usability with the indicators of popularity (range = rs = 0.15, p = 0.68 and rs = 0.59, p = 0.07).”
Quote in the paperNot a substantial claim- Rests on an assumption: The number of downloads, user ratings, and reviews are valid indicators of an app's popularity.
- Findingown data
The core ingredients of CBT included least often were monitoring physical sensations, monitoring behaviors, and conceptualization.
“The core ingredients included least often were: monitoring physical sensations, monitoring behaviors, and conceptualization.”
Quote in the paperNot a substantial claim- Rests on an assumption: The expert's ratings of adherence to CBT/BA core ingredients are valid.
- Findingnone given
The majority of apps that use CBT or BA do not come close to including the core ingredients of a CBT or BA program.
“Of those apps which do use CBT or BA, some apps may provide benefits by partially applying CBT or BA principles, but the majority do not come close to including the core ingredients of a CBT or BA program.”
Quote in the paperSubstantial claim- Rests on an assumption: The core ingredients of CBT and BA used in the evaluation are the necessary components for a CBT or BA program to be effective.
- Rests on an assumption: The evaluation of CBT/BA content by one expert is a valid measure of the apps' fidelity to CBT/BA models.
- Findingown data
The median level of adherence with BA principles among the CBT/BA apps was 18.75% (range = 6.25-25%).
“and the median level of adherence with the BA principles was 18.75% (range = 6.25–25%).”
Quote in the paperNot a substantial claim- Rests on an assumption: The expert's ratings of adherence to CBT/BA core ingredients are valid.
- Findingnone given
Four out of seven Android available CBT/BA apps achieved more than ten thousand downloads.
“The few apps that provide CBT or BA seem to be popular based on the number of downloads, with 4 out of 7 of the Android available apps achieving more than ten thousand downloads.”
Quote in the paperNot a substantial claim - Findingown data
The core ingredients of BA most commonly included were education about depression and depression ratings.
“The core ingredients of BA most commonly included were: education about depression and depression ratings”
Quote in the paperNot a substantial claim- Rests on an assumption: The expert's ratings of adherence to CBT/BA core ingredients are valid.
- Findingown data
The least typical types of self-help support delivered through the 117 apps were tracking (n = 10, 8.55%) and social support (n = 3, 2.56%).
“The least typical types of self-help support delivered through these 117 apps were tracking (n = 10, 8.55%) and social support (n = 3, 2.56%).”
Quote in the paperNot a substantial claim- Rests on an assumption: App store descriptions and manuscript descriptions accurately represent the type of support an app offers.
- Findingown data
No relationship was found between the level of adherence of the app to the theoretical BA model and indicators of popularity (rs = 0.57, p = 0.18).
“Also, no relationship was found between level of adherence of the app to the theoretical models and the indicators of popularity (range = rs = -0.02, p = 0.96 and rs = 0.57, p = 0.18), or between level of adherence of the app with the heuristic usability with the indicators of popularity (range = rs = 0.15, p = 0.68 and rs = 0.59, p = 0.07).”
Quote in the paperNot a substantial claim- Rests on an assumption: The number of downloads, user ratings, and reviews are valid indicators of an app's popularity.
- Findingown data
The best apps from a theoretical perspective were Depression CBT Self-Help Guide and eCBT Mood, meeting 75% and 55% of the qualifying criteria for CBT, respectively.
“The best apps from a theoretical perspective were Depression CBT Self-Help Guide and eCBT Mood meeting 75% and 55% of the qualifying criteria for CBT, respectively.”
Quote in the paperNot a substantial claim- Rests on an assumption: The expert's ratings of adherence to CBT/BA core ingredients are valid.
- Findingown data
No relationship was found between the level of adherence of the app with heuristic usability and indicators of popularity (rs = 0.15, p = 0.68 and rs = 0.59, p = 0.07).
“Also, no relationship was found between level of adherence of the app to the theoretical models and the indicators of popularity (range = rs = -0.02, p = 0.96 and rs = 0.57, p = 0.18), or between level of adherence of the app with the heuristic usability with the indicators of popularity (range = rs = 0.15, p = 0.68 and rs = 0.59, p = 0.07).”
Quote in the paperNot a substantial claim- Rests on an assumption: The number of downloads, user ratings, and reviews are valid indicators of an app's popularity.
- Findingown data
The usability of the available CBT/BA apps is highly variable.
“When evaluating the usability dimension, we have seen that the usability of the available CBT/BA apps is highly variable and likely serves as a barrier to adoption and regular usage for those apps that violate a large number of heuristics.”
Quote in the paperSubstantial claim- Rests on an assumption: The usability heuristics used are appropriate for evaluating the usability of CBT/BA apps.
- Findingcitation
Few of the phone apps designed to assist those with depression available through the commercial market utilize a CBT or BA approach.
“While there are a large number of phone apps designed to assist those with depression available through the commercial market, few of these utilize a CBT or BA approach despite these being the gold standard of first line psychological treatments [72].”
Quote in the paperSubstantial claim - Findingown data
The core ingredients of CBT most commonly included in the CBT/BA apps were education about depression and depression ratings.
“The core ingredients of CBT most commonly included in these CBT/BA apps were: education about depression and depression ratings.”
Quote in the paperNot a substantial claim- Rests on an assumption: The expert's ratings of adherence to CBT/BA core ingredients are valid.
- Findingown data
The median level of adherence with CBT principles among the CBT/BA apps was 15% (range = 0-75%).
“the median level of adherence with the CBT principles was 15% (range = 0–75%)”
Quote in the paperSubstantial claim- Rests on an assumption: The expert's ratings of adherence to CBT/BA core ingredients are valid.
- Findingown data
Five out of the 12 apps (41.66%) provide important safety information during crisis.
“Five out of the 12 apps (41.66%) provide important safety information during crisis.”
Quote in the paperNot a substantial claim - Findingown data
Only the eCBT app and the Depression CBT Self-Help Guide app offer a privacy policy.
“Only the eCBT app and the Depression CBT Self-Help Guide app offer a privacy policy.”
Quote in the paperNot a substantial claim - Findingown data
Thirty-five of the 48 apps identified from the literature were also identified through the commercial marketplace search.
“Thirty-five of these 48 apps were also identified through our search of the commercial marketplace.”
Quote in the paperNot a substantial claim- Rests on an assumption: The search query 'depression' and the two app stores (Canadian Apple App Store and Android Market) capture the relevant available apps for depression.
- Findingown data
The median level of adherence of the CBT/BA apps with the usability heuristics was 83% (range 42-98%).
“The usability heuristic evaluation found that the median level of adherence with the heuristics was 83% (range = 42–98%).”
Quote in the paperNot a substantial claim- Rests on an assumption: The usability heuristics used are appropriate for evaluating the usability of CBT/BA apps.
- Findingreasoning
There is no available information on the effectiveness of the CBT/BA apps.
“After evaluating the usefulness dimension of the CBT/BA apps taking into account the main usefulness criteria of ‘effectiveness’, we can see that there is no available information on effectiveness.”
Quote in the paperSubstantial claim - Findingown data
The rest of the CBT/BA apps presented less than 50% adherence for both the CBT and BA principles.
“The rest of the apps presented less than 50% of adherence for both the CBT and BA principles (see Table 5).”
Quote in the paperNot a substantial claim- Rests on an assumption: The expert's ratings of adherence to CBT/BA core ingredients are valid.
- Findingown data
Safety information is not always available in CBT/BA apps.
“When evaluating the integration and infrastructure dimension, we have seen that safety information is not always available in apps, and very rarely are users provided with a privacy policy.”
Quote in the paperSubstantial claim - Findingown data
The apps with the highest usability ratings were Mood Tools-Depression Aids, Activity Diary, and Depression on Cure-The Free 12 Week Course, scoring 98%, 98%, and 92% respectively.
“The apps associated with highest usability ratings were Mood Tools–Depression Aids, Activity Diary, and Depression on Cure–The Free 12 Week Course scoring 98%, 98%, and 92% respectively.”
Quote in the paperNot a substantial claim- Rests on an assumption: The usability heuristics used are appropriate for evaluating the usability of CBT/BA apps.
- Findingown data
Four apps in English that offer CBT or BA treatment for depression have been studied by researchers and published in scientific papers: Behavioural Activation Scheduling, Get Happy Program, CBT Mobilwork, and Mobilyze.
“We have identified through our systematic review four apps in English that offer CBT or BA treatment for depression and have been studied by researchers and published in scientific papers, the Behavioural Activation Scheduling [50], the Get Happy Program [40], CBT Mobilwork [65] and Mobilyze [45].”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [50] Both F, Cuijpers P, Hoogendoorn M, Klein M. Towards fully automated psychotherapy for adults-BAS-Behavioral Activation Scheduling via web and mobile phone. Healthinf 2010: Proceedings of the Third International Conference on Health Informatics, ed. A. Fred, J. Filipe, and H. Gamboa. 375–380. source
Not checked in the free run.
- cites [40] S Watts, 2013, CBT for depression: a pilot RCT comparing mobile phone vs. computer, BMC Psychiatry source
Not checked in the free run.
- cites [65] J.A. Callan, CBT mobilework: A technological solution to improve depression outcomes, Clinical and Translational Science
Not checked in the free run.
- Premisecitation
The number of apps intended to help people cope with depression is increasing rapidly, especially in the commercial marketplace.
“The number of apps intended to help people cope with depression is increasing rapidly, especially in the commercial marketplace [14,15];”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [14] N Shen, 2015, Finding a depression App: A review and content analysis of the depression app marketplace, J Med Internet Res
Not checked in the free run.
- cites [15] J Torous, 2015, Current research trends in the use of smartphone applications for mood disorders, Internet Interv source
Not checked in the free run.
- Study design and conductcitation
Inter-rater agreement at the second level of screening was substantial (kappa = 0.85).
“With substantial levels of agreement observed at this second level of screening (kappa = 0.85) [17], the remaining full-text articles were reviewed by only one reviewer (SR).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [17] JR Landis, 1977, The measurement of observer agreement for categorical data, Biometrics source
Not checked in the free run.
- Premisenone given
There are more Internet-based studies on CBT/BA than on other evidence-based models such as Interpersonal Therapy or Acceptance and Commitment Therapy.
“This type of therapy is suited for digital delivery as demonstrated by the fact that there are more Internet-based studies on CBT/BA than on other evidence-based models (e.g., Interpersonal Therapy or Acceptance and Commitment Therpy).”
Quote in the paperNot a substantial claim - Premisecitation
Depression can be treated successfully with CBT and BA based self-help interventions delivered over the Internet.
“More recent research indicates that depression can be treated successfully with CBT and BA based self-help interventions delivered over the Internet [8,9,10].”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [8] E Van't Hof, 2009, Self-help and Internet-guided interventions in depression and anxiety disorders: a systematic review of meta-analyses, CNS Spectr source
Not checked in the free run.
- cites [9] KH Ly, 2014, Behavioural activation versus mindfulness-based guided self-help treatment administered through a smartphone application: A randomised controlled trial, BMJ Open source
Not checked in the free run.
- cites [10] K Moss, 2012, A self-help behavioral activation treatment for geriatric depressive symptoms, Aging Ment Health source
Not checked in the free run.
- Interpretationcitation
It is unlikely that the traditional face-to-face approach can reach everyone with depression.
“However, with population estimates of Major Depression at 6.7% and even higher for Non-Major Depression [1], it is unlikely that this traditional approach can reach everyone.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [1] RC Kessler, 2005, Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication, Arch Gen Psychiatry source
Not checked in the free run.
- Study design and conductcitation
Inter-rater agreement on inclusion and exclusion criteria at the first level of screening was substantial (kappa = 0.69).
“With substantial levels of agreement (kappa = 0.69) observed [17], the remaining titles and abstracts were screened by only one reviewer (SR).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [17] JR Landis, 1977, The measurement of observer agreement for categorical data, Biometrics source
Not checked in the free run.
- Study design and conductcitation
The core ingredients for CBT and BA were derived by consulting two academic experts and one CBT clinician and by reviewing the literature for CBT and BA models in the treatment of depression.
“The core ingredients for CBT and BA were derived by consulting with two academic experts and one CBT clinician, as well as reviewing the literature for CBT and BA models in the treatment of depression [18,19].”
Quote in the paperSubstantial claim0 of 2 sources checked - Interpretationreasoning
There is a concerning lack of appropriate CBT or BA apps for depression, especially from a clinical and legal point of view.
“Despite the growing public demand, there is a concerning lack of appropiate CBT or BA apps, especially from a clinical and legal point of view.”
Quote in the paperSubstantial claim- Rests on an assumption: App store descriptions and manuscript descriptions accurately represent the type of support an app offers.
- Rests on an assumption: Cross-referencing with apps identified in the scientific literature is sufficient to determine whether any efficacy or effectiveness study exists for the included CBT/BA apps.
- Premisecitation
CBT and BA are the gold standard of first line psychological treatments.
“While there are a large number of phone apps designed to assist those with depression available through the commercial market, few of these utilize a CBT or BA approach despite these being the gold standard of first line psychological treatments [72].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [72] 2004, Depression: management of depression in primary and secondary care, Clinical Guideline
Not checked in the free run.
- Premisecitation
The lack of empirically tested apps identified during this review is consistent with observations in other health fields.
“The lack of empirically tested apps identified during this review is consistent with observations in other health fields [36] and raises concerns about relying on these tools to support treatment for depression.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [36] T Donker, 2013, Smartphones for smarter delivery of mental health programs: a systematic review, J Med Internet Res source
Not checked in the free run.
- Interpretationreasoning
There is no direct evidence demonstrating the efficacy of these CBT/BA apps.
“This means that we do not have any direct evidence demonstrating the efficacy of these CBT/BA apps and consequently we do not have direct scientific proof to support their use.”
Quote in the paperSubstantial claim - Interpretationreasoning
The low usability score of the Depression CBT Self-Help Guide app could complicate its use.
“For instance, the Depression CBT Self-Help Guide app has the highest fidely to CBT models, but the low usability score could complicate its use.”
Quote in the paperNot a substantial claim- Rests on an assumption: The usability heuristics used are appropriate for evaluating the usability of CBT/BA apps.
- Rests on an assumption: The evaluation of CBT/BA content by one expert is a valid measure of the apps' fidelity to CBT/BA models.
- Interpretationreasoning
The lack of fidelity to proven CBT or BA principles could hamper the efficacy of these programs.
“The lack of fidelity to proven CBT or BA principles could hamper the efficacy of these programs.”
Quote in the paperSubstantial claim- Rests on an assumption: The core ingredients of CBT and BA used in the evaluation are the necessary components for a CBT or BA program to be effective.
- Premisecitation
Lack of availability of privacy information seems to be an issue for mental health apps in general.
“This lack of availability of privacy information seems to be an issue for mental health apps in general [21].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [21] A Sunyaev, 2015, Availability and quality of mobile mental health app privacy policies, J Am Med Inform Assoc source
Not checked in the free run.
- Study design and conductown data
These four apps were not included in the full analysis because they are not currently available for download by the public, at least from within Canada.
“However, these four apps have not been included in our full analysis because they are not currently available for download by the public, at least from within Canada.”
Quote in the paperSubstantial claim - Interpretationreasoning
The lack of empirically tested apps raises concerns about relying on these tools to support treatment for depression.
“The lack of empirically tested apps identified during this review is consistent with observations in other health fields [36] and raises concerns about relying on these tools to support treatment for depression.”
Quote in the paperSubstantial claim - Interpretationreasoning
Usability likely serves as a barrier to adoption and regular usage for those CBT/BA apps that violate a large number of heuristics.
“When evaluating the usability dimension, we have seen that the usability of the available CBT/BA apps is highly variable and likely serves as a barrier to adoption and regular usage for those apps that violate a large number of heuristics.”
Quote in the paperSubstantial claim- Rests on an assumption: The usability heuristics used are appropriate for evaluating the usability of CBT/BA apps.
- Classified as probably not a substantial claim. 23
- May combine two claims. 15
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe literature search yielded 2,789 abstracts.
“The literature search yielded 2,789 abstracts”
Quote in the paperNot a substantial claim- Rests on an assumption: The search query 'depression' and the two app stores (Canadian Apple App Store and Android Market) capture the relevant available apps for depression.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataOne hundred and sixty full-text manuscripts were reviewed at the full-text level.
“160 full text manuscripts were reviewed at the full-text level.”
Quote in the paperNot a substantial claim- Rests on an assumption: The search query 'depression' and the two app stores (Canadian Apple App Store and Android Market) capture the relevant available apps for depression.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe 53 manuscripts included at the first screening stage mentioned a total of 253 native apps for people with depression.
“The 53 manuscripts included at this stage mentioned a total of 253 native apps for people with depression.”
Quote in the paperNot a substantial claimMay combine two claims.
FindingcitationAll apps identified through searching the scientific literature were simply cited in reviews and were not evaluated in primary research studies.
“All the apps identified through searching the scientific literature were simply cited in reviews [14]; they were not evaluated in primary research studies.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [14] N Shen, 2015, Finding a depression App: A review and content analysis of the depression app marketplace, J Med Internet Res
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe cost of the CBT/BA apps ranged from $0.00 to $8.99.
“The cost of these CBT/BA apps ranged from $0.00 to $8.99.”
Quote in the paperNot a substantial claim- Rests on an assumption: App store descriptions and manuscript descriptions accurately represent the type of support an app offers.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe Depression Cure: The Free 12 Week Course was not the app that received the highest number of reviews.
“However, this app was not the one that received the highest number of reviews.”
Quote in the paperNot a substantial claim- Rests on an assumption: App store descriptions and manuscript descriptions accurately represent the type of support an app offers.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe iPhone apps that received the highest number of reviews were Anti-depression and MoodTools-Depression Aid, both also available in the Google Play store.
“The iPhone apps that received the highest number of reviews were the Anti-depression and MoodTools–Depression Aid apps, both of them also available for download in the Google Play store.”
Quote in the paperNot a substantial claim- Rests on an assumption: App store descriptions and manuscript descriptions accurately represent the type of support an app offers.
May combine two claims.
Findingown dataThe Depression CBT Self-Help Guide app's privacy policy indicates what information is collected and for what purpose, whether this information is shared with others, but it does not address users control.
“Its privacy policy indicates what information is collected and for what purpose, whether this information is shared with others but it does not address users control.”
Quote in the paperNot a substantial claimMay combine two claims.
FindingreasoningThe few available apps that offer CBT or BA have either not been tested or the results derived from these tests have not been reported in the scientific literature.
“The few available apps that offer CBT or BA have either not been tested or the results derived from these tests have not been reported in the scientific literature.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe most typical types of self-help support delivered through the 117 apps were education (n = 32, 27.35%), diagnostic/screening (n = 30, 25.64%), and state induction (n = 18, 15.38%).
“The most typical type of self-help support delivered through these 117 apps was education (n = 32, 27.35%) and diagnostic/screening support (n = 30, 25.64%), followed by state induction (n = 18, 15.38%).”
Quote in the paperNot a substantial claim- Rests on an assumption: App store descriptions and manuscript descriptions accurately represent the type of support an app offers.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe Depression CBT Self-Help Guide app's privacy policy is available on the developer's homepage but is also available to users after they have downloaded the app.
“This policy is available on the developer’s homepage but is also available to users after they have downloaded the app.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataOf the 12 CBT/BA apps, 5 (41.67%) were available on iOS only and 5 (41.67%) on Android only.
“Five of the 12 CBT/BA apps (41.67%) were available on iOS only and 5 (41.67%) on Android only.”
Quote in the paperNot a substantial claim- Rests on an assumption: App store descriptions and manuscript descriptions accurately represent the type of support an app offers.
May combine two claims.
Findingown dataThe Depression CBT Self-Help Guide and MoodTools-Depression Aid received high user satisfaction ratings (4.2 and 4.3, respectively).
“The Depression CBT Self-Help Guide and The Mood Tools–Depression Aid were those Android apps with the highest number of downloads (i.e., between 100,000 and 500,000 downloads, and between 50,000 and 100,000, respectively) and received high user satisfaction ratings (average satisfaction rates were 4.2 and 4.3, respectively).”
Quote in the paperNot a substantial claim- Rests on an assumption: App store descriptions and manuscript descriptions accurately represent the type of support an app offers.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe iPhone app that received the highest user satisfaction rating was The Depression Cure: The Free 12 Week Course (average satisfaction rating = 4.5).
“The iPhone app that received the highest user satisfaction rating was The Depression Cure: The Free 12 Week Course app (average satisfaction rating = 4.5).”
Quote in the paperNot a substantial claim- Rests on an assumption: App store descriptions and manuscript descriptions accurately represent the type of support an app offers.
May combine two claims.
Findingown dataOf the 117 apps, 36 (30.77%) were available on iOS only, 74 (63.25%) on Android only, and 7 (5.98%) on both platforms.
“Out of the total 117 apps, 36 apps (30.77%) were available on iOS only, 74 (63.25%) were available on Android only, and 7 (5.98%) were available across both platforms.”
Quote in the paperNot a substantial claim- Rests on an assumption: App store descriptions and manuscript descriptions accurately represent the type of support an app offers.
May combine two claims.
Findingown dataThe eCBT app has a brief privacy policy stating that information collected in the app is only accessed by the application on the device and that the developer does not collect any information about the user or the use of the app.
“The eCBTapp has a brief privacy policy that states that the information collected in the app is only accessed by the application on the device and they do not collect any information about the user or the use of the app.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe Depression CBT Self-Help Guide app's privacy policy applies to this app in particular, but also other products of its developer (other apps and its homepage).
“The Depression CBT Self-Help Guide app’s privacy policy applies to this app in particular, but also other products of its developer (other apps and its homepage).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe Depression CBT Self-Help Guide and MoodTools-Depression Aid were the Android apps with the highest number of downloads (100,000-500,000 and 50,000-100,000, respectively).
“The Depression CBT Self-Help Guide and The Mood Tools–Depression Aid were those Android apps with the highest number of downloads (i.e., between 100,000 and 500,000 downloads, and between 50,000 and 100,000, respectively) and received high user satisfaction ratings (average satisfaction rates were 4.2 and 4.3, respectively).”
Quote in the paperNot a substantial claim- Rests on an assumption: App store descriptions and manuscript descriptions accurately represent the type of support an app offers.
Classified as probably not a substantial claim.
PremisecitationSmartphones have been used to facilitate the delivery of healthcare interventions, including treatment of mental health conditions.
“Smartphones have been used to facilitate the delivery of healthcare interventions including treatment of mental health conditions [13].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [13] Michael S. Application of smartphone technology in the management and treatment of mental illnesses. MPH. Thesis, University of Pittsburgh. 2011. Available: <ext-link xmlns:xlink="http://www.w3.org/1999/xlink" ext-link-type="uri" xlink:href="http://d-scholarship.pitt.edu/6783/" xlink:type="simple">http://d-scholarship.pitt.edu/6783/</ext-link>
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataInter-rater agreement when categorizing the apps was 0.92, indicating almost perfect agreement.
“The level of agreement between the two reviewers when categorizing the apps, using the Cohen’s kappa, was 0.92, indicating almost perfect agreement [17].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [17] JR Landis, 1977, The measurement of observer agreement for categorical data, Biometrics source
Not checked in the free run.
May combine two claims.
PremisecitationBoth CBT and BA have meta-analytic level evidence in the treatment of depression.
“Both CBT and BA have meta-analytic level of evidence in the treatment of depression[6,7].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [6] P Cuijpers, 2007, Behavioral activation treatments of depression: a meta-analysis, Clin Psychol Rev source
Not checked in the free run.
- cites [7] AC Butler, 2006, The empirical status of cognitive-behavioral therapy: A review of meta-analyses, Clin Psychol Rev source
Not checked in the free run.
May combine two claims.
Interpretationown dataThe utility of the identified CBT/BA apps is questionable, given the absence of effectiveness/efficacy studies and the low adherence to core CBT/BA ingredients.
“Taking into account the non existence of effectiveness/efficacy studies, and the low level of adherence to the core ingredients of the CBT/BA models, the utility of these CBT/BA apps are questionable.”
Quote in the paperSubstantial claim- Rests on an assumption: Cross-referencing with apps identified in the scientific literature is sufficient to determine whether any efficacy or effectiveness study exists for the included CBT/BA apps.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationPrivacy policy scope and transparency were assessed as done by Sunyaev et al.
“If a privacy policy was available the evaluator assessesed the scope and the level of transparency of the policy as done by Sunyaev et al. [21].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [21] A Sunyaev, 2015, Availability and quality of mobile mental health app privacy policies, J Am Med Inform Assoc source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationUsability was evaluated using a common list of usability heuristics proposed by Nielsen & Mack.
“He evaluated the user interface of each app using a common list of usability heuristics proposed by Nielsen & Mack [20].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [20] J Nielsen, 1994
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationInter-rater agreement at the third level of screening was almost perfect (kappa = 0.92).
“With almost perfect agreement observed at this third level of screening (kappa = 0.92) [17], the remaining apps were reviewed by only one reviewer (SR).”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [17] JR Landis, 1977, The measurement of observer agreement for categorical data, Biometrics source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataInter-rater agreement for commercial marketplace app eligibility was 0.89 using Cohen's Kappa.
“The level of agreement between both independent reviewers using the Cohen’s Kappa was 0.89.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationCBT and BA are accepted evidence-based first-line treatments for depression.
“Cognitive Behavioural Therapy (CBT) and Behavioural Activation (BA) are now an accepted evidence-based first-line treatment for depression [5].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [5] National Collaborating Centre for Mental Health. Depression: The treatment and management of depression in adults. 2010. Available: <ext-link xmlns:xlink="http://www.w3.org/1999/xlink" ext-link-type="uri" xlink:href="https://www.nice.org.uk/guidance/cg90/evidence/cg90-depression-in-adults-full-guidance2" xlink:type="simple">https://www.nice.org.uk/guidance/cg90/evidence/cg90-depression-in-adults-full-guidance2</ext-link>
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationreasoningUse of an expert panel instead of only one expert could have increased the credibility of the conclusions.
“Although expert opinion plays an important role when no research evidence exists, the use of an expert panel instead of only one expert could have increased the credibility of the conclusions.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPrevious reviews have found no relationship between the quality of the apps and consumers reviews or ratings.
“This finding is not surprising; previous reviews have found no relationship between the quality of the apps and consumers reviews or ratings [73,74].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [73] NF BinDhim, 2015, A systematic review of quality assessment methods for smartphone health apps
Not checked in the free run.
- cites [74] Kuehnhausen M, Frost VS. Trusting smartphone apps? To install or not to install, that is the question. Cognitive Methods in Situation Awareness and Decision Support; IEEE International Multi-Disciplinary Conference; 2013 Feb 25–28; San Diego, CA, USA. IEEE; 2013. pp. 30–37. source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThis review was limited to English downloadable apps in Canada and only looked at the two most popular platforms when exploring the commercial market.
“First, this review was limited to English downloadable apps in Canada and only looked at the two most popular platforms when exploring the commercial market.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe evaluation of the CBT and BA apps was based on the opinion of one expert.
“Second, the evaluation of the CBT and BA apps was based on the opinion of one expert.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningInformation available on the app download page can be misleading for judging the app.
“Therefore, users should be careful when using the information available on the app download page to judge the app, since this information can be misleading.”
Quote in the paperSubstantial claim- Rests on an assumption: The number of downloads, user ratings, and reviews are valid indicators of an app's popularity.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationChan et al. suggest evaluating apps on three broad dimensions: usefulness, usability, and integration/infrastructure.
“Chan et al. [16] suggest evaluating apps on three broad dimensions: usefulness, usability, and integration/infrastructure.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [16] S Chan, 2015, Towards a framework for evaluating mobile mental health apps, Telemed J E Health source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationnone givenLack of common constructs, outcome measures, definitions and/or standards for tracking, state induction, diagnostic/screening, and education apps make cross-case comparison of these different types of self-help apps impossible.
“Finally, although it was not the primary goal of this review, the lack of common constructs, outcome measures, definitions and/or standards for tracking, state induction, diagnostic/screening, and education apps make cross-case comparison of these different types of self-help apps impossible.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationChan et al. proposed a framework for patients and health care providers to evaluate existing mental health mobile apps and help them make informed choices about their use.
“Chan et al. [16] have recently proposed a framework that can be used for patients and health care providers to evaluate existing mental health mobile apps and help them make informed choices about their use.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [16] S Chan, 2015, Towards a framework for evaluating mobile mental health apps, Telemed J E Health source
Not checked in the free run.
The paper relies on this without support.
App store descriptions and manuscript descriptions accurately represent the type of support an app offers.
“The app description displayed in the stores and any available description provided in the manuscript was the only information used by the reviewers to base their decisions on which category each app fell into.”
The paper relies on this without support.
The development process, usability, feasibility, and efficacy of apps developed in the commercial marketplace are rarely assessed or reported.
“however the development process, usability, feasibility, and efficacy of these apps developed in the commercial marketplace are rarely assessed or reported.”
The paper relies on this without support.
The quality of available apps for depression had not been the subject of any systematic reviews before this review.
“The quality of the available apps has not been the subject of any systematic reviews, until now.”
The paper relies on this without support.
The search query 'depression' and the two app stores (Canadian Apple App Store and Android Market) capture the relevant available apps for depression.
“The search was restricted to apps available through the two most popular mobile phone platforms, The Canadian Apple App Store and Android Market (Google Play). The search was made in November 2015 using ‘depression’ as the search query.”
The paper relies on this without support.
Cross-referencing with apps identified in the scientific literature is sufficient to determine whether any efficacy or effectiveness study exists for the included CBT/BA apps.
“To evaluate the effectiveness of the apps, we cross-referenced with apps identified in the scientific literature to see whether there was any efficacy or effectiveness study on apps included in the review.”
The paper relies on this without support.
The expert's ratings of adherence to CBT/BA core ingredients are valid.
“an experienced academic CBT clinician (SR) evaluated the apps for their level of fidelity to theoretical CBT and BA principles by exploring what extent the apps included the core ingredients of these models.”
The paper relies on this without support.
The usability expert's ratings are valid.
“a user experience designer (MW), who regulary performs expert reviews on mobile apps and websites, where he applies heuristics and professional experience to evaluate user interfaces and suggest design improvements, evaluated the usability of the apps.”
The paper relies on this without support.
The number of downloads, user ratings, and reviews are valid indicators of an app's popularity.
“Also, no relationship was found between level of adherence of the app to the theoretical models and the indicators of popularity (range = rs = -0.02, p = 0.96 and rs = 0.57, p = 0.18), or between level of adherence of the app with the heuristic usability with the indicators of popularity (range = rs = 0.15, p = 0.68 and rs = 0.59, p = 0.07).”
The paper relies on this without support.
The core ingredients of CBT and BA used in the evaluation are the necessary components for a CBT or BA program to be effective.
“Of those apps which do use CBT or BA, some apps may provide benefits by partially applying CBT or BA principles, but the majority do not come close to including the core ingredients of a CBT or BA program.”
The paper relies on this without support.
The usability heuristics used are appropriate for evaluating the usability of CBT/BA apps.
“The usability heuristic evaluation found that the median level of adherence with the heuristics was 83% (range = 42–98%).”
The paper relies on this without support.
The evaluation of CBT/BA content by one expert is a valid measure of the apps' fidelity to CBT/BA models.
“Second, the evaluation of the CBT and BA apps was based on the opinion of one expert.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A meta-analysis of 117 CBT-based applications revealed that only 12 of them provided support aligned with the evidence-based tenets of CBT.
“However, a meta-analysis of 117 CBT-based applications revealed that only 12 of them provided support aligned with the evidence-based tenets of CBT.46”
From Mixed-methods feasibility outcomes for a novel ACT-based video game ‘ACTing Minds’ to support mental health (2024) · cites this paper as [46]
The Abstract reports that 12 of 117 depression self-help apps seemed consistent with evidence-based CBT or BA principles. The Results (App characteristics) clarify that the 117 were not all CBT-based: only 12 were classified as offering CBT or BA. The source is a systematic review, not a meta-analysis, and its finding includes BA as well as CBT. (LLM, conf=0.89)
Experimental — accuracy not yet measuredEven among apps purporting to be drawn from evidence-based therapies such as cognitive behavioural therapy, only a small percentage actually contain the core principles of those therapeutic traditions.
“Even among those purporting to be drawn from evidence-based therapies such as cognitive behavioural therapy, only a small percentage actually contain the core principles of those therapeutic traditions (19).”
From What Works and What Doesn’t Work? A Systematic Review of Digital Mental Health Interventions for Depression and Anxiety in Young People (2019) · cites this paper as [19]
This finding suggests that the observed effectiveness in earlier studies could potentially stem from participants' interaction with a DHI (perhaps as a form of distraction) rather than their proficient implementation of CBT principles.
“This finding suggests that the observed effectiveness in earlier studies could potentially stem from participants’ interaction with a DHI (perhaps as a form of distraction) rather than their proficient implementation of CBT principles.”
From Mixed-methods feasibility outcomes for a novel ACT-based video game ‘ACTing Minds’ to support mental health (2024) · cites this paper as [46]
The Discussion (page unavailable) says the reviewed CBT/BA apps lacked published efficacy evidence, while the Introduction reports that earlier internet-delivered CBT/BA interventions were effective. Neither passage, nor the passages searched for distraction and engagement, addresses whether earlier benefits came from interacting with a digital intervention rather than applying CBT principles. (LLM, conf=0.76)
Experimental — accuracy not yet measuredVery few Internet-based cognitive-behavioral therapy products that claim to be based on cognitive-behavioral therapy actually contain the core elements of cognitive-behavioral therapy.
“For iCBT, very few of the products that claim to be based on cognitive-behavioral therapy (CBT) actually contain the CBT core elements (56).”
From Banbury forum consensus statement on the path forward for digital mental health treatment (2021) · cites this paper as [56]
In the past 5 years, a number of literature reviews have highlighted an increase in research and development activities for eMental healthcare technologies for children and adolescents.
“In the past 5 years, a number of literature reviews have highlighted the increase in research and development activities for eMental healthcare technologies for children and adolescents [18-22].”
From eMental Healthcare Technologies for Anxiety and Depression in Childhood and Adolescence: Systematic Review of Studies Reporting Implementation Outcomes (2018) · cites this paper as [20]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Privacy-policy description conflicts with Table 7
MinorThe text says the Depression CBT Self-Help Guide policy does not address user control, while Table 7 marks its user-control category as addressed.
“it does not address users control.”
“| Depression CBT Self-Help Guide | ✓ | This app plus other apps created by the developer, and the homepage | ✓ | ✓ | ✓ | ✓ | ✗ | |”
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148 · see citing works
Kessler, R. C., Chiu, W. T., Demler, O., & Walters, E. E. (2005) · Archives of General Psychiatry · doi:10.1001/archpsyc.62.6.617 · Cited by 12,052 · see citing works
The empirical status of cognitive-behavioral therapy: A review of meta-analyses
Butler, A. C., Chapman, J. E., Forman, E. M., & Beck, A. T. (2006) · Clinical Psychology Review · doi:10.1016/j.cpr.2005.07.003 · Cited by 3,215 · see citing works
Mood disorders in children and adolescents: an epidemiologic perspective
Kessler, R. C., Avenevoli, S., & Ries Merikangas, K. (2001) · Biological Psychiatry · doi:10.1016/s0006-3223(01)01129-5 · Cited by 1,181 · see citing works
Behavioral activation treatments of depression: a meta-analysis.
Cuijpers, P., van Straten, A., & Warmerdam, L. (2007) · Clinical Psychology Review · doi:10.1016/j.cpr.2006.11.001 · Cited by 1,144 · see citing works
Smartphones for smarter delivery of mental health programs: A systematic review
Donker, T., Petrie, K., Proudfoot, J., Clarke, J., Birch, M.-R., & Christensen, H. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2791 · Cited by 1,136 · see citing works
National Collaborating Centre for Mental Health
Veale, D. M. (2006) · Research Portal (King's College London) · Cited by 941 · see citing works
Kessler, R. C. (2012) · Psychiatric Clinics of North America · doi:10.1016/j.psc.2011.11.005 · Cited by 871 · see citing works
The National Collaborating Centre for Mental Health
Kendall, T., & Pilling, S. (2004) · UCL Discovery (University College London) · Cited by 840 · see citing works
The Current State of Cognitive Therapy: A 40-Year Retrospective
Beck, A. T. (2005) · Archives of General Psychiatry · doi:10.1001/archpsyc.62.9.953 · Cited by 816 · see citing works
Saeb, S., Zhang, M., Karr, C. J., Schueller, S. M., Corden, M. E., Kording, K. P., & Mohr, D. C. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4273 · Cited by 782 · see citing works
Harnessing context sensing to develop a mobile intervention for depression
Burns, M. N., Begale, M., Duffecy, J., Gergle, D., Karr, C. J., Giangrande, E., & Mohr, D. C. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1838 · Cited by 646 · cited 2× · see citing works
“We have identified through our systematic review four apps in English that offer CBT or BA treatment for depression and have been studied by researchers and published in scientific papers, the Behavioural Activation Scheduling [50], the Get Happy Program [40], CBT Mobilwork [65] and Mobilyze [45].”
BackgroundMethodMartínez-Pérez, B., de la Torre-Díez, I., & López-Coronado, M. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2600 · Cited by 550 · see citing works
What is behavioral activation?
Kanter, J. W., Manos, R. C., Bowe, W. M., Baruch, D. E., Busch, A. M., & Rusch, L. C. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2010.04.001 · Cited by 515 · see citing works
mHealth: A Mechanism to Deliver More Accessible, More Effective Mental Health Care
Price, M., Yuen, E. K., Goetter, E. M., Herbert, J. D., Forman, E. M., Acierno, R., & Ruggiero, K. J. (2014) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.1855 · Cited by 488 · see citing works
Ben-Zeev, D., Scherer, E. A., Wang, R., Xie, H., & Campbell, A. T. (2015) · Psychiatric Rehabilitation Journal · doi:10.1037/prj0000130 · Cited by 429 · see citing works
Kauer, S. D., Reid, S. C., Crooke, A. H. D., Khor, A., Hearps, S. J. C., Jorm, A. F., Sanci, L., & Patton, G. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1858 · Cited by 404 · see citing works
Availability and quality of mobile health app privacy policies
Sunyaev, A., Dehling, T., Taylor, P. L., & Mandl, K. D. (2015) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002605 · Cited by 360 · see citing works
CBT for depression: a pilot RCT comparing mobile phone vs. computer
Watts, S., Mackenzie, A., Thomas, C., Griskaitis, A., Mewton, L., Williams, A., & Andrews, G. (2013) · BMC Psychiatry · doi:10.1186/1471-244x-13-49 · Cited by 329 · see citing works
Determinants of physicians’ technology acceptance for e-health in ambulatory care
Dünnebeil, S., Sunyaev, A., Blohm, I., Leimeister, J. M., & Krcmar, H. (2012) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2012.02.002 · Cited by 320 · see citing works
Finding a Depression App: A Review and Content Analysis of the Depression App Marketplace
Shen, N., Levitan, M.-J., Johnson, A., Bender, J. L., Hamilton-Page, M., Jadad, A. (. R., & Wiljer, D. (2015) · JMIR mHealth and uHealth · doi:10.2196/mhealth.3713 · Cited by 282 · see citing works
Ly, K. H., Trüschel, A., Jarl, L., Magnusson, S., Windahl, T., Johansson, R., Carlbring, P., & Andersson, G. (2014) · BMJ Open · doi:10.1136/bmjopen-2013-003440 · Cited by 269 · see citing works
Mobile mental health: Review of the emerging field and proof of concept study
Harrison, V., Proudfoot, J., Wee, P. P., Parker, G., Pavlovic, D. H., & Manicavasagar, V. (2011) · Journal of Mental Health · doi:10.3109/09638237.2011.608746 · Cited by 267 · see citing works
Roepke, A. M., Jaffee, S. R., Riffle, O. M., McGonigal, J., Broome, R., & Maxwell, B. (2015) · Games for Health Journal · doi:10.1089/g4h.2014.0046 · Cited by 265 · cited 1× · see citing works
“The 53 manuscripts [9,14,15,22, 23,24, 25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48, 49,50,51,52, 53,54,55, 56,57,58,59,60,61,62,63,64,65, 66,67,68,69,70,71] identified as relevant to the review made mention of a total of 48 unique apps that met our inclusion/exclusion criteria.”
BackgroundProudfoot, J., Clarke, J., Birch, M.-R., Whitton, A. E., Parker, G., Manicavasagar, V., Harrison, V., Christensen, H., & Hadzi-Pavlovic, D. (2013) · BMC Psychiatry · doi:10.1186/1471-244x-13-312 · Cited by 264 · see citing works
Torous, J., Staples, P., Shanahan, M., Lin, C., Peck, P., Keshavan, M., & Onnela, J.-P. (2015) · JMIR Mental Health · doi:10.2196/mental.3889 · Cited by 258 · see citing works
Mobile phones as medical devices in mental disorder treatment: an overview
Gravenhorst, F., Muaremi, A., Bardram, J. E., Grünerbl, A., Mayora, O., Wurzer, G., Frost, M. M., Osmani, V., Bert Arnrich, A., Lukowicz, P., et al. (2014) · Personal and Ubiquitous Computing · doi:10.1007/s00779-014-0829-5 · Cited by 196 · see citing works
Reid, S. C., Kauer, S. D., Hearps, S. J., Crooke, A. H., Khor, A. S., Sanci, L. A., & Patton, G. C. (2011) · BMC Family Practice · doi:10.1186/1471-2296-12-131 · Cited by 192 · see citing works
Agaku, I. T., Adisa, A. O., Ayo-Yusuf, O. A., & Connolly, G. N. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002079 · Cited by 180 · see citing works
Realizing the Potential of Mobile Mental Health: New Methods for New Data in Psychiatry
Torous, J. B., Staples, P. C., & Onnela, J.-P. “. (2015) · Current Psychiatry Reports · doi:10.1007/s11920-015-0602-0 · Cited by 173 · see citing works
Ly, K. H., Topooco, N., Cederlund, H., Wallin, A., Bergström, J., Molander, O., Carlbring, P., & Andersson, G. (2015) · PLOS ONE · doi:10.1371/journal.pone.0126559 · Cited by 171 · see citing works
Strategies for mHealth Research: Lessons from 3 Mobile Intervention Studies
Ben‐Zeev, D., Schueller, S. M., Begale, M. J., Duffecy, J. L., Kane, J. M., & Mohr, D. C. (2014) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-014-0556-2 · Cited by 167 · see citing works
Towards a Framework for Evaluating Mobile Mental Health Apps
Chan, S., Torous, J., Hinton, L., & Yellowlees, P. (2015) · Telemedicine and e-Health · doi:10.1089/tmj.2015.0002 · Cited by 163 · see citing works
Hof, E. V., Cuijpers, P., & Stein, D. J. (2009) · CNS Spectrums · doi:10.1017/s1092852900027279 · Cited by 156 · see citing works
Telemental health: A status update
Aboujaoude, E., Salame, W., & Naim, L. (2015) · World Psychiatry · doi:10.1002/wps.20218 · Cited by 154 · see citing works
Depression screening via a smartphone app: cross-country user characteristics and feasibility
BinDhim, N. F., Shaman, A. M., Trevena, L. J., Basyouni, M. H., Pont, L., & Alhawassi, T. M. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2014-002840 · Cited by 153 · see citing works
A Systematic Review of Quality Assessment Methods for Smartphone Health Apps
BinDhim, N. F., Hawkey, A., & Trevena, L. (2015) · Telemedicine and e-Health · doi:10.1089/tmj.2014.0088 · Cited by 142 · see citing works
Clarke, J., Proudfoot, J., Birch, M.-R., Whitton, A. E., Parker, G., Manicavasagar, V., Harrison, V., Christensen, H., & Hadzi-Pavlovic, D. (2014) · BMC Psychiatry · doi:10.1186/s12888-014-0272-1 · Cited by 138 · see citing works
Proudfoot, J. (2013) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867412471441 · Cited by 126 · see citing works
Current research and trends in the use of smartphone applications for mood disorders
Torous, J., & Powell, A. C. (2015) · Internet Interventions · doi:10.1016/j.invent.2015.03.002 · Cited by 121 · see citing works
Public attitudes toward health information exchange: perceived benefits and concerns.
Dimitropoulos, L. L., Patel, V., Scheffler, S., & Posnack, S. L. (2011) · PubMed · Cited by 111 · see citing works
The smart therapist: A look to the future of smartphones and mHealth technologies in psychotherapy.
Clough, B. A., & Casey, L. M. (2015) · Professional Psychology Research and Practice · doi:10.1037/pro0000011 · Cited by 106 · see citing works
A Mobile Health Application to Predict Postpartum Depression Based on Machine Learning
Jiménez-Serrano, S., Tortajada, S., & Garcia-Gomez, J. M. (2015) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2014.0113 · Cited by 105 · see citing works
Web-Based Interventions for Youth Internalizing Problems: A Systematic Review
Reyes-Portillo, J. A., Mufson, L., Greenhill, L. L., Gould, M. S., Fisher, P. W., Tarlow, N., & Rynn, M. A. (2014) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2014.09.005 · Cited by 88 · see citing works
Juengst, S. B., Graham, K. M., Pulantara, I. W., McCue, M. P., Whyte, E. M., Dicianno, B. E., Parmanto, B., Arenth, P. M., Skidmore, E. R., & Wagner, A. K. (2015) · Brain Injury · doi:10.3109/02699052.2015.1045031 · Cited by 86 · see citing works
Ly, K. H., Janni, E., Wrede, R., Sedem, M., Donker, T., Carlbring, P., & Andersson, G. (2015) · Internet Interventions · doi:10.1016/j.invent.2014.12.002 · Cited by 78 · see citing works
Reid, S. C., Kauer, S. D., Hearps, S. J. C., Crooke, A. H. D., Khor, A. S., Sanci, L. A., & Patton, G. C. (2013) · BMC Family Practice · doi:10.1186/1471-2296-14-84 · Cited by 73 · see citing works
Trusting smartphone Apps? To install or not to install, that is the question
Kuehnhausen, M., & Frost, V. S. (2013) · IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support · doi:10.1109/cogsima.2013.6523820 · Cited by 69 · see citing works
Ly, K. H., Carlbring, P., & Andersson, G. (2012) · Trials · doi:10.1186/1745-6215-13-62 · Cited by 61 · cited 1× · see citing works
“The 53 manuscripts [9,14,15,22, 23,24, 25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48, 49,50,51,52, 53,54,55, 56,57,58,59,60,61,62,63,64,65, 66,67,68,69,70,71] identified as relevant to the review made mention of a total of 48 unique apps that met our inclusion/exclusion criteria.”
BackgroundTecnología digital e intervenciones para la salud mental: oportunidades y barreras
Aguilera, A. (2015) · Arbor · doi:10.3989/arbor.2015.771n1012 · Cited by 60 · see citing works
Karasouli, E., & Adams, A. (2014) · JMIR Mental Health · doi:10.2196/mental.3708 · Cited by 59 · see citing works
Gordon, M., Henderson, R., Holmes, J. H., Wolters, M. K., Bennett, I. M., & the SPIRIT (Stress in Pregnancy: Improving Results with Interactive Technology) Group (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv109 · Cited by 57 · see citing works
Innovative ICT Solutions to Improve Treatment Outcomes for Depression: The ICT4Depression Project
Warmerdam, L., Riper, H. M., Michel, K., Pepijn, V. D. V., Rocha, A., Mario, R. H., Tousset, E., Hugo, S., Andersson, G., & Cuijpers, P. J. (2012) · Studies in health technology and informatics · doi:10.3233/978-1-61499-121-2-339 · Cited by 54 · see citing works
A self-help behavioral activation treatment for geriatric depressive symptoms
Moss, K., Scogin, F., Di Napoli, E., & Presnell, A. (2012) · Aging & Mental Health · doi:10.1080/13607863.2011.651435 · Cited by 51 · see citing works
Rozbroj, T., Lyons, A., Pitts, M. K., Mitchell, A., & Christensen, H. M. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3529 · Cited by 41 · see citing works
Utility of the T2 Mood Tracker Mobile Application Among Army Warrior Transition Unit Service Members
Bush, N. E., Ouellette, G., & Kinn, J. (2014) · Military Medicine · doi:10.7205/milmed-d-14-00271 · Cited by 39 · see citing works
Javelot, H., Spadazzi, A., Weiner, L., Garcia, S., Gentili, C., Kosel, M., & Bertschy, G. (2014) · BioMed Research International · doi:10.1155/2014/104658 · Cited by 34 · see citing works
Watanabe, N., Horikoshi, M., Yamada, M., Shimodera, S., Akechi, T., Miki, K., Inagaki, M., Yonemoto, N., Imai, H., Tajika, A., et al. (2015) · Trials · doi:10.1186/s13063-015-0805-z · Cited by 28 · see citing works
Schaffer, A., Kreindler, D., Reis, C., & Levitt, A. J. (2013) · Journal of Clinical Psychopharmacology · doi:10.1097/jcp.0b013e31829e8359 · Cited by 22 · see citing works
Mobile Apps for Psychiatric Nurses
Elias, B. L., Fogger, S. A., McGuinness, T. M., & D’Alessandro, K. R. (2014) · Journal of Psychosocial Nursing and Mental Health Services · doi:10.3928/02793695-20131126-07 · Cited by 22 · see citing works
Intelligent Mobile Computing to Assist in the Treatment of Depression
Wan, J., Hu, B., Moore, P. T., & Ashford, R. L. (2008) · International Conference on Pervasive Computing and Applications (ICPCA) · doi:10.1109/icpca.2008.4783691 · Cited by 13 · see citing works
Psychologist in a Pocket: Towards Depression Screening on Mobile Phones
Ágila, B. J., Ramos, R. M., Ix, T., Glenda, F.-C. P., & Wehrle, K. (2015) · Studies in health technology and informatics · doi:10.3233/978-1-61499-516-6-153 · Cited by 11 · see citing works
(2010) · Proceedings of the Third International Conference on Health Informatics · doi:10.5220/0002688503750380 · Cited by 10 · see citing works
Addressing Cultural Contexts in the Management of Stress via Narrative and Mobile Technology
Lee, M. D., Kang, X. L., & Hanrahan, N. P. (2014) · Studies in health technology and informatics · doi:10.3233/978-1-61499-401-5-173 · Cited by 9 · see citing works
Global mobile applications for monitoring health
Giridher, T., Wasilewska, A., Wong, J. L., & Rekhi, K. S. (2010) · Proceedings of the International Multiconference on Computer Science and Information Technology · doi:10.1109/imcsit.2010.5679857 · Cited by 6 · see citing works
Application of Smartphone Technology in the Management and Treatment of Mental Illnesses
Susick, M. (2011) · D-Scholarship@Pitt (University of Pittsburgh) · Cited by 5 · see citing works
Kauer, S. D., Reid, S., Crooke, A., Khor, A. S., Patton, G. C., Jorm, A. F., & Jackson, H. J. (2011) · Journal of Adolescent Health · doi:10.1016/j.jadohealth.2010.11.191 · Cited by 4 · see citing works
Focus on Clinical Practice: Improving the Quality of Care
Bowman, M. A., & Neale, A. V. (2012) · The Journal of the American Board of Family Medicine · doi:10.3122/jabfm.2012.03.120068 · Cited by 3 · see citing works
Mental health services for older people
Parish, C. (2000) · Nursing Standard · doi:10.7748/ns.14.29.31.s57 · Cited by 2 · see citing works
P.8.b.001 Secondary prevention in depression, bipolar and addiction via mobile phone
Carmi, L., & Zohar, J. (2014) · European Neuropsychopharmacology · doi:10.1016/s0924-977x(14)71183-6 · Cited by 1 · see citing works
Black-Box: snapshots of daily life Time series analysis in a depressed elderly patient
Dijk, S. V., & Voshaar, R. C. O. (2014) · American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2013.12.058 · Cited by 1 · see citing works
Parish, C. (2015) · Nursing Standard · doi:10.7748/ns.29.31.33.s33 · Cited by 1 · see citing works
Flagged first, then by citations
2 of the 74 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- J.A. Callan, CBT mobilework: A technological solution to improve depression outcomes, Clinical and Translational Science, 4, 120
- M Murugan, Mobile phone based supportive patient care, Indian Journal of Psychiatry, 55, S72
- 2004, Depression: management of depression in primary and secondary care, Clinical Guideline, 23
Cited by
Sorted by citations
Hollis, C., Falconer, C. J., Martin, J. L., Whittington, C., Stockton, S., Glazebrook, C., & Davies, E. B. (2017) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12663 · Cited by 846 · see citing works
Mental Health Mobile Apps for Preadolescents and Adolescents: A Systematic Review.
Grist, R., Porter, J., & Stallard, P. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7332 · Cited by 518 · see citing works
An overview of and recommendations for more accessible digital mental health services
Lattie, E. G., Stiles-Shields, C., & Graham, A. K. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-021-00003-1 · Cited by 446 · see citing works
The Use and Effectiveness of Mobile Apps for Depression: Results From a Fully Remote Clinical Trial
Arean, P. A., Hallgren, K. A., Jordan, J. T., Gazzaley, A., Atkins, D. C., Heagerty, P. J., & Anguera, J. A. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.6482 · Cited by 376 · see citing works
Garrido, S., Millington, C., Cheers, D., Boydell, K., Schubert, E., Meade, T., & Nguyen, Q. V. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00759 · Cited by 400 · see citing works
“Even among those purporting to be drawn from evidence-based therapies such as cognitive behavioural therapy, only a small percentage actually contain the core principles of those therapeutic traditions (19).”
BackgroundPhilippe, T. J., Sikder, N., Jackson, A., Koblanski, M. E., Liow, E., Pilarinos, A., & Vasarhelyi, K. (2022) · JMIR Mental Health · doi:10.2196/35159 · Cited by 436 · see citing works
Consumer Mobile Health Apps: Current State, Barriers, and Future Directions
Kao, C., & Liebovitz, D. M. (2017) · PM&R · doi:10.1016/j.pmrj.2017.02.018 · Cited by 350 · see citing works
Mental health monitoring with multimodal sensing and machine learning: A survey
Garcia-Ceja, E., Riegler, M., Nordgreen, T., Jakobsen, P., Oedegaard, K. J., & Tørresen, J. (2018) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2018.09.003 · Cited by 356 · see citing works
IntelliCare: An Eclectic, Skills-Based App Suite for the Treatment of Depression and Anxiety
Mohr, D. C., Tomasino, K. N., Lattie, E. G., Palac, H. L., Kwasny, M. J., Weingardt, K., Karr, C. J., Kaiser, S. M., Rossom, R. C., Bardsley, L. R., et al. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.6645 · Cited by 301 · see citing works
Using science to sell apps: Evaluation of mental health app store quality claims
Larsen, M. E., Huckvale, K., Nicholas, J., Torous, J., Birrell, L., Li, E., & Reda, B. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0093-1 · Cited by 334 · see citing works
Wang, K., Varma, D. S., & Prosperi, M. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2018.10.006 · Cited by 308 · see citing works
A randomized trial of online single-session interventions for adolescent depression during COVID-19
Schleider, J. L., Mullarkey, M. C., Fox, K. R., Dobias, M. L., Shroff, A., Hart, E. A., & Roulston, C. A. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01235-0 · Cited by 289 · see citing works
Van Ameringen, M., Turna, J., Khalesi, Z., Pullia, K., & Patterson, B. (2017) · Depression and Anxiety · doi:10.1002/da.22657 · Cited by 234 · see citing works
“While the mHealth app literature is limited, several review papers have been published on the topic (Donker et al., 2013; Gee, Griffiths, & Gulliver, 2016; Huguet et al., 2016; Olff, 2015; Singh et al., 2016; Torous & Powell, 2015; Turvey & Roberts, 2015).”
BackgroundBakker, D., & Rickard, N. (2018) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.11.016 · Cited by 231 · see citing works
Hollis, C., Sampson, S., Simons, L., Davies, E. B., Churchill, R., Betton, V., Butler, D., Chapman, K., Easton, K., Gronlund, T. A., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30296-7 · Cited by 186 · see citing works
Lipschitz, J., Miller, C. J., Hogan, T. P., Burdick, K. E., Lippin-Foster, R., Simon, S. R., & Burgess, J. (2019) · JMIR Mental Health · doi:10.2196/11334 · Cited by 183 · see citing works
Reviewing the data security and privacy policies of mobile apps for depression
O'Loughlin, K., Neary, M., Adkins, E. C., & Schueller, S. M. (2019) · Internet Interventions · doi:10.1016/j.invent.2018.12.001 · Cited by 179 · see citing works
Stawarz, K., Preist, C., Tallon, D., Wiles, N., & Coyle, D. (2018) · Journal of Medical Internet Research · doi:10.2196/10120 · Cited by 198 · see citing works
Rohani, D. A., Faurholt-Jepsen, M., Kessing, L. V., & Bardram, J. E. (2017) · doi:10.2196/preprints.9691 · Cited by 195 · see citing works
Lindner, P., Miloff, A., Hamilton, W., Reuterskiöld, L., Andersson, G., Powers, M. B., & Carlbring, P. (2017) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2017.1280843 · Cited by 187 · see citing works
Batra, S., Baker, R., Wang, T., Forma, F., DiBiasi, F., & Peters-Strickland, T. (2017) · Medical Devices: Evidence and Research · doi:10.2147/mder.s144158 · Cited by 169 · see citing works
Pocket Skills: A Conversational Mobile Web App To Support Dialectical Behavioral Therapy
Schroeder, J., Wilkes, C., Rowan, K., Toledo, A., Paradiso, A., Czerwinski, M., Mark, G., & Linehan, M. M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173972 · Cited by 175 · see citing works
Bauer, M., Glenn, T., Geddes, J., Gitlin, M., Grof, P., Kessing, L. V., Monteith, S., Faurholt-Jepsen, M., Severus, E., & Whybrow, P. C. (2020) · International Journal of Bipolar Disorders · doi:10.1186/s40345-019-0164-x · Cited by 163 · see citing works
Scaling evidence-based treatments through digital mental health.
Schueller, S. M., & Torous, J. B. (2020) · American Psychologist · doi:10.1037/amp0000654 · Cited by 174 · see citing works
Smartphone apps for the self-management of low back pain: A systematic review
Machado, G. C., Pinheiro, M. B., Lee, H., Ahmed, O. H., Hendrick, P. A., Williams, C. M., & Kamper, S. J. (2016) · Best Practice & Research Clinical Rheumatology · doi:10.1016/j.berh.2017.04.002 · Cited by 171 · see citing works
Wasil, A. R., Venturo-Conerly, K. E., Shingleton, R. M., & Weisz, J. R. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103498 · Cited by 160 · see citing works
Methods for Evaluating the Content, Usability, and Efficacy of Commercial Mobile Health Apps
Jake-Schoffman, D. E., Silfee, V. J., Waring, M. E., Boudreaux, E. D., Sadasivam, R. S., Mullen, S. P., Carey, J. L., Hayes, R. B., Ding, E. Y., Bennett, G. G., et al. (2017) · JMIR mHealth and uHealth · doi:10.2196/mhealth.8758 · Cited by 165 · see citing works
Kerst, A., Zielasek, J., & Gäebel, W. (2019) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-018-0974-3 · Cited by 158 · see citing works
Rathnayaka, P., Mills, N., Burnett, D., Silva, D. D., Alahakoon, D., & Gray, R. J. (2022) · Sensors · doi:10.3390/s22103653 · Cited by 169 · see citing works
Ramos, G., & Chavira, D. A. (2022) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2019.10.004 · Cited by 143 · see citing works
Security and Privacy of mHealth Applications: A Scoping Review
Nurgalieva, L., O'Callaghan, D., & Doherty, G. (2020) · IEEE Access · doi:10.1109/access.2020.2999934 · Cited by 132 · see citing works
“Studies covered a wide range of mental health conditions , which included stress [3], depression [55], [60], [71], sleep disorders [88], smoking cessation [55], [98], psychosis [4], anxiety [53], and cognitive decline such as dementia [58], [65].”
BackgroundSmartphone apps for the treatment of mental health conditions: status and considerations
Huckvale, K., Nicholas, J., Torous, J. B., & Larsen, M. E. (2020) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2020.04.008 · Cited by 137 · see citing works
Ethical perspectives on recommending digital technology for patients with mental illness
Bauer, M., Glenn, T., Monteith, S., Bauer, R., Whybrow, P. C., & Geddes, J. R. (2017) · International Journal of Bipolar Disorders · doi:10.1186/s40345-017-0073-9 · Cited by 132 · see citing works
Banbury forum consensus statement on the path forward for digital mental health treatment
Mohr, D. C., Azocar, F., Bertagnolli, A., Choudhury, T., Chrisp, P., Frank, R., Harbin, H., Histon, T., Kaysen, D., Nebeker, C., et al. (2021) · Psychiatric Services · doi:10.1176/appi.ps.202000561 · Cited by 115 · see citing works
Suganuma, S., Sakamoto, D., & Shimoyama, H. (2018) · JMIR Mental Health · doi:10.2196/10454 · Cited by 112 · see citing works
Mohr, D. C., Schueller, S. M., Tomasino, K. N., Kaiser, S. M., Alam, N., Karr, C., Vergara, J. L., Gray, E. L., Kwasny, M. J., & Lattie, E. G. (2019) · Journal of Medical Internet Research · doi:10.2196/13609 · Cited by 116 · see citing works
Scott, I. A., Scuffham, P., Gupta, D., Harch, T. M., Borchi, J., & Richards, B. (2018) · Australian Health Review · doi:10.1071/ah18064 · Cited by 120 · see citing works
Stawarz, K., Preist, C., & Coyle, D. (2019) · JMIR Mental Health · doi:10.2196/12546 · Cited by 107 · see citing works
Mobile health app usability and quality rating scales: a systematic review
Azad‐Khaneghah, P., Neubauer, N., Cruz, A. M., & Liu, L. (2020) · Disability and Rehabilitation Assistive Technology · doi:10.1080/17483107.2019.1701103 · Cited by 119 · see citing works
Najm, A., Nikiphorou, E., Kostine, M., Richez, C., Pauling, J. D., Finckh, A., Ritschl, V., Prior, Y., Balážová, P., Stones, S. R., et al. (2019) · RMD Open · doi:10.1136/rmdopen-2019-001014 · Cited by 114 · see citing works
Gamification and serious games in depression care: A systematic mapping study
Dias, L. P. S., Barbosa, J. L. V., & Vianna, H. D. (2018) · Telematics and Informatics · doi:10.1016/j.tele.2017.11.002 · Cited by 112 · see citing works
Understanding People’s Use of and Perspectives on Mood-Tracking Apps: Interview Study
Schueller, S. M., Neary, M., Lai, J., & Epstein, D. A. (2021) · JMIR Mental Health · doi:10.2196/29368 · Cited by 102 · see citing works
Leech, T., Dorstyn, D., Taylor, A. M., & Li, W. (2021) · Children and Youth Services Review · doi:10.1016/j.childyouth.2021.106073 · Cited by 102 · see citing works
Drissi, N., Ouhbi, S., Marques, G., Díez, I. D. L. T., Ghogho, M., & Idrissi, M. A. J. (2020) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2020.0287 · Cited by 102 · see citing works
Mental health apps in a college setting: openness, usage, and attitudes.
Kern, A., Hong, V., Song, J., Lipson, S., & Eisenberg, D. (2018) · mHealth · doi:10.21037/mhealth.2018.06.01 · Cited by 96 · see citing works
Computer-Assisted Cognitive-Behavior Therapy and Mobile Apps for Depression and Anxiety
Wright, J. H., Mishkind, M., Eells, T. D., & Chan, S. R. (2019) · Current Psychiatry Reports · doi:10.1007/s11920-019-1031-2 · Cited by 93 · see citing works
Porras-Segovia, A., Molina-Madueño, R. M., Berrouiguet, S., López-Castroman, J., Barrigón, M. L., Pérez-Rodríguez, M. S., Marco, J. H., Díaz-Oliván, I., de León, S., Courtet, P., et al. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.05.067 · Cited by 85 · see citing works
Hwang, W. J., & Jo, H. H. (2019) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph16214270 · Cited by 86 · see citing works
eHealth in Treatment of Offenders in Forensic Mental Health: A Review of the Current State
Kip, H., Bouman, Y. H. A., Kelders, S. M., & van Gemert-Pijnen, L. J. E. W. C. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00042 · Cited by 89 · see citing works
Wasil, A. R., Gillespie, S., Patel, R., Petre, A., Venturo-Conerly, K. E., Shingleton, R. M., Weisz, J. R., & DeRubeis, R. J. (2020) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000604 · Cited by 89 · see citing works
Howe, E. S., Suh, J., Morshed, M. B., McDuff, D., Rowan, K., Hernandez, J., Abdin, M., Ramos, G., Tran, T., & Czerwinski, M. P. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502027 · Cited by 90 · see citing works
Hilty, D. M., Chan, S. R., Hwang, T., Wong, A., & Bauer, A. M. (2017) · mHealth · doi:10.21037/mhealth.2017.06.02 · Cited by 86 · see citing works
Technology-enhanced human interaction in psychotherapy.
Imel, Z. E., Caperton, D. D., Tanana, M., & Atkins, D. C. (2017) · Journal of Counseling Psychology · doi:10.1037/cou0000213 · Cited by 88 · see citing works
Co-producing digital mental health interventions: A systematic review
Brotherdale, R., Berry, K., Branitsky, A., & Bucci, S. (2024) · Digital Health · doi:10.1177/20552076241239172 · Cited by 81 · see citing works
Marshall, J. M., Dunstan, D. A., & Bartik, W. J. (2019) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867419876700 · Cited by 84 · see citing works
Lindner, P., Hamilton, W., Miloff, A., & Carlbring, P. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00792 · Cited by 85 · see citing works
Functionality of Top-Rated Mobile Apps for Depression: Systematic Search and Evaluation
Qu, C., Sas, C., Roquet, C. D., & Doherty, G. J. (2019) · JMIR Mental Health · doi:10.2196/15321 · Cited by 84 · see citing works
Attention-deficit/ hyperactivity disorder mobile apps: a systematic review
Păsărelu, C. R., Andersson, G., & Dobrean, A. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104133 · Cited by 74 · see citing works
Bakker, D., & Rickard, N. S. (2019) · Australian Psychologist · doi:10.1111/ap.12383 · Cited by 72 · see citing works
Søgaard Neilsen, A., & Wilson, R. L. (2019) · International Journal of Mental Health Nursing · doi:10.1111/inm.12527 · Cited by 71 · see citing works
Bantjes, J. R., Kazdin, A. E., Cuijpers, P. J., Breet, E., Dunn‐Coetzee, M., Davids, C., Stein, D. J., & Kessler, R. C. (2021) · JMIR Mental Health · doi:10.2196/27400 · Cited by 76 · see citing works
Is There an App for That? A Review of Popular Apps for Depression, Anxiety, and Well-Being
Wasil, A. R., Palermo, E. H., Lorenzo‐Luaces, L., & DeRubeis, R. J. (2021) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2021.07.001 · Cited by 80 · see citing works
Toscos, T., Carpenter, M., Drouin, M., Roebuck, A., Kerrigan, C., & Mirro, M. (2018) · Telemedicine and e-Health · doi:10.1089/tmj.2017.0243 · Cited by 57 · see citing works
mHealth Assessment and Intervention of Depression and Anxiety in Older Adults
Grossman, J. T., Frumkin, M., Rodebaugh, T., & Lenze, E. (2020) · Harvard Review of Psychiatry · doi:10.1097/hrp.0000000000000255 · Cited by 45 · see citing works
Rubanovich, C. K., Mohr, D. C., & Schueller, S. M. (2017) · JMIR Mental Health · doi:10.2196/mental.7603 · Cited by 63 · see citing works
Lee, T. T., & Kesselheim, A. S. (2018) · Annals of Internal Medicine · doi:10.7326/m17-2715 · Cited by 62 · see citing works
Depression in multiple sclerosis: Is one approach for its management enough?
Jones, C. D., Motl, R. W., & Sandroff, B. M. (2021) · Multiple Sclerosis and Related Disorders · doi:10.1016/j.msard.2021.102904 · Cited by 63 · see citing works
Steinglass (2017) · doi:10.1002/da.22624 · Cited by 54 · see citing works
Zanella-Calzada, L. A., Galvan-Tejada, C. E., Chávez-Lamas, N. M., Gracia-Cortés, M. D. C., Magallanes‐Quintanar, R., Celaya-Padilla, J. M., Galván-Tejada, J. I., & Gamboa-Rosales, H. (2019) · Diagnostics · doi:10.3390/diagnostics9010008 · Cited by 60 · see citing works
Wozney, L., Newton, A. S., Gehring, N. D., Bennett, K., Huguet, A., Hartling, L., Dyson, M. P., & McGrath, P. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0474-9 · Cited by 58 · see citing works
Self-citationPendse, S. R., Sharma, A., Vashistha, A., Choudhury, M. D., & Kumar, N. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445410 · Cited by 59 · see citing works
Teepe, G. W., Fonseca, A. D., Kleim, B., Jacobson, N. C., Salamanca-Sanabria, A., Car, L. T., Fleisch, E., & Kowatsch, T. (2021) · Journal of Medical Internet Research · doi:10.2196/29412 · Cited by 60 · see citing works
Economides, M., Ranta, K., Nazander, A., Hilgert, O., Goldin, P. R., Raevuori, A., & Forman‐Hoffman, V. L. (2019) · JMIR mhealth and uhealth · doi:10.2196/14284 · Cited by 51 · see citing works
Melbye, S. A., Kessing, L. V., Bardram, J. E., & Faurholt‐Jepsen, M. (2020) · JMIR Mental Health · doi:10.2196/17453 · Cited by 57 · see citing works
Dam, N. T. V., van Vugt, M. K., Vago, D. R., Schmalzl, L., Saron, C. D., Olendzki, A., Meissner, T., Lazar, S. W., Gorchov, J., Fox, K. C. R., et al. (2017) · Perspectives on Psychological Science · doi:10.1177/1745691617727529 · Cited by 56 · see citing works
Martinengo, L., Stona, A., Griva, K., Dazzan, P., Pariante, C. M., von Wangenheim, F., & Car, J. (2021) · Journal of Medical Internet Research · doi:10.2196/27619 · Cited by 57 · see citing works
Bahadori, S., Wainwright, T. W., & Ahmed, O. H. (2018) · Disability and Rehabilitation · doi:10.1080/09638288.2018.1514661 · Cited by 53 · see citing works
Temple, J. L., Hostler, D. P., Martin‐Gill, C., Moore, C. G., Weiss, P. M., Sequeira, D. J., Condle, J. P., Lang, E., Higgins, J. S., & Patterson, P. D. (2018) · Prehospital Emergency Care · doi:10.1080/10903127.2017.1382624 · Cited by 53 · see citing works
MUBS: A Personalized Recommender System for Behavioral Activation in Mental Health
Rohani, D. A., Lopategui, A. Q., Tuxen, N., Faurholt‐Jepsen, M., Kessing, L. V., & Bardram, J. E. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376879 · Cited by 56 · see citing works
Hallgren, K. A., Bauer, A. M., & Atkins, D. C. (2017) · Depression and Anxiety · doi:10.1002/da.22640 · Cited by 45 · see citing works
Wozney, L., McGrath, P. J., Gehring, N. D., Bennett, K., Huguet, A., Hartling, L., Dyson, M. P., Soleimani, A., & Newton, A. S. (2018) · JMIR Mental Health · doi:10.2196/mental.9655 · Cited by 50 · see citing works
Self-citationDrissi, N., Ouhbi, S., Idrissi, M. A. J., & Ghogho, M. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104243 · Cited by 51 · see citing works
Mental Health Apps in Psychiatric Treatment: A Patient Perspective on Real World Technology Usage
Chiauzzi, E., & Newell, A. (2019) · JMIR Mental Health · doi:10.2196/12292 · Cited by 46 · see citing works
Mobile Mental Health: A Review of Applications for Depression Assistance
Teles, A. S., de Moura, I. R., Viana, D., Silva, F., Coutinho, L., Endler, M., & Rabêlo, R. A. L. (2019) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms.2019.00143 · Cited by 47 · see citing works
Mobile app development in health research: pitfalls and solutions
Siegler, A. J., Knox, J., Bauermeister, J. A., Golinkoff, J., Hightow‐Weidman, L., & Scott, H. M. (2021) · mHealth · doi:10.21037/mhealth-19-263 · Cited by 46 · see citing works
Conley, C. S., Raposa, E. B., Bartolotta, K., Broner, S. E., Hareli, M., Forbes, N., Christensen, K. M., & Assink, M. (2022) · JMIR Mental Health · doi:10.2196/34254 · Cited by 49 · see citing works
Cole, T., Robinson, L., Romero, L., & O’Brien, L. (2017) · Journal of Hand Therapy · doi:10.1016/j.jht.2017.11.040 · Cited by 45 · see citing works
Technology and Mental Health: State of the Art for Assessment and Treatment
Harvey, P. D., Depp, C. A., Rizzo, A., Strauss, G. P., Spelber, D., Carpenter, L. L., Kalin, N. H., Krystal, J. H., McDonald, W. M., Nemeroff, C. B., et al. (2022) · American Journal of Psychiatry · doi:10.1176/appi.ajp.21121254 · Cited by 46 · see citing works
Nan, Y., Zhang, J., Nisar, A., Huo, L., Yang, L., Yin, J., Wang, D., Rahman, A., Gao, Y., & Li, X. (2020) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-020-03087-4 · Cited by 44 · see citing works
Baumel, A., Torous, J. B., Edan, S., & Kane, J. M. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.05.011 · Cited by 40 · see citing works
Castro, Ó., Mair, J. L., Salamanca-Sanabria, A., Alattas, A., Keller, R., Zheng, S., Jabir, A. I., Lin, X., Frese, B. F., Lim, C. S., et al. (2023) · Frontiers in Digital Health · doi:10.3389/fdgth.2023.1039171 · Cited by 42 · see citing works
Burchert, S., Kerber, A., Zimmermann, J., & Knaevelsrud, C. (2021) · PLoS ONE · doi:10.1371/journal.pone.0244955 · Cited by 38 · see citing works
Callan, J. A., Jacob, J. D., Siegle, G. J., Dey, A. K., Thase, M. E., Dabbs, A. D., Kazantzis, N., Rotondi, A. J., Tamres, L. K., Slyke, A. V., et al. (2020) · Cognitive Therapy and Research · doi:10.1007/s10608-020-10159-4 · Cited by 38 · see citing works
Lyngs, U., Lukoff, K., Csuka, L., Slovák, P., Kleek, M. G. V., & Shadbolt, N. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102869 · Cited by 38 · see citing works
A gamified app on emotion recognition and anger management for pre-school children
Nicolaidou, I. A., Tozzi, F., & Antoniades, A. (2021) · International Journal of Child-Computer Interaction · doi:10.1016/j.ijcci.2021.100449 · Cited by 37 · see citing works
Social physics: uncovering human behaviour from communication
Bhattacharya, K., & Kaski, K. K. (2018) · Advances in Physics X · doi:10.1080/23746149.2018.1527723 · Cited by 37 · see citing works
Tuerk, P. W., Schaeffer, C. M., Mcguire, J. F., Larsen, M. A., Capobianco, N. B., & Piacentini, J. C. (2019) · Current Psychiatry Reports · doi:10.1007/s11920-019-1092-2 · Cited by 34 · see citing works
Povey, J., Sweet, M., Nagel, T., Lowell, A., Shand, F., Vigona, J., & Dingwall, K. M. (2022) · JMIR Formative Research · doi:10.2196/28342 · Cited by 26 · see citing works
What are you thinking?: Using CBT and Storytelling to Improve Mental Health Among College Students
Hamid, A., Arshad, R., & Shahid, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517603 · Cited by 35 · see citing works
Apps for Depression: Are They Ready to Work?
Porras‐Segovia, A., Díaz-Oliván, I., Gutiérrez-Rojas, L., Dunne, H., Moreno, M., & Baca‐García, E. (2020) · Current Psychiatry Reports · doi:10.1007/s11920-020-1134-9 · Cited by 33 · see citing works
- Authors
- Huguet, A., Rao, S., McGrath, P. J., Wozney, L., Wheaton, M., Conrod, J., & Rozario, S.
- Affiliations
- Dalhousie University (CA)
- Izaak Walton Killam Health Centre (CA)
- Corresponding author
- Huguet, Anna
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-05-02
- Volume
- 11
- Issue
- 5
- Pages
- e0154248
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4852920
- OpenAlex
- W2346033637
- Semantic Scholar
- d6006c7856b940bd31d69ba83edf42f73f09d783
- Cited by
- 328
- References
- 74
- Citations by year
- 2026: 10 · 2025: 16 · 2024: 26 · 2023: 31 · 2022: 38 · 2021: 44 · 2020: 53 · 2019: 40 · 2018: 34 · 2017: 30 · 2016: 6
- FWCI
- 5.96
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- depression, cognitive behavioral therapy, behavioral activation, mobile health apps, self-help apps, digital mental health, app usability
- MeSH terms
- Depression, Humans, Cognitive Behavioral Therapy, Cell Phone, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
