CoChat
Sign Up
Cited by 93Digital Mental Health InterventionsArtificial Intelligence in Healthcare and EducationPsychotherapy Techniques and Applications

Using Artificial Intelligence to Enhance Ongoing Psychological Interventions for Emotional Problems in Real- or Close to Real-Time: A Systematic Review

Open access
Gual-Montolio, Patricia, Jaén, Irene, Martínez-Borba, Verónica, Castilla, Diana, Suso-Ribera, Carlos·2022·International Journal of Environmental Research and Public Health
DOI 10.3390/ijerph19137737PMC PMC9266240
Open this paper in CoChat — free accountOpen PDF

Abstract

Emotional disorders are the most common mental disorders globally. Psychological treatments have been found to be useful for a significant number of cases, but up to 40% of patients do not respond to psychotherapy as expected. Artificial intelligence (AI) methods might enhance psychotherapy by providing therapists and patients with real- or close to real-time recommendations according to the patient’s response to treatment. The goal of this investigation is to systematically review the evidence on the use of AI-based methods to enhance outcomes in psychological interventions in real-time or close to real-time. The search included studies indexed in the electronic databases Scopus, Pubmed, Web of Science, and Cochrane Library. The terms used for the electronic search included variations of the words “psychotherapy”, “artificial intelligence”, and “emotional disorders”. From the 85 full texts assessed, only 10 studies met our eligibility criteria. In these, the most frequently used AI technique was conversational AI agents, which are chatbots based on software that can be accessed online with a computer or a smartphone. Overall, the reviewed investigations indicated significant positive consequences of using AI to enhance psychotherapy and reduce clinical symptomatology. Additionally, most studies reported high satisfaction, engagement, and retention rates when implementing AI to enhance psychotherapy in real- or close to real-time. Despite the potential of AI to make interventions more flexible and tailored to patients’ needs, more methodologically robust studies are needed.

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic review identified 10 studies using artificial intelligence (AI) to enhance ongoing psychological interventions for emotional problems in real- or close to real-time, most commonly via conversational AI agents or predictive feedback algorithms. Preliminary evidence suggested positive effects on depressive and anxiety symptoms and generally high satisfaction, but the literature is heterogeneous and methodologically limited.

Key findings

  • From 2059 records and 85 full texts assessed, only 10 studies met eligibility criteria; 5 used conversational AI agents (chatbots) and 5 used AI to predict mood/behavior and provide feedback.
  • Almost all studies reported positive effects: depressive symptoms decreased in pre-post studies (n = 4) and controlled studies (n = 5); anxiety decreased in controlled studies (n = 3) and uncontrolled studies (n = 2); only one study failed to show significant affect change after an AI chatbot intervention.
  • Engagement varied: sample retention ranged from approximately 20% to almost 52%; one study reported participants completed 53.3% of sessions, and another found mobile phone training decreased from a mean of 15.3 episodes (SD = 8.3) in week 1 to 4.8 episodes (SD = 4.6) in week 8.
  • Satisfaction was generally high, with AI users reporting greater satisfaction, learning skills, and therapeutic alliance than users of non-AI programs, though technical problems and limited natural interaction were also reported.

Methods

  • PRISMA-guided systematic review (PROSPERO CRD42022245856) searching Scopus, Pubmed, Web of Science, and Cochrane Library; last search 30 March 2022, using variations of “psychotherapy,” “artificial intelligence,” and “emotional disorders.”
  • Inclusion required an AI method implemented to improve an ongoing psychological treatment for people with emotional problems; exclusion included AI data not used to change treatment during therapy and protocols without results. Two investigators independently screened titles/abstracts and full texts, with consensus and a third reviewer when needed.
  • Quality/risk of bias was assessed with National Heart Lung and Blood Institute tools; Kappa for quality rating was 0.783 (SE = 0.201; 95% CI 0.388, 1.000), and inter-rater agreement was Kappa = 0.936 for screening and Kappa = 0.887 for study selection.

Limitations

  • The paper states that heterogeneity in sample sizes, measures, and methodologies limits generalizability and robustness; most studies tested AI tools within the same sample, limiting external validation; the review could not determine which algorithms or AI techniques performed best; and findings are limited to the authors’ interpretations.
  • The paper also states that no clear distinction was made between human-like AI and calculator AI, and insufficient/heterogeneous data prevented a meta-analytic calculation of pooled effects.
  • Evident from the text: only 10 studies were included, sample sizes ranged from 8 to 1234 participants, most investigations did not report sample size calculations, and risk-of-bias ratings varied (e.g., one controlled intervention study was rated “poor” at 2/14 points, one “fair” at 8/14, and two “good”; one case-control study was “fair” at 7/12).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/ijerph19137737

Cites

83 of 83 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 73 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Most included studies provided only cognitive behavioral therapy (CBT; n = 4).

    “Regarding the treatments offered in the included studies, most studies provided only cognitive behavioral therapy (CBT; n = 4; [82,83,85,86]).”
    Quote in the paperNot a substantial claim
  • Findingcitation

    Both controlled (n = 3) and uncontrolled (n = 2) studies including a measure of anxiety evidenced a reduction in anxiety when using AI.

    “In addition to changes in depression, both controlled (n = 3; [78,80,86]) and uncontrolled (n = 2; [77,82]) studies including a measure of anxiety evidenced a reduction in this symptom when using AI.”
    Quote in the paperSubstantial claim
  • Findingcitation

    Results were inconsistent for marital status and clinical symptomatology as factors for engagement.

    “Inconsistent results were found for marital status and clinical symptomatology.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Most included studies used the Patient Health Questionnaire-9 (n = 7) and the Generalized Anxiety Disorder-7 (n = 3) as primary measures.

    “Most studies included the measures of depression and anxiety, namely the Patient Health Questionnaire-9 (n = 7; [77,78,79,81,83,85,86]) and the Generalized Anxiety Disorder-7 (n = 3; [77,78,86]) as primary measures in their investigations.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Almost all included studies found positive effects when implementing AI to reduce psychopathology.

    “As reported in Table 2, almost all studies found positive effects when implementing AI to reduce psychopathology.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 35
  • May combine two claims. 37
  • The quote only partly states this claim. 1

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The search in four databases generated 2059 studies (PubMed = 266; Web of Science = 546; Scopus = 1119; Cochrane Library = 128).

    “The search in the four databases generated a total of 2059 studies (PubMed = 266; Web of Science = 546; Scopus = 1119; and Cochrane Library = 128).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In terms of design, four included studies were pre-post investigations, four were randomized controlled trials, one was a case-control study, and one was a non-concurrent multiple baseline single group.

    “In terms of design, four studies were pre–post investigations [77,81,82,83], four were randomized controlled trials (RCTs; [78,79,80,86]), one study was a case-control study ([84]), and one was a non-concurrent multiple baseline single group [85].”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Of the 10 included studies, 3 were published in the USA, 2 in the United Kingdom, and 1 each in Germany, Switzerland, Korea, Kenya, and China.

    “Of the 10 studies included in the systematic review, 3 were published in the USA [77,78,79], with the remaining studies being published in the United Kingdom (n = 2; [80,81]), Germany (n = 1; [82]), Switzerland (n = 1; [83]), Korea (n = 1; [84]), Kenya (n = 1; [85]), and China (n = 1; [86]).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    One study found high longitudinal retention rates in the short term (most participants engaged for more than 4 days), whereas three studies evidenced a reduction in engagement over the course of treatment.

    “Regarding continuous assessments retention, a study [81] found high longitudinal retention rates in the short term (most participants were engaged during more than 4 days), whereas three studies [77,79,86] evidenced a reduction in engagement over the course of the treatment.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Findings for remaining outcomes (manic/hypomanic episode severity and duration, overall psychological functioning, stress levels, affect, behavioral changes, constructive coping, and problem resolution) were also positive when implementing AI.

    “The findings in relation to the remaining outcomes, namely the severity and duration of manic/hypomanic episodes severity [84], overall psychological functioning [82], stress levels [79,80], affect [78], behavioral changes [84], constructive coping [79], and problem resolution [80], were also positive when implementing AI.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    Factors that may have contributed to low engagement included having children, problems at work, or illness; certain personality traits; difficulties in interpersonal relationships; poor treatment expectations; lack of university entrance qualifications; and being pregnant or employed.

    “The factors that may have contributed to low engagement according to the included studies were: having children, problems at work, or illness [79], certain personality traits (severe histrionic traits and not obsessive), difficulties on interpersonal relationships, poor treatment expectations, lack of university entrance qualifications [82], and being pregnant or employed [85].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Most studies focused their outcomes on depressive symptoms (n = 10) or anxiety symptoms (n = 5).

    “Most studies focused their outcomes on depressive (n = 10; [77,78,79,80,81,82,83,84,85,86]) or anxiety symptoms (n = 5; [77,78,80,82,86]).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Two included studies targeted volunteers with daily stressors that would impact their emotional status (n = 2).

    “Two studies targeted volunteers with daily stressors that would impact their emotional status (n = 2; [78,80]).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Participants in one study completed 53.3% of sessions.

    “while the participants in other study [77] completed 53.3% of sessions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [77] Burns, M.N., Begale, M., Duffecy, J., Gergle, D., Karr, C.J., Giangrande, E., and Mohr, D.C. (2011). Harnessing Context Sensing to Develop a Mobile Intervention for Depression. J. Med. Internet Res., 13. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    One study revealed that less severe anxiety and depressive symptoms were related with higher drop-out, and another indicated that more severe depressive symptoms were a risk factor for poor engagement.

    “Similarly, results in one study [82] revealed that less severe anxiety and depressive symptoms were related with higher drop-out, and another study indicated that more severe depressive symptoms were in fact a risk factor for poor engagement [85].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [82] Lutz, W., Rubel, J.A., Schwartz, B., Schilling, V., and Deisenhofer, A.K. (2019). Towards Integrating Personalized Feedback Research into Clinical Practice: Development of the Trier Treatment Navigator (TTN). Behav. Res. Ther., 120. source

      Not checked in the free run.

    • cites [85] Green, E.P., Lai, Y., Pearson, N., Rajasekharan, S., Rauws, M., Joerin, A., Kwobah, E., Musyimi, C., Jones, R.M., and Bhat, C. (2020). Expanding Access to Perinatal Depression Treatment in Kenya through Automated Psychological Support: Development and Usability Study. JMIR Form. Res., 4. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Only two studies compared drop-out rates by condition (AI vs random technique recommendations/control group), and both groups showed comparable drop-out rates.

    “Only two studies compared drop-out rates as a function of the condition’s assignment (AI vs Random technique recommendations/control group). The results indicated that both groups showed comparable drop-out rates [79,86]”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [79] Paredes, P., Berkeley, U.C., Gilad-bachrach, R., Czerwinski, M., Roseway, A., and Hernandez, J. (2014, January 20). PopTherapy: Coping with Stress through Pop-Culture. Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare, Oldenburg, Germany. source

      Not checked in the free run.

    • cites [86] Liu, H., Peng, H., Song, X., Xu, C., and Zhang, M. (2022). Using AI Chatbots to Provide Self-Help Depression Interventions for University Students: A Randomized Trial of Effectiveness. Internet Interv., 27. source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    The remaining five reviewed studies implemented AI to predict patient mood based on behavioral and self-reported data and provided feedback to participants to promote behavioral change.

    “The remaining studies (n = 5) implemented AI to predict patient mood based on behavioral and self-reported data and provided feedback to the participants to promote behavioral change (i.e., second type of AI).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Around 60% of participants completed at least one wellness tool in one study.

    “For example, the proportion of participants who completed at least one of the wellness tools proposed by one study [81] was around 60%”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [81] Inkster, B., Sarda, S., and Subramanian, V. (2018). An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods Study. JMIR mHealth uHealth, 6. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Users of AI systems reported greater satisfaction, learning skills, and therapeutic alliance than users with non-AI-based programs such as eBooks, bibliotherapy, or text-based programs.

    “Users of AI systems reported greater satisfaction, learning skills, and therapeutic alliance than users with non-AI-based programs, such as eBooks, bibliotherapy, or text-based programs [78,80,86].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [78] Fulmer, R., Joerin, A., Gentile, B., Lakerink, L., and Rauws, M. (2018). Using Psychological Artificial Intelligence (Tess) to Relieve Symptoms of Depression and Anxiety: Randomized Controlled Trial. JMIR Ment. Health, 5. source

      Not checked in the free run.

    • cites [80] Bird, 2018, Manage Your Life Online: A Web-Based Randomized Controlled Trial Evaluating the Effectiveness of a Problem-Solving Intervention in a Student Sample, Behav. Cogn. Psychother. source

      Not checked in the free run.

    • cites [86] Liu, H., Peng, H., Song, X., Xu, C., and Zhang, M. (2022). Using AI Chatbots to Provide Self-Help Depression Interventions for University Students: A Randomized Trial of Effectiveness. Internet Interv., 27. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Participants generally rated AI interventions to be helpful and satisfactory.

    “As reported in Table 2, the participants generally rated AI interventions to be helpful [81] and satisfactory [77].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [81] Inkster, B., Sarda, S., and Subramanian, V. (2018). An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods Study. JMIR mHealth uHealth, 6. source

      Not checked in the free run.

    • cites [77] Burns, M.N., Begale, M., Duffecy, J., Gergle, D., Karr, C.J., Giangrande, E., and Mohr, D.C. (2011). Harnessing Context Sensing to Develop a Mobile Intervention for Depression. J. Med. Internet Res., 13. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Both pre-post (n = 4) and controlled studies (n = 5) found a reduction in depressive symptoms after using a tool with integrated AI.

    “Both pre-post (n = 4; [77,81,82,83]) and controlled studies (n = 5; [78,79,80,84,86]) found a reduction in depressive symptoms after using a tool with integrated AI.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In general, all the investigations indicated that AI has some potential to enhance psychotherapy and help reduce clinical symptomatology (e.g., depressive symptoms, anxiety symptoms, psychological functioning, and stress, among others) or prevent mood episodes (e.g., manic/hypomanic episodes).

    “An important finding regarding the use of AI for the improvement of ongoing psychotherapy was that, in general, all the investigations indicated that AI has some potential to enhance psychotherapy and help reduce clinical symptomatology (e.g., depressive symptoms, anxiety symptoms, psychological functioning, and stress, among others) or prevent mood episodes (e.g., manic/hypomanic episodes).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    Users had a positive attitude toward the use of AI and trusted this system.

    “In terms of confidence, users had a positive attitude toward the use of AI and trusted this system [85].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [85] Green, E.P., Lai, Y., Pearson, N., Rajasekharan, S., Rauws, M., Joerin, A., Kwobah, E., Musyimi, C., Jones, R.M., and Bhat, C. (2020). Expanding Access to Perinatal Depression Treatment in Kenya through Automated Psychological Support: Development and Usability Study. JMIR Form. Res., 4. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In this systematic review, 85 full-text studies were assessed for eligibility after reviewing more than 1400 titles and abstracts.

    “In this systematic review, 85 full-text studies assessed for eligibility after reviewing more than 1400 titles and abstracts and only 10 studies met our inclusion criteria, which suggests that this is an infrequently researched topic.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Problems associated with AI-based programs included human vs programmed interaction issues such as limitation of non-natural conversations, impersonality, rigidity of response patterns, repetitiveness, irrelevance, perceived lack of specificity, poor interactivity, high simplicity, and misunderstandings.

    “The other referred to human vs programmed interactions and included problems such as the limitation of non-natural conversations, some degree of impersonality, the rigidity of some response patterns, repetitiveness, irrelevance of some interactions, perceived lack of specificity of contents, poor interactivity, high simplicity of interactions, or existence of misunderstandings”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [78] Fulmer, R., Joerin, A., Gentile, B., Lakerink, L., and Rauws, M. (2018). Using Psychological Artificial Intelligence (Tess) to Relieve Symptoms of Depression and Anxiety: Randomized Controlled Trial. JMIR Ment. Health, 5. source

      Not checked in the free run.

    • cites [86] Liu, H., Peng, H., Song, X., Xu, C., and Zhang, M. (2022). Using AI Chatbots to Provide Self-Help Depression Interventions for University Students: A Randomized Trial of Effectiveness. Internet Interv., 27. source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    In some cases, AI was also used to predict patient responses and changes based on patients' data and to provide feedback messages to them with the alerts created to promote a behavioral change (i.e., measurement-based care).

    “In some cases, however, AI was also used to predict patient responses and changes based on patients’ data and to provide feedback messages to them with the alerts created to promote a behavioral change (i.e., measurement-based care).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The four studies classified as before-after studies had a "good" quality, with total scores ranging from 8 to 10 points out of a maximum of 12 points.

    “Overall, the four studies classified as before-after studies had a “good” quality, with total scores ranging from 8 to 10 points out of a maximum of 12 points [77,81,82,83].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [77] Burns, M.N., Begale, M., Duffecy, J., Gergle, D., Karr, C.J., Giangrande, E., and Mohr, D.C. (2011). Harnessing Context Sensing to Develop a Mobile Intervention for Depression. J. Med. Internet Res., 13. source

      Not checked in the free run.

    • cites [81] Inkster, B., Sarda, S., and Subramanian, V. (2018). An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods Study. JMIR mHealth uHealth, 6. source

      Not checked in the free run.

    • cites [82] Lutz, W., Rubel, J.A., Schwartz, B., Schilling, V., and Deisenhofer, A.K. (2019). Towards Integrating Personalized Feedback Research into Clinical Practice: Development of the Trier Treatment Navigator (TTN). Behav. Res. Ther., 120. source

      Not checked in the free run.

    • cites [83] Wahle, F., Kowatsch, T., Fleisch, E., Rufer, M., and Weidt, S. (2016). Mobile Sensing and Support for People with Depression: A Pilot Trial in the Wild. JMIR mHealth uHealth, 4. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    AI users stated that chatbots were friendly, interesting, educational, and interactive.

    “AI users also stated that chatbots were friendly, interesting, educational, and interactive [86].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [86] Liu, H., Peng, H., Song, X., Xu, C., and Zhang, M. (2022). Using AI Chatbots to Provide Self-Help Depression Interventions for University Students: A Randomized Trial of Effectiveness. Internet Interv., 27. source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    The case-control study met only seven criteria of a maximum of 12, so its quality could only be rated as "fair".

    “Thus, it met only seven criteria of a maximum of 12, so its quality could only be rated as “fair”.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [84] Cho, C.H., Lee, T., Lee, J.B., Seo, J.Y., Jee, H.J., Son, S., An, H., Kim, L., and Lee, H.J. (2020). Effectiveness of a Smartphone App with a Wearable Activity Tracker in Preventing the Recurrence of Mood Disorders: Prospective Case-Control Study. JMIR Ment. Health, 7. source

      Not checked in the free run.

  • The quote only partly states this claim.

    Findingown data

    Qualitative flaws in the before-after studies were mostly no blinding of assessors.

    “Qualitative flaws were mostly found in missing power or effect size [77,83] and no blinding of assessors [77,82,83].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [77] Burns, M.N., Begale, M., Duffecy, J., Gergle, D., Karr, C.J., Giangrande, E., and Mohr, D.C. (2011). Harnessing Context Sensing to Develop a Mobile Intervention for Depression. J. Med. Internet Res., 13. source

      Not checked in the free run.

    • cites [82] Lutz, W., Rubel, J.A., Schwartz, B., Schilling, V., and Deisenhofer, A.K. (2019). Towards Integrating Personalized Feedback Research into Clinical Practice: Development of the Trier Treatment Navigator (TTN). Behav. Res. Ther., 120. source

      Not checked in the free run.

    • cites [83] Wahle, F., Kowatsch, T., Fleisch, E., Rufer, M., and Weidt, S. (2016). Mobile Sensing and Support for People with Depression: A Pilot Trial in the Wild. JMIR mHealth uHealth, 4. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Some factors that might negatively impact engagement include having children, problems at work, personality traits, and low treatment expectations.

    “A few investigations, however, indicated a decrease in engagement during the treatment and revealed some factors that might negatively impact engagement (e.g., having children, problems at work, personality traits, and low treatment expectations).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    In general, patients found AI-based psychological programs to be helpful and generally indicated that the AI system met their needs.

    “In general, patients found AI-based psychological programs to be helpful and generally indicated that the AI system met their needs.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    All included studies were independently rated for quality by three reviewers who reviewed papers in pairs.

    “All the studies were independently rated for quality by three reviewers (IJ, PGM, and VMB), who reviewed the quality of papers in pairs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Inter-rater agreement was excellent when screening records (Kappa = 0.936, SE = 0.023; 95% CI [0.891, 0.980]).

    “The inter-rater agreement was excellent both when screening records from the first time (Kappa = 0.936, SE = 0.023; 95% CI = [0.891, 0.980]) and when selecting included studies (Kappa = 0.887, SE = 0.079; 95% CI = [0.733, 1.000]).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    References from included articles were screened by two independent investigators to identify potentially important studies missed in the electronic search.

    “The references from the articles included were also screened by two independent investigators to identify potentially important studies that were not retrieved in the electronic search.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Disagreements in quality rating were resolved through discussion to reach consensus with a fourth reviewer.

    “Disagreements were resolved through discussion to reach consensus with a fourth reviewer (CSR).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The quality and risk of bias of eligible studies were assessed using the NHLBI study quality assessment tools.

    “The quality and risk of bias of the eligible studies was assessed using the study quality assessment tools from the National Heart Lung and Blood Institute (NHLBI, https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools, accessed on 10 April 2022), which includes six types of studies and specific criteria according to the study design”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    This study was retrospectively registered on PROSPERO (CRD42022245856).

    “This study was retrospectively registered on PROSPERO (CRD42022245856) with some adaptations to a preliminary version submitted to the platform that resulted from insights and novel knowledge acquired during the review process.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The use of AI can be especially useful in the context of a self-applied psychological therapy using technology because it could help investigate which ecological momentary interventions worked best for which patients to personalize interventions.

    “The use of AI can be especially useful in the context of a self-applied psychological therapy using technology since it could help investigate which EMIs worked best for which patients to personalize interventions.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Even for evidence-based psychotherapies carried out face-to-face, it is not recommended to adhere strictly to manualized treatment protocols nor to give predetermined responses to patient behaviors.

    “Even for evidence-based psychotherapies carried out face-to-face, it is not recommended to adhere strictly to more or less manualized treatment protocols nor to give predetermined responses to patient behaviors [57].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [57] Gallo, 2012, Factors Involved in Clinician Adoption and Nonadoption of Evidence-based Interventions in Mental Health, Clin. Psychol. Sci. Pract. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Two investigators independently analyzed titles and abstracts to exclude studies unrelated to the review topic.

    “Following the PICOS framework (P = participants; I = interventions; C = comparison; O = outcomes; and S = study design), two investigators independently analyzed the titles and abstracts of the retrieved studies to exclude those unrelated to the review topic.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Study selection was consensus-based, and a third reviewer was included when consensus was not reached.

    “The selection process was consensus-based. When a consensus was not reached, a third reviewer was included.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Inter-rater agreement was excellent when selecting included studies (Kappa = 0.887, SE = 0.079; 95% CI [0.733, 1.000]).

    “The inter-rater agreement was excellent both when screening records from the first time (Kappa = 0.936, SE = 0.023; 95% CI = [0.891, 0.980]) and when selecting included studies (Kappa = 0.887, SE = 0.079; 95% CI = [0.733, 1.000]).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    An approach based on symptom reduction has been shown to be more effective for patients with externalizing problems, while an insight-based therapy is generally more effective for patients with internalizing problems.

    “For example, an approach based on symptom reduction has been shown to be more effective for patients with externalizing problems, while an insight-based therapy is generally more effective for patients with internalizing problems [53], which would indicate that not all psychotherapies will be equally effective across individuals.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [53] Beutler, 2018, Fitting Psychotherapy to Patient Coping Style: A Meta-Analysis, J. Clin. Psychol. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    This systematic review followed the PRISMA guidelines.

    “This systematic review followed the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [76].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [76] Moher, 2009, Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement (Chinese Edition), J. Chin. Integr. Med. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    This is a field that requires more research, replicability, and generalizability to obtain robust findings.

    “Moreover, from the included studies, the sample sizes, study design, psychological interventions delivered, and type of AI technique used clearly varied across studies, which again suggests that this is a field that requires more research, replicability, and generalizability to obtain robust findings.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    All AI applications used might lie within the definition proposed by Russell and Norvig of agents that receive environmental percepts and respond to affect such an environment.

    “While all AI applications used might lie within the definition proposed by Russell and Norvig [88] of agents that receive environmental percepts and respond to affect such an environment, the two approaches used so far clearly differ from one another.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [88] Russell, S., and Norvig, P. (2009). Artificial Intelligence: A Modern Approach, Prentice Hall. [3rd ed.].

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Interpretationnone given

    AI might allow us to rapidly improve and personalize ongoing psychological interventions during the therapeutic process, as well as to enhance patient response to them.

    “The importance of the topic lies in the fact that AI might allow us to rapidly improve and personalize ongoing psychological interventions during the therapeutic process, as well as to enhance patient response to them.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductreasoning

    These findings should be interpreted with caution due to the scarce number of existing investigations, particularly high-quality ones (e.g., controlled studies), the heterogeneity of included populations, and the qualitative flaws found in the blinding procedure, sample size justification, and the overall drop-out of the included studies.

    “While these findings should be interpreted with caution due to the scarce number of existing investigations, particularly high-quality ones (e.g., controlled studies), the heterogeneity of included populations, and the qualitative flaws found in the blinding procedure, sample size justification, and the overall drop-out of the included studies, the results are promising regarding the use of AI to”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    This field requires further development.

    “A reduced number of studies implemented other techniques/algorithms, such as reinforcement learning algorithms, decision trees, random forest algorithm, and support vector machines, again supporting the idea that this field requires further development.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Overall, the reviewed investigations indicated significant positive consequences of using AI to enhance psychotherapy and reduce clinical symptomatology.

    “Overall, the reviewed investigations indicated significant positive consequences of using AI to enhance psychotherapy and reduce clinical symptomatology.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The two approaches used so far clearly differ from one another.

    “While all AI applications used might lie within the definition proposed by Russell and Norvig [88] of agents that receive environmental percepts and respond to affect such an environment, the two approaches used so far clearly differ from one another.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    A recent study proposed that psychotherapy can be supported by computation and claimed that AI should be understood as an additional resource for therapeutic work apart from the existing ones.

    “Congruently, a recent study proposed that psychotherapy can be supported by computation and claimed that AI should be understood as an additional resource for therapeutic work apart from the existing ones [69].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [69] De Mello, F.L., and de Souza, S.A. (2019). Psychotherapy and Artificial Intelligence: A Proposal for Alignment. Front. Psychol., 10. source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    No clear distinction was made in the included studies regarding the type of AI used (human-like AI and calculator AI) and the effect that this could have on its effectiveness because insufficient and too heterogeneous data were available to conduct a meta-analytic calculation of pooled effects.

    “Another limitation is that no clear distinction was made in the included studies regarding the type of AI used (human-like AI and calculator AI) and the effect that this could have on its effectiveness because insufficient and too heterogeneous data were available to conduct a meta-analytic calculation of pooled effects.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    There are some challenges regarding the use of the data from the AI tools used during psychotherapy as it is paramount to protect information from individuals.

    “Additionally, there are some challenges regarding the use of the data from the AI tools used during psychotherapy as it is paramount to protect information from individuals [72].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [72] Kamsteeg, 2021, A Scoping Review of Machine Learning in Psychotherapy Research, Psychother. Res. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    While the presented results are, overall, encouraging, especially to tailor and personalize treatments, more research is required in this field.

    “While the presented results are, overall, encouraging, especially to tailor and personalize treatments, more research is required in this field.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    With the implementation of AI in psychotherapy, protecting information from individuals becomes arduous, so emerging AI methods must be aware of this issue.

    “With the implementation of AI in psychotherapy, however, this task becomes arduous, so emerging AI methods must be aware of this issue.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Due to the heterogeneity of studies, in this review we could not discuss the types of algorithms or AI techniques that yielded the best performances.

    “In addition, due to the heterogeneity of studies, in this review we could not discuss the types of algorithms or AI techniques that yielded the best performances.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Methodologically robust studies are needed.

    “Despite the potential of AI in psychotherapy, this review evidenced that the integration of AI to enhance psychological treatments in real-time is still very rare and methodologically robust studies are needed.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The utility of AI algorithms lies in providing opportunities for tailoring treatments according to patients' needs (physiological states, EMAs, and EMIs) and making current interventions more flexible.

    “The utility of AI algorithms lies in providing opportunities for tailoring treatments according to patients’ needs (physiological states, EMAs, and EMIs) and making current interventions more flexible.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The findings of this review generally support the idea that the information provided by AI tools over the course of psychological interventions might help therapists clarify treatment processes, but also detect and rapidly react to changes in the patient trajectories when delivering psychotherapy.

    “While acknowledging these limitations, the findings of this review generally support the idea that the information provided by AI tools over the course of psychological interventions might help therapists clarify treatment processes, but also detect and rapidly react to changes in the patient trajectories when delivering psychotherapy.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In recent years, extensions and innovations of psychotherapy, such as treatments enhanced by ICT or AI, have been developed to facilitate the dissemination of psychotherapy considering the great number of limitations in existing resources of mental health care (i.e., long waiting lists in public health systems and inadequate resources).

    “In recent years, extensions and innovations of psychotherapy, such as treatments enhanced by ICT or AI, have been developed to facilitate the dissemination of psychotherapy considering the great number of limitations in existing resources of mental health care (i.e., long waiting lists in public health systems and inadequate resources) [15,16,17].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [15] Samele, C., Frew, S., and Urquía, N. (2013). Mental Health Systems in the European Union Member States, Status of Mental Health in Populations and Benefits to Be Expected from Investments into Mental Health, European Commission. Available online: https://ec.europa.eu/health/system/files/2016-11/europopp_full_en_0.pdf.

      Not checked in the free run.

    • cites [16] Purebl, G., Petrea, I., Shields, L., Tóth, M.D., Székely, A., Kurimay, T., McDaid, D., Arensman, E., Granic, I., and Abello, K.M. (2015). Joint Action on Mental Health and Well-Being: Situation Analysis and Recommendations for Action, European Union. Available online: https://ec.europa.eu/health/system/files/2017-07/2017_depression_suicide_ehealth_en_0.pdf.

      Not checked in the free run.

    • cites [17] Barbato, A., Vallarino, M., Rapisarda, F., Lora, A., and Caldas de Almeida, J.M. (2022, April 10). Access to Mental Health Care in Europe. Available online: https://www.researchgate.net/publication/319329050_Access_to_Mental_Health_Care_in_Europe.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    This emerging field does not aim to replace the role of the psychotherapist.

    “Even though AI is a great option to enhance effectiveness and make psychotherapy more accessible, it is crucial to mention that this emerging field does not aim to replace the role of the psychotherapist [70,89,90].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [70] Horn, 2020, Can Artificial Intelligence Improve Psychotherapy Research and Practice?, Adm. Policy Ment. Health Ment. Health Serv. Res. source

      Not checked in the free run.

    • cites [89] Miner, A.S., Shah, N., Bullock, K.D., Arnow, B.A., Bailenson, J., and Hancock, J. (2019). Key Considerations for Incorporating Conversational AI in Psychotherapy. Front. Psychiatry, 10. source

      Not checked in the free run.

    • cites [90] Sedlakova, J., and Trachsel, M. (2022). Conversational Artificial Intelligence in Psychotherapy: A New Therapeutic Tool or Agent?. Am. J. Bioeth., 1–10.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The intention of incorporating AI in the field of psychotherapy is to reach as many people in need as possible (i.e., increase reach), and to provide information on symptom progress over the course of psychological interventions to the therapists so that this information can be used to rapidly detect and react to problems that might occur during interventions.

    “The intention of incorporating AI in the field of psychotherapy is to reach as many people in need as possible (i.e., increase reach [44,70]), and to provide information on symptom progress over the course of psychological interventions to the therapists so that this information can be used to rapidly detect and react to problems that might occur during interventions (e.g., recovery trajectories”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [44] Andersson, 2019, Internet Interventions for Adults with Anxiety and Mood Disorders: A Narrative Umbrella Review of Recent Meta-Analyses, Can. J. Psychiatry source

      Not checked in the free run.

    • cites [70] Horn, 2020, Can Artificial Intelligence Improve Psychotherapy Research and Practice?, Adm. Policy Ment. Health Ment. Health Serv. Res. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    Technological interventions in real-time or close to real-time, such as EMIs supported by AI to may make short-term adjustments during the psychotherapy process, may be an excellent alternative to current rigid psychological interventions.

    “Particularly, technological interventions in real-time or close to real-time, such as EMIs supported by AI to may make short-term adjustments during the psychotherapy process, may be an excellent alternative to current rigid psychological interventions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    AI methods allow us to provide the best treatment for a particular patient with its idiosyncratic/idiographic information at the right time.

    “AI methods allow us to provide the best treatment for a particular patient with its idiosyncratic/idiographic information at the right time.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    While research to date shows some potential in this regard, further investigations are required to support the idea that AI may positively impact the job of psychotherapists by providing real-time or close to real-time information of patient progress and treatment recommendations.

    “However, while research to date shows some potential in this regard, further investigations are required to support the idea that AI may positively impact the job of psychotherapists by providing real-time or close to real-time information of patient progress and treatment recommendations.”
    Quote in the paperSubstantial claim
62 of 62

  • The paper relies on this without support.

    Randomized controlled trials are superior to uncontrolled studies because of their higher internal validity and robustness to explain causal relationships.

    “In particular, sufficiently powered RCTs with a-priori sample size calculations would be preferable because these designs are considered to be superior to uncontrolled studies because of their higher internal validity and robustness to explain causal relationships.”
1 of 1

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • AI can dramatically reduce the cost of offering personalized recommendations while increasing the level of personalization through integration with wearables.

    “AI can dramatically reduce the cost of offering personalized recommendations34,35 while increasing the level of personalization through integration with wearables36.”

    From Impact of digital health interventions on glycemic control and weight management. (2025) · cites this paper as [35]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Study [79] is called randomized despite reportedly lacking randomization

    Minor

    The review classifies study [79] as a randomized controlled trial, but its risk-of-bias assessment says the study did not follow the randomization criterion.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. Barbato, A., Vallarino, M., Rapisarda, F., Lora, A., and Caldas de Almeida, J.M. (2022, April 10). Access to Mental Health Care in Europe. Available online: https://www.researchgate.net/publication/319329050_Access_to_Mental_Health_Care_in_Europe.
  2. Russell, S., and Norvig, P. (2009). Artificial Intelligence: A Modern Approach, Prentice Hall. [3rd ed.].

Cited by

Sorted by citations

Related

Authors
Gual-Montolio, P., Jaén, I., Martínez-Borba, V., Castilla, D., & Suso-Ribera, C.
Affiliations
  • Universitat Jaume I (ES)
  • Department of Basic and Clinical Psychology and Psychobiology, Jaume I University, 12071 Castellon de la Plana, Spain
  • Universitat de València (ES)
ORCID
Venue
International Journal of Environmental Research and Public Health
Publisher
MDPI AG
Published
2022-06-24
Volume
19
Issue
13
Pages
7737
ISSN
1660-4601 (ISSN-L)
DOI
10.3390/ijerph19137737
PMC
PMC9266240
OpenAlex
W4283512721
Semantic Scholar
a7938afe6209165a55b545cc2f970dc7136abf02
Cited by
93
References
83
Citations by year
2026: 23 · 2025: 42 · 2024: 17 · 2023: 11
FWCI
3.11
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Environmental Research and Public Health (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • Repositori UJI (Universitat Jaume I) (submittedVersion, cc-by)
  • Repository of Digital Objects for Teaching Research and Culture (University of Valencia) (submittedVersion, cc-by-nc)
  • Zaguan (Universidad de Zaragoza) (submittedVersion, cc-by)
Funders
Universitat Jaume I (PREDOC-2019/05)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
Keywords
artificial intelligence, psychotherapy, chatbots, emotional disorders, treatment response, patient engagement, treatment retention, symptom relief
MeSH terms
Psychosocial Intervention, Artificial Intelligence, Mental Disorders, Humans, Intelligence, Psychotherapy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review