A Conversational Artificial Intelligence Agent for a Mental Health Care App: Evaluation Study of Its Participatory Design
Open accessPublished version of a preprint
The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index; 8 of the 27 sources its claims cite were checked for support.
Abstract
Background: Mobile apps for mental health are available on the market. Although they seem to be promising for improving the accessibility of mental health care, little is known about their acceptance, design methodology, evaluation, and integration into psychotherapy protocols. This makes it difficult for health care professionals to judge whether these apps may help them and their patients. Objective: Our aim is to describe and evaluate a protocol for the participatory design of mobile apps for mental health. In this study, participants and psychotherapists are engaged in the early phases of the design and development of the app empowered by conversational artificial intelligence (AI). The app supports interventions for stress management training based on cognitive behavioral theory. Methods: A total of 21 participants aged 33-61 years with mild to moderate levels of stress, anxiety, and depression (assessed by administering the Italian versions of the Symptom Checklist-90-Revised, Occupational Stress Indicator, and Perceived Stress Scale) were assigned randomly to 2 groups, A and B. Both groups received stress management training sessions along with cognitive behavioral treatment, but only participants assigned to group A received support through a mobile personal health care agent, designed for mental care and empowered by AI techniques. Psychopathological outcomes were assessed at baseline (T1), after 8 weeks of treatment (T2), and 3 months after treatment (T3). Focus groups with psychotherapists who administered the therapy were held after treatment to collect their impressions and suggestions. Results: Although the intergroup statistical analysis showed that group B participants could rely on better coping strategies, group A participants reported significant improvements in obsessivity and compulsivity and positive distress symptom assessment. The psychotherapists’ acceptance of the protocol was good. In particular, they were in favor of integrating an AI-based mental health app into their practice because they could appreciate the increased engagement of patients in pursuing their therapy goals. Conclusions: The integration into practice of an AI-based mobile app for mental health was shown to be acceptable to both mental health professionals and users. Although it was not possible in this experiment to show that the integration of AI-based conversational technologies into traditional remote psychotherapy significantly decreased the participants’ levels of stress and anxiety, the experimental results showed significant trends of reduction of symptoms in group A and their persistence over time. The mental health professionals involved in the experiment reported interest in, and acceptance of, the proposed technology as a promising tool to be included in a blended model of psychotherapy. ©Morena Danieli, Tommaso Ciulli, Seyed Mahed Mousavi, Giuseppe Riccardi.
Overview
CoChat Summary
TL;DR
This exploratory study evaluated a participatory-design protocol for a conversational AI mobile app (m-PHA/TEO) integrated with CBT-based stress management for work-related stress in 21 Italian workers. The integrated app was acceptable to therapists and users, but the study did not show a clear overall symptom advantage for the app-supported group over CBT-only treatment.
Key findings
- 21 participants aged 33-61 years were randomized to two groups: 11 received SMT-CBT plus the m-PHA app, and 10 received SMT-CBT only.
- In between-group analyses, group A (app support) showed significantly different scores at T1 for obsessivity/compulsivity (mean 62.5 vs 51.3; *P*=.006), at T2 for anxiety (mean 59.3 vs 49.6; *P*=.01), and at T3 for perceived stress (PSS mean 18 vs 10.3; *P*=.04).
- Nonparametric results also found T3 differences favoring group B on coping-related OSI scales: task-oriented (median 6 vs 7; *P*=.004) and involvement (median 6 vs 7.5; *P*=.02).
- Therapists’ acceptance was reported as good; they said the app increased patient engagement and helped patients complete ABC homework.
Methods
- Exploratory, randomized two-group study with repeated assessments at baseline (T1), after 8 weeks of treatment (T2), and 3 months after treatment (T3).
- Participants were recruited in Italy from aging workers with mild to high distress or mild to moderate anxiety/depression; 29 were assessed and 21 were recruited.
- Main analysis used parametric independent t tests / repeated-measures ANOVA and nonparametric Mann–Whitney, Friedman, and Wilcoxon tests on SCL-90-R, PSS, and OSI outcomes; therapists also took part in two post-treatment focus groups.
Limitations
- The paper states that the small number of participants did not allow allocation to more than two experimental groups, and that assigning a control group was not planned because the sample was small.
- The authors note the m-PHA had limited dialog capabilities at the time, aiming mainly to motivate ABC homework rather than sustain extended conversations.
- Evident from the text: the study was exploratory and coincided with the first COVID-19 lockdown in Italy, which the authors say may have influenced anxiety and stress results.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
7 Valid · 1 Not valid · 8 Not checked · 90 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Changing cognitions and thoughts can help to reduce symptoms [25].”
§ A Conversational Artificial Intelligence Agent for a Mental Health Care App: Evaluation Study of Its Participatory Design > Introduction > Objective
What the sources say
- Supportedcites [25] source
The paper’s Abstract states that CBT aims to reduce symptoms by challenging maladaptive cognitions and that strategies to change those cognitions lead to changes in emotional distress. This supports the citing sentence’s qualified claim that changing thoughts can help reduce symptoms. (LLM, conf=0.90)
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“4 meta-analyses showed how CBT performed better than the other interventions in the treatment of occupational stress [25].”
§ A Conversational Artificial Intelligence Agent for a Mental Health Care App: Evaluation Study of Its Participatory Design > Introduction > Objective
What the sources say
- Partly supportedcites [25] sourceExperimental — accuracy not yet measured
The Results > General Stress section says four meta-analyses examined occupational stress and that most found CBT more effective than other intervention types. It also says one of the four did not compare CBT with other interventions, so the claim overstates what all four showed. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“We chose the focus group technique because in the past this method has been found appropriate for evaluating attitudes of health care personnel, among others [19,40].”
§ A Conversational Artificial Intelligence Agent for a Mental Health Care App: Evaluation Study of Its Participatory Design > Results > Qualitative Evaluation of the Intervention
What the sources say
- Could not verifycites [19] sourceExperimental — accuracy not yet measured
The Results (Study Characteristics) discuss control groups, and the Discussion (Principal Findings) mentions clinicians’ attitudes toward digital interventions. Neither passage addresses whether focus groups are appropriate for evaluating health care personnel’s attitudes. Searches for “focus group” and related terms found no passage making that methodological claim. (LLM, conf=0.75)
- Not checked — no full text of the sourcecites [40] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the OSI coping strategies, only the task-oriented and involvement scales at T3 were significantly different.”
§ A Conversational Artificial Intelligence Agent for a Mental Health Care App: Evaluation Study of Its Participatory Design > Results > Parametric Data Analysis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In particular, they were in favor of integrating an AI-based mental health app into their practice because they could appreciate the increased engagement of patients in pursuing their therapy goals.”
§ A Conversational Artificial Intelligence Agent for a Mental Health Care App: Evaluation Study of Its Participatory Design > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
54 more claims
Create a free account to see all 106 claims and to verify the 5 references still open.
- Probably not a substantial claim. 27
- It may state two separate claims as one. 33
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Not confirmedz = -0.65reported p = .66 · recomputed p = .516
“Z=-0.65, p = .66”
- Not confirmedz = -1.72reported p = .16 · recomputed p = .085
“Z=-1.72, p = .16”
- Not confirmedz = -2.25reported p = .03 · recomputed p = .024
“Z=-2.25, p = .03”
- Consistentz = -2.3reported p = .02 · recomputed p = .021
“Z=-2.30, p = .02”
- Consistentz = -2.01reported p = .04 · recomputed p = .044
“Z=-2.01, p = .04”
29 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
A Global Measure of Perceived Stress
Cohen, S., Kamarck, T., & Mermelstein, R. (1983) · Journal of Health and Social Behavior · doi:10.2307/2136404 · Cited by 32,807 · cited 2×
“Description of the Questionnaires for Initial and Final Assessment The tests used for the initial assessment (T1) were the Italian versions of the SCL-90-R [28,29], the Perceived Stress Scale (PSS) [30,31], and the Occupational Stress Indicator (OSI) [32,33].”
(1962) · Medical Entomology and Zoology · Cited by 4,856
The Efficacy of Cognitive Behavioral Therapy: A Review of Meta-analyses
Hofmann, S. G., Asnaani, A., Vonk, I. J. J., Sawyer, A. T., & Fang, A. (2012) · Cognitive Therapy and Research · doi:10.1007/s10608-012-9476-1 · Cited by 3,445 · cited 2×
“Changing cognitions and thoughts can help to reduce symptoms [25].”
Analyzing and Interpreting Data From Likert-Type Scales
Sullivan, G. M., & Artino, A. R. (2013) · Journal of Graduate Medical Education · doi:10.4300/jgme-5-4-18 · Cited by 3,044 · cited 1×
“In addition, by following the suggestions made by an anonymous reviewer and by Sullivan and Artino [38], a parametric independent 2-tailed t test analysis of data was performed.”
Cognitive Therapy and the Emotional Disorders
Ince, L. P. (1977) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.1977.31.4.633 · Cited by 2,892
Rossi, R., Socci, V., Talevi, D., Mensi, S., Niolu, C., Pacitti, F., Di Marco, A., Rossi, A., Siracusano, A., & Di Lorenzo, G. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00790 · Cited by 1,448
Torous, J., Nicholas, J., Larsen, M. E., Firth, J., & Christensen, H. (2018) · Evidence Based Mental Health · doi:10.1136/eb-2018-102891 · Cited by 790 · cited 1×
“However, there is little consensus on the usability and effectiveness of such systems [13].”
An overview of the features of chatbots in mental health: A scoping review
Abd-alrazaq, A. A., Alajlani, M., Alalwan, A. A., Bewick, B. M., Gardner, P., & Househ, M. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.103978 · Cited by 605 · cited 1×
“In the mental health domain, chatbots are often based on scripted or Eliza-style dialogs [6,16].”
Overview of Friedman’s test and post-hoc analysis
Pereira, D. G., Afonso, A., & Medeiros, F. M. (2015) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610918.2014.931971 · Cited by 585 · cited 2×
“Nonparametric Data Analysis A nonparametric analysis of data was performed using the Friedman test (Pereira et al [39]), which allowed us to compare the different results reported in group B at T1, T2, and T3.”
Measuring and improving the quality of mental health care: a global perspective
Kilbourne, A. M., Beck, K., Spaeth‐Rublee, B., Ramanuj, P., O'Brien, R. W., Tomoyasu, N., & Pincus, H. A. (2018) · World Psychiatry · doi:10.1002/wps.20482 · Cited by 457 · cited 1×
“As the requirement for mental health services is widespread [8] and with the current COVID-19 pandemic creating a spike in demand (as stated by the World Health Organization surveys on October 10, 2020 [9,10]), there is a greater awareness of these apps among mental health professionals [11,12].”
E-Mental Health: A Rapid Review of the Literature
Lal, S., & Adair, C. E. (2014) · Psychiatric Services · doi:10.1176/appi.ps.201300009 · Cited by 417 · cited 1×
“As the requirement for mental health services is widespread [8] and with the current COVID-19 pandemic creating a spike in demand (as stated by the World Health Organization surveys on October 10, 2020 [9,10]), there is a greater awareness of these apps among mental health professionals [11,12].”
Montreal Cognitive Assessment (MoCA)-Italian version: regression based norms and equivalent scores
Conti, S., Bonazzi, S., Laiacona, M., Masina, M., & Coralli, M. V. (2015) · Neurological Sciences · doi:10.1007/s10072-014-1921-3 · Cited by 291 · cited 1×
“The Italian version of the MoCA was administered for assessing the absence of mild cognitive impairment [34].”
Insights from user reviews to improve mental health apps
Alqahtani, F., & Orji, R. (2020) · Health Informatics Journal · doi:10.1177/1460458219896492 · Cited by 286 · cited 2×
“Some independent research studies observed that often there is poor engagement from patients in continuing to use the apps after a few attempts [1]; others report concerns from the point of view of security, privacy, and ethical implications [14,15].”
Conversational agents in the treatment of mental health problems: mixed-method systematic review
Gaffney, H., Mansell, W., & Tai, S. (2019) · JMIR Mental Health · doi:10.2196/14166 · Cited by 275 · cited 3×
“In addition, Lim and Penn [18], who studied the potential of the application of digital technology in schizophrenia therapy, have stressed the need for reliable data, and the recent review by Gaffney et al [19] has highlighted the need for relying on unbiased data.”
Blumenthal, J. A., Sherwood, A., Smith, P. J., Watkins, L., Mabe, S., Kraus, W. E., Ingle, K., Miller, P., & Hinderliter, A. (2016) · Circulation · doi:10.1161/circulationaha.115.018926 · Cited by 257 · cited 1×
“The goal of SMT interventions is to reduce the intensity of demands and increase coping resources [26].”
A Mental Health Chatbot for Regulating Emotions (SERMO) - Concept and Usability Test
Denecke, K., Vaaheesan, S., & Arulnathan, A. (2021) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2020.2974478 · Cited by 237 · cited 1×
“Their functionalities range from (1) delivering questionnaires for mood self-monitoring [3,4] and (2) providing recommendations for emotion regulation [5] to (3) engaging users in rule-based interactions [6], sometimes with the support of web-based scripted dialogs [7].”
Security and Privacy Analysis of Mobile Health Applications: The Alarming State of Practice
Papageorgiou, A., Strigkos, M., Politou, E., Alepis, E., Solanas, A., & Patsakis, C. (2018) · IEEE Access · doi:10.1109/access.2018.2799522 · Cited by 236
Bendig, E., Erb, B., Schulze-Thuesing, L., & Baumeister, H. (2022) · Verhaltenstherapie · doi:10.1159/000501812 · Cited by 214 · cited 2×
“Bendig et al [17] have analyzed the results from 10 pilot studies published between 2009 and 2018.”
Anguera, J. A., Jordan, J. T., Castaneda, D., Gazzaley, A., & Areán, P. A. (2016) · BMJ Innovations · doi:10.1136/bmjinnov-2015-000098 · Cited by 172 · cited 1×
“Their functionalities range from (1) delivering questionnaires for mood self-monitoring [3,4] and (2) providing recommendations for emotion regulation [5] to (3) engaging users in rule-based interactions [6], sometimes with the support of web-based scripted dialogs [7].”
Patient safety and job-related stress: A focus group study
Berland, A., Natvig, G. K., & Gundersen, D. (2008) · Intensive and Critical Care Nursing · doi:10.1016/j.iccn.2007.11.001 · Cited by 123
A Decade of Ubiquitous Computing Research in Mental Health
Bardram, J. E., & Matic, A. (2020) · IEEE Pervasive Computing · doi:10.1109/mprv.2019.2925338 · Cited by 83 · cited 2×
“Their functionalities range from (1) delivering questionnaires for mood self-monitoring [3,4] and (2) providing recommendations for emotion regulation [5] to (3) engaging users in rule-based interactions [6], sometimes with the support of web-based scripted dialogs [7].”
Bennion, M. R., Hardy, G. E., Moore, R. K., Kellett, S., & Millings, A. (2020) · Journal of Medical Internet Research · doi:10.2196/16794 · Cited by 68 · cited 1×
“Their functionalities range from (1) delivering questionnaires for mood self-monitoring [3,4] and (2) providing recommendations for emotion regulation [5] to (3) engaging users in rule-based interactions [6], sometimes with the support of web-based scripted dialogs [7].”
Oltean, H.-R., Hyland, P., Vallières, F., & David, D. O. (2017) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465817000133 · Cited by 67 · cited 1×
“The basic theory is that it is not events (A) that directly generate certain emotions but how these events are cognitively processed and evaluated and how irrational or dysfunctional beliefs (B) influence this processing [35-37].”
SCL-90-R Symptom Checklist-90-RAdattamento italiano
Sarno, I., Preti, E., Prunas, A., & Madeddu, F. (2011) · BOA (University of Milano-Bicocca) · Cited by 52
A validation study of the OSI on a blue‐collar sample
Cooper, C. L., & Williams, J. (1991) · Stress Medicine · doi:10.1002/smi.2460070210 · Cited by 43 · cited 1×
“Description of the Questionnaires for Initial and Final Assessment The tests used for the initial assessment (T1) were the Italian versions of the SCL-90-R [28,29], the Perceived Stress Scale (PSS) [30,31], and the Occupational Stress Indicator (OSI) [32,33].”
On the Importance of Coping: A Model and New Directions for Research on Work and Family
Thompson, C. A., Poelmans, S., Allen, T. D., & Andreassi, J. K. (2007) · Research in occupational stress and well being · doi:10.1016/s1479-3555(06)06003-3 · Cited by 42
Current Regulation of Mobile Mental Health Applications.
Armontrout, J. A., Torous, J. B., Cohen, M., McNiel, D. E., & Binder, R. L. (2018) · PubMed · doi:10.29158/jaapl.003748-18 · Cited by 38
Pines, A. M. (2002) · Clinical Case Studies · doi:10.1177/1534650102001002005 · Cited by 32
When REBT Goes Difficult: Applying ABC-DEF to Personality Disorders
Sarracino, D., Dimaggio, G., Ibrahim, R., Popolo, R., Sassaroli, S., & Ruggiero, G. M. (2017) · Journal of Rational-Emotive & Cognitive-Behavior Therapy · doi:10.1007/s10942-016-0258-7 · Cited by 25 · cited 1×
“The basic theory is that it is not events (A) that directly generate certain emotions but how these events are cognitively processed and evaluated and how irrational or dysfunctional beliefs (B) influence this processing [35-37].”
Using Digital Technology in the Treatment of Schizophrenia
Lim, M. H., & Penn, D. L. (2018) · Schizophrenia Bulletin · doi:10.1093/schbul/sby081 · Cited by 23 · cited 1×
“In addition, Lim and Penn [18], who studied the potential of the application of digital technology in schizophrenia therapy, have stressed the need for reliable data, and the recent review by Gaffney et al [19] has highlighted the need for relying on unbiased data.”
What does COVID mean for UK mental health care?
Molodynski, A., McLellan, A., Craig, T., & Bhugra, D. (2020) · International Journal of Social Psychiatry · doi:10.1177/0020764020932592 · Cited by 23 · cited 1×
“As the requirement for mental health services is widespread [8] and with the current COVID-19 pandemic creating a spike in demand (as stated by the World Health Organization surveys on October 10, 2020 [9,10]), there is a greater awareness of these apps among mental health professionals [11,12].”
User’s perception on mental health applications: a qualitative analysis of user reviews
Thach, K. S. (2018) · NAFOSTED Conference on Information and Computer Science (NICS) · doi:10.1109/nics.2018.8606901 · Cited by 22 · cited 1×
“Background During the past 10 years, a multitude of mental health apps have been made available in the market [1,2].”
E-mental health applications for depression: an evidence-based ethical analysis
Rubeis, G. (2021) · European Archives of Psychiatry and Clinical Neuroscience · doi:10.1007/s00406-019-01093-y · Cited by 17 · cited 1×
“Some independent research studies observed that often there is poor engagement from patients in continuing to use the apps after a few attempts [1]; others report concerns from the point of view of security, privacy, and ethical implications [14,15].”
A systems psycho-dynamic perspective on burnout
Cilliers, F. (2003) · SA Journal of Industrial Psychology · doi:10.4102/sajip.v29i4.120 · Cited by 16 · cited 1×
“Although different psychological approaches to the treatment of stress and anxiety offer important insights into the roots of burnout and work-related stress, for example, individual psychology [20] and different declinations of psychodynamic theory [21,22], we chose to integrate the m-PHA support into a protocol for…”
Cognitive Behavioural Therapy Explained
Whitfield, G., & Davidson, A. (2018) · CRC Press eBooks · doi:10.1201/9781315377773 · Cited by 8 · cited 1×
“The basic theory is that it is not events (A) that directly generate certain emotions but how these events are cognitively processed and evaluated and how irrational or dysfunctional beliefs (B) influence this processing [35-37].”
Towards Development of a Mobile Application to Evaluate Mental Health: Systematic Literature Review
Solís-Galván, J. A., Vázquez-Reyes, S., Martínez‐Fierro, M. L., Velasco-Elizondo, P., Veloz, I. G., & Caldera-Villalobos, C. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-63329-5_16 · Cited by 3 · cited 1×
“Their functionalities range from (1) delivering questionnaires for mood self-monitoring [3,4] and (2) providing recommendations for emotion regulation [5] to (3) engaging users in rule-based interactions [6], sometimes with the support of web-based scripted dialogs [7].”
Research works printed but not linked: 6
- Armontrout, J, 2018, J Am Acad Psychiatry Law, 46, 000
- Matheny, K, 2000, J Indiv Psychol, 56, 74
- Thompson, C, 2007, Exploring the Work and Non-work Interface (Research in Occupational Stress and Well Being, Vol. 6), 73
- Derogatis, L, 1977, SCL-90: Administration Scoring and Procedures Manual-i and Other Instruments of the Psychopathology Rating Scale Series
- Cohen, S, 1988, The Social Psychology of Health: The Claremont Symposium on Applied Social Psychology, 31
- Cooper, C, 1988, Occupational Stress Indicator: Management Set
Cited by
Ni, Y., & Jia, F. (2025) · Healthcare · doi:10.3390/healthcare13101205 · Cited by 148
“[48] MH support Conversational AI agent A mental health app design involving participatory protocol with therapists and users for stress management.”
InfluentialDanieli, M., Ciulli, T., Mousavi, S. M., Silvestri, G., Barbato, S., Di Natale, L., & Riccardi, G. (2022) · JMIR Mental Health · doi:10.2196/38067 · Cited by 84
“A growing number of studies have indicated that the development of conversational AI systems (also known as chatbots) as applications in the mental health domain can improve access to mental health care support in an easy and inexpensive manner [8,11,12].”
InfluentialSelf-citationJabir, A. I., Lin, X., Martinengo, L., Sharp, G., Theng, Y.-L., & Tudor Car, L. (2024) · Journal of Medical Internet Research · doi:10.2196/48168 · Cited by 66
Polyportis, A., & Pahos, N. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-023-02464-6 · Cited by 64
Review of artificial intelligence‐based question‐answering systems in healthcare
Budler, L. C., Gosak, L., & Stiglic, G. (2023) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1487 · Cited by 52
Metadata
- Authors
- Danieli, M., Ciulli, T., Mousavi, S. M., & Riccardi, G.
- Affiliations
- University of Trento (IT)
- Corresponding author
- Danieli, Morena
- Venue
- JMIR Formative Research
- Publisher
- JMIR Publications Inc.
- Published
- 2021-12-01
- Volume
- 5
- Issue
- 12
- Pages
- e30053
- ISSN
- 2561-326X (ISSN-L)
- PMC
- PMC8686486
- OpenAlex
- W3200694407
- Cited by
- 64
- References
- 35
- Citations by year
- 2026: 15 · 2025: 23 · 2024: 18 · 2023: 5 · 2022: 3
- FWCI
- 5.84
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Formative Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Iris (University of Trento) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- European Commission (826266, HORIZON2020)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- conversational AI, mental health apps, participatory design, stress management training, cognitive behavioral therapy, obsessive-compulsive symptoms, anxiety and depression, remote psychotherapy, mobile health
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.