CoChat

Engagement and Effectiveness of a Healthy-Coping Intervention via Chatbot for University Students During the COVID-19 Pandemic: Mixed Methods Proof-of-Concept Study

Open access

Published version of a preprint

The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index; 50 of the 67 sources its claims cite were checked for support.

Abstract

Background: University students are increasingly reporting common mental health problems, such as stress, anxiety, and depression, and they frequently face barriers to seeking psychological support because of stigma, cost, and availability of mental health services. This issue is even more critical in the challenging time of the COVID-19 pandemic. Digital mental health interventions, such as those delivered via chatbots on mobile devices, offer the potential to achieve scalability of healthy-coping interventions by lowering cost and supporting prevention. Objective: The goal of this study was to conduct a proof-of-concept evaluation measuring the engagement and effectiveness of Atena, a psychoeducational chatbot supporting healthy coping with stress and anxiety, among a population of university students. Methods: In a proof-of-concept study, 71 university students were recruited during the COVID-19 pandemic; 68% (48/71) were female, they were all in their first year of university, and their mean age was 20.6 years (SD 2.4). Enrolled students were asked to use the Atena psychoeducational chatbot for 4 weeks (eight sessions; two per week), which provided healthy-coping strategies based on cognitive behavioral therapy, positive psychology, and mindfulness techniques. The intervention program consisted of conversations combined with audiovisual clips delivered via the Atena chatbot. Participants were asked to complete web-based versions of the 7-item Generalized Anxiety Disorder scale (GAD-7), the 10-item Perceived Stress Scale (PSS-10), and the Five-Facet Mindfulness Questionnaire (FFMQ) at baseline and postintervention to assess effectiveness. They were also asked to complete the User Engagement Scale-Short Form at week 2 to assess engagement with the chatbot and to provide qualitative comments on their overall experience with Atena postintervention. Results: Participants engaged with the Atena chatbot an average of 78 (SD 24.8) times over the study period. A total of 61 out of 71 (86%) participants completed the first 2 weeks of the intervention and provided data on engagement (10/71, 14% attrition). A total of 41 participants out of 71 (58%) completed the full intervention and the postintervention questionnaires (30/71, 42% attrition). Results from the completer analysis showed a significant decrease in anxiety symptoms for participants in more extreme GAD-7 score ranges (t39=0.94; P=.009) and a decrease in stress symptoms as measured by the PSS-10 (t39=2.00; P=.05) for all participants postintervention. Participants also improved significantly in the describing and nonjudging facets, based on their FFMQ subscale scores, and asked for some improvements in the user experience with the chatbot. Conclusions: This study shows the benefit of deploying a digital healthy-coping intervention via a chatbot to support university students experiencing higher levels of distress. While findings collected during the COVID-19 pandemic show promise, further research is required to confirm conclusions. © 2021 JMIR Publications. All rights reserved.

Overview

CoChat Summary

TL;DR

This proof-of-concept mixed-methods study found that a 4-week psychoeducational chatbot (Atena) was engaging for many university students and was associated with reductions in anxiety and stress, especially among students with more severe baseline symptoms. The authors conclude the results are promising but preliminary, with no control group and limited generalizability.

Key findings

  • The study recruited 71 university students; 68% (48/71) were female, all were in their first year, and mean age was 20.6 years (SD 2.4).
  • Engagement was moderate: participants interacted with Atena an average of 78 times (SD 24.8) over 4 weeks; 86% (61/71) completed the week-2 engagement measure, but only 58% (41/71) completed postintervention questionnaires.
  • In completer analyses, anxiety symptoms decreased significantly for participants in the more extreme GAD-7 range (t39=0.94; P=.009), and stress decreased for all participants on the PSS-10 (t39=2.00; P=.05).
  • Mindfulness outcomes improved: the describing and nonjudging facets of the FFMQ increased significantly, and participants’ feedback highlighted the value of videos and mindfulness content, while also requesting UX improvements.

Methods

  • Mixed-methods proof-of-concept study aligned with ORBIT Phase I and Phase IIa; intervention tested over 4 weeks with 8 sessions (2 per week), each about 10 minutes.
  • Sample: 71 first-year university students from the University of Trento, Italy, recruited during the COVID-19 pandemic; all were Italian speakers in northeastern Italy and used Telegram-accessed Atena voluntarily.
  • Main analysis: descriptive statistics, Shapiro tests for normality, paired-samples t tests for pre/post GAD-7, PSS-10, and FFMQ; engagement measured with the UES-SF at week 2; qualitative comments analyzed thematically.

Limitations

  • The paper states that findings are limited by being a preliminary proof-of-concept, involving a homogeneous university-student sample and no control group, which limits generalizability.
  • The authors note that because log data were deidentified, it was difficult to assess how deeply users engaged with each session’s content; they say more objective behavioral monitoring would be preferable.
  • Evident from the text: there was substantial attrition (42% at postintervention), and the intervention was run during the second wave of COVID-19 in Italy, which may have affected participants’ mental well-being and the intervention’s observed effects.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/27965

Cites

43 of 43 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Definition of the healthy-coping behavioral intervention and clinical outcome.
Figure 1 Definition of the healthy-coping behavioral intervention and clinical outcome.CC BY · PubMed Central
Figure 2 Screenshot from session 1 of the Atena chatbot intervention.
Figure 2 Screenshot from session 1 of the Atena chatbot intervention.CC BY · PubMed Central
Figure 3 Overall percentage of completed sessions over the study's 4-week duration.
Figure 3 Overall percentage of completed sessions over the study's 4-week duration.CC BY · PubMed Central

Verification

Claims the paper makes

19 Valid · 2 Not valid · 6 Not checked · 68 Own results and interpretations

25 more claims

Create a free account to see all 95 claims and to verify the 3 references still open.

See all claims — free account
  • Probably not a substantial claim. 39
  • It may state two separate claims as one. 35
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

1 more claim about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 43 references

Research works printed but not linked: 2

  1. Beck, JS, 2011, Cognitive Behavior Therapy: Basics and Beyond. 2nd edition
  2. 10.4172/2167-1044.1000330 · doi:10.4172/2167-1044.1000330

Cited by

Metadata

Authors
Gabrielli, S., Rizzi, S., Bassi, G., Carbone, S., Maimone, R., Marchesoni, M., & Forti, S.
Affiliations
Fondazione Bruno Kessler (IT)
Corresponding author
Gabrielli, Silvia
Venue
JMIR mHealth and uHealth
Published
2021-05-28
Volume
9
Issue
5
Pages
e27965
ISSN
2291-5222 (ISSN-L)
OpenAlex
W3159362230
Cited by
125
References
40
Citations by year
2026: 17 · 2025: 27 · 2024: 29 · 2023: 33 · 2022: 9 · 2021: 10
FWCI
17.98
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR mhealth and uhealth (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Padua Research Archive (University of Padova) (submittedVersion, cc-by)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Social Sciences › Psychology › Clinical Psychology › COVID-19 and Mental Health
  • Social Sciences › Psychology › Developmental and Educational Psychology › Student Stress and Coping
Keywords
chatbots, psychoeducational intervention, university students, COVID-19 pandemic, anxiety, stress, mindfulness, cognitive behavioral therapy, positive psychology, digital mental health, user engagement, attrition, ATENA, mobile health, mental health stigma, first-year students, intervention effectiveness, GAD-7, PSS-10
MeSH terms
COVID-19, SARS-CoV-2, Adaptation, Psychological, Adult, Female, Humans, Male, Students, Universities, Young Adult, Pandemics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.