CoChat

Development and Evaluation of a Smartphone-Based Chatbot Coach to Facilitate a Balanced Lifestyle in Individuals With Headaches (BalanceUP App): Randomized Controlled Trial

Open access
Ulrich, Sandra, Gantenbein, Andreas R, Zuber, Viktor, Von Wyl, Agnes, Kowatsch, Tobias, Künzli, Hansjörg · 2024 · Journal of Medical Internet Research

Published version of a preprint

The DOI matches its Crossref record; 146 of the 146 references it cites were found in a registry or an index; 44 of the 79 sources its claims cite were checked for support.

Abstract

Background: Primary headaches, including migraine and tension-type headaches, are widespread and have a social, physical, mental, and economic impact. Among the key components of treatment are behavior interventions such as lifestyle modification. Scalable conversational agents (CAs) have the potential to deliver behavior interventions at a low threshold. To our knowledge, there is no evidence of behavioral interventions delivered by CAs for the treatment of headaches. Objective: This study has 2 aims. The first aim was to develop and test a smartphone-based coaching intervention (BalanceUP) for people experiencing frequent headaches, delivered by a CA and designed to improve mental well-being using various behavior change techniques. The second aim was to evaluate the effectiveness of BalanceUP by comparing the intervention and waitlist control groups and assess the engagement and acceptance of participants using BalanceUP. Methods: In an unblinded randomized controlled trial, adults with frequent headaches were recruited on the web and in collaboration with experts and allocated to either a CA intervention (BalanceUP) or a control condition. The effects of the treatment on changes in the primary outcome of the study, that is, mental well-being (as measured by the Patient Health Questionnaire Anxiety and Depression Scale), and secondary outcomes (eg, psychosomatic symptoms, stress, headache-related self-efficacy, intention to change behavior, presenteeism and absenteeism, and pain coping) were analyzed using linear mixed models and Cohen d. Primary and secondary outcomes were self-assessed before and after the intervention, and acceptance was assessed after the intervention. Engagement was measured during the intervention using self-reports and usage data. Results: A total of 198 participants (mean age 38.7, SD 12.14 y; n=172, 86.9% women) participated in the study (intervention group: n=110; waitlist control group: n=88). After the intervention, the intention-to-treat analysis revealed evidence for improved well-being (treatment: β estimate=-3.28, 95% CI -5.07 to -1.48) with moderate between-group effects (Cohen d=-0.66, 95% CI -0.99 to -0.33) in favor of the intervention group. We also found evidence of reduced somatic symptoms, perceived stress, and absenteeism and presenteeism, as well as improved headache management self-efficacy, application of behavior change techniques, and pain coping skills, with effects ranging from medium to large (Cohen d=0.43-1.05). Overall, 64.8% (118/182) of the participants used coaching as intended by engaging throughout the coaching and completing the outro. Conclusions: BalanceUP was well accepted, and the results suggest that coaching delivered by a CA can be effective in reducing the burden of people who experience headaches by improving their well-being. Trial Registration: German Clinical Tr i a l s Register DRKS00017422; https://trialsearch.who.int/Trial2.aspx?TrialID=DRKS00017422. © Sandra Ulrich, Andreas R Gantenbein, Viktor Zuber, Agnes Von Wyl, Tobias Kowatsch, Hansjörg Künzli.

Overview

CoChat Summary

## TL;DR

BalanceUP is a fully unguided smartphone chatbot coaching intervention for people with frequent headaches that was tested in an unblinded randomized controlled trial. It improved mental well-being and several secondary outcomes versus waitlist control, and was generally well accepted.

## Key findings

  • The trial enrolled 198 participants (mean age 38.7 years; 86.9% women) and randomized them to intervention (n=110) or waitlist control (n=88).
  • For the primary outcome, the intervention improved mental well-being measured by PHQ-ADS with a medium between-group effect (Cohen d = –0.66, 95% CI –0.99 to –0.33); the intention-to-treat model showed a treatment effect (t342 = –3.6; P < .001).
  • Secondary outcomes showed treatment effects for depression (t342 = –3.56; P < .001), somatic symptoms (t348 = –4.48; P < .001), stress (t341 = –3.07; P = .003), headache-related self-efficacy (t342 = 4.08; P < .001), application of behavior change techniques (t342 = 4.82; P < .001), presenteeism/absenteeism (t317 = –3.00; P = .003), and pain coping; anxiety showed suggestive evidence (t342 = –2.73; P = .007).
  • Engagement and acceptance were high: 64.8% (118/182) used coaching as intended by completing the outro; among those with intended use, 86.3% (101/117) completed all 7 modules, and the app’s overall rating was 3.91/5 (SD 0.67).

## Methods

  • Design: Unblinded, 2-arm randomized controlled trial comparing BalanceUP with a waitlist control; postintervention assessment occurred 24 to 60 days after randomization.
  • Sample/data: Adults (≥18 years) in Switzerland, Germany, and Austria with a smartphone, fluent German, and regular headaches for at least 3 months with a minimum of 4 incidents per month; 198 participants were randomized.
  • Analysis: Primary outcome was mental well-being measured by PHQ-ADS; outcomes were analyzed with linear mixed models in intention-to-treat and per-protocol analyses, with effect sizes reported as Cohen d. Engagement was analyzed using usage data, repeated-measures ANOVA, linear regression, and qualitative content analysis for free-text responses.

## Limitations

  • The paper states that guidelines recommend headache frequency as the primary outcome, but this study used mental well-being instead; the authors note that disability, quality of life, and associated symptoms should be considered in future nonpharmacological interventions.
  • The authors state that self-selection limits generalizability and introduces self-selection bias; they caution that participants may not represent the broader population.
  • The paper also notes a nonrepresentative sample skewed toward women and university-educated participants, a digital placebo effect risk because the trial was unblinded, exclusion of people without smartphones, and technical issues in 28/198 (14.1%) participants.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/50132

Cites

146 of 146 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Screenshots of the BalanceUP app: (A) conversational agent (CA) selection screen, (B) screen displaying a chat with the CA, (C) screen displaying push reminder from the CA, and (D) sidebar of the app.
Figure 1 Screenshots of the BalanceUP app: (A) conversational agent (CA) selection screen, (B) screen displaying a chat with the CA, (C) screen displaying push reminder from the CA, and (D) sidebar of the app.CC BY · PubMed Central
Figure 2 Elements of coaching and engagement.
Figure 2 Elements of coaching and engagement.CC BY · PubMed Central
Figure 3 Structure of modules and units.
Figure 3 Structure of modules and units.CC BY · PubMed Central
Figure 4 Examples of session flow.
Figure 4 Examples of session flow.CC BY · PubMed Central
Figure 5 Study procedure.
Figure 5 Study procedure.CC BY · PubMed Central
Figure 6 Participant flowchart. ITT: intention-to-treat; PP: per-protocol.
Figure 6 Participant flowchart. ITT: intention-to-treat; PP: per-protocol.CC BY · PubMed Central
Figure 7 The thematic map illustrates the positive aspects of BalanceUP. Numbers in parentheses indicate the frequency of mentions for each topic.
Figure 7 The thematic map illustrates the positive aspects of BalanceUP. Numbers in parentheses indicate the frequency of mentions for each topic.CC BY · PubMed Central
Figure 8 The thematic map illustrates suggestions to improve BalanceUP. Numbers in parentheses indicate the frequency of mentions for each topic.
Figure 8 The thematic map illustrates suggestions to improve BalanceUP. Numbers in parentheses indicate the frequency of mentions for each topic.CC BY · PubMed Central

Verification

Claims the paper makes

33 Valid · 4 Not valid · 8 Not checked · 86 Own results and interpretations

61 more claims · 3 with their cited sources checked

Create a free account to see all 131 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 36
  • It may state two separate claims as one. 45

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Research works printed but not linked: 3

  1. Laux, G, 2017, J Neurol Neurosurg Psychiatry, 18, 16
  2. Geisser, E, 2001, Fragebogen zur Erfassung der Schmerzverarbeitung
  3. Wilmers, F, 2018, Klin Diagnostik u Evaluation, 1, 343

Cited by

See all 36

Metadata

Authors
Ulrich, S., Gantenbein, A. R., Zuber, V., Von Wyl, A., Kowatsch, T., & Künzli, H.
Affiliations
  • University Hospital Zurich (CH)
  • ZHAW Zurich University of Applied Sciences (CH)
  • University of Zurich (CH)
Venue
Journal of Medical Internet Research
Published
2024-01-24
Volume
26
Pages
e50132
ISSN
1438-8871 (ISSN-L)
OpenAlex
W4389613191
Cited by
36
References
135
Citations by year
2026: 9 · 2025: 20 · 2024: 7
FWCI
9.26
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by-nc-nd)
  • Alexandria (UniSG) (University of St.Gallen) (submittedVersion)
  • Zürcher Hochschule für Angewandte Wissenschaften digital collection (Zurich University of Applied Sciences) (submittedVersion, cc-by-nc-nd)
  • Zürcher Hochschule für Angewandte Wissenschaften digital collection (Zurich University of Applied Sciences) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
  • ETH Zürich Research Collection
  • Zurich Open Repository and Archive (University of Zurich)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Medicine › Clinical Neurology › Migraine and Headache Studies
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
Keywords
chatbots, smartphone-based intervention, primary headache, migraine, tension-type headache, psychological well-being, anxiety, depression, behaviour change techniques, lifestyle modification, pain coping, self-efficacy, presenteeism, absenteeism, stress, psychosomatic symptoms, user engagement, user acceptance, digital health interventions
MeSH terms
Smartphone, Adult, Female, Headache, Humans, Life Style, Male, Pain, Mobile Applications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.