Cited by 72Digital Mental Health InterventionsMindfulness and Compassion InterventionsMobile Health and mHealth Applications

Efficacy and Conflicts of Interest in Randomized Controlled Trials Evaluating Headspace and Calm Apps: Systematic Review

Open access
O'Daffer, Alison, Colt, Susannah F, Wasil, Akash R, Lau, Nancy·2022·JMIR Mental Health

Abstract

Background: Although there are thousands of mental health apps, 2 apps, Headspace and Calm, claim a large percentage of the marketplace. These two mindfulness and meditation apps have reached tens of millions of active users. To guide consumers, clinicians, and researchers, we performed a systematic review of randomized controlled trials (RCTs) of Headspace and Calm. Objective: Our study aimed to evaluate intervention efficacy, risk of bias, and conflicts of interest (COIs) in the evidence base for Headspace and Calm, the two most popular mental health apps at the time of our search. Methods: To identify studies, we searched academic databases (Google Scholar, MEDLINE, and PsycINFO) and the websites of Headspace and Calm in May 2021 for RCTs of Headspace and Calm testing efficacy via original data collection, published in English in peer-reviewed journals. For each study, we coded (1) study characteristics (eg, participants, sample size, and outcome measures), (2) intervention characteristics (eg, free vs paid version of the app and intended frequency of app usage), (3) all study outcomes, (4) Cochrane risk of bias variables, and (5) COI variables (eg, presence or absence of a preregistration and the presence or absence of a COI statement involving the company). Results: We identified 14 RCTs of Headspace and 1 RCT of Calm. Overall, 93% (13/14) of RCTs of Headspace and 100% (1/1) of RCTs of Calm recruited participants from a nonclinical population. Studies commonly measured mindfulness, well-being, stress, depressive symptoms, and anxiety symptoms. Headspace use improved depression in 75% of studies that evaluated it as an outcome. Findings were mixed for mindfulness, well-being, stress, and anxiety, but at least 40% of studies showed improvement for each of these outcomes. Studies were generally underpowered to detect “small” or “medium” effect sizes. Furthermore, 50% (7/14) of RCTs of Headspace and 0% (0/1) of RCTs of Calm reported a COI that involved Headspace or Calm (the companies). The most common COI was the app company providing premium app access for free for participants, and notably, 14% (2/14) of RCTs of Headspace reported Headspace employee involvement in study design, execution, and data analysis. Only 36% (5/14) of RCTs of Headspace were preregistered, and the 1 RCT of Calm was not preregistered. Conclusions: The empirical research on Headspace appears promising, whereas there is an absence of randomized trials on Calm. Limitations of this study include an inability to compare Headspace and Calm owing to the dearth of RCTs studying Calm and the reliance on author reports to evaluate COIs. When determining whether or not mental health apps are of high quality, identification of high-quality apps and evaluation of their effectiveness and investigators’ COIs should be ensured.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/40924

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [24]Turner E Matthews A Linardatos E Tell R Rosenthal R Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy N Engl J Med 2008 01 17 358 3 252 260 10.1056/NEJMsa065779 18199864 · DOI · Paper page

Cited by

Sorted by citations

Authors
O'Daffer, A., Colt, S. F., Wasil, A. R., & Lau, N.
Affiliations
  • University of Pennsylvania (US)
  • University of Washington (US)
Venue
JMIR Mental Health
Published
2022-09-20
Volume
9
Issue
9
Pages
e40924
ISSN
2368-7959 (ISSN-L)
OpenAlex
W4293397205
Cited by
72
References
45
Citations by year
2026: 7 · 2025: 27 · 2024: 20 · 2023: 18
FWCI
2.88
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Mental Health (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (acceptedVersion)
Funders
  • National Heart, Lung, and Blood Institute (K12HL137940)
  • National Cancer Institute (K08CA263474)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Social Sciences › Psychology › Clinical Psychology › Mindfulness and Compassion Interventions
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
Keywords
headspace, CALM, mental health apps, mindfulness apps, meditation apps, app efficacy, conflicts of interest, risk of bias, randomized controlled trials, depression, anxiety, stress, well-being, mindfulness, preregistration, underpowered studies, digital mental health interventions, consumer health information

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review