CoChat
Sign Up
Cited by 71Mental Health Research TopicsDigital Mental Health InterventionsMobile Health and mHealth Applications

Experienced burden of and adherence to smartphone-based ecological momentary assessment in persons with affective disorders

Open access
van Genugten, Claire R., Schuurmans, Josien, Lamers, Femke, Riese, Harriëtte, Penninx, Brenda W. J. H., Schoevers, Robert A., Riper, Heleen M., Smit, Johannes H.·2020·Journal of Clinical Medicine
DOI 10.3390/jcm9020322PMC PMC7073581
Open this paper in CoChat — free accountOpen PDF

Abstract

(1) Background: The use of smartphone-based ecological momentary assessment (EMA) questionnaires in affective disorder research has rapidly increased. Though, a thorough understanding of experienced burden of and adherence to EMA is crucial in determining the usefulness of EMA. (2) Methods: Persons with current affective disorders (n = 100), remitted persons (n = 190), and healthy controls (n = 94) participated in a smartphone-based EMA two-week monitoring period. Our primary outcomes were (momentary) perceived burden of and adherence to EMA. (3) Results: In the whole sample, lower positive and higher negative affect were associated with slightly higher levels of perceived momentary burden (B = −0.23 [95%CI = −0.27–0.19], B = 0.30 [95%CI = 0.24–0.37], respectively). The persons with current affective disorders reported slightly higher levels of experienced momentary burden (Mdn = 1.98 [IQR = 1.28–2.57]), than the remitted persons (Mdn = 1.64 [IQR = 1.11–2.24]) and healthy controls (Mdn = 1.28 [IQR = 1.04–1.92]). Nevertheless, the persons with current affective disorders still showed very high adherence rates (Mdn = 94.3% [IQR = 87.9–97.1]), at rates on a par with the remitted persons (Mdn = 94.3% [IQR = 90.0–97.1]) and healthy controls (Mdn = 94.3% [IQR = 90.0–98.6]). (4) Discussion: Frequent momentary questionnaires of mental well-being are slightly more burdensome to the persons with current affective disorders, but this does not seem to have a negative impact on adherence. Their high rate of adherence to EMA—which was similar to that in remitted persons and healthy controls —suggests that it is feasible to apply (short-duration) EMA.

AI snapshot

Written from the paper’s full text.

TL;DR

In a two-week smartphone EMA study, persons with current affective disorders reported slightly higher momentary burden than remitted persons and healthy controls, but all groups showed very high adherence. The authors conclude that short-duration smartphone EMA is feasible for persons with affective disorders despite modestly greater momentary burden.

Key findings

  • Momentary burden was significantly higher in current affective disorders (Mdn = 1.98, IQR = 1.28–2.57) than in remitted persons (Mdn = 1.64, IQR = 1.11–2.24; p = 0.01) and healthy controls (Mdn = 1.28, IQR = 1.04–1.92; p < 0.0001).
  • Adherence was identical across groups: all had a median of 66/70 questionnaires completed, or 94.3% (current IQR = 63.5–69.0; remitted IQR = 63.0–68.0; healthy controls IQR = 62.0–69.0; H[2] = 0.08, p = 0.98).
  • In the whole sample, lower positive affect (B = −0.23, 95%CI = −0.27–0.19) and higher negative affect (B = 0.30, 95%CI = 0.24–0.37 per abstract) were associated with slightly higher momentary burden; burden remained stable over time (B = 0.00, p = 0.09).
  • Overall burden over the whole monitoring period did not differ significantly between groups (current Mdn = 2.5, remitted Mdn = 2.25, healthy controls Mdn = 2.0; H[2] = 5.56, p = 0.062).

Methods

  • NESDA-EMAA subsample from the Netherlands Study of Depression and Anxiety: 384 participants, including current affective disorders (n = 100), remitted persons (n = 190), and healthy controls (n = 94).
  • Participants completed five smartphone-based EMA questionnaires per day for 14 days (70 total), at fixed 3-hour intervals, with up to 31 questions including a 13-item momentary affect scale.
  • Outcomes were momentary burden (single item), overall burden (4-item addendum; Cronbach’s α = 0.924), and adherence (percentage of completed questionnaires). Analyses used Kruskal–Wallis, Bonferroni-adjusted Mann–Whitney, chi-square/likelihood ratio tests, and GEE models in SPSS 25.

Limitations

  • The paper states that NESDA-EMAA participants were likely highly motivated after over nine years in NESDA, and the relatively short two-week monitoring period may not generalize to longer EMA settings.
  • The paper states that momentary burden was measured with one single item, so its psychometric properties could not be assessed.
  • The paper notes that adherence may have been influenced by allowing responses within 60 minutes after prompts; evident from inclusion criteria, participants also had to be familiar with smartphone use and willing to wear actigraphy, which may limit generalizability.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/jcm9020322

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 10 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The association between reported momentary burden and time did not change significantly over the study period (B = 0.00, 95%CI = 0.00-0.00, p = 0.09).

    “we found that the strength of the association between reported levels of burden as a result of the questionnaires did not change significantly over time (B = 0.00, 95%CI = 0.00-0.00, p = 0.09).”
    Quote in the paperNot a substantial claim
  • Findingown data

    No significant inter-group differences were found in average overall experienced burden over the whole two-week EMA monitoring period (H[2] = 5.56, p = 0.062).

    “No significant inter-group differences regarding average overall experienced burden were found (H[2] = 5.56, p = 0.062).”
    • Also stated in: § 4. Discussion · plain
      “when asked to evaluate the overall experienced burden of the whole two-week monitoring period, no significant inter-group differences were found.”
    Quote in the paperSubstantial claim
  • Findingown data

    In the whole sample, higher positive affect was associated with slightly lower momentary burden (B = −0.23, 95%CI = −0.27-0.19).

    “First, we found a significant negative association between positive affect and momentary burden (B = −0.23, 95%CI = −0.27–0.19, p < 0.0001)”
    • Also stated in: § Abstract · plain
      “In the whole sample, lower positive and higher negative affect were associated with slightly higher levels of perceived momentary burden (B = −0.23 [95%CI = −0.27–0.19], B = 0.30 [95%CI = 0.24–0.37], respectively).”
    Quote in the paperNot a substantial claim
  • Findingown data

    In the whole sample, higher negative affect was associated with slightly higher momentary burden (B = 0.30, 95%CI = 0.24-0.37).

    “and a significant positive association between negative affect and momentary burden (B = 0.30, 95%CI = −0.24–0.37, p < 0.0001)”
    • Also stated in: § Abstract · plain
      “In the whole sample, lower positive and higher negative affect were associated with slightly higher levels of perceived momentary burden (B = −0.23 [95%CI = −0.27–0.19], B = 0.30 [95%CI = 0.24–0.37], respectively).”
    Quote in the paperNot a substantial claim
  • Findingown data

    There was no significant difference in adherence rates between the current affective disorder, remitted, and healthy control groups (H[2] = 0.08, p = 0.98).

    “meaning that there is no significant difference between the persons with current affective disorders (IQR = 63.5–69.0), the remitted persons (IQR = 63.0–68.0), and the healthy controls (IQR = 62.0–69.0; (H[2] = 0.08, p = 0.98).”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 6
  • May combine two claims. 6

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A meta-analysis showed a pooled average adherence rate of 71% in substance abusers.

    “A meta-analysis showed a pooled average adherence rate of 71% in substance abusers [30].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30] Jones, 2018, Compliance with Ecological Momentary Assessment Protocols in Substance Users: A Meta-Analysis, Addiction source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    Adherence rates in this study are higher than in previous studies with a similar population, although those studies used different EMA methods (e.g., paper-and-pencil diaries, PDAs).

    “Moreover, the adherence rates found in this study are even higher than in previous ones with a similar population, although these studies used different EMA methods (e.g., paper-and-pencil diaries, PDAs) [8,14,15].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [8] Wenze, 2010, Use of Ecological Momentary Assessment in Mood Disorders Research, Clin. Psychol. Rev. source

      Not checked in the free run.

    • cites [14] Hogenelst, 2012, Mood Disorders in Everyday Life: A Systematic Review of Experience Sampling and Ecological Momentary Assessment Studies, Clin. Psychol. Rev. source

      Not checked in the free run.

    • cites [15] Colombo, D., Fernández-Álvarez, J., Patané, A., Semonella, M., Kwiatkowska, M., García-Palacios, A., Cipresso, P., Riva, G., and Botella, C. (2019). Current State and Future Directions of Technology-Based Ecological Momentary Assessment and Intervention for Major Depressive Disorder: A Systematic Review. J. Clin. Med., 8. source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductcitation

    GEE models adjust for dependency of repeated measures within one participant and can handle non-normally distributed data and missing data.

    “We chose GEE models because EMA data have a hierarchical structure; this type of model adjusts for dependency of repeated measures within one participant and can handle non-normally distributed data. GEE models are also suitable for dealing with missing data; it is not required to exclude participants with missing questionnaires, nor should missing questionnaires be imputed beforehand.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [28] Lu, 2007, A comparison of two bias-corrected covariance estimators for generalized estimating equations, Biometrics source

      Not checked in the free run.

    • cites [29] Twisk, 2002, Attrition in Longitudinal Studies: How to Deal with Missing Data, J. Clin. Epidemiol. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    The results from this relatively short EMA monitoring period might not be readily generalized to other settings that rely on longer EMA monitoring periods.

    “In addition, the results of our a relatively short EMA monitoring period might not be readily generalized to other settings that rely on longer EMA monitoring periods.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Composite International Diagnostic Interview (CIDI Version 2.1) has high validity for the assessment of mental disorders.

    “This has high validity for the assessment of mental disorders [26].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] Wittchen, 1991, Cross-Cultural Feasibility, Reliability and Sources of Variance of the Composite International Diagnostic Interview (CIDI), Br. J. Psychiatr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The study was carried out in accordance with the Declaration of Helsinki and received ethical approval from the Ethical Review Board of VU University Medical Centre Amsterdam and local review boards; all participants provided written consent.

    “The study was carried out in accordance with the latest version of the Declaration of Helsinki. The Ethical Review Board of VU University Medical Centre Amsterdam and the local review boards of the participating centres provided ethical approval and all participants provided written consent.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The clinical picture of current affective disorders involves a persistent depressed mood and/or diminished interest or pleasure in general.

    “These findings could possibly be explained by the fact that the clinical picture of the current affective disorders involves a persistent depressed mood and/or diminished interest or pleasure in general [17,18].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [17] American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental Disorders, American Psychiatric Association. [5th ed.]. source

      Not checked in the free run.

    • cites [18] Beck, A.T., and Alford, B.A. (2009). Depression: Causes and Treatment, University of Pennsylvania Press. [2nd ed.]. source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductreasoning

    Momentary burden was collected with one single item, so the psychometric properties of this main outcome measure could not be assessed.

    “Another point is that momentary burden was collected with one single item. As a result, the psychometric properties of this main outcome measure could not be assessed.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The findings provide important insights into the acceptability of short-duration EMA methods for persons with affective disorders.

    “Although our conclusions highlight the need to examine this topic further, they do already provide important insights into the acceptability of (short-duration) EMA methods for persons with affective disorders.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Persons with current affective disorders were still able and willing to provide real-time information on their mental well-being.

    “even though the persons with current affective disorders did report experiencing more burden as a result of the EMA questionnaires when asked about this at the moment of assessment, this was not the case when asked to evaluate the whole monitoring period, and they were still able and willing to provide real-time information on their mental well-being.”
    Quote in the paperSubstantial claim
10 of 10

  • The authors state this as an assumption.

    Participants in the NESDA-EMAA sample are highly motivated because they have been taking part in the NESDA study for over nine years.

    “Most notably, it can be reasonably assumed that the participants in the NESDA-EMAA sample are highly motivated as they have been taking part in the NESDA study for over nine years.”

  • The authors state this as an assumption.

    This study is the first to systematically examine user experiences of EMA questionnaires in persons with current affective disorders while making a direct comparison with remitted persons and healthy controls in a single cohort.

    “This study is, to the best of our knowledge, the first to systematically examine user experiences of EMA questionnaires in persons with current affective disorders while making a direct comparison with remitted persons and healthy controls in a single cohort.”
2 of 2

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • While the review supports the general acceptability of EMA in suicide research, the burden of EMA measures may be less tolerable for those currently experiencing very severe symptoms, analogous to findings in individuals with depressive disorders.

    “While our review supports the general acceptability of EMA in suicide research, the burden of EMA measures may be less tolerable for those currently experiencing very severe symptoms, analog to findings in individuals with depressive disorders (102).”

    From Don't Miss the Moment: A Systematic Review of Ecological Momentary Assessment in Suicide Research (2022) · cites this paper as [102]

  • Studies that apply non-personalized experience sampling method questionnaires in clinical samples, including those from our own group, have used a similar or higher number of questions, and compliance rates suggest a manageable burden for participants.

    “Studies that apply (non-personalized) ESM questionnaires in clinical samples, including those from our own group, have used a similar or higher number of questions [32–35], and compliance rates suggest a manageable burden for participants.”

    From Using person-specific networks in psychotherapy: challenges, limitations, and how we could use them anyway (2020) · cites this paper as [35]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Female percentage does not match the reported count

    Minor

    The reported 238 female participants out of 384 is 62.0%, not 67.0%.

    Reported 67 · recomputed 61.979167

  • Negative-affect confidence interval has conflicting lower bounds

    Minor

    The abstract reports a lower bound of 0.24, whereas the Results section reports −0.24 for the same coefficient.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Schoevers, R.A., Van Borkulo, C.D., Lamers, F., Servaas, M.N., Bastiaansen, J.A.C.J., Beekman, A.T., Van Hemert, A.M., Smit, J.H., Penninx, B.W.J.H., and Riese, H. Momentary affect fluctuations differ between patients with current depression and anxiety disorders, remitters and healthy controls, Psychol. Med., (under review).

Cited by

Sorted by citations

Related

Authors
van Genugten, C. R., Schuurmans, J., Lamers, F., Riese, H., Penninx, B. W. J. H., Schoevers, R. A., Riper, H. M., & Smit, J. H.
Affiliations
  • Department of Research and Innovation, GGZ inGeest, Specialized Mental Health Care, 1081 Amsterdam, The Netherlands
  • Amsterdam Public Health (NL)
  • University Medical Center Groningen (NL)
  • University of Southern Denmark (DK)
ORCID
Venue
Journal of Clinical Medicine
Publisher
MDPI AG
Published
2020-01-23
Volume
9
Issue
2
Pages
322
ISSN
2077-0383 (ISSN-L)
DOI
10.3390/jcm9020322
PMC
PMC7073581
OpenAlex
W3001799799
Cited by
71
References
32
Citations by year
2026: 9 · 2025: 17 · 2024: 9 · 2023: 8 · 2022: 14 · 2021: 6 · 2020: 8
FWCI
8.71
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Clinical Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Pure Amsterdam UMC (publishedVersion, cc-by)
  • University of Groningen research database (University of Groningen / Centre for Information Technology) (publishedVersion, cc-by)
  • VU Research Portal (publishedVersion, other-oa)
  • University of Southern Denmark Research Portal (University of Southern Denmark) (acceptedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, cc-by)
  • Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
Funders
  • Amsterdam University Medical Centers (10-000-1002)
  • European Commission (115902, 2007–2013, FP7 2007-2013, FP7/2007, 334065)
  • Universiteit Leiden (10‐000‐1002)
  • Rijksuniversiteit Groningen (10‐000‐1002)
  • ZonMw (10‐000‐1002)
  • GGZ inGeest
  • Rivierduinen
  • GGZ Friesland
  • GGZ Drenthe
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
Keywords
ecological momentary assessment, smartphone-based assessment, mood disorders, perceived burden, adherence, remitted depression, positive and negative affect, feasibility

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review