A Gamified Mobile App That Helps People Develop the Metacognitive Skills to Cope With Stressful Situations and Difficult Emotions: Formative Assessment of the InsightApp
Open accessPublished version of a preprint
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index; 5 of the 18 sources its claims cite were checked for support.
Abstract
Background: Ecological momentary interventions open up new and exciting possibilities for delivering mental health interventions and conducting research in real-life environments via smartphones. This makes designing psychotherapeutic ecological momentary interventions a promising step toward cost-effective and scalable digital solutions for improving mental health and understanding the effects and mechanisms of psychotherapy. Objective: The first objective of this study was to formatively assess and improve the usability and efficacy of a gamified mobile app, the InsightApp, for helping people learn some of the metacognitive skills taught in cognitive behavioral therapy, acceptance and commitment therapy, and mindfulness-based interventions. The app aims to help people constructively cope with stressful situations and difficult emotions in everyday life. The second objective of this study was to test the feasibility of using the InsightApp as a research tool for investigating the efficacy of psychological interventions and their underlying mechanisms. Methods: We conducted 2 experiments. In experiment 1 (n=65; completion rate: 63/65, 97%), participants (mean age 27, SD 14.9; range 19-55 years; 41/60, 68% female) completed a single session with the InsightApp. The intervention effects on affect, belief endorsement, and propensity for action were measured immediately before and after the intervention. Experiment 2 (n=200; completion rate: 142/200, 71%) assessed the feasibility of conducting a randomized controlled trial using the InsightApp. We randomly assigned participants to an experimental or a control condition, and they interacted with the InsightApp for 2 weeks (mean age 37, SD 12.16; range 20-78 years; 78/142, 55% female). Experiment 2 included all the outcome measures of experiment 1 except for the self-reported propensity to engage in predefined adaptive and maladaptive behaviors. Both experiments included user experience surveys. Results: In experiment 1, a single session with the app seemed to decrease participants’ emotional struggle, the intensity of their negative emotions, their endorsement of negative beliefs, and their self-reported propensity to engage in maladaptive coping behaviors (P<.001 in all cases; average effect size=−0.82). Conversely, participants’ endorsement of adaptive beliefs and their self-reported propensity to act in accordance with their values significantly increased (P<.001 in all cases; average effect size=0.48). Experiment 2 replicated the findings of experiment 1 (P<.001 in all cases; average effect size=0.55). Moreover, experiment 2 identified a critical obstacle to conducting a randomized controlled trial (ie, asymmetric attrition) and how it might be overcome. User experience surveys suggested that the app’s design is suitable for helping people apply psychotherapeutic techniques to cope with everyday stress and anxiety. User feedback provided valuable information on how to further improve app usability. Conclusions: In this study, we tested the first prototype of the InsightApp. Our encouraging preliminary results show that it is worthwhile to continue developing the InsightApp and further evaluate it in a randomized controlled trial. ©Victoria Amo, Mike Prentice, Falk Lieder.
Overview
CoChat Summary
TL;DR
The paper presents formative testing of the InsightApp, a gamified smartphone intervention that combines CBT, ACT, and mindfulness elements to train metacognitive coping skills. In two experiments, the app showed immediate pre–post improvements in emotional and cognitive outcomes, but the longitudinal prototype also revealed a major feasibility problem: asymmetric attrition.
Key findings
- In experiment 1 (n=65; final analyzed sample 63), a single session significantly decreased perceived likelihood of unwanted actions (V=88; P<.001; effect size=0.79), emotional intensity (V=1; P<.001; 0.87), emotional struggle (V=43; P<.001; 0.83), and endorsement of maladaptive beliefs (V=53; P<.001; 0.82), while increasing perceived likelihood of beneficial actions (V=1549; P<.001; 0.68) and endorsement of adaptive beliefs (V=761; P<.001; 0.27).
- In experiment 2 (n=200; final sample 142), the daily morning practice again improved outcomes from pre- to post-session: emotion intensity (t316.7=−5.959; P<.001; Cohen d=−0.67), emotional struggle (t236.4=−5.048; P<.001; d=−0.66), discouraging belief endorsement (t610.6=−7.385; P<.001; d=−0.60), and encouraging belief endorsement (t656.1=3.642; P<.001; d=0.28).
- Attrition in experiment 2 was much higher in the experimental group than the control group: 41/100 (41%) vs 17/100 (17%), χ²1=12.8, P<.001.
- User experience was positive: in experiment 1, 98% (62/63) agreed they developed a valuable daily-life skill, the app averaged 4.7/5 stars, and 95% (60/63) said nothing was confusing; in experiment 2, 86% (51/59) agreed they developed a valuable skill, the app averaged 4/5 stars, and 83% (49/59) said nothing was confusing.
Methods
- Two formative experiments were conducted with iPhone-owning, English-speaking adults recruited and compensated via Prolific; experiment 1 used a pretest-posttest single-session design, and experiment 2 used a 2-week randomized design with experimental and control conditions.
- The app intervention combined CBT (ABCDE cognitive restructuring), ACT (values reflection, cognitive defusion, acceptance), and mindfulness-based strategies through a reflection module plus two “metacognitive coaches” (meta-reasoning and meta-awareness).
- Experiment 1 analyzed pre/post changes with nonparametric paired-sample Wilcoxon tests and summarized user experience with descriptive statistics; experiment 2 used linear mixed models for repeated pre/post morning-practice scores and a chi-square test to compare attrition.
Limitations
- The paper explicitly states that experiment 1 lacked a control condition and that experiment 2 had asymmetric attrition, which prevented comparison of the experimental and control conditions.
- The paper notes reliance on single-item measures as a possible limitation, although it says test-retest reliability was examined in a follow-up study.
- Evident from the text: the studies measured immediate pre–post effects after practice, so they do not establish longer-term efficacy or clinical outcomes. The paper also says it did not measure whether participants actually performed the mindful breathing exercise, only the effect of being told to do it.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
5 Valid · 0 Not valid · 9 Not checked · 71 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“EMIs can help solve this issue as they allow people to learn new responses to old triggers by training in the same context in which the maladaptive behavior occurs [10].”
§ A Gamified Mobile App That Helps People Develop the Metacognitive Skills to Cope With Stressful Situations and Difficult Emotions: Formative Assessment of the InsightApp > Introduction > Background
What the sources say
- Supportedcites [10] Versluis A Verkuil B Spinhoven P van der Ploeg MM Brosschot JF Changing mental health and positive psychological well-being using ecological momentary interventions: a systematic review and meta-analysis J Med Internet Res 2016 06 27 18 6 e152 10.2196/jmir.5642 27349305 27349305 PMC4940607 source
The Introduction says EMIs may train people in the context where maladaptive behavior occurs, potentially forming a new, more adaptive association between context and behavior. This supports the citing sentence's qualified claim; no page number is provided in our copy. (LLM, conf=0.92)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“These effects are noticeably larger than the average effects reported in previous studies evaluating the effectiveness of EMIs in improving mental health [10], mental health apps [35], and mindfulness interventions [7].”
§ A Gamified Mobile App That Helps People Develop the Metacognitive Skills to Cope With Stressful Situations and Difficult Emotions: Formative Assessment of the InsightApp > Discussion > Comparison With Prior Work
What the sources say
- Supportedcites [10] Versluis A Verkuil B Spinhoven P van der Ploeg MM Brosschot JF Changing mental health and positive psychological well-being using ecological momentary interventions: a systematic review and meta-analysis J Med Internet Res 2016 06 27 18 6 e152 10.2196/jmir.5642 27349305 27349305 PMC4940607 source
The Between-Subject Analyses report an average EMI effect on mental health of g=0.40, or g=0.23 after correction for publication bias. This supports the EMI benchmark attributed to this source; the comparison with InsightApp's effects is the citing authors' own. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [35] Tamir M Halperin E Porat R Bigman YE Hasson Y When there's a will, there's a way: disentangling the effects of goals and means in emotion regulation J Pers Soc Psychol 2019 05 116 5 795 816 10.1037/pspp0000232 30614729 2019-00280-001 30614729 source
- Not checked — no full text of the sourcecites [7] Gál É Ștefan S Cristea IA The efficacy of mindfulness meditation apps in enhancing users' well-being and mental health related outcomes: a meta-analysis of randomized controlled trials J Affect Disord 2021 01 15 279 131 42 10.1016/j.jad.2020.09.134 33049431 S0165-0327(20)32831-7 33049431 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“the adherence score for the experimental condition was comparable with the adherence scores for mobile mental health apps targeting the clinical population (ie, 56.6%) and mindfulness apps targeting the general public (ie, 59%), as reported by Jakob et al [37].”
§ A Gamified Mobile App That Helps People Develop the Metacognitive Skills to Cope With Stressful Situations and Difficult Emotions: Formative Assessment of the InsightApp > Discussion > Comparison With Prior Work
What the sources say
- Partly supportedcites [37] Jakob R Harperink S Rudolf AM Fleisch E Haug S Mair JL Salamanca-Sanabria A Kowatsch T Factors influencing adherence to mHealth apps for prevention or management of noncommunicable diseases: systematic review J Med Internet Res 2022 05 25 24 5 e35371 10.2196/35371 35612886 v24i5e35371 35612886 PMC9178451 sourceExperimental — accuracy not yet measured
The Results sections on Mental Health and Mindfulness report mean adherence scores of 56.6% and 59.0%, respectively. The Mindfulness section says most included apps aimed to improve well-being or reduce work-related stress rather than treat a chronic mental condition, but the review does not characterize the entire mindfulness group as targeting the general public or the entire mental-health group as targeting a clinical population. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Conversely, participants’ endorsement of adaptive beliefs and their self-reported propensity to act in accordance with their values significantly increased (P<.001 in all cases; average effect size=0.48).”
§ A Gamified Mobile App That Helps People Develop the Metacognitive Skills to Cope With Stressful Situations and Difficult Emotions: Formative Assessment of the InsightApp > Abstract > Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“User experience surveys suggested that the app’s design is suitable for helping people apply psychotherapeutic techniques to cope with everyday stress and anxiety.”
§ A Gamified Mobile App That Helps People Develop the Metacognitive Skills to Cope With Stressful Situations and Difficult Emotions: Formative Assessment of the InsightApp > Abstract > Results
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
44 more claims
Create a free account to see all 85 claims and to verify the 3 references still open.
- Probably not a substantial claim. 27
- It may state two separate claims as one. 18
- The quote only partly states this claim. 1
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Gamified Mobile App That Helps People Develop the Metacognitive Skills to Cope With Stressful Situations and Difficult Emotions: Formative Assessment of the InsightApp > Methods > Experiment 1
“In addition, the app asks participants to use a 6-point Likert scale (strongly disagree=1; strongly agree=6) to specify the degree to which they agree with the following statement: “I developed a valuable skill for my daily life.””
§ A Gamified Mobile App That Helps People Develop the Metacognitive Skills to Cope With Stressful Situations and Difficult Emotions: Formative Assessment of the InsightApp > Results > Experiment 1
“When answering regarding the degree to which they agreed with the following statement—“You developed valuable skills for your daily life” (panel B in Multimedia Appendix 2)—98% (62/63) of the participants agreed that the app taught them a valuable skill (14/63, 22% mostly agreed; 27/63, 43% agreed; and 21/63, 33% strongly agreed).”
Citations
Cites
Fitting Linear Mixed-Effects Models Using lme4
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835
lmerTest Package: Tests in Linear Mixed Effects Models
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017) · Journal of Statistical Software · doi:10.18637/jss.v082.i13 · Cited by 24,346 · cited 1×
“Significance was calculated using the lmerTest package [28], which applies the Satterthwaite method to estimate df and generate P values for mixed models.”
MethodSelf-Efficacy Beliefs of Adolescents
(2006) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-60752-750-3 · Cited by 3,506
The Efficacy of Cognitive Behavioral Therapy: A Review of Meta-analyses
Hofmann, S. G., Asnaani, A., Vonk, I. J. J., Sawyer, A. T., & Fang, A. (2012) · Cognitive Therapy and Research · doi:10.1007/s10608-012-9476-1 · Cited by 3,445
Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.01.006 · Cited by 2,957 · cited 1×
“Compared with other platforms for web-based studies, Prolific is known for providing superior data quality [24].”
BackgroundHow important are the common factors in psychotherapy? An update
Wampold, B. E. (2015) · World Psychiatry · doi:10.1002/wps.20238 · Cited by 1,728 · cited 1×
“Some research has focused on understanding common factors across psychotherapies (eg, therapeutic alliance, empathy and listening skills, and cultural adaptation) [19].”
BackgroundHeron, K. E., & Smyth, J. M. (2010) · British Journal of Health Psychology · doi:10.1348/135910709x466063 · Cited by 1,438 · cited 1×
“Ecological momentary interventions (EMIs), for example, are medical treatments delivered via mobile devices that aim to help people integrate intervention strategies into their everyday lives and real-life settings [8].”
BackgroundMetacognitive therapy for anxiety and depression
Veeraraghavan, V. (2009) · Anxiety, Stress & Coping · doi:10.1080/10615800902833770 · Cited by 1,165
(2002) · Journal of Psychiatric and Mental Health Nursing · doi:10.1046/m.1365-2850.2002.00006.x · Cited by 1,143
Constructing Confidence Sets Using Rank Statistics
Bauer, D. F. (1972) · Journal of the American Statistical Association · doi:10.1080/01621459.1972.10481279 · Cited by 848 · cited 4×
“ACT: acceptance and commitment therapy CBT: cognitive behavioral therapy EMI: ecological momentary intervention MBI: mindfulness-based intervention PSW: paired-sample Wilcoxon test JMIR Form Res 2023 | vol. 7 | e44429 | p. 16https://formative.jmir.org/2023/1/e44429 (page number not for citation purposes) XSL•FO Render…”
MethodInfluential
Research works printed but not linked: 3
- Segal ZV Williams JM Teasdale JD Mindfulness-Based Cognitive Therapy for Depression: A New Approach to Preventing Relapse. 2nd edition 2018 New York, NY, USA Guilford Press
- Leahy RL Cognitive Therapy Techniques: A Practitioner's Guide. 2nd edition 2017 New York, NY, USA Guilford Press
- Harris R ACT Made Simple: An Easy-To-Read Primer on Acceptance and Commitment Therapy. 2nd edition 2019 Oakland, CA, USA New Harbinger
Cited by
The use of serious games for psychological education and training: a systematic review
Facchino, A. P., Marchetti, D., Colasanti, M., Fontanesi, L., & Verrocchio, M. C. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1511729 · Cited by 44
Ngabo-Woods, H., Dunai, L., Verdú, I. S., & Tîrșu, V. (2026) · Information · doi:10.3390/info17020168 · Cited by 7
Amo, V., & Lieder, F. (2024) · JMIR Mental Health · doi:10.2196/57201 · Cited by 4
“We used the measure of single-item sliding scales proposed by Amo et al [19] for momentary and daily levels of anxiety, struggle with anxiety, unwanted action, valued action, and the endorsement of the maladaptive and adaptive beliefs.”
InfluentialSelf-citationAmo, V., & Lieder, F. (2023) · doi:10.2196/preprints.52280
Self-citationThe Impact of Mindfulness Apps and Cognitive Behavioral Therapy on Prisoners' Mental Health
Xiao, X. (2024) · Lecture Notes in Education Psychology and Public Media · doi:10.54254/2753-7048/57/20240102
Ramirez-Montoya, M. S., Patiño, A., & Hernandez, L. E. (2025) · Lecture notes in educational technology · doi:10.1007/978-981-96-5658-5_6 · Cited by 1
Therapeutic Alliance and Effectiveness of Digital Mental Health Interventions
Arakkal, M. A., & Thangal, M. R. (2026) · Wiley eBooks · doi:10.1002/9781394429660.ch5 · Cited by 1
Zamer, F. N. A. M., & Mahamarowi, N. H. (2026) · International Conference on e-Education, e-Business, e-Management and e-Learning · doi:10.1109/ic4e69618.2026.11631406
“In the view of [8], lacks of application has appeared that merges journaling, emotion tracking and calendar scheduling for students.”
Metadata
- Authors
- Amo, V., Prentice, M., & Lieder, F.
- Affiliations
- Max Planck Society (DE)
- Corresponding author
- Amo, Victoria
- Venue
- JMIR Formative Research
- Publisher
- JMIR Publications Inc.
- Published
- 2023-06-16
- Volume
- 7
- Pages
- e44429
- ISSN
- 2561-326X (ISSN-L)
- PMC
- PMC10337330
- OpenAlex
- W4327884691
- Semantic Scholar
- d0525d2b52a17372db989677597d8e03d8e7f509
- Cited by
- 9
- References
- 33
- Citations by year
- 2026: 3 · 2025: 2 · 2024: 2 · 2023: 2
- FWCI
- 2.32
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Formative Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Clinical Psychology › Mindfulness and Compassion Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Keywords
- metacognitive skills, ecological momentary intervention, digital mental health interventions, cognitive behavioral therapy, acceptance and commitment therapy, mindfulness-based interventions, stress coping, difficult emotions, negative beliefs, user experience, usability, mobile application, emotion regulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.