A Randomized Controlled Trial of Multiple Versions of an Acceptance and Commitment Therapy Matrix App for Well-Being
The DOI matches its Crossref record, with a small difference in title or year; 27 of the 27 references it cites were found in a registry or an index; 2 of the 9 sources its claims cite were checked for support.
Abstract
Mobile apps may be useful in teaching psychological skills in a high-frequency, low-intensity intervention. The acceptance and commitment therapy (ACT) matrix is a visual tool to help develop psychological flexibility by categorizing moment-to-moment experience and is well suited to a mobile app. This pilot study tested the effects of a simple and complex version of a novel app using the ACT matrix in two distinct samples: help-seeking individuals (n = 35) and students receiving SONA credit (n = 63). Findings indicated no differences between app conditions and a waitlist condition in the SONA credit sample. However, in the help-seeking sample, improvements were found on well-being and valued action in participants who used the app, with greater improvements and app adoption for those using a complex version with additional skills. A mobile app based on the ACT matrix has benefits for help-seeking individuals, but supplementary features may be necessary to support consistent use and benefits. © The Author(s) 2017.
Overview
CoChat Summary
TL;DR
This pilot RCT tested simple and complex versions of an ACT matrix mobile app against waitlist in two samples and found no effects in a SONA credit sample, but improvements in distress, anxiety, and valued action among help-seeking adults, especially with the complex app. The authors conclude that a matrix app may benefit help-seeking/distressed individuals, but motivation and additional features may be important for engagement and effects.
Key findings
- In the help-seeking sample (n = 35), significant time-by-condition interactions were found for distress, F(4, 20.30) = 3.92, p = .02, and anxiety, F(4, 20.82) = 3.07, p = .04, with trends for depression and stress (both p = .07); the complex matrix condition improved significantly on distress from baseline to posttreatment, t(21.08) = 3.51, p < .01, d = 1.26, while the simple matrix condition improved significantly on anxiety, t(20.74) = 2.95, p = .01, d = 1.11.
- A significant time-by-condition effect was found for progress toward values in the help-seeking sample, F(4, 22.56) = 3.16, p = .03; the probability of reporting a toward move increased by 0.5% per app notification (exp(γ) = 1.005, p = .03), translating to a 15.00% increase after four weeks of consistent use.
- In the SONA credit sample (n = 63), there were no effects between conditions over time on any outcome or process measure, and notification number did not predict the probability of making a toward move (p = .98).
- App engagement differed by condition in the help-seeking sample: median total app contacts were 5 for simple matrix (M = 42.93, SD = 50.59) versus 42.5 for complex matrix (M = 53.60, SD = 45.58); 42.9% of simple-app participants never used the app, while all complex-app participants used it at least once and 40% used it 70+ times.
Methods
- Pilot randomized controlled trial with 98 adults who owned a mobile phone: a SONA credit sample (n = 63) and a help-seeking community sample (n = 35), randomized to simple matrix app, complex matrix app, or waitlist for four weeks.
- Measures included the DASS-21, MHC-SF, Valuing Questionnaire, System Usability Scale, additional satisfaction items, and automatically recorded app usage; analyses used mixed model repeated measures (MMRM) with intent-to-treat and hierarchical generalized linear models (HGLM) for toward/away move probability.
- The simple app delivered 5 random daily notifications asking participants to identify toward versus away moves; the complex app added a daily check-in, goal setting, values activities, and mindfulness/acceptance activities.
Limitations
- Stated: the help-seeking sample was relatively small, limiting power; results should be replicated in a fully powered RCT; researcher contact made the study less than naturalistic; no clinical or severity cutoffs were used; and both samples were young and ethnically homogeneous.
- Stated: as a low-intensity intervention, small effect sizes would be reasonable and may have been difficult to detect.
- Evident from the text: adherence was relatively low, the complex app’s additional skills were rarely used (median 2.5 other skills exercises), and the SONA sample was incentivized to use the app, which may have contributed to the different results.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 0 Not valid · 6 Not checked · 91 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“condition did not interact with number of notifications ( p = 0.19) in predicting the probability of making a toward move.”
plainn = 22- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There were no effects between conditions over time on any outcome or process measure (see Table 3 for descriptive statistics by condition and time).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There were no statistically significant differences between the simple and complex matrix app users on the SUS or the individual satisfaction items.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There was no significant time by condition interaction in the higher-distress group for depression, progress towards values, values obstruction, or positive mental health.”
plainn = 49- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Again, the conditions were not significantly different in anxiety at posttreatment.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
36 more claims
Create a free account to see all 99 claims and every source verdict.
- It may state two separate claims as one. 45
- Probably not a substantial claim. 20
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, participants who used a more complex ACT matrix health app made greater health improvements and used the app more than those who used a simpler version [27], suggesting that users may prefer a variety of novel toolbox-like activities.”
From An Acceptance and Commitment Therapy Prototype Mobile Program for Individuals with a Visible Difference: Mixed Methods Feasibility Study (2022) · cites this paper as [27]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“There is evidence supporting the efficacy of acceptance and commitment therapy (ACT) in decreasing psychological symptoms in undergraduates, which is thought to be facilitated through increases in psychological flexibility (PF) and decreases in psychological inflexibility (PIF).”
From The effect of acceptance and commitment therapy on the psychological flexibility and inflexibility of undergraduate students: A systematic review and three-level meta-analysis (2023) · cites this paper as Krafft et al., 2019
Experimental — accuracy not yet measured
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- ConsistentF(2, 32) = 4.26reported p = .02 · recomputed p = .023
“F (2, 32) = 4.26, p = .02”
- Consistentt(22) = 3.36reported p < .01 · recomputed p = .003
“t (22) = 3.36, p < .01”
- ConsistentF(4, 20.3) = 3.92reported p = .02 · recomputed p = .017
“F (4, 20.30) = 3.92, p = .02”
- ConsistentF(4, 20.82) = 3.07reported p = .04 · recomputed p = .039
“F (4, 20.82) = 3.07, p = .04”
- ConsistentF(4, 20.38) = 2.61reported p = .07 · recomputed p = .066
“F (4, 20.38) = 2.61, p = .07”
28 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Help-seeking sample
“Again, the conditions were not significantly different in anxiety at posttreatment.”
§ References
“-1.01 [-1.54, -0.48] | 1.01 [0.36, 1.66]”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Participants
“Sample 2, consisting of helpseeking participants, was slightly older ( M = 24.57, SD = 7.68)”
§ Results Preliminary analyses
“The SONA credit sample was significantly younger ( M = 20.24, SD = 3.88) than the help-seeking sample ( M = 24.57, SD = 7.86, Mann-Whitney U, p < 0.001).”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Help-seeking sample
“t (20.70) = 1.77, p = 0.09, pre-post d = 0.67”
§ References
“0.67† [0.12, 1.22]”
Citations
Cites
Acceptance and commitment therapy: model, processes and outcomes
Hayes, S. C., Luoma, J. B., Bond, F. W., Masuda, A., & Lillis, J. (2006) · Behaviour Research and Therapy · doi:10.1016/j.brat.2005.06.006 · Cited by 6,587 · cited 1×
“…set of mechanisms of change, referred to in combination as psychological flexibility, which is “the ability to contact the present moment more fully as a conscious human being, and to change or persist in behavior when doing so serves valued ends” (Hayes, Luoma, Bond, Masuda, & Lillis, 2006, p.7).”
Henry, J. D., & Crawford, J. R. (2005) · British Journal of Clinical Psychology · doi:10.1348/014466505x29657 · Cited by 5,801 · cited 1×
“The 21-item DASS (Lovibond & Lovibond, 1995) was used to measure depression, anxiety, stress, and general distress (Henry & Crawford, 2005).”
An Empirical Evaluation of the System Usability Scale
Bangor, A., Kortum, P. T., & Miller, J. T. (2008) · International Journal of Human–Computer Interaction · doi:10.1080/10447310802205776 · Cited by 5,181 · cited 7×
“The 10-item SUS (Tullis & Albert, 2008) is a measure of usability and acceptability of technology-based programs.”
InfluentialMental illness and/or mental health? Investigating axioms of the complete state model of health
Keyes, C. L. M. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.3.539 · Cited by 3,357
Depression Anxiety Stress Scales
Lovibond, S. H., & Lovibond, P. F. (2011) · PsycTESTS Dataset · doi:10.1037/t01004-000 · Cited by 2,416
Heron, K. E., & Smyth, J. M. (2010) · British Journal of Health Psychology · doi:10.1348/135910709x466063 · Cited by 1,438
Evaluating the psychometric properties of the mental health continuum-short form (MHC-SF)
Lamers, S. M. A., Westerhof, G. J., Bohlmeijer, E. T., ten Klooster, P. M., & Keyes, C. L. M. (2011) · Journal of Clinical Psychology · doi:10.1002/jclp.20741 · Cited by 1,402 · cited 1×
“The MHC-SF has acceptable reliability and validity (Lamers, Westerhof, Bohlmeijer, Ten Klooster, & Keyes, 2011) and has been found to be sensitive to online self-guided ACT interventions (Levin, Haeger, et al., 2017).”
Barriers to mental health treatment: results from the WHO World Mental Health surveys
Andrade, L. H., Alonso, J., Mneimneh, Z., Wells, J. E., Al-Hamzawi, A., Borges, G., Bromet, E., Bruffaerts, R., de Girolamo, G., de Graaf, R., et al. (2014) · Psychological Medicine · doi:10.1017/s0033291713001943 · Cited by 1,289 · cited 1×
“Common barriers that prevent individuals in need from accessing treatment include financial limitations, lack of local providers, and concerns about mental health stigma (Andrade et al., 2014).”
Adherence in Internet Interventions for Anxiety and Depression
Christensen, H., Griffiths, K. M., & Farrer, L. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1194 · Cited by 1,133
Hayes, S. C., Levin, M. E., Plumb-Vilardaga, J., Villatte, J. L., & Pistorello, J. (2013) · Behavior Therapy · doi:10.1016/j.beth.2009.08.002 · Cited by 1,120 · cited 1×
“Throughout the development and evaluation of ACT, there has been a strong emphasis on understanding mechanisms of change and treatment components (Hayes, Levin, Plumb-Vilardaga, Villatte, & Pistorello, 2013).”
Self-citation
Research works printed but not linked: 2
- Polk K. L., 2014, The ACT Matrix: A new approach to building psychological flexibility across settings and populations
- Twohig M. P., 2017, Psychiatric Clinics, 40, 751
Cited by
Linardon, J., Cuijpers, P., Carlbring, P., Messer, M., & Fuller‐Tyszkiewicz, M. (2019) · World Psychiatry · doi:10.1002/wps.20673 · Cited by 860
A systematic review and meta-analysis of psychological interventions to improve mental wellbeing
van Agteren, J., Iasiello, M., Lo, L., Bartholomaeus, J., Kopsaftis, Z., Carey, M., & Kyrios, M. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01093-w · Cited by 510
Eisenstadt, M., Liverpool, S., Infanti, E., Ciuvat, R. M., & Carlsson, C. (2021) · JMIR Mental Health · doi:10.2196/31170 · Cited by 182
“A total of 52 articles [42,43,55-104] (see Multimedia Appendix 2) describing 48 interventions, published between 2008 and 2020, met the inclusion criteria.”
Linardon, J. (2020) · Behavior Therapy · doi:10.1016/j.beth.2019.10.002 · Cited by 133
Six, S. G., Byrne, K. A., Tibbett, T. P., & Pericot-Valverde, I. (2021) · JMIR Mental Health · doi:10.2196/32199 · Cited by 95
“The following data were extracted from each article: first author, year of publication, participants’ characteristics (ie, gender and age), population, and study length (Multimedia Appendix 3), as well as app name, app classification (ie, mobile or internet), presence of gamification elements, type of gamification ele…”
MethodResult
Metadata
- Authors
- Krafft, J., Potts, S., Schoendorff, B., & Levin, M. E.
- Affiliations
- Utah State University (US)
- Institut universitaire en santé mentale de Montréal (CA)
- Corresponding author
- Krafft, Jennifer
- Venue
- Behavior Modification
- Publisher
- SAGE Publications
- Published
- 2017-12-20
- Volume
- 43
- Issue
- 2
- Pages
- 246-272
- ISSN
- 0145-4455 (ISSN-L)
- 1552-4167
- OpenAlex
- W2777934163
- Semantic Scholar
- 7175c7d3aa3abadbc149801f1ff6996b2b49bedd
- Cited by
- 45
- References
- 24
- Citations by year
- 2026: 4 · 2025: 6 · 2024: 4 · 2023: 8 · 2022: 10 · 2021: 9 · 2020: 1 · 2019: 3
- FWCI
- 1.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Behavior Modification (publishedVersion)
- PubMed (publishedVersion)
- Digital Commons - USU (Utah State University) (submittedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Mindfulness and Compassion Interventions
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Keywords
- acceptance and commitment therapy, mobile application, psychological flexibility, well-being, help-seeking, valued action, app adoption, low-intensity intervention, mental training, students
- MeSH terms
- Help-Seeking Behavior, Adult, Anxiety, Attitude to Health, Canada, Depression, Female, Humans, Male, Personal Satisfaction, Pilot Projects, Surveys and Questionnaires, Stress, Psychological, Students, United States, Universities, Young Adult, Mobile Applications, Acceptance and Commitment Therapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-26, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-26 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Randomized Controlled Trial of Multiple Versions of an Acceptance and Commitment Therapy Matrix App for Well-Being. https://app.cochat.ai/papers/10.1177/0145445517748561
- The paper
- Krafft, J., Potts, S., Schoendorff, B., & Levin, M. E. (2019). A Randomized Controlled Trial of Multiple Versions of an Acceptance and Commitment Therapy Matrix App for Well-Being. Behavior Modification. https://doi.org/10.1177/0145445517748561
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.