MindSurf: a pilot study to assess the usability and acceptability of a smartphone app designed to promote contentment, wellbeing, and goal achievement
Open accessAbstract
BACKGROUND: The Method of Levels (MOL) is a transdiagnostic cognitive therapy that promotes contentment, wellbeing, and goal achievement through the resolution of internal conflicts underlying psychological distress. MOL, based on Perceptual Control Theory (PCT), was developed in routine clinical practice and has been used effectively across different health services by different practitioners. Access to MOL-style questions through a smartphone app could, potentially, help both the general public maintain robust mental health, and also be a useful adjunct to therapy for clinical populations. The app is called MindSurf because of its focus on helping people explore their thinking. Prior to developing the app and using it with different populations it was necessary to determine whether such an idea would be usable for and acceptable to potential app users. Therefore, a pilot study was conducted with a non-clinical sample to assess the usability and acceptability of the app including monitoring whether the questions delivered in this way were associated with any adverse events. METHODS: A pilot study using quantitative as well as qualitative methods and incorporating a repeated measures, A-B design was conducted. RESULTS: The 23 participants were healthy adult volunteers who were all either undergraduate students, postgraduate students, or staff of the University of Manchester. They received MOL-style questions on their mobile phones over a 1-week period. Qualitative results were encouraging and indicated that the format and style of questioning were acceptable to participants and did not lead to increased worry or concern. A one-way, repeated measures ANOVA indicated that there was a nonsignificant decrease in scores on the 21-item Depression, Anxiety, and Stress Scale (DASS21) over a 2 week period. DISCUSSION: The results of the pilot study justified development of MindSurf and further testing once it is available for use. A power analysis indicated that the pilot study was underpowered to detect significant effects but provided important information regarding the appropriate sample size for future research. The pilot study also indicated that future research should investigate the effects of receiving more than three questions per day. CONCLUSION: Results of the pilot study indicate that MindSurf will be a usable and acceptable app. Its benefits should be further explored through longer studies with larger sample of both the general population as well as clinical populations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The nine research articles that examined the 10 apps were made up of seven randomized controlled trials and two feasibility or pilot studies.
“The nine research articles [47-55] that examined the 10 apps were made up of seven randomized controlled trials (RCTs)—the Flett et al article contained one RCT with 2 of the listed apps: Headspace and Smiling Mind [52]—and two feasibility or pilot studies: Kinderman et al [48] and Carey et al [55].”
From Apps with maps-anxiety and depression mobile apps with evidence-based frameworks: Systematic search of major app stores (2020) · cites this paper as [55]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Lovibond, P. F., & Lovibond, S. H. (1995) · Behaviour Research and Therapy · doi:10.1016/0005-7967(94)00075-u · Cited by 14,523 · see citing works
(1995) · Cited by 6,909 · see citing works
Henry, J. D., & Crawford, J. R. (2005) · British Journal of Clinical Psychology · doi:10.1348/014466505x29657 · Cited by 5,801 · see citing works
Behavior: The Control of Perception
Kuhn, A., & Powers, W. T. (1975) · Contemporary Sociology · doi:10.2307/2063243 · Cited by 2,398 · see citing works
Text messaging as a tool for behavior change in disease prevention and management
Cole-Lewis, H., & Kershaw, T. (2010) · Epidemiologic Reviews · doi:10.1093/epirev/mxq004 · Cited by 1,190 · see citing works
Smartphones for smarter delivery of mental health programs: A systematic review
Donker, T., Petrie, K., Proudfoot, J., Clarke, J., Birch, M.-R., & Christensen, H. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2791 · Cited by 1,136 · see citing works
Mapping mHealth research: a decade of evolution
Fiordelli, M., Diviani, N., & Schulz, P. J. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2430 · Cited by 570 · see citing works
Howells, A., Ivtzan, I., & Eiroa-Orosa, F. J. (2016) · Journal of Happiness Studies · doi:10.1007/s10902-014-9589-1 · Cited by 407 · see citing works
Clara, I. P., Cox, B. J., & Enns, M. W. (2001) · Journal of Psychopathology and Behavioral Assessment · doi:10.1023/a:1011095624717 · Cited by 403 · see citing works
Torous, J., Chan, S. R., Yee-Marie Tan, S., Behrens, J., Mathew, I., Conrad, E. J., Hinton, L., Yellowlees, P., & Keshavan, M. (2014) · JMIR Mental Health · doi:10.2196/mental.4004 · Cited by 228 · see citing works
Quantifying App Store Dynamics: Longitudinal Tracking of Mental Health Apps
Larsen, M. E., Nicholas, J., & Christensen, H. (2016) · JMIR mHealth and uHealth · doi:10.2196/mhealth.6020 · Cited by 177 · see citing works
(2006) · Flinders Academic Commons (Flinders University) · Cited by 150 · see citing works
Self-citationRandall, W. M., Rickard, N. S., & Vella‐Brodrick, D. (2014) · Musicae Scientiae · doi:10.1177/1029864914536430 · Cited by 142 · see citing works
Carey, T. A., Tai, S. J., & Stiles, W. B. (2013) · Professional Psychology Research and Practice · doi:10.1037/a0035038 · Cited by 107 · see citing works
Assessing the Statistical and Personal Significance of the Method of Levels
Carey, T. A., Carey, M., Mullan, R. J., Spratt, C. G., & Spratt, M. B. (2009) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465809005232 · Cited by 86 · see citing works
Self-citationUsing Smartphone Apps to Promote Psychiatric and Physical Well-Being
Macias, C., Panch, T., Hicks, Y. M., Scolnick, J., Weene, D. L., Öngür, D., & Cohen, B. (2015) · Psychiatric quarterly · doi:10.1007/s11126-015-9337-7 · Cited by 78 · see citing works
A Transdiagnostic Approach to CBT using Method of Levels Therapy
Mansell, W., Carey, T. A., & Tai, S. J. (2012) · doi:10.4324/9780203081334 · Cited by 50 · see citing works
Self-citationA Transdiagnostic Approach to CBT using Method of Levels Therapy: Distinctive Features
Mansell, W., Carey, T. A., & Tai, S. J. (2012) · Research Explorer (The University of Manchester) · Cited by 44 · see citing works
Self-citationPrinciples-Based Counselling and Psychotherapy: A Method of Levels approach
Carey, T. A., Mansell, W., & Tai, S. J. (2015) · doi:10.4324/9781315695778 · Cited by 37 · see citing works
Self-citationCarey, T. A. (2008) · International Journal of Cognitive Therapy · doi:10.1521/ijct.2008.1.3.237 · Cited by 36 · see citing works
Self-citationIntroduction and Historical Overview
Kazantzis, N., & L’Abate, L. (2006) · Springer eBooks · doi:10.1007/978-0-387-29681-4_1 · Cited by 8 · see citing works
Effect Of Telephone Calls And Text Messages On Goal Attainment In A Ehealth Coaching Service
Brivio, E., Gatti, F. M., Galimberti, C., Paolo, G., & Binello, M. (2015) · Studies in health technology and informatics · doi:10.3233/978-1-61499-595-1-135 · Cited by 1 · see citing works
Flagged first, then by citations
1 of the 23 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Lattie, E. G., Adkins, E. C., Winquist, N., Stiles-Shields, C., Wafford, Q. E., & Graham, A. K. (2019) · Journal of Medical Internet Research · doi:10.2196/12869 · Cited by 843 · see citing works
“As seen in Multimedia Appendix 55,64,66,98,114,117]) or did not report on psychological outcomes and focused on program usage or usability (n=6 [116,118,119,121,126,129]).”
Lau, N., O'Daffer, A., Colt, S., Yi-Frazier, J. P., Palermo, T. M., McCauley, E., & Rosenberg, A. R. (2020) · JMIR mhealth and uhealth · doi:10.2196/17798 · Cited by 174 · see citing works
“Of the 10 apps with feasibility or usability studies, the majority (8/10 apps) reported that users found the apps to be enjoyable, accessible, and acceptable (Multimedia Appendix 5) [39,59-67].”
BackgroundSlater, H., Campbell, J. M., Stinson, J. N., Burley, M. M., & Briggs, A. M. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8888 · Cited by 155 · see citing works
“Reasons for exclusion included not being a research paper [42], not being qualitative or having a qualitative component [43-50], investigating the wrong phenomena of interest [51-53], not meeting the definition of mHealth [54,55], the population being outside the target age band [19,56-62], and the population being af…”
The Digital Psychiatrist: In Search of Evidence-Based Apps for Anxiety and Depression
Marshall, J. M., Dunstan, D. A., & Bartik, W. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00831 · Cited by 107 · see citing works
Usability Evaluations of Mobile Mental Health Technologies: Systematic Review
İnal, Y., Wake, J. D., Guribye, F., & Nordgreen, T. (2019) · Journal of Medical Internet Research · doi:10.2196/15337 · Cited by 104 · see citing works
“…et al, 2014 [24]; Barrio et al, 2017 [25]; Bauer et al, 2018 [26]; Ben-Zeev et al, 2013 [27]; Boman and Bartfai, 2015 [29]; Boyd et al, 2017 [30]; Carey et al, 2016 [57]; Corden et al, 2016 [31]; de Korte et al, 2018 [59]; Deady et al, 2018 [32]; Dulin et al, 2014 [33]; Fuller-Tyszkiewicz et al,…”
InfluentialMarshall, J. M., Dunstan, D. A., & Bartik, W. (2020) · JMIR Mental Health · doi:10.2196/16525 · Cited by 83 · see citing works
“Mean age (SD): not given Carey et al [55] MindSurf”
BackgroundDodd, A. L., Priestley, M., Tyrrell, K., Cygan, S., Newell, C., & Byrom, N. C. (2021) · Journal of Mental Health · doi:10.1080/09638237.2021.1875419 · Cited by 89 · see citing works
College Student Perspectives of Telemental Health: a Review of the Recent Literature
Hadler, N. L., Bu, P., Winkler, A., & Alexander, A. W. (2021) · Current Psychiatry Reports · doi:10.1007/s11920-020-01215-7 · Cited by 93 · see citing works
21-Day Stress Detox: Open Trial of a Universal Well-Being Chatbot for Young Adults
Williams, R. M., Hopkins, S. A., Frampton, C., Holt‐Quick, C., Merry, S., & Stasiak, K. (2021) · Social Sciences · doi:10.3390/socsci10110416 · Cited by 43 · see citing works
“We have found evidence of partial efficacy, similar to a number of other chatbots (Carey et al. 2016).”
ResultRossetto, F., Borgnis, F., Isernia, S., Foglia, E., Garagiola, E., Realdon, O., & Baglio, F. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1154481 · Cited by 36 · see citing works
Do young men and women differ in well-being apps usage? Findings from a randomised trial
Antezana, G., Venning, A., Smith, D., & Bidargaddi, N. P. (2022) · Health Informatics Journal · doi:10.1177/14604582211064825 · Cited by 31 · see citing works
Kasson, E. M., Vázquez, M. M., Doroshenko, C., Fitzsimmons‐Craft, E. E., Wilfley, D. E., Taylor, C. B., & Cavazos‐Rehg, P. A. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18157979 · Cited by 21 · see citing works
Shahsavar, Y., & Choudhury, A. (2025) · PLoS ONE · doi:10.1371/journal.pone.0319983 · Cited by 16 · see citing works
“[80] To evaluate an app that uses metacognitive questions to address mental health problems.”
InfluentialWakimizu, R., Sasaki, K., Yoshimoto, M., Miyazaki, A., & Saito, Y. (2022) · Frontiers in Pediatrics · doi:10.3389/fped.2022.919865 · Cited by 15 · see citing works
Muhideen, S., Chu, J., Akintunde, T. Y., Rufai, O. H., Amosun, T. S., & George-Ufot, G. (2022) · Smart Health · doi:10.1016/j.smhl.2022.100286 · Cited by 13 · see citing works
“In the findings, most treated reliance on mHealth is availability (Carey et al., 2016), device and communication affordance (Bauer et al., 2018; De Korte et al., 2018), the novelty of mHealth, and user acceptance (Barrio et al., 2017; De Korte et al., 2018), positive scientific knowledge of mHealth…”
Nabunya, P., Cavazos‐Rehg, P. A., Mugisha, J., Kasson, E. M., Namuyaba, O. I., Najjuuko, C., Nsubuga, E., Filiatreau, L. M., Mwebembezi, A., & Ssewamala, F. M. (2024) · JMIR Research Protocols · doi:10.2196/54635 · Cited by 8 · see citing works
“This is within the range of sample sizes for other pilot tests of digital therapeutics [77,89], and the suggested number of participants for pilot studies [90].”
BackgroundNewCope: A Theory-Linked Mobile Application for Stress Education and Management
Carter, L., Rogith, D., Amy, F., & Myneni, S. (2019) · Studies in health technology and informatics · doi:10.3233/shti190406 · Cited by 8 · see citing works
“Method of Levels transdiagnostic cognitive therapy was used in the development of the “MindSurf” app, which showed positive results in preliminary function and usability testing [29].”
MethodAntezana, G., Venning, A., Smith, D., & Bidargaddi, N. P. (2022) · Digital Health · doi:10.1177/20552076221144104 · Cited by 8 · see citing works
“Participant characteristics Most studies (18/22) reported higher female participation36–39,41–43,45–47,49,51–57; this is consistent with existing literature.(55) Three studies reported mostly male(44,48,50) (one of them focusing solely on males(48)), and one study did not report on gender.”
BackgroundMethodResultInfluentialUsability Testing an mHealth Program with Tailored Motivational Messages for Early Adolescents
Lin, C. A., Vosburgh, K. L., Roy, D., & Duffy, V. B. (2023) · Nutrients · doi:10.3390/nu15030574 · Cited by 7 · see citing works
“The sample size of past usability studies conducted for testing mHealth apps varies from 10 [51] to 23 [52].”
BackgroundThe development and evaluation of the Depth and Duration of Awareness Coding Scheme (D-DACS)
Higginson, S., & Mansell, W. (2018) · The Cognitive Behaviour Therapist · doi:10.1017/s1754470x1800020x · Cited by 5 · see citing works
“…of Levels therapy (MOL; Carey, 2006; Alsawy et al., 2014), group courses (Morris et al., 2016), and digital applications (Gaffney et al., 2014; Carey et al., 2016), thereby making it important to build the methodological foundations of how to measure their potential mechanism of change; (3)…”
BackgroundSelf-citationAlSideiri, A., Alsharida, R. A. F., Hammood, M. M., Shakir, M., Cob, Z. C., Shamsi, I. R. A., Muttar, A. K., Abood, R., & Sheker, M. T. (2022) · ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS) · doi:10.1109/icetsis55481.2022.9888852 · Cited by 4 · see citing works
“Saraswat[7] proposed a system that records the attendance of students through a fingerprint.”
BackgroundJoshi, A., Kaur, K., Bhatt, A., Surapaneni, K. M., Grover, A., & Pandya, A. K. (2023) · JMIR Research Protocols · doi:10.2196/39779 · Cited by 1 · see citing works
“Various digital interventions, including computer-based [26,27], web-based [28,29], mobile-based (SMS text message–based and app-based), and virtual reality–based [30-32] interventions have been used for depression, anxiety, stress, sleep quality, psychological well-being, and mindfulness.”
BackgroundPCT Within Psychological Interventions
Morris, L., McEvoy, P., Wallwork, T., Bates, R., Comiskey, J., & Mansell, W. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-813989-9.00003-8
Self-citationMuhideen, S., Chu, J., Akintunde, T. Y., Rufai, O. H., Shahani, R., & Jin, M. (2022) · Digital Health · doi:10.1177/20552076221132092
“41 The method employed was to overcome crowd-gathering of high transmission risk of COVID-19 and to abide by social distancing demands in the lives of social settings.”
MethodDas Smartphone als Medium in der Psychotherapie
Caplazi, S. (2023) · Psychotherapie-Wissenschaft · doi:10.30820/1664-9583-2023-1-73
Moharam, M. M. R. (2024) · Studies in systems, decision and control · doi:10.1007/978-3-031-65207-3_5
Tawalbeh, A., & Moharam, M. M. R. (2025) · Studies in systems, decision and control · doi:10.1007/978-3-031-84636-6_22
Heimbruch, H. H. H., Onwudiwe, A. O., Ogundare, S. A., & Eke, D. O. (2026) · Epidemiology and Health Data Insights · doi:10.63946/ehdi/18571
- Authors
- Carey, T. A., Haviland, J., Tai, S. J., Vanags, T., & Mansell, W.
- Affiliations
- Flinders University (AU)
- University of Manchester (GB)
- Corresponding author
- Carey, Timothy A.
- Venue
- BMC Psychiatry
- Published
- 2016-12
- Volume
- 16
- Issue
- 1
- Pages
- 442-442
- ISSN
- 1471-244X (ISSN-L)
- PMC
- PMC5153894
- OpenAlex
- W2566086561
- Cited by
- 29
- References
- 23
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 2 · 2023: 4 · 2022: 6 · 2021: 4 · 2020: 2 · 2019: 4 · 2018: 2 · 2017: 2
- FWCI
- 4.11
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Psychiatry (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (submittedVersion, other-oa)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- mobile application, perceptual control theory, contentment, well-being, goal achievement, usability, acceptability, mental health promotion, mobile mental health, depression, anxiety and stress, adverse events
- MeSH terms
- Smartphone, Adult, Female, Goals, Health Education, Health Promotion, Humans, Male, Pilot Projects, Quality of Life, Cell Phone, Young Adult, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
