Developing an Internet-Based Cognitive Behavioral Therapy Intervention for Adolescents With Anxiety Disorders: Design, Usability, and Initial Evaluation of the CoolMinds Intervention
Open accessPublished version of a preprint
Abstract
Background: Digital mental health interventions may help increase access to psychological treatment for adolescents with anxiety disorders. However, many clinical evaluations of digital treatments report low adherence and engagement and high dropout rates, which remain challenges when the interventions are implemented in routine care. Involving intended end users in the development process through user-centered design methods may help maximize user engagement and establish the validity of interventions for implementation. Objective: This study aimed to describe the methods used to develop a new internet-based cognitive behavioral therapy intervention, CoolMinds, within a user-centered design framework. Methods: The development of intervention content progressed in three iterative design phases: (1) identifying needs and design specifications, (2) designing and testing prototypes, and (3) running feasibility tests with end users. In phase 1, a total of 24 adolescents participated in a user involvement workshop exploring their preferences on graphic identity and communication styles as well as their help-seeking behavior. In phase 2, a total of 4 adolescents attended individual usability tests in which they were presented with a prototype of a psychoeducational session and asked to think aloud about their actions on the platform. In phase 3, a total of 7 families from the feasibility trial participated in a semistructured interview about their satisfaction with and initial impressions of the platform and intervention content while in treatment. Activities in all 3 phases were audio recorded, transcribed, and coded using thematic analysis and qualitative description design. The intervention was continuously revised after each phase based on the feedback. Results: In phase 1, adolescent feedback guided the look and feel of the intervention content (ie, color scheme, animation style, and communication style). Participants generally liked content that was relatable and age appropriate and felt motivating. Animations that resembled “humans” received more votes as adolescents could better “identify” themselves with them. Communication should preferably be “supportive” and feel “like a friend” talking to them. Statements including praise—such as “You’re well on your way. How are you today?”—received the most votes (12 votes), whereas directive statements such as “Tell us how your day has been?” and “How is practicing your steps going?” received the least votes (2 and 0 votes, respectively). In phase 2, adolescents perceived the platform as intuitive and easy to navigate and the session content as easy to understand but lengthy. In phase 3, families were generally satisfied with the intervention content, emphasizing the helpfulness of graphic material to understand therapeutic content. Their feedback helped identify areas for further improvement, such as editing down the material and including more in-session breaks. Conclusions: Using user involvement practices in the development of interventions helps ensure continued alignment of the intervention with end-user needs and may help establish the validity of the intervention for implementation in routine care practice.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Previous studies with participatory approaches when developing CBT interventions for youth anxiety have primarily focused on digital interventions.
“When developing CBT interventions for youth anxiety, previous studies with participatory approaches have primarily been focused on digital interventions [18–23].”
From Developing a primary care-based cognitive behavioral intervention for anxiety in children through a participatory approach: a qualitative study. (2026) · cites this paper as [22]
The Introduction (Background; page unavailable) identifies previous studies that used end-user involvement or co-design to develop internet-based CBT interventions for young people with anxiety. It does not compare digital with nondigital participatory studies, so it does not establish that previous work has primarily focused on digital interventions. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Harris, P. A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N., & Conde, J. G. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.08.010 · Cited by 51,384 · cited 1× · see citing works
“…an online consent form distributed using REDCap (Research Electronic Data Capture; Vanderbilt University), a secure, web-based software platform designed to support data capture for research studies [38,39] hosted at the Open Patient Data Exploratory Network within the Region of Southern Denmark.”
The REDCap consortium: building an international community of software platform partners
Harris, P. A., Taylor, R., Minor, B. L., Elliott, V., Fernandez, M., O'Neal, L., McLeod, L., Delacqua, G., Delacqua, F., Kirby, J., et al. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103208 · Cited by 24,013 · cited 1× · see citing works
“…an online consent form distributed using REDCap (Research Electronic Data Capture; Vanderbilt University), a secure, web-based software platform designed to support data capture for research studies [38,39] hosted at the Open Patient Data Exploratory Network within the Region of Southern Denmark.”
Social Cognitive Theory: An Agentic Perspective
Bandura, A. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.1 · Cited by 15,510 · see citing works
THE ROLE OF TUTORING IN PROBLEM SOLVING <sup>*</sup>
Wood, D., Bruner, J. S., & Ross, G. (1976) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.1976.tb00381.x · Cited by 8,851 · cited 2× · see citing works
“Scaffolding refers to the process “that enables a child or novice to solve a task or achieve a goal that would be beyond his unassisted efforts” [42].”
Gulliver, A., Griffiths, K. M., & Christensen, H. (2010) · BMC Psychiatry · doi:10.1186/1471-244x-10-113 · Cited by 3,292 · see citing works
Qualitative description – the poor cousin of health research?
Neergaard, M. A., Olesen, F., Andersen, R. S., & Søndergaard, J. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-52 · Cited by 2,099 · see citing works
Staniszewska, S., Brett, J., Simera, I., Seers, K., Mockford, C., Goodlad, S., Altman, D. G., Moher, D., Barber, R., Denegri, S., et al. (2017) · BMJ · doi:10.1136/bmj.j3453 · Cited by 1,983 · cited 1× · see citing works
“Reporting of the study followed the Guidance for Reporting Involvement of Patients and the Public, and a Guidance for Reporting Involvement of Patients and the Public long-form checklist is available in Multimedia Appendix 1 [25].”
Staniszewska, S., Brett, J., Simera, I., Seers, K., Mockford, C., Goodlad, S., Altman, D. G., Moher, D., Barber, R., Denegri, S., et al. (2017) · Research Involvement and Engagement · doi:10.1186/s40900-017-0062-2 · Cited by 1,643 · see citing works
Yardley, L., Morrison, L., Bradbury, K., & Muller, I. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4055 · Cited by 1,442 · cited 1× · see citing works
“User-centered design processes typically involve three phases of development: (1) identifying needs and design specifications (with end users), (2) designing prototypes of the intervention and testing their usability, and (3) running feasibility tests with end users [17,18].”
Evidence Base Update: 50 Years of Research on Treatment for Child and Adolescent Anxiety
Higa-McMillan, C. K., Francis, S. E., Rith-Najarian, L., & Chorpita, B. F. (2016) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2015.1046177 · Cited by 557 · cited 1× · see citing works
“Several reviews and meta-analyses suggest that key CBT components such as exposure therapy (in vivo and imaginal) and cognitive restructuring are some of the main components driving the treatment effects [27,32,33].”
User-Centered Design and Interactive Health Technologies for Patients
DABBS, A. D. V., MYERS, B. A., MC CURRY, K. R., DUNBAR-JACOB, J., HAWKINS, R. P., BEGEY, A., & DEW, M. A. (2009) · CIN: Computers, Informatics, Nursing · doi:10.1097/ncn.0b013e31819f7c7c · Cited by 517 · cited 1× · see citing works
“User-centered design processes typically involve three phases of development: (1) identifying needs and design specifications (with end users), (2) designing prototypes of the intervention and testing their usability, and (3) running feasibility tests with end users [17,18].”
Garrido, S., Millington, C., Cheers, D., Boydell, K., Schubert, E., Meade, T., & Nguyen, Q. V. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00759 · Cited by 400 · cited 1× · see citing works
“This is in line with previous research emphasizing the importance of including relatable and age-appropriate content [16,35,36,41] and, thus, including adolescents in the design process to clarify which types of content are seen as relevant.”
A systematic review of online youth mental health promotion and prevention interventions
Clarke, A. M., Kuosmanen, T., & Barry, M. M. (2015) · Journal of Youth and Adolescence · doi:10.1007/s10964-014-0165-0 · Cited by 381 · cited 1× · see citing works
“However, many evaluations of ICBT programs for children and adolescents with anxiety report high rates of dropout and that participants often fail to complete assessments or treatment sessions [8-12].”
March, S., Spence, S. H., & Donovan, C. L. (2009) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsn099 · Cited by 315 · see citing works
Vigerland, S., Lenhard, F., Bonnert, M., Lalouni, M., Hedman, E., Ahlen, J., Olén, O., Serlachius, E., & Ljótsson, B. (2016) · Clinical Psychology Review · doi:10.1016/j.cpr.2016.09.005 · Cited by 292 · see citing works
Pennant, M. E., Loucas, C. E., Whittington, C., Creswell, C., Fonagy, P., Fuggle, P., Kelvin, R., Naqvi, S., Stockton, S., & Kendall, T. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.01.009 · Cited by 269 · see citing works
Whiteside, S. P. H., Sim, L. A., Morrow, A. S., Farah, W. H., Hilliker, D. R., Murad, M. H., & Wang, Z. (2020) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-019-00303-2 · Cited by 190 · cited 2× · see citing works
“It was decided to develop a cognitive behavioral therapy (CBT)–based program as this is the most well-researched approach and has shown good results in previous evaluations using the internet-based format [27-31].”
Bennett, S. D., Cuijpers, P., Ebert, D. D., McKenzie Smith, M., Coughtrey, A. E., Heyman, I., Manzotti, G., & Shafran, R. (2019) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.13010 · Cited by 162 · cited 1× · see citing works
“It is also well documented that internet-based cognitive behavioral therapy (ICBT) with support from a therapist (eg, via telephone or email) is effective for a variety of mental health problems, including anxiety in children and adolescents [4-7].”
Vigerland, S., Ljótsson, B., Thulin, U., Öst, L.-G., Andersson, G., & Serlachius, E. (2016) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.11.006 · Cited by 136 · cited 1× · see citing works
“It was decided to develop a cognitive behavioral therapy (CBT)–based program as this is the most well-researched approach and has shown good results in previous evaluations using the internet-based format [27-31].”
Nearchou, F. A., Bird, N., Costello, A., Duggan, S., Gilroy, J., Long, R., McHugh, L., & Hennessy, E. (2018) · Journal of Adolescence · doi:10.1016/j.adolescence.2018.05.003 · Cited by 135 · see citing works
A randomized controlled trial of the cool teens CD-ROM computerized program for adolescent anxiety
Wuthrich, V. M., Rapee, R. M., Cunningham, M. J., Lyneham, H. J., Hudson, J. L., & Schniering, C. A. (2012) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2011.12.002 · Cited by 124 · cited 1× · see citing works
“However, many evaluations of ICBT programs for children and adolescents with anxiety report high rates of dropout and that participants often fail to complete assessments or treatment sessions [8-12].”
Radež, J., Reardon, T. C., Creswell, C., Orchard, F., & Waite, P. (2021) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-020-01707-0 · Cited by 110 · see citing works
Spence, S. H., Donovan, C. L., March, S., Kenardy, J. A., & Hearn, C. S. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.12.003 · Cited by 101 · see citing works
Fleming, T. M., Merry, S., Stasiak, K., Hopkins, S. A., Patolo, T., Ruru, S., Latu, M., Shepherd, M., Christie, G., & Goodyear‐Smith, F. (2019) · JMIR Mental Health · doi:10.2196/12656 · Cited by 98 · cited 1× · see citing works
“However, working experts in the field of ICBT for children and young people propose that the use of user-centered design methods may help maximize user engagement [14,15].”
March, S., Spence, S. H., Donovan, C. L., & Kenardy, J. A. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9211 · Cited by 84 · see citing works
Jolstedt, M., Wahlund, T., Lenhard, F., Ljótsson, B., Mataix-Cols, D., Nord, M., Öst, L.-G., Högström, J., Serlachius, E., & Vigerland, S. (2018) · The Lancet Child & Adolescent Health · doi:10.1016/s2352-4642(18)30275-x · Cited by 76 · see citing works
Nordh, M., Wahlund, T., Jolstedt, M., Sahlin, H., Bjureberg, J., Åhlén, J., Lalouni, M., Salomonsson, S., Vigerland, S., Lavner, M., et al. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2021.0469 · Cited by 75 · see citing works
Stjerneklar, S., Hougaard, E., McLellan, L. F., & Thastum, M. (2019) · PLOS ONE · doi:10.1371/journal.pone.0222485 · Cited by 71 · see citing works
Horvath, K. J., Ecklund, A. M., Hunt, S., Nelson, T. F., & Toomey, T. L. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.3770 · Cited by 63 · cited 1× · see citing works
“The purpose of these meetings was to facilitate the decision-making and implementation processes by assembling relevant team members [26].”
Wickersham, A., Barack, T., Cross, L., & Downs, J. (2022) · Journal of Medical Internet Research · doi:10.2196/29842 · Cited by 49 · see citing works
Spence, S. H., March, S., & Donovan, C. L. (2019) · Internet Interventions · doi:10.1016/j.invent.2019.100268 · Cited by 45 · cited 3× · see citing works
“It was also decided to create a separate parent program and offer the treatment as a family-based intervention as social support from parents may increase treatment adherence and response when treatment is delivered in a digital format [37].”
InfluentialHill, C., Creswell, C., Vigerland, S., Nauta, M. H., March, S., Donovan, C., Wolters, L., Spence, S. H., Martin, J. L., Wozney, L., et al. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.02.002 · Cited by 44 · cited 3× · see citing works
“However, working experts in the field of ICBT for children and young people propose that the use of user-centered design methods may help maximize user engagement [14,15].”
Online Support and Intervention for Child Anxiety (OSI): Development and Usability Testing
Hill, C., Reardon, T., Taylor, L., & Creswell, C. (2022) · JMIR Formative Research · doi:10.2196/29846 · Cited by 43 · cited 1× · see citing works
“In an initial evaluation of the clinical effectiveness, Hill et al [21] found that the dropout rate for their intervention was lower (13”
Ludlow, K., Russell, J. K., Ryan, B., Brown, R. L., Joynt, T., Uhlmann, L. R., Smith, G. E., Donovan, C., Hides, L., Spence, S. H., et al. (2023) · DIGITAL HEALTH · doi:10.1177/20552076231216410 · Cited by 42 · cited 6× · see citing works
“In a study by Ludlow et al [16], the research team used co-design methods to understand preferences and perceptions of the design, functions, and engagement features within a new digital mental health platform.”
InfluentialMcCall, H. C., Hadjistavropoulos, H. D., & Sundström, C. (2021) · Journal of Medical Internet Research · doi:10.2196/26939 · Cited by 39 · cited 1× · see citing works
“Some research conducted with adult populations indicates that using user-centered practices may help increase engagement [46,47], but more research is generally needed.”
Wozney, L., Baxter, P., & Newton, A. S. (2015) · BMC Pediatrics · doi:10.1186/s12887-015-0534-1 · Cited by 38 · cited 1× · see citing works
“To the best our knowledge, only a few studies describe the inclusion of end users in the design of ICBT interventions for children and adolescents with anxiety [16,19,20].”
Stjerneklar, S., Hougaard, E., & Thastum, M. (2019) · Internet Interventions · doi:10.1016/j.invent.2019.01.003 · Cited by 38 · cited 1× · see citing works
“However, in a predictor analysis by Stjerneklar et al [43], the time spent by parents helping their adolescents complete treatment did not significantly predict treatment outcome.”
Waite, P., Marshall, T., & Creswell, C. (2019) · Child and Adolescent Mental Health · doi:10.1111/camh.12311 · Cited by 36 · cited 2× · see citing works
“%) than those reported in other studies conducted in routine care (32% in the study by Moor et al [22], 21% in the study by Waite et al [13], and 17% in the study by Vigerland et al [23]).”
Radomski, A. D., Bagnell, A., Curtis, S., Hartling, L., & Newton, A. S. (2020) · JMIR Mental Health · doi:10.2196/15795 · Cited by 30 · cited 2× · see citing works
“This is also in line RenderX with findings from qualitative evaluations of ICBT treatments, where adolescents preferred easy-to-use techniques [35,36].”
Moor, S., Williman, J., Drummond, S., Fulton, C., Mayes, W., Ward, N., Dovenberg, E., Whitaker, C., & Stasiak, K. (2019) · Internet Interventions · doi:10.1016/j.invent.2019.100249 · Cited by 26 · cited 1× · see citing works
“%) than those reported in other studies conducted in routine care (32% in the study by Moor et al [22], 21% in the study by Waite et al [13], and 17% in the study by Vigerland et al [23]).”
Luo, A., & McAloon, J. (2020) · Depression and Anxiety · doi:10.1002/da.23116 · Cited by 16 · cited 1× · see citing works
“Several reviews and meta-analyses suggest that key CBT components such as exposure therapy (in vivo and imaginal) and cognitive restructuring are some of the main components driving the treatment effects [27,32,33].”
CBT for Childhood Anxiety: Reviewing the State of Personalised Intervention Research
Bertie, L., & Hudson, J. L. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.722546 · Cited by 16 · cited 1× · see citing works
“Evidence on the efficacy of coping strategies is scarce compared to that for the key components [34].”
Hill, C., Chessell, C., Percy, R., & Creswell, C. (2022) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465822000157 · Cited by 13 · see citing works
Smart, K., Smith, L., Harvey, K., & Waite, P. (2021) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-021-01903-6 · Cited by 13 · cited 2× · see citing works
“This is also in line RenderX with findings from qualitative evaluations of ICBT treatments, where adolescents preferred easy-to-use techniques [35,36].”
Patterson, V. C., Rossi, M. A., Pencer, A., & Wozney, L. (2022) · JMIR Formative Research · doi:10.2196/33374 · Cited by 11 · cited 1× · see citing works
“Some research conducted with adult populations indicates that using user-centered practices may help increase engagement [46,47], but more research is generally needed.”
Ludlow, K., March, S., Russell, J. K., Ryan, B., Brown, R. L., Hides, L., Donovan, C., Spence, S. H., Saxby, J., & Cobham, V. E. (2024) · DIGITAL HEALTH · doi:10.1177/20552076241263693 · Cited by 7 · cited 1× · see citing works
“This is in line with findings from a co-design study by Ludlow et al [45] with caregivers of young people aged 7 to 17 years.”
Vigerland, S., Fredlander, S., Aspvall, K., Jolstedt, M., Lenhard, F., Mataix‐Cols, D., Ljótsson, B., & Serlachius, E. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100738 · Cited by 3 · cited 1× · see citing works
“%) than those reported in other studies conducted in routine care (32% in the study by Moor et al [22], 21% in the study by Waite et al [13], and 17% in the study by Vigerland et al [23]).”
Skaarnes, H., Sørensen, N. M., Wisnewski, A. H., Lomholt, J. J., Thastum, M., McLellan, L. F., & Mathiasen, K. (2024) · Trials · doi:10.1186/s13063-024-08511-0 · Cited by 3 · cited 1× · see citing works
“The feasibility trial and RCT were ethically approved by the Medical Research Ethics Committees in Denmark (ID: VMK 2211954), and the protocol for these studies was preregistered in ClinicalTrials.gov (NCT06076964) and published in Trials [24].”
Self-citationStjerneklar, S., Hougaard, E., & Thastum, M. (2019) · Internet Interventions · doi:10.1016/j.invent.2019.100253 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Isaksson, A., Åhlén, J., Hasson, H., & Eriksson, L. (2026) · BMC Primary Care · doi:10.1186/s12875-026-03191-y
Sørensen, N. M., Skaarnes, H., Mathiasen, K., McLellan, L. F., Thastum, M., & Lomholt, J. J. (2025) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2025.2522995 · Cited by 2 · see citing works
“In addition, the content of the first four sessions was edited down significantly (Sørensen et al., 2025).”
InfluentialSelf-citationOhu, F. C., & Jones, L. A. (2025) · Scientific Bulletin · doi:10.2478/bsaft-2025-0024
“Evidence from Social Identity Theory and Cognitive Behavioral Theory shows that peer conformity and maladaptive thinking patterns heighten risk but also provide intervention pathways for resilience (Pérez-Torres, 2024; Sørensen et al., 2025).”
- Authors
- Sørensen, N. M., Skaarnes, H., Mathiasen, K., Thastum, M., & Lomholt, J. J.
- Affiliations
- Aarhus University (DK)
- University of Southern Denmark (DK)
- Corresponding author
- Sørensen, Nikita Marie
- Venue
- JMIR Formative Research
- Publisher
- JMIR Publications Inc.
- Published
- 2025-04-08
- Volume
- 9
- Pages
- e66966
- ISSN
- 2561-326X (ISSN-L)
- PMC
- PMC12015348
- OpenAlex
- W4408081789
- Cited by
- 3
- References
- 49
- Citations by year
- 2026: 1 · 2025: 2
- FWCI
- 2.40
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Formative Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- University of Southern Denmark Research Portal (University of Southern Denmark) (acceptedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- TrygFonden
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Keywords
- internet-based cognitive behavioral therapy, adolescents, anxiety disorders, user-centered design, usability testing, help-seeking, animation style, family satisfaction, digital mental health interventions, intervention development, dropout, psychoeducation, help-seeking barriers, help-seeking facilitators, help-seeking self-efficacy
- MeSH terms
- Internet-Based Intervention, User-Centered Design, Adolescent, Anxiety Disorders, Feasibility Studies, Female, Humans, Male, Patient Acceptance of Health Care, Cognitive Behavioral Therapy, Patient Preference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
