Examining the Usage, User Experience, and Perceived Impact of an Internet-Based Cognitive Behavioral Therapy Program for Adolescents With Anxiety: Randomized Controlled Trial

“Adolescents were ineligible for participation if they (1) screened as high risk for self-harm via four items from the Ask Suicide-Screening Questionnaire [43] (a yes answer to thoughts about killing oneself in the past week or a prior attempt), (2) indicated the possible presence of a psychosis-related disorder via th…”

“Considering that multiple iCBT components work together to form a complex intervention [99], we recommend connecting the persuasive system design features known to relate to a positive user experience (program reminders, progress and feedback tools, multimedia demonstrations, and flexible program…”

“This GRCS change of 2 points on an 11-point scale is consistent with the MCID (change) of half a standard deviation from a large systematic review of health care outcome studies [62].”

“Anxiety disorders are the most prevalent mental health concern in children and adolescents, affecting about 8% to 11% of youth [1-3].”

“Consistent with the literature, Breathe respondents described difficulty remembering to work on the program [29,52], concerns with privacy and stigma (eg, others knowing about or judging their help seeking) [30,71,72], time constraints, and conflicting commitments [31,73-75], and delaying or avoiding tasks they found…”

“Inclusion criteria were as follows: (1) a minimum score of 25 on the Screen for Child Anxiety Related Disorders [42], indicating the presence of clinical anxiety symptoms; (2) the ability to read and write English; (3) regular access to a telephone and a computer system with high-speed internet service; and (4) the ab…”

“Attrition is said to be a fundamental characteristic and methodological limitation of longitudinal iCBT studies [121-123]; however, our attrition rates are consistent with dropouts in outpatient therapy settings [9].”

“These features included interactive surveys and graphs (designed to provide feedback, increase adolescents’ awareness of their changes over time, and help with goal setting [95-97]), and video clips showing in-vivo exposure and diaphragmatic breathing (designed to provide step-by-step peer simulations of therapeutic a…”

“Third, we identified the mean change in MASC-2 scores for adolescents who reported experiencing a small change in their anxiety (ie, a +2 response rating on the GRCS, somewhat better) to provide the final MCID estimate [35,60,61].”

“The composite and ranges can be corroborated using Delphi (eg, clinical or expert opinion) or distribution-based methods (eg, effect size and standard error of measurement) [59,116], triangulating multiple approaches to calculating the MCID to improve the robustness of the estimate [101].”

“02), providing face validity for the GRCS to indicate changes in adolescents’ anxiety symptoms [55].”

“Currently, there is considerable heterogeneity in how the user experience is defined and evaluated, with most research being conducted with adult populations [65,69,110,111].”

“Unlike face-to-face CBT where treatment may involve use of a workbook and in-person meetings with a therapist, iCBT provides therapeutic content and strategies through structured modules and activities (Web-based or offline) that involve the use of multimedia (eg, video and audio) and other technological features (eg,…”

“Yet, low usage patterns have been consistently reported in the literature, with typically more than 50% of participants not completing an iCBT program as part of a research study [14,17,32-34].”

“Perceived impact may also be based on unique individual factors, such as treatment expectancy, preintervention anxiety severity, self-regulation abilities, or motivational factors [69,102,109], factors that we did not assess.”

“Use of factor analysis [90] or multiple regression [91] could help to illuminate how different constructs of user experience relate to one another and to intervention use and how the constructs change over the course of treatment.”

“…relationship between iCBT use and change in anxiety (a dose-response relationship) exists—some studies have found evidence for this relationship [102,103], whereas others have not [104,105]; however, there is consensus that some degree of program use is required to reduce users’ symptoms [106-108].”

“Although other studies have looked to user demographics to provide explanations in low program use, explanations have been mixed [13,15,18], which suggests new approaches to understanding program use are needed.”

“Unlike face-to-face CBT where treatment may involve use of a workbook and in-person meetings with a therapist, iCBT provides therapeutic content and strategies through structured modules and activities (Web-based or offline) that involve the use of multimedia (eg, video and audio) and other technological features (eg,…”

“Thus, the MCID estimate calculated can vary between samples with different participant characteristics (eg, baseline severity and previous treatment experiences) [55,59,118].”

“Attrition is said to be a fundamental characteristic and methodological limitation of longitudinal iCBT studies [121-123]; however, our attrition rates are consistent with dropouts in outpatient therapy settings [9].”

“Such evaluations may reveal how the user experience changes over time, how it can be optimized [89], and how to improve the accuracy of collected data on the user experience (eg, reduce recall bias and link user experience domains to specific program sessions).”

“The composite and ranges can be corroborated using Delphi (eg, clinical or expert opinion) or distribution-based methods (eg, effect size and standard error of measurement) [59,116], triangulating multiple approaches to calculating the MCID to improve the robustness of the estimate [101].”

“Recent systematic reviews and meta-analyses demonstrate that in reducing anxiety in adolescents, iCBT has comparable effectiveness with traditional, face-to-face CBT [10,12-14] and is more effective than waiting for treatment [10,13,15-18].”

“Third, we identified the mean change in MASC-2 scores for adolescents who reported experiencing a small change in their anxiety (ie, a +2 response rating on the GRCS, somewhat better) to provide the final MCID estimate [35,60,61].”

“These features included interactive surveys and graphs (designed to provide feedback, increase adolescents’ awareness of their changes over time, and help with goal setting [95-97]), and video clips showing in-vivo exposure and diaphragmatic breathing (designed to provide step-by-step peer simulations of therapeutic a…”

“The composite and ranges can be corroborated using Delphi (eg, clinical or expert opinion) or distribution-based methods (eg, effect size and standard error of measurement) [59,116], triangulating multiple approaches to calculating the MCID to improve the robustness of the estimate [101].”

“For iCBT programs designed and delivered to adolescents with anxiety, program evaluations should aim to understand how iCBT is experienced by adolescents to further ensure its relevance, use, and impact as a self-help treatment [63-66].”

“A composite MCID estimate can then be generated by amalgamating MCID data across multiple studies to increase the generalizability and validity of the estimate [120] or provide a range of critical MCID values can be provided.”

“It may be that this type of support as well as the degree of support provided may help adolescents manage their time and complete challenging program activities [27,81,83-85].”

“Yet, low usage patterns have been consistently reported in the literature, with typically more than 50% of participants not completing an iCBT program as part of a research study [14,17,32-34].”

“…of cognitive behavior therapy (CBT) as first-line treatment of mild-to-moderate child and adolescent anxiety disorders with number needed to treat ranging from 3 to 6, but also some evidence that CBT is not significantly more effective than active control with support and education materials [6,7].”

“Evaluations of adolescent experiences with various iCBT program delivery and content formats have revealed good program usability (eg, program had few errors and it was easy to learn to use) [21-24], moderate-to-strong credibility (eg, the program contained expert and reliable information), promising treatment expecta…”

“There is a trend in the literature that some type of program support can increase program use or effectiveness of iCBT for children and adolescents [10]; however, inconsistent evidence is published [16,28,86,87], and what type of support, such as when it should be provided and by whom, that improves outcomes is unclea…”

“Internet-based CBT (iCBT), with its self-help format, can increase the access and availability of CBT for adolescents with mild-to-moderate anxiety [10,11].”

“Yet, low usage patterns have been consistently reported in the literature, with typically more than 50% of participants not completing an iCBT program as part of a research study [14,17,32-34].”

“There is a trend in the literature that some type of program support can increase program use or effectiveness of iCBT for children and adolescents [10]; however, inconsistent evidence is published [16,28,86,87], and what type of support, such as when it should be provided and by whom, that improves outcomes is unclea…”

“Consistent with the literature, Breathe respondents described difficulty remembering to work on the program [29,52], concerns with privacy and stigma (eg, others knowing about or judging their help seeking) [30,71,72], time constraints, and conflicting commitments [31,73-75], and delaying or avoiding tasks they found…”

“Evaluations of adolescent experiences with various iCBT program delivery and content formats have revealed good program usability (eg, program had few errors and it was easy to learn to use) [21-24], moderate-to-strong credibility (eg, the program contained expert and reliable information), promising treatment expecta…”

“There is a trend in the literature that some type of program support can increase program use or effectiveness of iCBT for children and adolescents [10]; however, inconsistent evidence is published [16,28,86,87], and what type of support, such as when it should be provided and by whom, that improves outcomes is unclea…”

“Anxiety disorders are the most prevalent mental health concern in children and adolescents, affecting about 8% to 11% of youth [1-3].”

“Similar to other iCBT studies, Breathe respondents reported that the program was easy to understand [92], met their needs [79], and that they were satisfied overall [29,93,94].”

“These features included interactive surveys and graphs (designed to provide feedback, increase adolescents’ awareness of their changes over time, and help with goal setting [95-97]), and video clips showing in-vivo exposure and diaphragmatic breathing (designed to provide step-by-step peer simulations of therapeutic a…”

“The overall intent of these objectives was to examine self-reported user experience data and automatically captured program usage data together for a better understanding of the relationship between behavioral (objective usage) and experiential (subjective usage, user experience, and MCID) data [38-40] to explain and…”

“There is a trend in the literature that some type of program support can increase program use or effectiveness of iCBT for children and adolescents [10]; however, inconsistent evidence is published [16,28,86,87], and what type of support, such as when it should be provided and by whom, that improves outcomes is unclea…”

“Recent systematic reviews and meta-analyses demonstrate that in reducing anxiety in adolescents, iCBT has comparable effectiveness with traditional, face-to-face CBT [10,12-14] and is more effective than waiting for treatment [10,13,15-18].”

“To target our anticipated participants, we used current, key literature [30,52,53,70,112-114] to develop the UEQII.”

“Previous iCBT studies have used clinical severity ratings (ratings have ranged from 0=none to 8=extremely severe) as a proximal indicator of treatment response [27,29,79,81].”

“Recent systematic reviews and meta-analyses demonstrate that in reducing anxiety in adolescents, iCBT has comparable effectiveness with traditional, face-to-face CBT [10,12-14] and is more effective than waiting for treatment [10,13,15-18].”

“Children and adolescents with anxiety disorders are at increased risk of academic and social difficulties and have an increased likelihood of developing secondary anxiety disorders and depression [4,5].”

“For iCBT programs designed and delivered to adolescents with anxiety, program evaluations should aim to understand how iCBT is experienced by adolescents to further ensure its relevance, use, and impact as a self-help treatment [63-66].”

“Previous iCBT studies have used clinical severity ratings (ratings have ranged from 0=none to 8=extremely severe) as a proximal indicator of treatment response [27,29,79,81].”

“…relationship between iCBT use and change in anxiety (a dose-response relationship) exists—some studies have found evidence for this relationship [102,103], whereas others have not [104,105]; however, there is consensus that some degree of program use is required to reduce users’ symptoms [106-108].”

“Consistent with the literature, Breathe respondents described difficulty remembering to work on the program [29,52], concerns with privacy and stigma (eg, others knowing about or judging their help seeking) [30,71,72], time constraints, and conflicting commitments [31,73-75], and delaying or avoiding tasks they found…”

“Most previously studied iCBT programs with completion rates greater than 50% involved regular therapist or parent involvement to support program use [26,29,78-82].”

“Standardized interviews and patient-reported measures (eg, Ratings of Perceived Helpfulness in Behavior Change [74,100]) may also help researchers determine how iCBT program features have or have not engaged adolescents in behavior change, the reliability of adolescents’ self-awareness/reports on…”

“To target our anticipated participants, we used current, key literature [30,52,53,70,112-114] to develop the UEQII.”

“In the study, we recognized that multiple interacting components influence the user experience [67-69].”

“…relationship between iCBT use and change in anxiety (a dose-response relationship) exists—some studies have found evidence for this relationship [102,103], whereas others have not [104,105]; however, there is consensus that some degree of program use is required to reduce users’ symptoms [106-108].”

“There is a trend in the literature that some type of program support can increase program use or effectiveness of iCBT for children and adolescents [10]; however, inconsistent evidence is published [16,28,86,87], and what type of support, such as when it should be provided and by whom, that improves outcomes is unclea…”

“UEQII items were informed by previously published questionnaires and key literature on user experiences [51-53].”

“Evaluations of adolescent experiences with various iCBT program delivery and content formats have revealed good program usability (eg, program had few errors and it was easy to learn to use) [21-24], moderate-to-strong credibility (eg, the program contained expert and reliable information), promising treatment expecta…”

“Similar to other iCBT studies, Breathe respondents reported that the program was easy to understand [92], met their needs [79], and that they were satisfied overall [29,93,94].”

“Currently, there is considerable heterogeneity in how the user experience is defined and evaluated, with most research being conducted with adult populations [65,69,110,111].”

“It may be that this type of support as well as the degree of support provided may help adolescents manage their time and complete challenging program activities [27,81,83-85].”

“The overall intent of these objectives was to examine self-reported user experience data and automatically captured program usage data together for a better understanding of the relationship between behavioral (objective usage) and experiential (subjective usage, user experience, and MCID) data [38-40] to explain and…”

“Evaluations of adolescent experiences with various iCBT program delivery and content formats have revealed good program usability (eg, program had few errors and it was easy to learn to use) [21-24], moderate-to-strong credibility (eg, the program contained expert and reliable information), promising treatment expecta…”

“Evaluations of adolescent experiences with various iCBT program delivery and content formats have revealed good program usability (eg, program had few errors and it was easy to learn to use) [21-24], moderate-to-strong credibility (eg, the program contained expert and reliable information), promising treatment expecta…”

“Evaluations of adolescent experiences with various iCBT program delivery and content formats have revealed good program usability (eg, program had few errors and it was easy to learn to use) [21-24], moderate-to-strong credibility (eg, the program contained expert and reliable information), promising treatment expecta…”

“…relationship between iCBT use and change in anxiety (a dose-response relationship) exists—some studies have found evidence for this relationship [102,103], whereas others have not [104,105]; however, there is consensus that some degree of program use is required to reduce users’ symptoms [106-108].”

“There is a trend in the literature that some type of program support can increase program use or effectiveness of iCBT for children and adolescents [10]; however, inconsistent evidence is published [16,28,86,87], and what type of support, such as when it should be provided and by whom, that improves outcomes is unclea…”

“Consistent with the literature, Breathe respondents described difficulty remembering to work on the program [29,52], concerns with privacy and stigma (eg, others knowing about or judging their help seeking) [30,71,72], time constraints, and conflicting commitments [31,73-75], and delaying or avoiding tasks they found…”

“It may be that this type of support as well as the degree of support provided may help adolescents manage their time and complete challenging program activities [27,81,83-85].”

“Internet-based CBT (iCBT), with its self-help format, can increase the access and availability of CBT for adolescents with mild-to-moderate anxiety [10,11].”

“Evaluations of adolescent experiences with various iCBT program delivery and content formats have revealed good program usability (eg, program had few errors and it was easy to learn to use) [21-24], moderate-to-strong credibility (eg, the program contained expert and reliable information), promising treatment expecta…”

“Considering that multiple iCBT components work together to form a complex intervention [99], we recommend connecting the persuasive system design features known to relate to a positive user experience (program reminders, progress and feedback tools, multimedia demonstrations, and flexible program support) with propose…”

“The overall intent of these objectives was to examine self-reported user experience data and automatically captured program usage data together for a better understanding of the relationship between behavioral (objective usage) and experiential (subjective usage, user experience, and MCID) data [38-40] to explain and…”

“Evaluations of adolescent experiences with various iCBT program delivery and content formats have revealed good program usability (eg, program had few errors and it was easy to learn to use) [21-24], moderate-to-strong credibility (eg, the program contained expert and reliable information), promising treatment expecta…”

“…the program [29,52], concerns with privacy and stigma (eg, others knowing about or judging their help seeking) [30,71,72], time constraints, and conflicting commitments [31,73-75], and delaying or avoiding tasks they found challenging [76,77] as the biggest obstacles to program adherence and use.”

“To target our anticipated participants, we used current, key literature [30,52,53,70,112-114] to develop the UEQII.”

“Reporting whether changes in anxiety across different programs met an MCID can also assist adolescents, parents, and clinicians in deciding which program best matches their expected treatment response [37,101].”

“For iCBT programs designed and delivered to adolescents with anxiety, program evaluations should aim to understand how iCBT is experienced by adolescents to further ensure its relevance, use, and impact as a self-help treatment [63-66].”

“The MASC-2 is one of the most widely used self-report measures in trials involving adolescents with anxiety because of the brevity of the measure and simplicity of its administration [50].”

“For iCBT programs designed and delivered to adolescents with anxiety, program evaluations should aim to understand how iCBT is experienced by adolescents to further ensure its relevance, use, and impact as a self-help treatment [63-66].”

“Most previously studied iCBT programs with completion rates greater than 50% involved regular therapist or parent involvement to support program use [26,29,78-82].”

“It may be that this type of support as well as the degree of support provided may help adolescents manage their time and complete challenging program activities [27,81,83-85].”

“Previous iCBT studies have measured whether program participation was perceived as effective or useful by adolescents [81,92] but have not formally measured the degree of meaningful change in anxiety as experienced by users of a program.”

“…is no standard for how to calculate the MCID; therefore, a variety of methods exist and can be used depending on the study sample and data collected (for a review of the different methods, refer to the studies by Copay et al [59], Wells et al [115], Beaton et al [116], and Ebrahim et al [117]).”

“Use of factor analysis [90] or multiple regression [91] could help to illuminate how different constructs of user experience relate to one another and to intervention use and how the constructs change over the course of treatment.”

“The Breathe program for mild-to-moderate anxiety symptoms among adolescents is described in detail elsewhere [46].”

“Consistent with the literature, Breathe respondents described difficulty remembering to work on the program [29,52], concerns with privacy and stigma (eg, others knowing about or judging their help seeking) [30,71,72], time constraints, and conflicting commitments [31,73-75], and delaying or avoiding tasks they found…”

“…the program [29,52], concerns with privacy and stigma (eg, others knowing about or judging their help seeking) [30,71,72], time constraints, and conflicting commitments [31,73-75], and delaying or avoiding tasks they found challenging [76,77] as the biggest obstacles to program adherence and use.”

“Perceived impact may also be based on unique individual factors, such as treatment expectancy, preintervention anxiety severity, self-regulation abilities, or motivational factors [69,102,109], factors that we did not assess.”

“(73) RCT with 6 week follow-up CBT 6 weeks Anxiety, Experience of E-Health Interventions Clinically moderated (one coaching session”

“…such as a telephone or computer; digital mental health interventions [80]; web, computer, online-or internet-based interventions [57, 58, 71, 72, 74, 81–89]; websites [76, 90]; e-therapy programs [88]; CD-ROMS or DVDs [74, 76, 83, 91]; ecounsel-ling [90]; applications or “apps” [74, 79, 89,…”

“Paraprofessionals provided guidance on one intervention (Radomski et al., 2020) while clinical psychology students provided guidance in another intervention (Garnefski & Kraaij, 2023).”

“Thus, key design problems could affect the generalizability of the results of effective clinical interventions at the time of their implementation in the community context (25, 26).”

“So far a teletherapeutic approach is often used in psychotherapeutic sessions with a high level of satisfaction and compliance [9-11].”

“The domains were based on a preliminary literature review of user experiences in eHealth studies that was conducted by one of the authors [23].”

“Apart from performance-based usability testing, user perception and acceptability have been employed to evaluate user health interventions in healthcare [19].”

“Supplementary analysis To compare youth that participated and completed an interview in this qualitative study (N = 6) to invited youth that did not complete the interview (N = 14), we examined scores on the satisfaction and acceptability construct of the User Experience Questionnaire for Internet-based Interventions…”

“…mindfulness strategies (eg, meditations) may be more easily applied by youth in an app-based platform without therapist guidance than other interventions [40,41], which is important given the high prevalence of dysmenorrhea yet limited availability of nonpharmacological treatment options [17,42].”

““Please click on the appropriate QR code above” [34]; Schleider’s [36] and Radomski’s [35] posts directed potential participants to a trial website to view more information.”

“…came to a collaborative decision to allocate the LICBT module to first work with, further intervention modifications could include providing more module-specific information, for example, a module preview, which may also facilitate starting with a module that meets their needs and preferences [79].”

“Both automated and human support has been found to be efficacious for enhancing engagement in iCBT programs (Olthuis et al. 2016; Radomski et al. 2020).”

“, 2020 [46] E-gift card/voucher CA $25 Schleider et al.”

“This is also in line RenderX with findings from qualitative evaluations of ICBT treatments, where adolescents preferred easy-to-use techniques [35,36].”

“Previous research has relied on clinician judgment and clinical theory to determine their “anchor” on which to base the MCID [13].”

“More specifically, providing users with their scores on assessment tools and allowing comparisons with aggregated results from other users could further enhance self-awareness and engagement, as suggested in studies of apps targeting anxiety and depression [75,78,79].”

“Thirteen trials sent reminders to participants 305 [28, 32-33, 37-38, 40-43, 45-46] for enrolment, retention, or follow-up data collection.”

“Several parents and children found the breathing exercise helpful, in line with previous findings that youth prefer treatment techniques that are easy to learn, such as breathing and relaxation exercises [49].”

“While most technology-based interventions for mental health have been developed and researched for adults, the few studies on anxiety treatment in adolescents show effectiveness, though youth engagement and completion rates remain very low [21–24].”