Design and Delivery Features That May Improve the Use of Internet-Based Cognitive Behavioral Therapy for Children and Adolescents With Anxiety: A Realist Literature Synthesis With a Persuasive Systems Design Perspective
Open accessThe DOI matches its Crossref record; 100 of the 101 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Internet-based cognitive behavioral therapy (iCBT) is a persuasive system as its design combines therapeutic content, technological features, and interactions between the user and the program to reduce anxiety for children and adolescents. How iCBT is designed and delivered differs across programs. Although iCBT is considered an effective approach for treating child and adolescent anxiety, rates of program use (eg, module completion) are highly variable for reasons that are not clear. As the extent to which users complete a program can impact anxiety outcomes, understanding what iCBT design and delivery features improve program use is critical for optimizing treatment effects. OBJECTIVE: The objectives of this study were to use a realist synthesis approach to explore the design and delivery features of iCBT for children and adolescents with anxiety as described in the literature and to examine their relationship to program use outcomes. METHODS: A search of published and gray literature was conducted up to November 2017. Prespecified inclusion criteria identified research studies, study protocols, and program websites on iCBT for child and adolescent anxiety. Literature was critically appraised for relevance and methodological rigor. The persuasive systems design (PSD) model, a comprehensive framework for designing and evaluating persuasive systems, was used to guide data extraction. iCBT program features were grouped under 4 PSD categories-Primary task support, Dialogue support, System credibility support, and Social support. iCBT design (PSD Mechanisms) and delivery features (Context of use) were linked to program use (Outcomes) using meta-ethnographic methods; these relationships were described as Context-Mechanism-Outcome configurations. For our configurations, we identified key PSD features and delivery contexts that generated moderate-to-high program use based on moderate-to-high quality evidence found across multiple iCBT programs. RESULTS: A total of 44 documents detailing 10 iCBT programs were included. Seven iCBT programs had at least one document that scored high for relevance; most studies were of moderate-to-high methodological rigor. We developed 5 configurations that highlighted 8 PSD features (Tailoring, Personalization [Primary task supports]; Rewards, Reminders, Social role [Dialogue supports]; and Trustworthiness, Expertise, Authority [System credibility supports]) associated with moderate-to-high program use. Important features of delivery Context were adjunct support (a face-to-face, Web- or email-based communications component) and whether programs targeted the prevention or treatment of anxiety. Incorporating multiple PSD features may have additive or synergistic effects on program use. CONCLUSIONS: The Context-Mechanism-Outcome configurations we developed suggest that, when delivered with adjunct support, certain PSD features contribute to moderate-to-high use of iCBT prevention and treatment programs for children and adolescents with anxiety. Standardization of the definition and measurement of program use, formal testing of individual and combined PSD features, and use of real-world design and testing methods are important next steps to improving how we develop and deliver increasingly useful treatments to target users.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“There is some support to suggest that the inclusion of social networking facilities in digital interventions for individuals with serious mental illnesses can assist in fostering a sense of social connectedness [35,140,141], a sense of cooperation (shared goals), and increased accountability toward treatment [137].”
From The therapeutic alliance in digital mental health interventions for serious mental illnesses: narrative review (2020) · cites this paper as [137]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“multiple authors indicated that it is always important to use a systematic approach when planning and executing any sort of literature review to ensure completeness of the results”
From Methods for Human-Centered eHealth Development: Narrative Scoping Review (2022) · cites this paper as [167]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“As adolescents are sensitive to peer influence and can be motivated by interactions with peers [40-42]”
From A Smartphone Serious Game for Adolescents (Grow It! App): Development, Feasibility, and Acceptance Study (2022) · cites this paper as [40]
Experimental — accuracy not yet measuredIn the citing paper
“Credibility support is associated with perceived authority, expertise, and trustworthiness among users of digital interventions, and findings show that its deliberate implementation is also associated with engagement [137].”
From The therapeutic alliance in digital mental health interventions for serious mental illnesses: narrative review (2020) · cites this paper as [137]
In the citing paper
“multiple authors pointed out that often a lot of time has passed between initial data collection and publication of studies—which is especially relevant for the rapidly changing domain of eHealth.”
From Methods for Human-Centered eHealth Development: Narrative Scoping Review (2022) · cites this paper as [167]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Interrater reliability: The kappa statistic
McHugh, M. L. (2012) · Biochemia Medica · doi:10.11613/bm.2012.031 · Cited by 19,877 · cited 1×
“During this stage, we randomly selected 100 citations to assess inter-rater agreement for inclusion or exclusion decisions; Cohen kappa was 0.74 between raters, reflecting substantial agreement [36].”
MethodInfluentialRelation of the therapeutic alliance with outcome and other variables: a meta-analytic review
Martin, D. J., Garske, J. P., & Davis, M. K. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.68.3.438 · Cited by 3,355 · cited 1×
“It has been suggested that adjunct support may leverage the advantages of therapeutic alliance [101,102], which might include principles of persuasion (eg, users feel the need to respond to social cues [18]), it may establish process expectations and social accountability [103], or it may develop a…”
BackgroundRealist review - a new method of systematic review designed for complex policy interventions
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005) · Journal of Health Services Research & Policy · doi:10.1258/1355819054308530 · Cited by 3,190 · cited 3×
“We used a multistep approach to data analysis that was structured according to Pawson’s techniques [27,28] and meta-ethnography [43,44] The second step in the analysis involved reciprocal translation analysis, a meta-ethnographic technique that involved reviewing the Context-Mechanism-Outcome…”
MethodNoblit, G., & Hare, R. (1988) · doi:10.4135/9781412985000 · Cited by 2,723
Webb, T. L., Joseph, J., Yardley, L., & Michie, S. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1376 · Cited by 2,517 · cited 1×
“Together, these steps led to the generation of 5 candidate Context-Mechanism-Outcome configurations (Multimedia Appendix 1) [30-34].”
BackgroundPersuasive technology: using computers to change what we think and do
Fogg, B. J. (2002) · Ubiquity · doi:10.1145/764008.763957 · Cited by 1,554
Pace, R., Pluye, P., Bartlett, G., Macaulay, A. C., Salsberg, J., Jagosh, J., & Seller, R. (2012) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.07.002 · Cited by 1,331 · cited 1×
“The MMAT is a reliable, efficient, and valid tool that provides different sections for assessing studies of qualitative, randomized, nonrandomized studies, descriptive studies, and mixed-method designs [38-40].”
MethodPersuasive System Design Does Matter: a Systematic Review of Adherence to Web-based Interventions
Kelders, S. M., Kok, R. N., Ossebaard, H. C., & Van Gemert-Pijnen, J. E. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2104 · Cited by 1,308 · cited 7×
“This method would allow for iCBT program use to not only be measured at end of the intervention but also throughout the program access period to assess usage patterns over time [15], when certain design or delivery features may be more or less activated or present.”
BackgroundMethodResultInfluentialAdherence in Internet Interventions for Anxiety and Depression
Christensen, H., Griffiths, K. M., & Farrer, L. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1194 · Cited by 1,133 · cited 3×
“However, it was rare to have information on treatment or technology preferences of users (eg, early completers [22,122]), their psychological characteristics or cognitions (eg, motivation, personality, expectations, and treatment perceptions [123]), or personal circumstances (eg, program access…”
BackgroundA Holistic Framework to Improve the Uptake and Impact of eHealth Technologies
van Gemert-Pijnen, J. E., Nijland, N., van Limburg, M., Ossebaard, H. C., Kelders, S. M., Eysenbach, G., & Seydel, E. R. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1672 · Cited by 1,126 · cited 1×
“Together, these steps led to the generation of 5 candidate Context-Mechanism-Outcome configurations (Multimedia Appendix 1) [30-34].”
BackgroundPluye, P., Gagnon, M.-P., Griffiths, F., & Johnson-Lafleur, J. (2009) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2009.01.009 · Cited by 1,119 · cited 3×
“A strength of this synthesis is the inclusion of diverse and high-quality evidence (ie, MMAT scores>75% [38]) from both the published and gray literature.”
BackgroundMethodMohr, D. C., Cuijpers, P., & Lehman, K. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1602 · Cited by 1,097 · cited 1×
“…include principles of persuasion (eg, users feel the need to respond to social cues [18]), it may establish process expectations and social accountability [103], or it may develop a sense of legitimacy or credibility of the program [103] (see Santarossa et al’s study [104] for further suggestions).”
BackgroundCollins, L. M., Murphy, S. A., & Strecher, V. (2007) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2007.01.022 · Cited by 1,082
Reducing Risks for Mental Disorders: Frontiers for Preventive Intervention Research
Novins, D. K. (1995) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-199503000-00028 · Cited by 1,076
Conducting a meta-ethnography of qualitative literature: Lessons learnt
Atkins, S., Lewin, S., Smith, H., Engel, M., Fretheim, A., & Volmink, J. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-21 · Cited by 981
RAMESES II reporting standards for realist evaluations
Wong, G., Westhorp, G., Manzano, A., Greenhalgh, J., Jagosh, J., & Greenhalgh, T. (2016) · BMC Medicine · doi:10.1186/s12916-016-0643-1 · Cited by 880 · cited 1×
“This realist synthesis was conducted using steps recommended by Pawson and Tilley [27,28] and is reported in accordance with the Realist and Meta-narrative Evidence Synthesis: Evolving Standards (RAMESES II) [29].”
MethodMerry, S. N., Stasiak, K., Shepherd, M., Frampton, C., Fleming, T., & Lucassen, M. F. G. (2012) · BMJ · doi:10.1136/bmj.e2598 · Cited by 716 · cited 1×
“Recent efforts into improving the gamification of technology-based CBT for pediatric mental health (see SPARX [108]), where incorporating game-design elements such as Praise and Rewards are regularly used to enhance program use and engagement [109], demonstrate the potential benefits that the use…”
BackgroundCOMPTON, S. N., MARCH, J. S., BRENT, D., ALBANO, A. M., WEERSING, V. R., & CURRY, J. (2004) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/01.chi.0000127589.57468.bf · Cited by 616 · cited 1×
“Cognitive behavioral therapy (CBT) is recommended as a first-line treatment for children and adolescents with anxiety [1-4].”
BackgroundDropout from Internet‐based treatment for psychological disorders
Melville, K. M., Casey, L. M., & Kavanagh, D. J. (2010) · British Journal of Clinical Psychology · doi:10.1348/014466509x472138 · Cited by 597 · cited 1×
“…early completers [22,122]), their psychological characteristics or cognitions (eg, motivation, personality, expectations, and treatment perceptions [123]), or personal circumstances (eg, program access [124])—factors that are also considered critical to program use and could be used to construct…”
BackgroundConnolly, S. D., & Bernstein, G. A. (2007) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/01.chi.0000246070.23695.06 · Cited by 548 · cited 1×
“Therapeutic content in programs was coded according to the 5 main CBT components found in the American Academy of Child and Adolescent Psychiatry (AACAP) practice parameter [42]: psychoeducation, somatic management skills, cognitive restructuring, exposure methods, and relapse prevention.”
MethodEfficacy and effectiveness of school-based prevention and early intervention programs for anxiety
Neil, A. L., & Christensen, H. (2009) · Clinical Psychology Review · doi:10.1016/j.cpr.2009.01.002 · Cited by 523 · cited 1×
“iCBT is recognized as an important treatment option [9-11] to meet the increasing demands of a population where anxiety is often undetected and untreated, but with whom early access to care can improve long-term outcomes [3,12-14].”
BackgroundFry, J. P., & Neff, R. A. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1138 · Cited by 447 · cited 1×
“This realist synthesis not only provides support for incorporating some of the well-studied and highly used PSD features into iCBT (ie, Reminders [93,105,106]) but also draws attention to underutilized features that can be incorporated in the designs of new treatments.”
BackgroundTitov, N., Andrews, G., Davies, M., McIntyre, K., Robinson, E., & Solley, K. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010939 · Cited by 406 · cited 1×
“Although some evidence suggests that layperson support is as effective as clinician support [100], this synthesis suggests that the person providing support as well as the intensity of their support activities (frequency; inclusion of therapeutic elements) may have a noteworthy effect on program…”
BackgroundA Systematic Framework for Designing and Evaluating Persuasive Systems
Oinas-Kukkonen, H., & Harjumaa, M. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-68504-3_15 · Cited by 405 · cited 2×
“The persuasive systems design (PSD) model [17] emerged as a key framework for understanding how iCBT, as a persuasive system, was intended to work, and we used this model to develop the initial list of Context-Mechanism-Outcome configurations.”
MethodWaller, R., & Gilbody, S. (2009) · Psychological Medicine · doi:10.1017/s0033291708004224 · Cited by 393 · cited 1×
“…or cognitions (eg, motivation, personality, expectations, and treatment perceptions [123]), or personal circumstances (eg, program access [124])—factors that are also considered critical to program use and could be used to construct and validate more intricate Context-Mechanism-Outcome…”
BackgroundAnxiety disorders in adolescents and psychosocial outcomes at age 30
Essau, C. A., Lewinsohn, P. M., Olaya, B., & Seeley, J. R. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.12.033 · Cited by 384 · cited 1×
“…demands of a population where anxiety is often undetected and untreated, but with whom early access to care can improve long-term outcomes [3,12-14]. iCBT uses technological features (ie, multimedia and email) to deliver treatment content through the Web or a software application to…”
BackgroundA randomized controlled trial of online versus clinic-based CBT for adolescent anxiety.
Spence, S. H., Donovan, C. L., March, S., Gamble, A., Anderson, R. E., Prosser, S., & Kenardy, J. (2011) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0024512 · Cited by 381 · cited 2×
“5/10 modules completed (high use); 39% of users completed all 10 modules (low use) Same as above Same as above [80]”
BackgroundA systematic review of online youth mental health promotion and prevention interventions
Clarke, A. M., Kuosmanen, T., & Barry, M. M. (2015) · Journal of Youth and Adolescence · doi:10.1007/s10964-014-0165-0 · Cited by 381 · cited 2×
“…and meta-analyses have found a range of iCBT programs to be effective at reducing anxiety in children and adolescents [9,10,20,21]; however, poor and highly variable rates of completion can be found across programs (up to 50% of participants not reaching the end of a program) [5,7,10,21,22].”
BackgroundSkinner, H., Biscope, S., Poland, B., & Goldberg, E. (2003) · Journal of Medical Internet Research · doi:10.2196/jmir.5.4.e32 · Cited by 377 · cited 1×
“…to preserve adolescent autonomy, appeal to user preferences, reduce health care system costs, and improve the time it takes to receive treatment [6-8]. iCBT is recognized as an important treatment option [9-11] to meet the increasing demands of a population where anxiety is often undetected and…”
BackgroundCollins, L. M., Dziak, J. J., Kugler, K. C., & Trail, J. B. (2014) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2014.06.021 · Cited by 335 · cited 1×
“Modeling, factorial designs or the multiphase optimization strategy [110-112] (see also Baker et al’s study [113] for other suggestions) can be used to evaluate the best set (individual or combination) of program features to use under different conditions (eg, delivery setting and start or end of…”
MethodSieverink, F., Kelders, S. M., & van Gemert-Pijnen, J. E. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8578 · Cited by 331 · cited 1×
“Adoption of a recent standardized definition and calculation of adherence [118] can clarify what is meant by specific program use terms and allow for comparisons of outcomes across programs. iCBT programs’ widespread implementation and ability to meet the health goals of users will involve an…”
BackgroundArnberg, F. K., Linton, S. J., Hultcrantz, M., Heintz, E., & Jonsson, U. (2014) · PLoS ONE · doi:10.1371/journal.pone.0098118 · Cited by 330 · cited 1×
“Cognitive behavioral therapy (CBT) is recommended as a first-line treatment for children and adolescents with anxiety [1-4].”
BackgroundCalear, A. L., Christensen, H., Mackinnon, A., Griffiths, K. M., & O'Kearney, R. (2009) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0017391 · Cited by 327 · cited 1×
“…Program 10: MoodGYM Average of 3.2/5 modules completed (moderate use) Dialogue support: Social role Adolescent users; Teacher support: in-person [53] Average of 9.4/28 exercises completed (low use); >25% of users completed all modules (low use) Same as above Same as above [25] Average of 3.1/28…”
BackgroundInfluentialMarch, S., Spence, S. H., & Donovan, C. L. (2009) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsn099 · Cited by 315 · cited 2×
“…2: Reminders to increase awareness of program availability and progress • “Participants receive automated, computer-generated, standardized, weekly e-mails both before each module (as a reminder to complete their modules) and after each module (to congratulate them on finishing their module)” [78].”
BackgroundMethodThe Multiphase Optimization Strategy for Engineering Effective Tobacco Use Interventions
Collins, L. M., Baker, T. B., Mermelstein, R. J., Piper, M. E., Jorenby, D. E., Smith, S. S., Christiansen, B. A., Schlam, T. R., Cook, J. W., & Fiore, M. C. (2011) · Annals of Behavioral Medicine · doi:10.1007/s12160-010-9253-x · Cited by 300 · cited 1×
“Modeling, factorial designs or the multiphase optimization strategy [110-112] (see also Baker et al’s study [113] for other suggestions) can be used to evaluate the best set (individual or combination) of program features to use under different conditions (eg, delivery setting and start or end of…”
MethodVigerland, S., Lenhard, F., Bonnert, M., Lalouni, M., Hedman, E., Ahlen, J., Olén, O., Serlachius, E., & Ljótsson, B. (2016) · Clinical Psychology Review · doi:10.1016/j.cpr.2016.09.005 · Cited by 292 · cited 2×
“…and meta-analyses have found a range of iCBT programs to be effective at reducing anxiety in children and adolescents [9,10,20,21]; however, poor and highly variable rates of completion can be found across programs (up to 50% of participants not reaching the end of a program) [5,7,10,21,22].”
Background(2012) · PubMed · Cited by 291
Richardson, T., Stallard, P., & Velleman, S. (2010) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-010-0069-9 · Cited by 272 · cited 1×
“…and meta-analyses have found a range of iCBT programs to be effective at reducing anxiety in children and adolescents [9,10,20,21]; however, poor and highly variable rates of completion can be found across programs (up to 50% of participants not reaching the end of a program) [5,7,10,21,22].”
BackgroundPennant, M. E., Loucas, C. E., Whittington, C., Creswell, C., Fonagy, P., Fuggle, P., Kelvin, R., Naqvi, S., Stockton, S., & Kendall, T. (2015) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.01.009 · Cited by 269 · cited 1×
“Recent systematic reviews and meta-analyses have found a range of iCBT programs to be effective at reducing anxiety in children and adolescents [9,10,20,21]; however, poor and highly variable rates of completion can be found across programs (up to 50% of participants not reaching the end of a…”
BackgroundCognitive-Behavioral Therapy for Anxiety Disorders in Youth
Seligman, L. D., & Ollendick, T. H. (2011) · Child and Adolescent Psychiatric Clinics of North America · doi:10.1016/j.chc.2011.01.003 · Cited by 263 · cited 1×
“Cognitive behavioral therapy (CBT) is recommended as a first-line treatment for children and adolescents with anxiety [1-4].”
BackgroundSpence, S. H., Holmes, J. M., March, S., & Lipp, O. V. (2006) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.74.3.614 · Cited by 261 · cited 2×
“For example, in-person therapist sessions or telephone calls provided opportunities for Reminders, Personalized feedback, or Praise to be conveyed to users [49,50,54,55].”
BackgroundHowgego, I. M., Yellowlees, P., Owen, C., Meldrum, L., & Dark, F. (2003) · Australian & New Zealand Journal of Psychiatry · doi:10.1046/j.1440-1614.2003.01131.x · Cited by 252 · cited 1×
“It has been suggested that adjunct support may leverage the advantages of therapeutic alliance [101,102], which might include principles of persuasion (eg, users feel the need to respond to social cues [18]), it may establish process expectations and social accountability [103], or it may develop a…”
BackgroundPrevention of childhood anxiety disorders
Donovan, C. L., & Spence, S. H. (2000) · Clinical Psychology Review · doi:10.1016/s0272-7358(99)00040-9 · Cited by 241 · cited 1×
“…demands of a population where anxiety is often undetected and untreated, but with whom early access to care can improve long-term outcomes [3,12-14]. iCBT uses technological features (ie, multimedia and email) to deliver treatment content through the Web or a software application to…”
BackgroundWhat Makes Internet Therapy Work?
Andersson, G., Carlbring, P., Berger, T., Almlöv, J., & Cuijpers, P. (2009) · Cognitive Behaviour Therapy · doi:10.1080/16506070902916400 · Cited by 234 · cited 1×
“In the literature, little is known about how much, when, and what type of support is necessary for enhancing program use and efficacy [98,99].”
BackgroundBaker, T. B., Gustafson, D. H., & Shah, D. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2925 · Cited by 221 · cited 1×
“Modeling, factorial designs or the multiphase optimization strategy [110-112] (see also Baker et al’s study [113] for other suggestions) can be used to evaluate the best set (individual or combination) of program features to use under different conditions (eg, delivery setting and start or end of…”
MethodCalear, A. L., & Christensen, H. (2010) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2010.tb03686.x · Cited by 220 · cited 2×
“…to preserve adolescent autonomy, appeal to user preferences, reduce health care system costs, and improve the time it takes to receive treatment [6-8]. iCBT is recognized as an important treatment option [9-11] to meet the increasing demands of a population where anxiety is often undetected and…”
Mechanisms of change in interpersonal therapy (IPT)
Lipsitz, J. D., & Markowitz, J. C. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2013.09.002 · Cited by 214 · cited 1×
“Many programs also integrated interpersonal therapy skills [57], such as assertiveness training and problem solving, to reduce environmental stressors and enhance social support.”
BackgroundHealthy Living with Persuasive Technologies: Framework, Issues, and Challenges
Chatterjee, S., & Price, A. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2859 · Cited by 205
Lehto, T., & Oinas-Kukkonen, H. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1559 · Cited by 203 · cited 2×
“This realist synthesis identified [6,15,89,92,93].”
MethodAlbano, A. M., & Kendall, P. C. (2002) · International Review of Psychiatry · doi:10.1080/09540260220132644 · Cited by 189 · cited 1×
“…demands of a population where anxiety is often undetected and untreated, but with whom early access to care can improve long-term outcomes [3,12-14]. iCBT uses technological features (ie, multimedia and email) to deliver treatment content through the Web or a software application to…”
BackgroundByng, R., Norman, I., & Redfern, S. (2005) · Evaluation · doi:10.1177/1356389005053198 · Cited by 186 · cited 2×
“…that was structured according to Pawson’s techniques [27,28] and meta-ethnography [43,44] The second step in the analysis involved reciprocal translation analysis, a meta-ethnographic technique that involved reviewing the Context-Mechanism-Outcome configurations across iCBT programs [44,45].”
MethodHow to increase reach and adherence of web-based interventions: a design research viewpoint
Ludden, G. D., van Rompay, T. J., Kelders, S. M., & van Gemert-Pijnen, J. E. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4201 · Cited by 185 · cited 1×
“Together, these steps led to the generation of 5 candidate Context-Mechanism-Outcome configurations (Multimedia Appendix 1) [30-34].”
BackgroundGamification and Adherence to Web-Based Mental Health Interventions: A Systematic Review
Brown, M., O'Neill, N., van Woerden, H., Eslambolchilar, P., Jones, M., & John, A. (2016) · JMIR Mental Health · doi:10.2196/mental.5710 · Cited by 178 · cited 1×
“…the gamification of technology-based CBT for pediatric mental health (see SPARX [108]), where incorporating game-design elements such as Praise and Rewards are regularly used to enhance program use and engagement [109], demonstrate the potential benefits that the use of these features may have.”
BackgroundNew MRC guidance on evaluating complex interventions
Anderson, R. (2008) · BMJ · doi:10.1136/bmj.a1937 · Cited by 172 · cited 1×
“Moreover, involving target users in the (co)design and testing of treatments is recommended to improve the acceptability, feasibility, and effectiveness of iCBT with children and adolescents [19,94,95].”
BackgroundSantarossa, S., Kane, D., Senn, C. Y., & Woodruff, S. J. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.8480 · Cited by 172
Orlowski, S. K., Lawn, S., Venning, A., Winsall, M., Jones, G. M., Wyld, K., Damarell, R. A., Antezana, G., Schrader, G., Smith, D., et al. (2015) · JMIR Human Factors · doi:10.2196/humanfactors.4361 · Cited by 165 · cited 1×
“Moreover, involving target users in the (co)design and testing of treatments is recommended to improve the acceptability, feasibility, and effectiveness of iCBT with children and adolescents [19,94,95].”
BackgroundThe SmartCAT: An m-Health Platform for Ecological Momentary Intervention in Child Anxiety Treatment
Pramana, G., Parmanto, B., Kendall, P. C., & Silk, J. S. (2014) · Telemedicine and e-Health · doi:10.1089/tmj.2013.0214 · Cited by 162 · cited 8×
“A total of 3 iCBT programs regularly incorporated Rewards into modules to encourage ongoing program use and promote the completion of essential treatment exercises [46,47,51].”
BackgroundMethodInfluentialLentferink, A. J., Oldenhuis, H. K., de Groot, M., Polstra, L., Velthuijsen, H., & van Gemert-Pijnen, J. E. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7288 · Cited by 162 · cited 1×
“The extent of iCBT program use may be a fundamental indication of its persuasiveness and its potential to assist the user with their goals of the program [15,88-91].”
BackgroundThe Law of Attrition Revisited
Christensen, H., & Mackinnon, A. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.3.e20 · Cited by 161 · cited 1×
“However, it was rare to have information on treatment or technology preferences of users (eg, early completers [22,122]), their psychological characteristics or cognitions (eg, motivation, personality, expectations, and treatment perceptions [123]), or personal circumstances (eg, program access…”
BackgroundPersuasiveness of a Mobile Lifestyle Coaching Application Using Social Facilitation
Gasser, R., Brodbeck, D., Degen, M., Luthiger, J., Wyss, R., & Reichlin, S. (2006) · Lecture notes in computer science · doi:10.1007/11755494_5 · Cited by 160 · cited 1×
“Together, these steps led to the generation of 5 candidate Context-Mechanism-Outcome configurations (Multimedia Appendix 1) [30-34].”
BackgroundWhat's the Point?: A Review of Reward Systems Implemented in Gamification Interventions
Lewis, Z. H., Swartz, M. C., & Lyons, E. J. (2016) · Games for Health Journal · doi:10.1089/g4h.2015.0078 · Cited by 156 · cited 1×
“For example, Rewards were only occasionally used by iCBT programs included in this study (4 out of 10 programs) but are more commonly used techniques for increasing program use in internet-based interventions targeting physical activity or dietary behaviors [107].”
MethodNeil, A. L., Batterham, P., Christensen, H., Bennett, K., & Griffiths, K. M. (2009) · Journal of Medical Internet Research · doi:10.2196/jmir.1050 · Cited by 152 · cited 5×
“…support: in-person [53] Average of 9.4/28 exercises completed (low use); >25% of users completed all modules (low use) Same as above Same as above [25] Average of 3.1/28 activities completed (very low use) Same as above Adolescent users [25] <1% of users completed all activities (very low use)…”
BackgroundInfluentialSelf-citationMoller, A. C., Merchant, G., Conroy, D. E., West, R., Hekler, E., Kugler, K. C., & Michie, S. (2017) · Journal of Behavioral Medicine · doi:10.1007/s10865-016-9818-7 · Cited by 151 · cited 1×
“Designing studies that incorporate in vivo, objective measurements or automatic data capture of program use [115,116] could improve our awareness of program use predictors beyond user demographics (ie, age), for example, to actual usage behaviors (ie, number of Web pages viewed).”
BackgroundRooksby, M., Elouafkaoui, P., Humphris, G., Clarkson, J., & Freeman, R. (2015) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2014.11.006 · Cited by 143 · cited 2×
“…and meta-analyses have found a range of iCBT programs to be effective at reducing anxiety in children and adolescents [9,10,20,21]; however, poor and highly variable rates of completion can be found across programs (up to 50% of participants not reaching the end of a program) [5,7,10,21,22].”
BackgroundVigerland, S., Ljótsson, B., Thulin, U., Öst, L.-G., Andersson, G., & Serlachius, E. (2016) · Behaviour Research and Therapy · doi:10.1016/j.brat.2015.11.006 · Cited by 136 · cited 1×
“Through email the therapist provided “written feedback on worksheets” and was available to “answer questions and clarify treatment content, increase motivation and to help solve problems” [83].”
BackgroundBaumel, A., & Yom-Tov, E. (2018) · Translational Behavioral Medicine · doi:10.1093/tbm/ibx037 · Cited by 128 · cited 1×
“The extent of iCBT program use may be a fundamental indication of its persuasiveness and its potential to assist the user with their goals of the program [15,88-91].”
BackgroundSpence, S. H., Donovan, C. L., March, S., Gamble, A., Anderson, R., Prosser, S., Kercher, A., & Kenardy, J. (2008) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s135246580800444x · Cited by 120 · cited 4×
“Studies found that both parent and child ratings of therapeutic alliance and program use were high [76,81].”
BackgroundInfluentialAdherence to the MoodGYM program: outcomes and predictors for an adolescent school-based population
Calear, A. L., Christensen, H., Mackinnon, A., & Griffiths, K. M. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2012.11.036 · Cited by 105 · cited 2×
“Studies of iCBT in children and adolescents with anxiety [25,26] have found that certain participant demographics (eg, gender, age, location, and anxiety severity) and delivery features (eg, parental or therapist support) relate to or predict program use.”
BackgroundAnderson, R. E. E., Spence, S. H., Donovan, C. L., March, S., Prosser, S., & Kenardy, J. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1848 · Cited by 102 · cited 5×
“One of these studies correlated therapeutic alliance with program use and found a significant, positive relationship [81].”
BackgroundInfluentialSpence, S. H., Donovan, C. L., March, S., Kenardy, J. A., & Hearn, C. S. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.12.003 · Cited by 101 · cited 1×
“Understanding the factors that influence iCBT program use is important as there have been indications that greater program use is associated with better outcomes [23,24].”
BackgroundWildeboer, G., Kelders, S. M., & van Gemert-Pijnen, J. E. W. C. (2016) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2016.04.005 · Cited by 98 · cited 3×
“…internet-based interventions for mental health demonstrated that determining the amount and type of persuasive principles to include may be a delicate balance, as some principles seem to work together, whereas when other principles co-occur, they may have an unapparent or diminishing effect [90].”
BackgroundTesting in the Wild: The Social and Organisational Dimensions of Real World Practice
Rooksby, J., Rouncefield, M., & Sommerville, I. (2009) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-009-9098-7 · Cited by 95 · cited 1×
“These participatory research efforts may provide important guidance on the usefulness and functionality of select (PSD) features of iCBT programs in the real world and from the user’s perspective [96]; therefore, facilitating greater program use [97].”
BackgroundConaughton, R. J., Donovan, C. L., & March, S. (2017) · Journal of Affective Disorders · doi:10.1016/j.jad.2017.04.032 · Cited by 94 · cited 1×
“7 /10 modules completed (moderate use); 19% of users completed all 10 modules (very low use) Same as above Same as above [82]d”
BackgroundWozney, L., Huguet, A., Bennett, K., Radomski, A. D., Hartling, L., Dyson, M., McGrath, P. J., & Newton, A. S. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.7573 · Cited by 93 · cited 2×
“This realist synthesis identified [6,15,89,92,93].”
BackgroundSelf-citationCrutzen, R., Cyr, D., & de Vries, N. K. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.1922 · Cited by 91 · cited 1×
“Together, these steps led to the generation of 5 candidate Context-Mechanism-Outcome configurations (Multimedia Appendix 1) [30-34].”
BackgroundA Meta-Analysis on the Efficacy of Technology Mediated CBT for Anxious Children and Adolescents
Podina, I. R., Mogoase, C., David, D., Szentagotai, A., & Dobrean, A. (2016) · Journal of Rational-Emotive & Cognitive-Behavior Therapy · doi:10.1007/s10942-015-0228-5 · Cited by 83 · cited 2×
“Recent systematic reviews and meta-analyses have found a range of iCBT programs to be effective at reducing anxiety in children and adolescents [9,10,20,21]; however, poor and highly variable rates of completion can be found across programs (up to 50% of participants not reaching the end of a program) [5,7,10,21,22].”
BackgroundGunther, E. (2013) · Studies in Health Technology and Informatics · doi:10.3233/978-1-61499-289-9-657 · Cited by 75 · cited 1×
“As PSD features are modifiable, iCBT program designers and developers can”
Jolstedt, M., Ljótsson, B., Fredlander, S., Tedgård, T., Hallberg, A., Ekeljung, A., Högström, J., Mataix-Cols, D., Serlachius, E., & Vigerland, S. (2018) · Internet Interventions · doi:10.1016/j.invent.2017.11.003 · Cited by 75 · cited 1×
“exercise; moderate use) Same as above Same as above [85]”
MethodSchueller, S. M., Tomasino, K. N., & Mohr, D. C. (2017) · Clinical Psychology: Science and Practice · doi:10.1037/h0101740 · Cited by 70 · cited 1×
“In the literature, little is known about how much, when, and what type of support is necessary for enhancing program use and efficacy [98,99].”
BackgroundInternet-delivered CBT for children with specific phobia: a pilot study
Vigerland, S., Thulin, U., Ljótsson, B., Svirsky, L., Öst, L.-G., Lindefors, N., Andersson, G., & Serlachius, E. (2013) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2013.844201 · Cited by 67 · cited 4×
“…to improve perceived value of information or support Indicated prevention and treatment programs with adjunct therapist support • Participants received “comments and feedback from their therapist on all exercises, and the technical platform also allowed participants to comment on worksheets” [50]).”
BackgroundInfluentialCluster randomised controlled trial of the e-couch Anxiety and Worry program in schools
Calear, A. L., Batterham, P. J., Poyser, C., Mackinnon, A. J., Griffiths, K. M., & Christensen, H. M. (2016) · Journal of Affective Disorders · doi:10.1016/j.jad.2016.02.049 · Cited by 66 · cited 5×
“…36% of users completed all modules (low use) Same as above Adolescent users; Teacher support: in-person; Mental health provider support: in-person [24] Program 10: MoodGYM Average of 3.2/5 modules completed (moderate use) Dialogue support: Social role Adolescent users; Teacher support: in-person…”
BackgroundInfluentialXu, Y., Poole, E. S., Miller, A. D., Eiriksdottir, E., Kestranek, D., Catrambone, R., & Mynatt, E. D. (2012) · Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145330 · Cited by 65 · cited 1×
“Finally, like others [119-121], we recognized the outcomes of persuasive systems depended on multiple factors, many of which were not examined in this synthesis.”
BackgroundHill, C., Creswell, C., Vigerland, S., Nauta, M. H., March, S., Donovan, C., Wolters, L., Spence, S. H., Martin, J. L., Wozney, L., et al. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.02.002 · Cited by 44 · cited 3×
“Although recent efforts have been made to provide guidance on the design and delivery features of iCBT [19], considerable differences exist across programs both in terms of their features and the health outcomes they produce.”
BackgroundMethodSelf-citationVigerland, S., Serlachius, E., Thulin, U., Andersson, G., Larsson, J., & Ljótsson, B. (2016) · Behaviour Research and Therapy · doi:10.1016/j.brat.2016.12.008 · Cited by 39 · cited 1×
“83% of users completed ≥9 of 11 modules (high use) Same as above Same as above [54,84]”
BackgroundStasiak, K., Merry, S. N., Frampton, C., & Moor, S. (2018) · Psychotherapy Research · doi:10.1080/10503307.2016.1244617 · Cited by 36 · cited 1×
“88/10 modules completed (low use) Same as above Same as above [79]”
MethodMotivational affordances and personality types in personal informatics
Karanam, Y., Filko, L., Kaser, L., Alotaibi, H., Makhsoom, E., & Voida, S. (2014) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2638728.2638800 · Cited by 32
Silfvernagel, K., Gren-Landell, M., Emanuelsson, M., Carlbring, P., & Andersson, G. (2015) · Internet Interventions · doi:10.1016/j.invent.2015.07.002 · Cited by 30
Nine key principles to guide youth mental health: development of service models in New South Wales
Howe, D., Batchelor, S., Coates, D., & Cashman, E. (2014) · Early Intervention in Psychiatry · doi:10.1111/eip.12096 · Cited by 25 · cited 1×
“These participatory research efforts may provide important guidance on the usefulness and functionality of select (PSD) features of iCBT programs in the real world and from the user’s perspective [96]; therefore, facilitating greater program use [97].”
BackgroundCalear, A. L., Batterham, P. J., Griffiths, K. M., & Christensen, H. M. (2016) · Stigma and Health · doi:10.1037/sah0000046 · Cited by 23 · cited 1×
“…k Teacher facilitated program administration and was available for general guidance or if questions arose but did not provide an active therapeutic role. l Mental health service provider was present in 1 study of the program to facilitate program administration or address student questions [56].”
BackgroundA pilot randomized controlled trial of the e-couch anxiety and worry program in schools
Calear, A. L., Christensen, H., Brewer, J., Mackinnon, A., & Griffiths, K. M. (2016) · Internet Interventions · doi:10.1016/j.invent.2016.08.003 · Cited by 21
Silfvernagel, K., Wassermann, C., & Andersson, G. (2017) · Internet Interventions · doi:10.1016/j.invent.2017.03.002 · Cited by 20
Alfonsson, S., Olsson, E., & Hursti, T. (2015) · Internet Interventions · doi:10.1016/j.invent.2015.07.005 · Cited by 14 · cited 1×
“Studies with multiple, active treatment arms would also allow the examination of the comparative effects of select program features [114] or in different delivery contexts (ie, varying the type of adjunct support).”
BackgroundThe Y-Worri Project: study protocol for a randomised controlled trial
Calear, A. L., Christensen, H. M., Griffiths, K. M., & Mackinnon, A. J. (2013) · Trials · doi:10.1186/1745-6215-14-76 · Cited by 12 · cited 1×
“Most iCBT programs were adaptations of previously developed mental health prevention or treatment resources [46-55].”
BackgroundRocha, A., Camacho, R., Ruwaard, J., & Riper, H. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.03.003 · Cited by 7
Patwardhan, M. (2016) · Cited by 5
March, S. (2008) · The University of Queensland · doi:10.14264/179806 · Cited by 3 · cited 2×
““In this way, the parent was empowered to help their child acquire and use the skills presented in the program, and to handle situations in which their child became anxious” [77].”
Silfvernagel, K. (2017) · Linköping studies in behavioural science · doi:10.3384/diss.diva-135823 · Cited by 3 · cited 1×
“5/9 modules completed (moderate use) Primary task support: Tailoring and Personalization; Dialogue support: Social role and Reminders; System credibility support: Authority, Expertise, and Trustworthiness Adolescent users; Therapist support: in-person, email, phone [55, 86]”
Vigerland, S. (2015) · Figshare · Cited by 2
Promoting Pediatric Healthcare through Persuasive Information Systems: A Qualitative Study
Langrial, S. U., & Araimi, F. A. F. A. (2017) · Journal of the Association for Information Systems · Cited by 2
(2016) · PubMed Central
Research works printed but not linked: 10
- Calear, AL, 2010, Med J Aust, 192, S12
- Spence, S, 2016, Internet-based therapies for child adolescent emotional behavioral problems, 197
- Pawson, R, 1997, Realistic Evaluation
- Walker, JL, 2012, Can J Cardiovasc Nurs, 22, 37
- 2013, Diagnostic and Statistical Manual of Mental Disorders, 5th Edition: DSM-5
- Keller, ML, 2011, An Internet Cognitive-Behavioral Skills-Based Program for Child Anxiety
- 10.1111/cpsp.12173 · doi:10.1111/cpsp.12173 · Paper page
- Martin, DJ, 2000, J Consult Clin Psychol, 68, 438, 10.1037/0022-006X.68.3.438 · doi:10.1037/0022-006x.68.3.438 · Paper page
- Eysenbach, G, 2013, Stud Health Technol Inform, 192, 657
- HalkoSKientzJAPersonality and persuasive technology: an exploratory study on health-promoting mobile applications2010International conference on persuasive technologyJune 7, 2010Copenhagen, DenmarkBerlin, HeidelbergSpringer150161
Cited by
Badawy, S. M., & Radovic, A. (2020) · JMIR Pediatrics and Parenting · doi:10.2196/20049 · Cited by 242
“Importantly, these programs showed moderate to high use, especially with adjunct support from coaches, teachers, or therapists, to demonstrate credibility and to help users complete their target behavior [43].”
BackgroundTremain, H., McEnery, C., Fletcher, K., & Murray, G. (2020) · JMIR Mental Health · doi:10.2196/17204 · Cited by 193
Louwen, C., Reidlinger, D. P., & Milne, N. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04003-y · Cited by 136
Methods for Human-Centered eHealth Development: Narrative Scoping Review
Kip, H., Keizer, J., da Silva, M. C., Beerlage-de Jong, N., Köhle, N., & Kelders, S. M. (2022) · Journal of Medical Internet Research · doi:10.2196/31858 · Cited by 67
“Consequently, not all state-of-the-art knowledge on technologies is published at the point of conducting the review [99,158,163,167,172].”
Rachamim, L., Zimmerman-Brenner, S., Rachamim, O., Mualem, H., Zingboim, N., & Rotstein, M. (2022) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-020-01686-2 · Cited by 47
Metadata
- Authors
- Radomski, A. D., Wozney, L., McGrath, P., Huguet, A., Hartling, L., Dyson, M. P., Bennett, K., & Newton, A. S.
- Affiliations
- University of Alberta (CA)
- Izaak Walton Killam Health Centre (CA)
- Dalhousie University (CA)
- McMaster University (CA)
- Corresponding author
- Newton, Amanda S
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-02-05
- Volume
- 21
- Issue
- 2
- Pages
- e11128
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC6379818
- OpenAlex
- W2912197217
- Cited by
- 40
- References
- 101
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 5 · 2023: 5 · 2022: 6 · 2021: 9 · 2020: 5 · 2019: 3
- FWCI
- 0.81
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Alberta Innovates
- Canadian Institutes of Health Research
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Social Sciences › Psychology › Clinical Psychology › Psychological Treatments and Assessments
- Keywords
- internet-based cognitive behavioral therapy, child and adolescent anxiety, persuasive system design, program use, medication adherence, dialogue support, social support, personalization, tailoring, anxiety prevention, anxiety treatment, digital mental health interventions, context-mechanism-outcome configurations, meta-ethnography, realist synthesis
- MeSH terms
- Adolescent, Adult, Anxiety, Child, Female, Humans, Male, Cognitive Behavioral Therapy, Telemedicine, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.