Internet-Delivered Cognitive Behavioral Therapy for Anxiety Disorders in Open Community Versus Clinical Service Recruitment: Meta-Analysis
Open accessThe DOI matches its Crossref record; 110 of the 113 references it cites were found in a registry or an index; 9 of the 34 sources its claims cite were checked for support.
Abstract
BACKGROUND: Ample studies have shown the effectiveness of internet-delivered cognitive behavioral therapy (iCBT) for anxiety disorders. These studies recruited their participants mainly from the community and, to a lesser extent, from within routine care services. Little is known about whether different recruitment strategies lead to different treatment effects. OBJECTIVE: This meta-analysis compared clinical results obtained in trials with recruitment from the community versus results obtained in trials with clinical service recruitment and explored factors that may mediate differences in treatment outcome. METHODS: We included randomized controlled trials in which the clinical effects of iCBT for anxiety disorders were compared with a control condition (waitlist controls or face-to-face cognitive behavioral therapy). We classified trials as open recruitment trials (recruitment from the community) or clinical service recruitment trials (recruitment through outpatient clinics). Pooled effect sizes based on measures examining anxiety symptoms, depressive symptoms, and quality of life were computed for each type of trial. Subgroup analyses examined whether clinical results from open recruitment trials differed from those obtained in clinical service recruitment trials. Additional analyses explored which demographic, clinical, and treatment-related factors contributed to differences in effect sizes of open recruitment versus clinical service recruitment trials. RESULTS: We included 42 studies with 53 comparisons (43 open recruitment comparisons and 10 clinical recruitment comparisons). Analyses of anxiety measures revealed, first, that iCBT open recruitment studies with waitlist control comparators showed a significantly higher effect size for decrease in anxiety symptoms than did those with clinical recruitment (Q=10.09; P=.001). This association between recruitment method and effect size was no longer significant in a multivariate metaregression with treatment adherence and exclusion of patients with depressive symptoms entered as additional predictors of effect size. Second, effect size for decrease in anxiety symptoms did not differ significantly between clinical recruitment and open recruitment studies with face-to-face cognitive behavioral therapy comparators. The effects of open recruitment trials and clinical recruitment trials did not differ significantly for the secondary outcomes, compared with face-to-face cognitive behavioral therapy and waitlist controls. CONCLUSIONS: iCBT was effective in samples recruited in clinical practice, but effect sizes were smaller than those found in trials with an open recruitment method for studies with waitlist control comparators. Hence, for patients with anxiety disorders in routine care, the impact of iCBT may not be as positive as for study participants recruited from the community. The difference between open recruitment trials and clinical service recruitment trials might be partly explained by patients' greater therapy adherence in open recruitment trials and the stricter exclusion of patients with severe depressive symptoms in these studies. Since most trials in this meta-analysis applied an open recruitment method, more studies with routine care populations are needed to further validate these findings.
Overview
CoChat Summary
TL;DR
This meta-analysis of 42 RCTs (53 comparisons, 3714 participants) found internet-delivered CBT (iCBT) for anxiety disorders was effective versus waitlist control, but effect sizes were smaller in clinical service recruitment (CSR) trials than open recruitment (OR) trials for waitlist comparisons. The recruitment-method difference was no longer significant in multivariate metaregression with treatment adherence and exclusion of severely depressed patients entered as predictors.
Key findings
- Overall iCBT versus waitlist control reduced anxiety symptoms with g=0.72 (95% CI 0.60-0.83; P<.001; I²=53%; NNT=3). iCBT versus face-to-face CBT showed a nonsignificant difference, g=0.12 (95% CI –0.02 to 0.26; P=.11; NNT=15).
- For waitlist-controlled studies, OR trials had a large effect on anxiety (g=0.79; P<.001) while CSR trials had a small effect (g=0.28; P=.003); the between-group difference was significant (Q=10.09; P=.001). For face-to-face CBT comparisons, OR (g=0.19; P=.09) and CSR (g=0.06; P=.51) did not differ significantly (Q=0.82; P=.37).
- Secondary outcomes showed no significant OR versus CSR differences for depression or quality of life. iCBT versus waitlist control effects were g=0.61 for depressive symptoms and g=0.44 for quality of life.
- In separate analyses, exclusion of severely depressed patients (Q=8.06; P=.005), treatment dose (slope=0.10; P=.003), and treatment adherence (slope=0.01; P<.001) were associated with effect size for waitlist comparators. In multivariate metaregression, recruitment type (slope=0.30; P=.14), treatment adherence (slope=0.01; P=.23), and exclusion of severely depressed patients (slope=0.13; P=.27) were no longer significant.
Methods
- PRISMA-guided systematic review and meta-analysis; searched PubMed, PsycINFO, and EMBASE through December 2017 for RCTs of guided, unguided, or blended iCBT for adults with a primary anxiety disorder diagnosis by structured interview, compared with waitlist control or face-to-face CBT; excluded OCD and PTSD.
- Included 42 trials with 53 comparisons (43 OR, 10 CSR) and 3714 participants; outcomes were anxiety symptoms, depressive symptoms, and quality of life.
- Used random-effects Hedges g, subgroup/mixed-effects analyses, metaregression, Cochrane risk-of-bias assessment, and Egger/trim-and-fill publication-bias checks. OR trials recruited self-referred community participants; CSR trials recruited patients already seeking treatment in outpatient clinical practices.
Limitations
- Stated by the authors: few trials had face-to-face CBT comparators, so null findings may be underpowered; clinical representativeness involves criteria beyond recruitment type; 3 trials used mixed recruitment, which may have reduced the OR-CSR difference; subgroup and metaregression results were observational and require direct comparisons.
- Evident in the text: all included studies were rated high risk of bias for blinding of participants and personnel, and selective outcome reporting was low risk in only 4/33 OR trials and 2/9 CSR trials.
- Evident in the text: heterogeneity was moderate for the primary waitlist-controlled anxiety analysis (I²=53%), which the authors note complicates interpretation of the pooled effect size.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
5 Valid · 0 Not valid · 11 Not checked · 57 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Often these studies apply strict inclusion and exclusion criteria; for example, they may exclude patients taking psychoactive medication, patients with comorbid disorders, or severely depressed patients [8,9].”
§ Internet-Delivered Cognitive Behavioral Therapy for Anxiety Disorders in Open Community Versus Clinical Service Recruitment: Meta-Analysis > Introduction > Background
What the sources say
- Not checked yetcites [8] Westen, D, 2001, J Consult Clin Psychol source
- Partly supportedcites [9] sourceExperimental — accuracy not yet measured
Stewart & Chambless (2009) support only part of the attributed content: the Introduction (Abstract section, p. 1) states that in research settings 'patients with comorbid disorders are often excluded to achieve homogeneous diagnostic samples' (a characterisation it credits to Westen & Morrison, 2001), and the Method (Studies) section treats 'allowance of medication' and 'clinically representative patients (i.e., no exclusion criteria aside from psychosis, suicidality, organic brain disease, or substance abuse)' as markers separating effectiveness from efficacy studies — which implies, but does not state, that medication-taking patients are excluded. Nothing in our copy says that severely depressed patients are excluded; the paper speaks only of practice patients being 'purported to be more severe'. Also, the selection argument in this source concerns randomized efficacy/RCT samples, not open-recruitment trials as the citing sentence frames it. (LLM, conf=0.72)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Only a minority of the patient samples in trials with an OR method correspond to patient populations in a regular clinical setting in terms of sociodemographic characteristics, motivation for treatment, level of suffering, and clinical characteristics such as severity of anxiety, comorbidity, or medical history [11-13].”
§ Internet-Delivered Cognitive Behavioral Therapy for Anxiety Disorders in Open Community Versus Clinical Service Recruitment: Meta-Analysis > Introduction > Background
What the sources say
- Partly supportedcites [11] Persons, JB, 1998, J Consult Clin Psychol sourceExperimental — accuracy not yet measured
Silberschatz's section 'Results of RCTs Are Useless to Clinicians' (and his Rebuttal) states the core of the claim: RCTs lack external/ecological validity, 'RCTs do not focus on the types of patients seen in practice', research subjects 'differ markedly from psychotherapy patients', and because single-diagnosis, homogeneous samples are required, 'the type of patient most likely to be screened out of the RCT is the patient seen by the typical practicing therapist' — i.e., comorbidity/multiple problems and general level of functioning are covered. However, our copy nowhere gives the quantified 'only a minority' figure, and it does not address the other enumerated dimensions the claim attributes to this source: sociodemographic characteristics, motivation for treatment, level of suffering, severity of anxiety, or medical history (the closest is Seligman's five properties of practice, which concern treatment duration, self-correction, therapist choice, multiple problems, and general functioning — not patient demographics or morbidity). Persons' rebuttal likewise concedes only that RCT samples are homogeneous and thus 'not easily generalized'. (LLM, conf=0.72)
- Not checked yetcites [12] source
- Partly supportedcites [13] sourceExperimental — accuracy not yet measured
The only passage addressing the claim's point is the Abstract Background, which hedges: people seeking Internet treatment 'may not resemble the general population or have less severe disorders than patients attending outpatient clinics', so 'the response to treatment in Internet based trials might not generalize'. That is a concern about generalization, not a finding that 'only a minority' of trial samples fail to match regular clinical settings; the paper studies no trials at all, comparing one Internet clinic (n=774), one specialist outpatient clinic (n=454) and a national survey (n=627) on demographics and symptom severity only (Abstract Methodology; Measurements). Nothing in our copy supports the claim's specific elements of motivation for treatment, comorbidity, or medical history, and the paper's own results run the other way for severity and demographics (Abstract Principal Findings/Conclusions; Discussion Conclusions: Internet sample as severe as the outpatient clinic, demographics more consistent with the national sample, 'benefits of Internet treatment could apply to the wider population'). (LLM, conf=0.65)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“A meta-analysis [14] found a strong and positive relationship (r=.70) between the number of exclusion criteria and the rate of clinically improved participants for studies on iCBT for anxiety disorders.”
§ Internet-Delivered Cognitive Behavioral Therapy for Anxiety Disorders in Open Community Versus Clinical Service Recruitment: Meta-Analysis > Introduction > Background
What the sources say
- Supportedcites [14] source
The Abstract (Results section of the abstract) states 'a large relationship between number of exclusion criteria and proportion clinically changed in the treatment group for anxiety studies (r = 0.70)', and the Results (Correlation Analyses) describe 'the strong, positive correlation found between the number of exclusion criteria and the proportion clinically changed... for anxiety studies'; the Discussion ('Exclusion and Outcome') repeats that the number of exclusion criteria 'was strongly and positively correlated with the proportion of cCBT participants experiencing clinically significant change for anxiety studies.' Magnitude, direction, both variables and the anxiety-studies subset all match. Two minor caveats the citing sentence drops: the source labels the intervention cCBT (computerized CBT, a superset of iCBT, its anxiety subset being largely internet-delivered trials) and flags the correlation as preliminary given only 23 anxiety studies, but neither changes what is attributed to it. (LLM, conf=0.85)
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The difference in overall effect size for the decrease in anxiety symptoms between iCBT and face-to-face CBT at posttest was nonsignificant (g=0.12, 95% CI –0.02 to 0.26; P=.11; I²=0%, 95% CI 0-75; NNT=15).”
- Also stated in: § Background · hedged
“Studies also suggest that it is as effective as face-to-face cognitive behavioral therapy (CBT) in improving symptoms of anxiety [1-5], although these studies are limited in number.”
§ Internet-Delivered Cognitive Behavioral Therapy for Anxiety Disorders in Open Community Versus Clinical Service Recruitment: Meta-Analysis > Results > Overall Effect
What the sources say
- Partly supportedcites [1] sourceExperimental — accuracy not yet measured
Our copy supports the qualitative core: comparison 3.2 (Reduction in disorder-specific anxiety symptom severity) reports a non-significant SMD of 0.06 (95% CI -0.25 to 0.37) for therapist-supported ICBT versus face-to-face CBT at post-treatment (seven studies, 228 vs 222 participants), and 3.3 (general anxiety) reports SMD 0.06 (95% CI -0.42 to 0.55); the Discussion ('Summary of main results') likewise states no significant differences versus face-to-face group and individual CBT. However, none of the specific statistics attributed to this source — g = 0.12, 95% CI -0.02 to 0.26, P = .11, I² = 0%, NNT = 15 — appear anywhere in our copy (searched for these values, NNT/number needed to treat, and the face-to-face comparisons); the review instead reports substantial heterogeneity (I² = 60% and 78%) at post-treatment, not I² = 0%. So the direction of the finding is supported while the effect size, CI, p-value, heterogeneity and NNT are not this review's numbers. (LLM, conf=0.78)
- Not checked yetcites [2] source
- Not checked yetcites [3] source
- Not checked yetcites [4] source
- Not checked yetcites [5] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Background · hedged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Baseline Beck Anxiety Inventory score, mean (SD) | Baseline Beck Anxiety Inventory score, mean (SD) | 25.4 (11.8) | 29.0 (11.1) | t1208=–6.2 | <.001”
§ Internet-Delivered Cognitive Behavioral Therapy for Anxiety Disorders in Open Community Versus Clinical Service Recruitment: Meta-Analysis > Results > Description of Included Studies
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
43 more claims
Create a free account to see all 73 claims and to verify the 25 references still open.
- It may state two separate claims as one. 16
- Probably not a substantial claim. 13
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In addition, participants in our study appear to be a clinically representative sample, in view of the large proportion of patients with comorbid disorders, lower education levels and severe anxiety symptoms at baseline, in comparison with self-referred samples recruited from the community [12].”
From Acceptability, effectiveness and cost-effectiveness of blended cognitive-behavioural therapy (bCBT) versus face-to-face CBT (ftfCBT) for anxiety disorders in specialised mental health care: A 15-week randomised controlled trial with 1-year follow-up (2021) · cites this paper as [12]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“There is strong evidence supporting internet-delivered cognitive behaviour therapy (iCBT) and consequently growing demand for iCBT in Canada.”
The cited source says what the claim attributes to it.
In the citing paper
“while a small effect size (g = 0.28) was found in the same comparison in routine care populations.”
The cited source says what the claim attributes to it.
In the citing paper
“in a recent meta-analysis [12], a large effect size (g = 0.79) was found for anxiety symptom reduction by iCBT as compared with waitlisted controls in samples recruited from the community”
The cited source says what the claim attributes to it.
In the citing paper
“iCBT has been found effective for several anxiety disorders [10–13], and there are indications for its cost-effectiveness [14].”
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
The CES-D Scale: A Self-Report Depression Scale for Research in the General Population
Radloff, L. S. (1977) · Applied Psychological Measurement · doi:10.1177/014662167700100306 · Cited by 53,267
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
An inventory for measuring depression
BECK, A. T. (1961) · Archives of General Psychiatry · doi:10.1001/archpsyc.1961.01710120031004 · Cited by 38,032
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.10.1092 · Cited by 31,765
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
(1970) · Cited by 24,983
Research works printed but not linked: 8
- Riper, H, 2013, Preventie & eMental-health. Onderzoek dat leidt, technologie die verleidt, preventie die bereikt en beklijft. Kennissynthese 2013 in opdracht van ZonMw
- Beck, AT, 1979, Cognitive Therapy of Depression
- Gallego, M, 2011, Int J Clin Heal Psychol, 11, 459
- Bruinsma, A, 2016, Tijdschr Psychiatr, 58, 361
- Cohen, J, 1988, Statistical Power Analysis for the Behavioral Sciences. 2nd edition
- Orsini, N, 2006, Heterogi: Stata Module to Quantify Heterogeneity in a Meta-Analysis
- Allison, Pd, 1999, Logistic Regression Using SAS: Theory And Application. 1st edition
- Weisz, J, 1995, J Consult Clin Psychol, 63, 688, 10.1037/0022-006x.63.5.688 · DOI · Paper page
Cited by
Etzelmueller, A., Vis, C., Karyotaki, E., Baumeister, H., Titov, N., Berking, M., Cuijpers, P., Riper, H., & Ebert, D. D. (2020) · Journal of Medical Internet Research · doi:10.2196/18100 · Cited by 356
“They also found that effects were smaller in samples recruited in clinical practice than in samples recruited with an open recruitment method compared with waitlist-control groups [13], which raises the question of the effects of iCBT when implemented in routine practice.”
BackgroundResultInfluentialSelf-citationMilgrom, J., Danaher, B. G., Seeley, J. R., Holt, C. J., Holt, C., Ericksen, J., Tyler, M. S., Gau, J. M., & Gemmill, A. W. (2021) · Journal of Medical Internet Research · doi:10.2196/17185 · Cited by 96
Evaluation of the Use of Digital Mental Health Platforms and Interventions: Scoping Review
Balcombe, L., & De Leo, D. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20010362 · Cited by 76
Domhardt, M., Letsch, J., Kybelka, J., Koenigbauer, J., Doebler, P., & Baumeister, H. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.06.059 · Cited by 75
Hadjistavropoulos, H. D., Peynenburg, V., Thiessen, D. L., Nugent, M., Karin, E., Staples, L., Dear, B. F., & Titov, N. (2022) · The Canadian Journal of Psychiatry · doi:10.1177/07067437211006873 · Cited by 52
Self-citation
Metadata
- Authors
- Romijn, G., Batelaan, N., Kok, R., Koning, J., van Balkom, A., Titov, N., & Riper, H.
- Affiliations
- Amsterdam Public Health (NL)
- University of Southern Denmark (DK)
- Pro Persona (NL)
- Macquarie University (AU)
- Corresponding author
- Romijn, Geke
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2019-04-17
- Volume
- 21
- Issue
- 4
- Pages
- e11706
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC6492068
- OpenAlex
- W2938191752
- Cited by
- 52
- References
- 113
- Citations by year
- 2026: 1 · 2025: 7 · 2024: 4 · 2023: 4 · 2022: 8 · 2021: 12 · 2020: 13 · 2019: 3
- FWCI
- 1.27
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- VU Research Portal (publishedVersion, other-oa)
- University of Southern Denmark Research Portal (University of Southern Denmark) (publishedVersion, cc-by)
- Pure Amsterdam UMC (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Keywords
- internet-based cognitive behavioral therapy, anxiety disorders, recruitment strategies, community-based recruitment, waitlist control, medication adherence, depressive symptoms, quality of life, usual care, effect size heterogeneity, moderators of treatment outcome, digital mental health interventions, primary care settings, generalizability of trial results
- MeSH terms
- Adult, Anxiety Disorders, Female, Humans, Male, Middle Aged, Quality of Life, Cognitive Behavioral Therapy, Treatment Outcome, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.