Internet-Delivered Cognitive-Behavioral Therapy for Social Anxiety Disorder in Romania: A Randomized Controlled Trial
Open accessThe DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index.
Abstract
BACKGROUND AND AIMS: Internet-based cognitive-behavioral therapy (iCBT) for social anxiety disorder has been found effective, as attested by independently conducted randomized controlled trials in four languages. The study aim is to test the efficacy of an iCBT program in a culture where it was not tested before (i.e. Romania). METHODS: Participants (n = 76) were recruited, screened and randomized to either a nine-week guided iCBT or a wait-list control group in April and May 2012. Self-report measures were collected before (April 2012) and after the intervention (July 2012), as well as six months later (January 2013). Although social anxiety was assessed with multiple measures, the Liebowitz Social Anxiety Scale - Self Report version (LSAS-SR) and Social Phobia Inventory (SPIN) were used as the primary outcome measures. RESULTS: A significant difference with a large between-group effect size in favor of iCBT was found (Cohen's d = 1.19 for LSAS-SR and d = 1.27 for SPIN). Recovery rates show that 36.8% (n = 14) in the treatment group score below the SPIN clinical cut-off compared to only 2.6% (n = 1) in the wait-list control group. Post-intervention clinical interviews also revealed that 34.2% (n = 13) of the treatment group was completely recovered (full remission) while additionally 36.8% (n = 14) retained some social anxiety symptoms (partial remission). However, an important study limitation is that post-intervention interviewers were not blinded to the study conditions. The program also effectively reduced depression and dysfunctional thinking (between-group Cohen's d = 0.84 for depression and d = 0.63 for dysfunctional thinking). Moreover, the iCBT intervention appears to have a long-term impact for participants' functioning, as the treatment gains were maintained six months later. CONCLUSIONS: Internet-delivered interventions display a high potential to quickly and widely disseminate effective evidence-based programs around the world. This study provides support for guided iCBT as a promising treatment approach in Romania. TRIAL REGISTRATION: ClinicalTrials.gov NCT01557894.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Studies show that programs delivered on the Iterapi platform reduce psychological problems for a range of disorders from both somatic problems such as hearing loss (Thorén et al., 2014) and tinnitus (Hesser et al., 2012) to more traditional psychological difficulties such as anxiety (Tulbure et al., 2015) and depression (Johansson et al., 2012).”
From Features and functionality of the Iterapi platform for internet-based psychological treatment (2016) · cites this paper as Tulbure et al., 2015
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Diagnostic and Statistical Manual of Mental Disorders
Williams, J. B. W., & First, M. (2013) · Encyclopedia of Social Work · doi:10.1093/acrefore/9780199975839.013.104 · Cited by 109,519
Jacobson, N. S., & Truax, P. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.12 · Cited by 7,770
(1962) · Medical Entomology and Zoology · Cited by 4,856
Mattick, R. P., & Clarke, J. C. (1998) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(97)10031-6 · Cited by 3,367
Structured clinical interview for DSM-IV axis I disorders : SCID-I : clinical version : scoresheet
First, M. B., Spitzer, R. L., Gibbon, M., & Williams, J. B. W. (1997) · American Psychiatric Press eBooks · Cited by 3,101
A cognitive model of social phobia
Adrian Wells, A. (1995) · Research Explorer (The University of Manchester) · Cited by 2,701
Liebowitz, M. R. (2015) · Modern Trends in Pharmacopsychiatry · doi:10.1159/000414022 · Cited by 2,266
Analysing controlled trials with baseline and follow up measurements
Vickers, A. J., & Altman, D. G. (2001) · BMJ · doi:10.1136/bmj.323.7321.1123 · Cited by 2,116
(2012) · SpringerReference · doi:10.1007/springerreference_179754 · Cited by 2,089
Psychometric properties of the Social Phobia Inventory (SPIN): New self-rating scale
Connor, K. M., Davidson, J. R. T., Churchill, L. E., Sherwood, A., Weisler, R. H., & Foa, E. (2000) · British Journal of Psychiatry · doi:10.1192/bjp.176.4.379 · Cited by 1,624
Cognitive self-statements in depression: Development of an automatic thoughts questionnaire
Hollon, S. D., & Kendall, P. C. (1980) · Cognitive Therapy and Research · doi:10.1007/bf01178214 · Cited by 1,480
Social Phobia: Diagnosis, Assessment, and Treatment
Amada, G. (1996) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.1996.50.3.373 · Cited by 1,248
Social phobia : diagnosis, assessment, and treatment
Heimberg, R. G. (1995) · Guilford Press eBooks · Cited by 1,067
Hofmann, S. G. (2007) · Cognitive Behaviour Therapy · doi:10.1080/16506070701421313 · Cited by 1,064 · cited 1×
“, public speaking) and generalized SAD [4].”
Agoraphobia, simple phobia, and social phobia in the National Comorbidity Survey
Magee, W. J. (1996) · Archives of General Psychiatry · doi:10.1001/archpsyc.1996.01830020077009 · Cited by 996
FRESCO, D. M., COLES, M. E., HEIMBERG, R. G., LIEBOWITZ, M. R., HAMI, S., STEIN, M. B., & GOETZ, D. (2001) · Psychological Medicine · doi:10.1017/s0033291701004056 · Cited by 874 · cited 1×
“, all included participants), as well as the level had to be within two SDs from a non-clinical reference group [30, 51].”
BackgroundDisability and Quality of Life in Social Phobia: Epidemiologic Findings
Stein, M. B., & Kean, Y. M. (2000) · American Journal of Psychiatry · doi:10.1176/appi.ajp.157.10.1606 · Cited by 685
The Liebowitz social anxiety scale as a self-report instrument: a preliminary psychometric analysis
Baker, S. L., Heinrichs, N., Kim, H.-J., & Hofmann, S. G. (2002) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(01)00060-2 · Cited by 640
Review of Motivational Interviewing in promoting health behaviors
Martins, R. K., & McNeil, D. W. (2009) · Clinical Psychology Review · doi:10.1016/j.cpr.2009.02.001 · Cited by 584 · cited 1×
“Also, a motivational interviewing could be added before the intervention to enhance treatment adherence [54, 56].”
BackgroundFluoxetine, Comprehensive Cognitive Behavioral Therapy, and Placeboin Generalized Social Phobia
Davidson, J. R. T., Foa, E. B., Huppert, J. D., Keefe, F. J., Franklin, M. E., Compton, J. S., Zhao, N., Connor, K. M., Lynch, T. R., & Gadde, K. M. (2004) · Archives of General Psychiatry · doi:10.1001/archpsyc.61.10.1005 · Cited by 415 · cited 1×
“Thus CBT interventions are mainly focused on altering these beliefs using both cognitive and behavioral strategies such as cognitive restructuring, behavioral experiments, exposure, applied relaxation, and social skills training[19].”
BackgroundA Cognitive Behavioral Model of Social Anxiety Disorder
Heimberg, R. G., Brozovich, F. A., & Rapee, R. M. (2010) · Social Anxiety · doi:10.1016/b978-0-12-375096-9.00015-8 · Cited by 407 · cited 1×
“Cognitive models of SAD emphasize the importance of dysfunctional beliefs in generating the disorder, by transforming innocuous social cues into threats [17, 18].”
BackgroundSocial phobia: overview of community surveys
Furmark, T. (2002) · Acta Psychiatrica Scandinavica · doi:10.1034/j.1600-0447.2002.1r103.x · Cited by 386
Andersson, G., Carlbring, P., Holmström, A., Sparthan, E., Furmark, T., Nilsson-Ihrfelt, E., Buhrman, M., & Ekselius, L. (2006) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.74.4.677 · Cited by 382
Self-citationSize and burden of social phobia in Europe
Fehm, L., Pelissolo, A., Furmark, T., & Wittchen, H.-U. (2005) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2005.04.002 · Cited by 373
Social phobia in the general population: prevalence and sociodemographic profile
Furmark, T., Tillfors, M., Everz, P.-O., Marteinsdottir, I., Gefvert, O., & Fredrikson, M. (1999) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s001270050163 · Cited by 358
Barriers to the Treatment of Social Anxiety
Olfson, M., Guardino, M., Struening, E., Schneier, F. R., Hellman, F., & Klein, D. F. (2000) · American Journal of Psychiatry · doi:10.1176/appi.ajp.157.4.521 · Cited by 356 · cited 3×
“Embarrassment associated with help-seeking and fear of what others might think or say have been found to prevent individuals with SAD from seeking treatment [7].”
BackgroundCognitive-behavioral therapy for social anxiety disorder: Current status and future directions
Heimberg, R. G. (2002) · Biological Psychiatry · doi:10.1016/s0006-3223(01)01183-0 · Cited by 351
Psychological treatment of social anxiety disorder: a meta-analysis
Acarturk, C., Cuijpers, P., van Straten, A., & de Graaf, R. (2009) · Psychological Medicine · doi:10.1017/s0033291708003590 · Cited by 333
Cognitive therapy for social phobia: individual versus group treatment
Stangier, U., Heidenreich, T., Peitz, M., Lauterbach, W., & Clark, D. M. (2003) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(02)00176-6 · Cited by 326 · cited 1×
“Evidence indicates that individually administered CBT is equally effective [14] or more effective [15]than group CBT and superior to standard psychiatric treatment with medication and emotional support [16].”
BackgroundPsychometric properties of the social phobia inventory: Further evaluation
Antony, M. M., Coons, M. J., McCabe, R. E., Ashbaugh, A., & Swinson, R. P. (2006) · Behaviour Research and Therapy · doi:10.1016/j.brat.2005.08.013 · Cited by 299 · cited 1×
“The scale has generally good to excellent psychometric proprieties[30, 43]and was considered an evidence based measure for treatment monitoring and treatment outcome[41].”
BackgroundSocial anxiety : clinical, developmental, and social perspectives
Hofmann, S. G., & DiBartolo, P. M. (2014) · Cited by 293
Carlbring, P., Gunnarsdóttir, M., Hedensjö, L., Andersson, G., Ekselius, L., & Furmark, T. (2007) · British Journal of Psychiatry · doi:10.1192/bjp.bp.105.020107 · Cited by 293
Self-citationBerger, T., Caspar, F., Richardson, R., Kneubühler, B., Sutter, D., & Andersson, G. (2011) · Behaviour Research and Therapy · doi:10.1016/j.brat.2010.12.007 · Cited by 265 · cited 1×
“In most studies, a guided self-help approach has been used, where a web-based presented program is combined with minimal, but regular therapist contact by e-mail or phone [24].”
BackgroundSelf-citationGuided and unguided self-help for social anxiety disorder: randomised controlled trial
Furmark, T., Carlbring, P., Hedman, E., Sonnenstein, A., Clevberger, P., Bohman, B., Eriksson, A., Hållén, A., Frykman, M., Holmström, A., et al. (2009) · British Journal of Psychiatry · doi:10.1192/bjp.bp.108.060996 · Cited by 256 · cited 2×
“Finally, the original SOFIE interventions conducted in Sweden reveal similar medium to large ES for both primary and secondary outcome measures [25, 37, 38].”
MethodSelf-citationMörtberg, E., Clark, D. M., Sundin, Ö., & Åberg Wistedt, A. (2007) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2006.00839.x · Cited by 213 · cited 1×
“Evidence indicates that individually administered CBT is equally effective [14] or more effective [15]than group CBT and superior to standard psychiatric treatment with medication and emotional support [16].”
BackgroundHedman, E., Andersson, E., Ljótsson, B., Andersson, G., Rück, C., & Lindefors, N. (2011) · Behaviour Research and Therapy · doi:10.1016/j.brat.2011.07.009 · Cited by 183 · cited 1×
“Internet-based CBT protocols have since been developed and tested by research groups in Australia, Spain, Sweden, and Switzerland, with results supporting the short- and long-term clinical benefits and financial advantages of these interventions[26].”
BackgroundSelf-citationA Guide to Assessments that Work
(2008) · Oxford University Press eBooks · doi:10.1093/med:psych/9780195310641.001.0001 · Cited by 176
A Guide to assessments that work
(2008) · Choice Reviews Online · doi:10.5860/choice.46-0328 · Cited by 169
Andersson, G., Carlbring, P., Furmark, T., & on behalf of the S. O. F. I. E. Research Group (2012) · PLoS ONE · doi:10.1371/journal.pone.0037411 · Cited by 166 · cited 1×
“, LSAS-SR, SPIN, BDI-II); and similar procedures were previously used in other clinical trials for selecting SAD participants [25, 36].”
MethodSelf-citationThe lifelong course of social anxiety disorder: a clinical perspective
Keller, M. B. (2003) · Acta Psychiatrica Scandinavica · doi:10.1034/j.1600-0447.108.s417.6.x · Cited by 165
Situational domains of social phobia
Holt, C. S., Heimberg, R. G., Hope, D. A., & Liebowitz, M. R. (1992) · Journal of Anxiety Disorders · doi:10.1016/0887-6185(92)90027-5 · Cited by 160
Heimberg, R. G., Stein, M. B., Hiripi, E., & Kessler, R. C. (2000) · European Psychiatry · doi:10.1016/s0924-9338(00)00213-3 · Cited by 145
Spek, V., Cuijpers, P., Nyklíček, I., Smits, N., Riper, H., Keyzer, J., & Pop, V. (2008) · Psychological Medicine · doi:10.1017/s0033291707002590 · Cited by 139
Alcohol abuse in social phobia
Schneier, F. R., Martin, L. Y., Liebowitz, M. R., Gorman, J. M., & Fyer, A. J. (1989) · Journal of Anxiety Disorders · doi:10.1016/0887-6185(89)90025-x · Cited by 122
Dagöö, J., Asplund, R. P., Bsenko, H. A., Hjerling, S., Holmberg, A., Westh, S., Öberg, L., Ljótsson, B., Carlbring, P., Furmark, T., et al. (2014) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2014.02.003 · Cited by 121
Self-citationWong, J., Gordon, E. A., & Heimberg, R. G. (2012) · Handbook of Evidence-Based Practice in Clinical Psychology · doi:10.1002/9781118156391.ebcp002027 · Cited by 96
Månsson, K. N. T., Carlbring, P., Frick, A., Engman, J., Olsson, C.-J., Bodlund, O., Furmark, T., & Andersson, G. (2013) · Psychiatry Research: Neuroimaging · doi:10.1016/j.pscychresns.2013.08.012 · Cited by 92 · cited 1×
“The Romanian iCBT intervention, named iSOFIE, was based on a Swedish internetdelivered CBT protocol which had been tested in previous studies [25, 39, 52].”
MethodSelf-citationEfficacy of a Manualized and Workbook-Driven Individual Treatment for Social Anxiety Disorder
Ledley, D. R., Heimberg, R. G., Hope, D. A., Hayes, S. A., Zaider, T. I., Dyke, M. V., Turk, C. L., Kraus, C., & Fresco, D. M. (2009) · Behavior Therapy · doi:10.1016/j.beth.2008.12.001 · Cited by 91 · cited 1×
“Evidence indicates that individually administered CBT is equally effective [14] or more effective [15]than group CBT and superior to standard psychiatric treatment with medication and emotional support [16].”
BackgroundSocial Phobia or Social Anxiety Disorder: What's in a Name?
Liebowitz, M. R., Heimberg, R. G., Fresco, D. M., Travers, J., & Stein, M. B. (2000) · Archives of General Psychiatry · doi:10.1001/archpsyc.57.2.191-a · Cited by 88
Social Phobia (Social Anxiety Disorder)
Andersson, G., & Carlbring, P. (2010) · Springer eBooks · doi:10.1007/978-1-4419-6510-3_6 · Cited by 80
Self-citationStepped Care and e-Health: Practical Applications to Behavioral Disorders
Draper, C., & O’Donohue, W. (2010) · Springer eBooks · doi:10.1007/978-1-4419-6510-3 · Cited by 72
Internet Treatment for Social Phobia Reduces Comorbidity
Titov, N., Gibson, M., Andrews, G., & McEvoy, P. (2009) · Australian & New Zealand Journal of Psychiatry · doi:10.1080/00048670903001992 · Cited by 58 · cited 1×
“Also, a motivational interviewing could be added before the intervention to enhance treatment adherence [54, 56].”
BackgroundTulbure, B. T., Szentágotai, A., Dobrean, A., & David, D. O. (2012) · Child Psychiatry & Human Development · doi:10.1007/s10578-012-0297-y · Cited by 56
Self-citationA comparative outcome study of four cognitive therapies in the treatment of social anxiety
Digiuseppe, R. A., McGowan, L., Sutton-Simon, K., & Gardner, F. L. (1990) · Journal of Rational-Emotive & Cognitive-Behavior Therapy · doi:10.1007/bf01066280 · Cited by 32
Handbook of evidence-based psychotherapies : a guide for research and practice
Freeman, C., & Power, M. (2007) · Wiley eBooks · Cited by 31
Tulbure, B. T. (2011) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2011.10.108 · Cited by 20
Self-citationSpecific Phobia and Social Phobia
Rowa, K., McCabe, R. E., & Antony, M. M. (2008) · Oxford University Press eBooks · doi:10.1093/med:psych/9780195310641.003.0010 · Cited by 13
State-of-the-Art Treatment via the Internet: An Optimistic Vision of the Future
Carlbring, P., Andersson, G., & Kaldo, V. (2011) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2011.575591 · Cited by 10
Self-citation
Research works printed but not linked: 2
- A Ellis, 1994, Revised and Expanded Edition ed
- RP Mattick, 1998, Development and validation of measures of social phobia scrutiny fear and social interaction anxiety, Behav Res Ther, 36, 455
Cited by
Andrews, G., Basu, A., Cuijpers, P., Craske, M. G., McEvoy, P., English, C. L., & Newby, J. M. (2018) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.01.001 · Cited by 922
Internet-delivered psychological treatments: from innovation to implementation
Andersson, G., Titov, N., Dear, B. F., Rozental, A., & Carlbring, P. (2019) · World Psychiatry · doi:10.1002/wps.20610 · Cited by 750
Self-citationInternet-Delivered Psychological Treatments
Andersson, G. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093006 · Cited by 595
Self-citationNaslund, J. A., Aschbrenner, K. A., Araya, R., Marsch, L. A., Unützer, J., Patel, V., & Bartels, S. J. (2017) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30096-2 · Cited by 638
Etzelmueller, A., Vis, C., Karyotaki, E., Baumeister, H., Titov, N., Berking, M., Cuijpers, P., Riper, H., & Ebert, D. D. (2020) · Journal of Medical Internet Research · doi:10.2196/18100 · Cited by 356
Metadata
- Authors
- Tulbure, B. T., Szentagotai, A., David, O., Ștefan, S., Månsson, K. N. T., David, D., & Andersson, G.
- Affiliations
- West University of Timişoara (RO)
- Babeș-Bolyai University (RO)
- Linköping University (SE)
- Corresponding author
- Tulbure, Bogdan Tudor
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-05-04
- Volume
- 10
- Issue
- 5
- Pages
- e0123997
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4418823
- OpenAlex
- W263243887
- Semantic Scholar
- c8e6050e604cfe5236e751805404211ed0252cf3
- Cited by
- 81
- References
- 59
- Citations by year
- 2026: 6 · 2025: 5 · 2024: 4 · 2023: 21 · 2022: 9 · 2021: 5 · 2020: 5 · 2019: 8 · 2018: 5 · 2017: 5 · 2016: 7 · 2015: 1
- FWCI
- 9.72
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DiVA (Linkoping University) (submittedVersion, other-oa)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- European Social Fund (2007–2013, POSDRU 89/1.5/S/60189)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Social Sciences › Psychology › Clinical Psychology › Psychological Treatments and Assessments
- Keywords
- internet-based cognitive behavioral therapy, social anxiety disorder, Romania, Liebowitz Social Anxiety Scale, Social Phobia Inventory, depression, dysfunctional thinking, dissemination of evidence-based treatment
- MeSH terms
- Adult, Anxiety Disorders, Demography, Female, Humans, Male, Patient Compliance, Romania, Cognitive Behavioral Therapy, Case-Control Studies, Treatment Outcome, Patient Satisfaction, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.