Negative effects associated with internet-delivered cognitive behaviour therapy: An analysis of client emails
Open accessAbstract
Internet-delivered cognitive behaviour therapy (ICBT) is an efficacious form of treatment for anxiety and depression, yet it is still possible for clients to experience negative effects associated with treatment. In the ICBT literature, the term negative effects is broadly used to refer to all potentially adverse or unwanted events or experiences that are perceived as undesirable by the client and may or may not be associated with long-term symptoms or distress. Previous ICBT studies have asked clients to retrospectively describe negative effects at post-treatment; however, no research has examined the content of clients' emails to their therapist to see whether clients are reporting negative effects as they arise. In the current study, 96 clients (80 completers; 16 non-completers) were randomly selected from a published ICBT trial and directed content analysis was used to examine client emails for mention of negative effects. In addition, correlational analyses were used to examine the relationship between negative effects and: 1) demographic characteristics; 2) treatment engagement; 3) treatment satisfaction; 4) working alliance; and 5) symptom outcomes among completers. The results indicated that 61.5% of clients experienced at least one negative effect during treatment, although total number of negative effects was not significantly correlated with client demographic characteristics, lessons completed, working alliance, treatment satisfaction, or symptom outcomes. Among completers, technical difficulties, implementation problems, and negative emotional states were the most commonly reported negative effects, whereas dropout was the most commonly reported negative effect by non-completers. Negative effects that have been identified in previous research, such as symptom deterioration, novel symptoms, and severe adverse events, were not identified in client emails. The high incidence of negative effects in the current study suggests there may be value in systematically monitoring client emails for negative effects throughout treatment as a supplement to retrospective post-treatment reports. This will give therapists the opportunity to intervene as negative effects occur and potentially mitigate any impact they have on treatment outcomes. Future research, both qualitative and quantitative, is needed to gain a more nuanced understanding of negative effects associated with ICBT.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Serious adverse events are unlikely with exercise and CBT.
“Serious adverse events are unlikely with exercise 45 and CBT 46”
From Management of chronic pain associated with temporomandibular disorders: a clinical practice guideline (2023) · cites this paper as [46]
The Discussion reports that clients did not mention severe adverse events, such as suicidality or hospitalization, in emails to their therapists. That supports an absence of reported events in this internet-delivered CBT sample, but it does not establish that serious adverse events are generally unlikely with CBT. The exercise portion is attached to a different citation and is not judged here. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971 · see citing works
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910 · see citing works
The qualitative content analysis process
Elo, S., & Kyngäs, H. (2008) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2007.04569.x · Cited by 22,617 · see citing works
Bunney, P. E., Zink, A. N., Holm, A. A., Billington, C. J., & Kotz, C. M. (2017) · Physiology & Behavior · doi:10.1016/j.physbeh.2017.03.040 · Cited by 2,104 · see citing works
Development and validation of a revised short version of the working alliance inventory
Hatcher, R. L., & Gillaspy, J. A. (2006) · Psychotherapy Research · doi:10.1080/10503300500352500 · Cited by 1,355 · see citing works
Barak, A., Hen, L., Boniel-Nissim, M., & Shapira, N. (2008) · Journal of Technology in Human Services · doi:10.1080/15228830802094429 · Cited by 1,283 · see citing works
The Lancet Psychiatry Commission on psychological treatments research in tomorrow's science
Holmes, E. A., Ghaderi, A., Harmer, C. J., Ramchandani, P. G., Cuijpers, P., Morrison, A. P., Roiser, J. P., Bockting, C. L. H., O'Connor, R. C., Shafran, R., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30513-8 · Cited by 624 · see citing works
Internet-Delivered Psychological Treatments
Andersson, G. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093006 · Cited by 595 · see citing works
Ebert, D. D., Van Daele, T., Nordgreen, T., Karekla, M., Compare, A., Zarbo, C., Brugnera, A., Øverland, S., Trebbi, G., Jensen, K. L., et al. (2018) · European Psychologist · doi:10.1027/1016-9040/a000318 · Cited by 437 · see citing works
Consensus statement on defining and measuring negative effects of internet interventions
Rozental, A., Andersson, G., Boettcher, J., Ebert, D. D., Cuijpers, P., Knaevelsrud, C., Ljótsson, B., Kaldo, V., Titov, N., & Carlbring, P. (2014) · Internet Interventions · doi:10.1016/j.invent.2014.02.001 · Cited by 371 · see citing works
Self-citationICBT in routine care: A descriptive analysis of successful clinics in five countries
Titov, N., Dear, B., Nielssen, O., Staples, L., Hadjistavropoulos, H., Nugent, M., Adlam, K., Nordgreen, T., Bruvik, K. H., Hovland, A., et al. (2018) · Internet Interventions · doi:10.1016/j.invent.2018.07.006 · Cited by 353 · see citing works
Self-citationHow would we know if psychotherapy were harmful?
Dimidjian, S., & Hollon, S. D. (2010) · American Psychologist · doi:10.1037/a0017299 · Cited by 314 · see citing works
Rozental, A., Kottorp, A., Forsström, D., Månsson, K., Boettcher, J., Andersson, G., Furmark, T., & Carlbring, P. (2019) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465819000018 · Cited by 230 · see citing works
Dear, B. F., Staples, L. G., Terides, M. D., Karin, E., Zou, J., Johnston, L., Gandy, M., Fogliati, V. J., Wootton, B. M., McEvoy, P. M., et al. (2015) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2015.09.003 · Cited by 230 · see citing works
Self-citationPatient experience of negative effects of psychological treatment: results of a national survey
Crawford, M. J., Thana, L., Farquharson, L., Palmer, L., Hancock, E., Bassett, P., Clarke, J., & Parry, G. D. (2016) · British Journal of Psychiatry · doi:10.1192/bjp.bp.114.162628 · Cited by 225 · see citing works
Rozental, A., Kottorp, A., Boettcher, J., Andersson, G., & Carlbring, P. (2016) · PLOS ONE · doi:10.1371/journal.pone.0157503 · Cited by 223 · see citing works
Titov, N., Dear, B. F., Staples, L. G., Terides, M. D., Karin, E., Sheehan, J., Johnston, L., Gandy, M., Fogliati, V. J., Wootton, B. M., et al. (2015) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2015.08.002 · Cited by 223 · see citing works
Self-citationRozental, A., Boettcher, J., Andersson, G., Schmidt, B., & Carlbring, P. (2015) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2015.1008033 · Cited by 189 · see citing works
Bendelin, N., Hesser, H., Dahl, J., Carlbring, P., Nelson, K. Z., & Andersson, G. (2011) · BMC Psychiatry · doi:10.1186/1471-244x-11-107 · Cited by 173 · cited 1× · see citing works
“Several studies have interviewed clinical trial participants after treatment to ask about a number of topics including negative effects (Bendelin et al., 2011; Fernández-Álvarez et al., 2017; Olsson Halmetoja et al., 2014).”
BackgroundEbert, D. D., Donkin, L., Andersson, G., Andrews, G., Berger, T., Carlbring, P., Rozenthal, A., Choi, I., Laferton, J. A. C., Johansson, R., et al. (2016) · Psychological Medicine · doi:10.1017/s0033291716001562 · Cited by 154 · see citing works
Self-citationRozental, A., Magnusson, K., Boettcher, J., Andersson, G., & Carlbring, P. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000158 · Cited by 137 · see citing works
Rozental, A., Castonguay, L., Dimidjian, S., Lambert, M., Shafran, R., Andersson, G., & Carlbring, P. (2018) · BJPsych Open · doi:10.1192/bjo.2018.42 · Cited by 124 · see citing works
Hadjistavropoulos, H. D., Nugent, M. M., Alberts, N. M., Staples, L., Dear, B. F., & Titov, N. (2016) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2016.05.006 · Cited by 103 · see citing works
Self-citationClinicians' Perspectives on Negative Effects of Psychological Treatments
Bystedt, S., Rozental, A., Andersson, G., Boettcher, J., & Carlbring, P. (2014) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2014.939593 · Cited by 78 · see citing works
Rozental, A., Andersson, G., & Carlbring, P. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00589 · Cited by 74 · see citing works
Hadjistavropoulos, H. D., Schneider, L. H., Klassen, K., Dear, B. F., & Titov, N. (2018) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2018.1457079 · Cited by 61 · see citing works
Self-citationFernández-Álvarez, J., Díaz-García, A., González-Robles, A., Baños, R., García-Palacios, A., & Botella, C. (2017) · Internet Interventions · doi:10.1016/j.invent.2017.09.001 · Cited by 60 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- Andersson G. Titov N. Advantages and limitations of internet-based interventions for common mental disorders World Psychiatry 13 2014 4 11 24497236 PMC3918007
- Andersson G. Titov N. Dear B.F. Rozental A. Carlbring P. Internet-delivered psychological treatments: from innovation to implementation World Psychiatry 18 2019 20 28 30600624 PMC6313242
- Eubanks C.F. Burckell L.A. Goldfried M.R. Clinical consensus strategies to repair ruptures in the therapeutic alliance J. Psychother. Integr. 28 1 2018 60 76 29805243 PMC5966286
- Hsieh H.-F. Shannon S.E. Three approaches to qualitative content analysis Qual. Health Res. 15 9 2005 1277 1288 16204405
- Parry G.D. Crawford M.J. Duggan C. Iatrogenic harm from psychological therapist - time to move on Br. J. Psychiatry 208 2016 210 212 26932481
- Scott J. Young A.H. Psychotherapies should be assessed for both benefit and harm Br. J. Psychiatry 208 208–209 2016 26932480
- Spitzer R.L. Kroenke K. Williams J.B.W. Lowe B. A brief measure for assessing generalized anxiety disorder Arch. Intern. Med. 166 2006 1092 1097 16717171
Cited by
Sorted by citations
Busse, J. W., Casassus, R., Carrasco-Labra, A., Durham, J., Mock, D., Zakrzewska, J. M., Palmer, C., Samer, C. F., Coen, M., Guevremont, B., et al. (2023) · BMJ · doi:10.1136/bmj-2023-076227 · Cited by 152 · see citing works
“• Serious adverse events are unlikely with exercise 45 and CBT 46 • Long term opioids, NSAIDs, and invasive or irreversible procedures are associated with a small risk of serious, potentially catastrophic, harms”
BackgroundInfluentialThe Safety of Digital Mental Health Interventions: Systematic Review and Recommendations
Taher, R., Hsu, C.-W., Hampshire, C., Fialho, C., Heaysman, C., Stahl, D., Shergill, S., & Yiend, J. (2023) · JMIR Mental Health · doi:10.2196/47433 · Cited by 68 · see citing works
Miller, C. B., Gu, J., Henry, A. L., Davis, M. L., Espie, C. A., Stott, R., Heinz, A. J., Bentley, K. H., Goodwin, G. M., Gorman, B. S., et al. (2021) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2020.101609 · Cited by 40 · see citing works
Leterme, A., Béhal, H., Demarty, A., Barasino, O., Rougegrez, L., Labreuche, J., Duhamel, A., Vaiva, G., & Servant, D. (2020) · Internet Interventions · doi:10.1016/j.invent.2020.100329 · Cited by 18 · see citing works
Duffy, D. J., Richards, D., Hisler, G. C., & Timulák, L. (2024) · Journal of Medical Internet Research · doi:10.2196/47927 · Cited by 34 · see citing works
Beck, C. D., Soucy, J., & Hadjistavropoulos, H. (2020) · Internet Interventions · doi:10.1016/j.invent.2020.100311 · Cited by 10 · see citing works
“Moreover, it would be valuable to explore the impact of MI on patient behaviour during ICBT, for example, the level of participant engagement and completion rates (Gullickson et al., 2019; Soucy et al., 2018a, 2018b; Soucy et al., 2019).”
BackgroundSelf-citationBørtveit, L., Nordgreen, T., & Nordahl-Hansen, A. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1236895 · Cited by 9 · see citing works
“Similar to our findings, to base the delivery of treatment content on presenting text to the patients is commonly reported to be associated with some challenges (Sinha Deb et al., 2018; Gullickson et al., 2019; Mol et al., 2020; Borghouts et al., 2021).”
ResultClients' negative experiences in psychological treatment.
Vybíral, Z., Řiháček, T., Borovička, P., & Ogles, B. (2025) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2025.102175 · Cited by 8 · see citing works
Taher, R., Hall, C. L., Bergin, A. D. G., Gupta, N., Heaysman, C., Jacobsen, P., Kabir, T., Kalnad, N., Keppens, J., Hsu, C.-W., et al. (2024) · Trials · doi:10.1186/s13063-024-08421-1 · Cited by 8 · see citing works
“This framework was informed by three key publications arising from the literature review that discussed a range of classes of negative effects in psychotherapy [18, 32, 33].”
MethodAsplund, M., Lenhard, F., Rück, C., Andersson, E., Grimlund, T., Nilsson, M., Sarachu-Nilsson, M., Sundh, L., & Ivanov, V. (2024) · The Behavior Therapist · doi:10.1016/j.beth.2024.04.006 · Cited by 6 · see citing works
Young, D. K. W., Carlbring, P., Petrus, Y. N. N. G., & Chen, Q.-R. J. (2022) · Research on Social Work Practice · doi:10.1177/10497315221079236 · Cited by 7 · see citing works
Kling, J., Zelihić, D., Williamson, H., & Feragen, K. B. (2022) · Body image · doi:10.1016/j.bodyim.2022.07.012 · Cited by 5 · see citing works
Khan, Z. A., Heggelund, J., Lydersen, S., Kidholm, K., Pedersen, H., Janszky, I., Nordgreen, T., & Halsteinli, V. (2025) · Journal of Medical Internet Research · doi:10.2196/73067 · Cited by 3 · see citing works
“Moreover, real-world effectiveness may be influenced by a range of factors, including organizational differences [20,21], therapists’ attitudes [17,22,23], patients’ perceptions [24,25], and the ease of using the technology [26].”
Bendelin, N. (2023) · Linköping University medical dissertations · doi:10.3384/9789180750288
Technological Interventions for Anxiety Disorders
Baños, R., Vara, M., Mira, A., García-Palacios, A., & Botella, C. (2020) · Reference Module in Neuroscience and Biobehavioral Psychology · doi:10.1016/b978-0-12-818697-8.00008-x · Cited by 2 · see citing works
Fenski, F., Heinrich, M., Schaeuffele, C., Zagorscak, P., Knaevelsrud, C., & Boettcher, J. (2025) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2025.2565674 · Cited by 1 · see citing works
Ingeman, K., Frydendal, D. H., Frostholm, L., Bjerre-Nielsen, E., Wellnitz, K., Onghena, P., Wright, K. D., & Rask, C. (2024) · JMIR Formative Research · doi:10.2196/65396 · Cited by 1 · see citing works
“In general, temporary distress from psychological interventions is common as a result of confronting distressing thoughts and feelings [67-69].”
Fernández, O., Pérez, J. C., Alamo, N., Fernández, S., Franco, P., Coo, S., García, M. I., & Aravena, M. (2024) · Digital Health · doi:10.1177/20552076241261894 · Cited by 1 · see citing works
“30 For example, a linguistic analysis of communication in therapist-assisted, internet-delivered treat-ment, conducted in Canada by Dirkse and cols. reported that clients used words of unpleasant content (i.e., indicative of anxiety and negative emotions), and that their use decreased throughout…”
Gatov, E., Strudwick, G., Wiljer, D., Bhattacharyya, O., & Kurdyak, P. (2026) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2026.2624695
Kince-Laus, I., Sīle, L., Novickis, J., Romanovska, E., Dambiņa, L., & Taube, M. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2025.1657309 · Cited by 1 · see citing works
“Gullickson et al. (36) reported negative emotional effects in 16% of cases, including anxiety, frustration, hopelessness, stress, and discomfort.”
Internet dan Kebahagiaan di Masa Pandemi
Masrura, A., Djuwita, R., & Hudiyana, J. (2022) · Jurnal Psikologi Jurnal Ilmiah Fakultas Psikologi Universitas Yudharta Pasuruan · doi:10.35891/jip.v9i1.2584 · Cited by 1 · see citing works
Fournier, A., Rivest, M.-C., Price, J. A. B., & Hadjistavropoulos, H. D. (2026) · Revue québécoise de psychologie · doi:10.7202/1126758ar
Self-citation
- Authors
- Gullickson, K. M., Hadjistavropoulos, H. D., Dear, B. F., & Titov, N.
- Affiliations
- University of Regina (CA)
- Macquarie University (AU)
- Corresponding author
- Hadjistavropoulos, Heather D.
- Venue
- Internet Interventions
- Publisher
- Elsevier BV
- Published
- 2019-12
- Volume
- 18
- Pages
- 100278
- ISSN
- 2214-7829 (ISSN-L)
- PMC
- PMC6926340
- OpenAlex
- W2971381309
- Cited by
- 26
- References
- 36
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 4 · 2023: 5 · 2022: 4 · 2020: 4
- FWCI
- 2.26
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Internet Interventions (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Saskatchewan Health Research Foundation
- Canadian Institutes of Health Research
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Keywords
- internet-based cognitive behavioral therapy, adverse outcomes, adverse events, patient-therapist communication, treatment discontinuation, technical challenges, treatment engagement, treatment satisfaction, therapeutic relationship, symptom deterioration, online therapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
