The DOI matches its Crossref record; 93 of the 94 references it cites were found in a registry or an index.
Negative Effects of Psychological Treatments: An Exploratory Factor Analysis of the Negative Effects Questionnaire for Monitoring and Reporting Adverse and Unwanted Events
Open accessAbstract
Research conducted during the last decades has provided increasing evidence for the use of psychological treatments for a number of psychiatric disorders and somatic complaints. However, by focusing only on the positive outcomes, less attention has been given to the potential of negative effects. Despite indications of deterioration and other adverse and unwanted events during treatment, little is known about their occurrence and characteristics. Hence, in order to facilitate research of negative effects, a new instrument for monitoring and reporting their incidence and impact was developed using a consensus among researchers, self-reports by patients, and a literature review: the Negative Effects Questionnaire. Participants were recruited via a smartphone-delivered self-help treatment for social anxiety disorder and through the media (N = 653). An exploratory factor analysis was performed, resulting in a six-factor solution with 32 items, accounting for 57.64% of the variance. The derived factors were: symptoms, quality, dependency, stigma, hopelessness, and failure. Items related to unpleasant memories, stress, and anxiety were experienced by more than one-third of the participants. Further, increased or novel symptoms, as well as lack of quality in the treatment and therapeutic relationship rendered the highest self-reported negative impact. In addition, the findings were discussed in relation to prior research and other similar instruments of adverse and unwanted events, giving credence to the items that are included. The instrument is presently available in eleven different languages and can be freely downloaded and used from www.neqscale.com.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“On the one hand, this result fits literature that shows, that psychological interventions, online and offline, can initially evoke negative effects and symptom deteriorations (Ladwig et al., 2014; Rozental et al., 2016).”
From Feasibility of a Software agent providing a brief Intervention for Self-help to Uplift psychological wellbeing (“SISU”). A single-group pretest-posttest trial investigating the potential of SISU to act as therapeutic agent (2021) · cites this paper as Rozental et al., 2016
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Adverse events and harms during the RCT are measured with the Negative Experiences Questionnaire (NEQ), [61]”
From Transdiagnostic and tailored internet intervention to improve mental health among university students: Research protocol for a randomized controlled trial (2024) · cites this paper as [61]
The cited source says what the claim attributes to it.
In the citing paper
“The internal consistency for the full instrument is excellent (Cronbach's α = 0.95), and is scored on several dimensions; 1) “Did you experience this?” (yes/no), 2) “If yes – here is how negatively it affected me”, from 0 (“Not at all”) to 4 (“Extremely”), and 3) “Probably caused by” (“The treatment I received” or “Other circumstances”).”
From A study protocol of a randomized controlled study of internet-based cognitive behavioral therapy for adult attention deficit hyperactivity disorder (2023) · cites this paper as Rozental et al., 2016
The cited source says what the claim attributes to it.
In the citing paper
“Internal Consistency of the NEQ is reported to be α = 0.95 (90).”
The cited source says what the claim attributes to it.
In the citing paper
“Others have also implied that non-response, dropout, and interpersonal difficulties may be perceived as negative effects (Dimidjian 2010; Rozental 2016).”
From Psychotherapeutic treatments for depression in older adults (2024) · cites this paper as Rozental 2016
In the citing paper
“and trials have already begun (Miloff et al. 2019) to include and report results from broader measures of negative effects used elsewhere in psychotherapy research (Rozental et al. 2016).”
From Better, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy (2021) · cites this paper as Rozental et al. 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ryan, R. M., & Deci, E. L. (2000) · American Psychologist · doi:10.1037//0003-066x.55.1.68 · Cited by 54,599
An inventory for measuring depression
BECK, A. T. (1961) · Archives of General Psychiatry · doi:10.1001/archpsyc.1961.01710120031004 · Cited by 38,032
(1998) · The Journal of Clinical Psychiatry · Cited by 19,139
The Scree Test For The Number Of Factors
Cattell, R. B. (1966) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr0102_10 · Cited by 13,554
The application of electronic computers to factor analysis
Kaiser, H. F. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000116 · Cited by 12,459
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195
(1998) · PubMed · Cited by 10,216
Evaluating the use of exploratory factor analysis in psychological research.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.3.272 · Cited by 9,587
A Rationale and Test for the Number of Factors in Factor Analysis
Horn, J. L. (1965) · Psychometrika · doi:10.1007/bf02289447 · Cited by 8,965
Selective Cortical Undercutting As a Means of Modifying and Studying Frontal Lobe Function in Man
Scoville, W. B. (1949) · Journal of neurosurgery · doi:10.3171/jns.1949.6.1.0065 · Cited by 7,374
Flagged first, then by citations
1 of the 94 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- I Ladwig, 2014, What are the Risks and Side Effects to Psychotherapy?—Development of an Inventory for the Assessment of Negative Effects of Psychotherapy (INEP), Verhaltenstherapie, 24, 252
- S Moritz, 2015, No pain, no gain? Adverse effects of psychotherapy in obsessive-compulsive disorder and its relationship to treatment gains, J Obsess-Compuls Rel, 5, 61
Cited by
Sorted by citations
The Lancet Psychiatry Commission on psychological treatments research in tomorrow's science
Holmes, E. A., Ghaderi, A., Harmer, C. J., Ramchandani, P. G., Cuijpers, P., Morrison, A. P., Roiser, J. P., Bockting, C. L. H., O'Connor, R. C., Shafran, R., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30513-8 · Cited by 624
Rozental, A., Kottorp, A., Forsström, D., Månsson, K., Boettcher, J., Andersson, G., Furmark, T., & Carlbring, P. (2019) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465819000018 · Cited by 230
“In relation to the factors obtained by Rozental et al. (2016), the first component is associated with symptoms, while the other is linked to four separate factors.”
BackgroundMethodResultInfluentialSelf-citationThe Return of the Repressed: The Persistent and Problematic Claims of Long-Forgotten Trauma
Otgaar, H., Howe, M. L., Patihis, L., Merckelbach, H. L. G. J., Lynn, S. J., Lilienfeld, S. O., & Loftus, E. F. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619862306 · Cited by 211
“, & Rubin, D. C. (2003). Confidence, not consistency, characterizes flashbulb memories.”
Lindner, P., Miloff, A., Hamilton, W., Reuterskiöld, L., Andersson, G., Powers, M. B., & Carlbring, P. (2017) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2017.1280843 · Cited by 187
“If VRET is to be considered for integration into regular health care settings alongside other psychological treatments, RCT quality needs to be improved, including the systematic reporting of negative effects (Rozental, Kottorp, Boettcher, Andersson, & Carlbring, 2016).”
BackgroundSelf-citationLindner, P., Miloff, A., Fagernäs, S., Andersen, J., Sigeman, M., Andersson, G., Furmark, T., & Carlbring, P. (2019) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.07.003 · Cited by 175
Self-citationMiloff, A., Lindner, P., Dafgård, P., Deak, S., Garke, M., Hamilton, W., Heinsoo, J., Kristoffersson, G., Rafi, J., Sindemark, K., et al. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.04.004 · Cited by 155
Self-citationRozental, A., Magnusson, K., Boettcher, J., Andersson, G., & Carlbring, P. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000158 · Cited by 137
Self-citationRozental, A., Castonguay, L., Dimidjian, S., Lambert, M., Shafran, R., Andersson, G., & Carlbring, P. (2018) · BJPsych Open · doi:10.1192/bjo.2018.42 · Cited by 124
Self-citationBetter, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy
Lindner, P. (2021) · International Journal of Cognitive Therapy · doi:10.1007/s41811-020-00090-7 · Cited by 120
Negative experiences in psychotherapy from clients’ perspective: A qualitative meta-analysis
Vybíral, Z., Ogles, B. M., Řiháček, T., Urbancová, B., & Gocieková, V. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2226813 · Cited by 116
“…but there are also quantitative studies that use standardized measures of negative effects, such as the Negative Effects Questionnaire (NEQ; Rozental et al., 2016) and the Inventory for the Assessment of Negative Effects of Psychotherapy (INEP; Ladwig et al., 2014), as well as…”
BackgroundHerzog, P., Lauff, S., Rief, W., & Brakemeier, E. (2019) · Brain and Behavior · doi:10.1002/brb3.1447 · Cited by 95
“…therapy Beneficial therapy Idealization of therapist 5 points (strongly disagree–strongly agree) Ireland English Negative Effects Questionnaire (NEQ) Rozental et al. (2016) 32 Symptoms Quality Dependency Stigma Hopelessness Failure 2 points (yes–no) 5 points (perceived burden) 2 points…”
BackgroundInfluentialStrategies for de-implementation of low-value care—a scoping review
Ingvarsson, S., Hasson, H., Schwarz, U. V. T., Nilsén, P., Powell, B. J., Lindberg, C., & Augustsson, H. (2022) · Implementation Science · doi:10.1186/s13012-022-01247-y · Cited by 96
“For instance, the problem of overprescribing antibiotics was defined in parallel with the development of the medication [106], whereas the side effects within psychotherapy research have only been investigated in recent years [107].”
BackgroundDeterioration rates in Virtual Reality Therapy: An individual patient data level meta-analysis
Fernández-Álvarez, J., Rozental, A., Carlbring, P., Colombo, D., Riva, G., Anderson, P. L., Baños, R. M., Benbow, A. A., Bouchard, S., Bretón-López, J. M., et al. (2019) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.06.005 · Cited by 79
Self-citationLindner, P., Dagöö, J., Hamilton, W., Miloff, A., Andersson, G., Schill, A., & Carlbring, P. (2020) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2020.1795240 · Cited by 84
“The Negative Effects Questionnaire (Rozental et al., 2016) was administered at the final post-assessment to screen for 32 possible negative effects of psychological treatment (Rozental et al., 2014).”
MethodSelf-citationRozental, A., Andersson, G., & Carlbring, P. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00589 · Cited by 74
“In addition, there is also an ongoing debate on how to define and measure adverse and unwanted events occurring in treatment, with different taxonomies having been proposed, which makes it difficult to systematically assess and report such incidents across studies (Rozental et al., 2016).”
BackgroundDepression in primary care: part 2—management
Ramanuj, P., Ferenchick, E. K., & Pincus, H. A. (2019) · BMJ · doi:10.1136/bmj.l835 · Cited by 68
Hoffmann, D., Rask, C. U., Hedman-Lagerlöf, E., Jensen, J. S., & Frostholm, L. (2021) · Psychological Medicine · doi:10.1017/s0033291720001312 · Cited by 63
“Regarding negative effects during iACT treatment, the most frequently reported factor was ’symptoms’ among the six factors encompassing symptoms, hopelessness, failure, stigma, dependency, and quality (Rozental et al., 2016).”
BackgroundPapaioannou, D. E., Cooper, C. L., Mooney, C., Glover, R., & Coates, E. J. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.03.002 · Cited by 65
Scheenen, M. E., Visser-Keizer, A. C., de Koning, M. E., van der Horn, H. J., van de Sande, P., van Kessel, M. E., van der Naalt, J., & Spikman, J. M. (2017) · Journal of Neurotrauma · doi:10.1089/neu.2016.4885 · Cited by 51
“11 It is possible that our CBTi sessions in groups unintentionally had a medicalizing and reinforcing effect on the perceived complaints, 43 whereas the individual TC sessions were more low key and more suited for giving information and reassurance.”
The evidence base of Acceptance and Commitment Therapy (ACT) in psychosis: A systematic review
Wakefield, S., Roebuck, S., & Boyden, P. (2018) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2018.07.001 · Cited by 49
Maroti, D., Lumley, M. A., Schubiner, H., Lilliengren, P., Bileviciute-Ljungar, I., Ljótsson, B., & Johansson, R. (2022) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2022.111068 · Cited by 40
Palermo, T. M., Slack, K., Loren, D., Eccleston, C., & Jamison, R. N. (2020) · Pain · doi:10.1097/j.pain.0000000000001772 · Cited by 42
Spanhel, K., Hovestadt, E., Lehr, D., Spiegelhalder, K., Baumeister, H., Bengel, J., & Sander, L. B. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.832196 · Cited by 37
“Negative Effects of the Intervention At T3, negative effects of the intervention were assessed within the IG with the Negative Effects Questionnaire, self-adapted to digital interventions (89, 90) [NEQ; 18 items; 3 scales per item: I.”
MethodHonkalampi, K. M., Urhonen, H.-R., & Virtanen, M. (2024) · Psychotherapy Research · doi:10.1080/10503307.2024.2301972 · Cited by 40
“Rozental et al. (2016) developed the Negative Effects Questionnaire (NEQ).”
Afsharnejad, B., Falkmer, M., Black, M. H., Alach, T., Lenhard, F., Fridell, A., Coco, C., Milne, K., Bölte, S., & Girdler, S. J. (2021) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-021-01814-6 · Cited by 36
“The adolescents’ self-report data from the adverse event measure (NEQ) demonstrated no significant changes over time between the groups (p = .44), with 28 (44”
BackgroundMethodInfluentialEye movement desensitization and reprocessing (EMDR) practitioners’ beliefs about memory.
Houben, S. T. L., Otgaar, H., Roelofs, J., Wessel, I., Patihis, L., & Merckelbach, H. L. G. J. (2019) · Psychology of Consciousness Theory Research and Practice · doi:10.1037/cns0000211 · Cited by 33
Bergin, A., Valentine, A. Z., Rennick‐Egglestone, S., Slade, M., Hollis, C. P., & Hall, C. L. (2022) · JMIR Mental Health · doi:10.2196/42501 · Cited by 34
“Some work has already started on the specific checklists for psychological interventions [50]; however, to the best of our knowledge, this has not been done digitally or online.”
Baumeister, H., Bauereiss, N., Zarski, A.-C., Braun, L., Buntrock, C., Hoherz, C., Idrees, A. R., Kraft, R., Meyer, P., Nguyen, T. B. D., et al. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.660534 · Cited by 30
“Additionally, possible negative effects of the psychotherapies are assessed by means of the 20-item version of the Negative Effects Questionnaire [NEQ; 20 items; (90, 91)] which measures the frequency (score range: 0–20) and impact (score range: 0–80) of several different negative effects during the treatment period.”
MethodGerke, L., Meyrose, A.-K., Ladwig, I., Rief, W., & Nestoriuc, Y. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.02144 · Cited by 33
Maroti, D., Ek, J., Widlund, R.-M., Schubiner, H. H., Lumley, M. A., Lilliengren, P., Bilevičiūtė-Ljungar, I., Ljótsson, B., & Johansson, R. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.620359 · Cited by 32
“Feasibility Measures The 32-item Negative Effect Questionnaire [NEQ; (47)] assessed negative effects following treatment.”
Adverse effects of meditation: A review of observational, experimental and case studies
Lambert, D., van den Berg, N. H., & Mendrek, A. (2023) · Current Psychology · doi:10.1007/s12144-021-01503-2 · Cited by 31
Fritz, J. M., Rhon, D. I., Teyhen, D. S., Kean, J. T., Vanneman, M. E., Garland, E. L., Lee, I. E., Thorp, R. E., & Greene, T. (2020) · Pain Medicine · doi:10.1093/pm/pnaa338 · Cited by 31
Bendig, E., Erb, B., Meißner, E., Bauereiß, N., & Baumeister, H. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100377 · Cited by 24
“On the one hand, this result fits literature that shows, that psychological interventions, online and offline, can initially evoke negative effects and symptom deteriorations (Ladwig et al., 2014; Rozental et al., 2016).”
Resultvan de Leur, J. C., Buhrman, M., Åhs, F., Rozental, A., & Jansen, G. B. (2020) · BMC Psychiatry · doi:10.1186/s12888-020-02907-3 · Cited by 30
Self-citationContextualizing the negative effects of psychotherapy
Rosendahl, J., Klatte, R., Frenzl, D., Castonguay, L. G., Vybíral, Z., & Strauss, B. M. (2025) · Nature Reviews Psychology · doi:10.1038/s44159-025-00472-8 · Cited by 33
“On a macro level, culture and the societal views of mental illness and psychotherapy might contribute to negative effects 26,45 , such as stigmatization and self-stigmatization, which can deter individuals from seeking help or lead to internalized shame about undergoing psychotherapy 34,46,47 .”
InfluentialNegative Effekte von Psychotherapie: Häufigkeit und Korrelate in einer klinischen Stichprobe
Abeling, B., Müller, A., Stephan, M., Pollmann, I., & de Zwaan, M. (2017) · PPmP - Psychotherapie · Psychosomatik · Medizinische Psychologie · doi:10.1055/s-0043-117604 · Cited by 27
“[10] Rozental A, Kottorp A, Boettcher J et al.”
BackgroundTauber, N. M., O’Toole, M. S., Jensen, A. B., Butow, P. N., Thewes, B., Elkjær, E., Knutzen, S., von Heymann, A., Johansen, C., & Zachariae, R. (2023) · Psycho-Oncology · doi:10.1002/pon.6193 · Cited by 26
“Possible adverse effects of the intervention were assessed with the Hopelessness and Failure Subscale of the Negative Effects Questionnaire (NEQ).(33) Working alliance and group cohesion were assessed after each session with the Working Alliance Inventory Revised Short Form(34) and the Therapeutic Factors Inventory Co…”
MethodKallestad, H., Vedaa, Ø., Scott, J. L., Morken, G., Pallesen, S., Harvey, A. G., Gehrman, P., Thorndike, F. P., Ritterband, L. M., Stiles, T. C., et al. (2018) · BMJ Open · doi:10.1136/bmjopen-2018-025152 · Cited by 25
Houben, S. T. L., Otgaar, H., Roelofs, J., Smeets, T., & Merckelbach, H. L. G. J. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103546 · Cited by 27
Gullickson, K. M., Hadjistavropoulos, H. D., Dear, B. F., & Titov, N. (2019) · Internet Interventions · doi:10.1016/j.invent.2019.100278 · Cited by 26
Wiklund, T., Molander, P., Lindner, P., Andersson, G., Gerdle, B. U. C., & Dragioti, E. (2022) · Journal of Medical Internet Research · doi:10.2196/29258 · Cited by 25
Self-citationAsplund, M., Lenhard, F., Andersson, E., & Ivanov, V. Z. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100573 · Cited by 25
Klein, J. P., Hauer, A., Berger, T., Faßbinder, E., Mayer, J., Borgwardt, S., Wellhöfer, B., Schweiger, U., & Jacob, G. A. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047771 · Cited by 24
Otgaar, H., Houben, S. T. L., Rassin, E., & Merckelbach, H. L. G. J. (2021) · Memory · doi:10.1080/09658211.2021.1966043 · Cited by 22
Munsch, S., Wyssen, A., Vanhulst, P., Lalanne, D., Steinemann, S. T., & Tuch, A. N. (2019) · Trials · doi:10.1186/s13063-019-3192-z · Cited by 21
Mechler, J., Lindqvist, K., Magnusson, K., Ringström, A., Krafman, J. D., Alvinzi, P., Kassius, L., Sowa, J., Andersson, G., & Carlbring, P. (2024) · npj Mental Health Research · doi:10.1038/s44184-024-00063-0 · Cited by 22
“Still,thisshouldbe furtherinvestigatedusingbroadersetsofquestionssuchastheNegative EffectsQuestionnaire 61 .”
Self-citationEvaluating the Unified Protocol in the Treatment of Dissociative Identify Disorder
Mohajerin, B., Lynn, S. J., Bakhtiyari, M., & Dolatshah, B. (2019) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2019.07.012 · Cited by 21
Blampied, M., Bell, C., Gilbert, C., & Rucklidge, J. J. (2020) · Expert Review of Neurotherapeutics · doi:10.1080/14737175.2020.1740595 · Cited by 23
Skolasky, R. L., Wegener, S. T., Aaron, R. V., Ephraim, P. L., Brennan, G. P., Greene, T., Lane, E., Minick, K. I., Hanley, A. W., Garland, E. L., et al. (2020) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-020-03324-z · Cited by 21
Predictors and impact of psychotherapy side effects in young adults
Lorenz, T. K. (2020) · Counselling and Psychotherapy Research · doi:10.1002/capr.12356 · Cited by 19
Niemeyer, H., Knaevelsrud, C., Schumacher, S., Engel, S., Kuester, A., Burchert, S., Muschalla, B., Weiss, D., Spies, J., Rau, H., et al. (2020) · BMC Psychiatry · doi:10.1186/s12888-020-02595-z · Cited by 22
“Rozental A, Kottorp A, Boettcher J, Andersson G, Carlbring P. Negative Effects of Psychological Treatments: An Exploratory Factor Analysis of the Negative Effects Questionnaire for Monitoring and Reporting Adverse and Unwanted Events.”
MethodLodewyk, K., Bagnell, A. L., Courtney, D., & Newton, A. S. (2023) · Child and Adolescent Mental Health · doi:10.1111/camh.12661 · Cited by 20
Verkooyen, R., Broers, N. J., & Dandachi-FitzGerald, B. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1378456 · Cited by 18
Moyes, E., Nutman, G., & Mirman, J. H. (2022) · Journal of Child & Adolescent Trauma · doi:10.1007/s40653-022-00454-5 · Cited by 19
Schubert, K., Lohse, J., Kalder, M., Ziller, V., & Weise, C. (2022) · Trials · doi:10.1186/s13063-022-06204-0 · Cited by 17
Vernmark, K., Buhrman, M., Carlbring, P., Hedman‐Lagerlöf, E., Kaldo, V., & Andersson, G. (2024) · Digital Health · doi:10.1177/20552076241287059 · Cited by 19
“140 Another excluded area in this review was internet-based psychodynamic treatments, which was fi rst studied by Swedish researchers 141,142 and has continued to interest other research groups in Sweden.”
BackgroundSelf-citationDietrichkeit, M., Hagemann-Goebel, M., Nestoriuc, Y., Moritz, S., & Jelinek, L. (2021) · Scientific Reports · doi:10.1038/s41598-021-87198-8 · Cited by 16
Dissociative identity disorder: validity and use in the criminal justice system
Paris, J. (2019) · BJPsych Advances · doi:10.1192/bja.2019.12 · Cited by 17
Kocur, M., Dechant, M., Wolff, C., Nothdurfter, C., Wetter, T. C., Rupprecht, R., & Shiban, Y. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.608997 · Cited by 15
Dahlin, M., Johansson, A., Romare, H., Carlbring, P., & Andersson, G. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100531 · Cited by 15
“The exception was that we did not administer the alcohol use disorder identification test (AUDIT) (Saunders et al., 1993), added the Negative Effects Questionnaire (NEQ) (Rozental et al., 2016) and questions about experiences of the treatment.”
Self-citationSundell, K., & Olsson, T. (2017) · Oxford Bibliographies in Social Work · doi:10.1093/obo/9780195389678-0254 · Cited by 21
Dandachi‐FitzGerald, B., Houben, S. T. L., Broers, N. J., & Merckelbach, H. L. G. J. (2024) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.3014 · Cited by 17
Jensen, K. H. R., Dam, V. H., Ganz, M., Fisher, P. M., Ip, C.-T., Sankar, A., Marstrand-Joergensen, M. R., Ozenne, B., Osler, M., Penninx, B. W. J. H., et al. (2023) · BMC Psychiatry · doi:10.1186/s12888-023-04618-x · Cited by 16
“, new symptoms, dependency, stigma, hopelessness, and the experienced quality of treatment [74, 75].”
BackgroundRubel, J. A., Väth, T., Hanraths, S., Pruessner, L., Timm, C., Hartmann, S. F., Barnow, S., & Lalk, C. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100716 · Cited by 16
“Adverse treatment effects were collected with the Negative Effects Questionnaire (NEQ; Rozental et al., 2016) as an exploratory measure.”
BackgroundMethodInfluentialBech, P., Lindberg, L., & Møeller, S. B. (2018) · Nordic Journal of Psychiatry · doi:10.1080/08039488.2018.1463393 · Cited by 15
“Clinically significant unwanted side effects are important to be measured in much more intense forms of psychological treatments than we have used [19].”
BackgroundAfsharnejad, B., Falkmer, M., Black, M. H., Alach, T., Lenhard, F., Fridell, A., Coco, C., Milne, K., Chen, N. T. M., Bölte, S., et al. (2020) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-020-04477-5 · Cited by 19
Mathiesen, A. S., Zoffmann, V., Skytte, T. B., Jakobsen, J. C., Gluud, C. N., Lindschou, J., Rasmussen, B., Marqvorsen, E. H. S., Thomsen, T., & Rothmann, M. J. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047037 · Cited by 13
Negative effects during multicomponent group-based treatment: A multisite study
Pourová, M., Řiháček, T., Chvála, Ľ., Vybíral, Z., & Boehnke, J. R. (2022) · Psychotherapy Research · doi:10.1080/10503307.2022.2095237 · Cited by 14
“Nevertheless, they were lower than in other studies (see, e.g., Moreira et al., 2015, for the ECR-RS and Rozental et al., 2016, for the NEQ).”
BackgroundMethodResultInfluentialSpada, F., Caruso, R., Maria, M. D., Karma, E., Oseku, A., Pata, X., Prendi, E., Rocco, G., Notarnicola, I., & Stievano, A. (2022) · Healthcare · doi:10.3390/healthcare10091698 · Cited by 14
“Items that supported the highest reliability of the scale for determining the correlation matrix for the EFA were retained, also avoiding the risk of multicollinearity [35].”
BackgroundPatient experience of supported computerized CBT in an inner-city IAPT service: a qualitative study
Perera-Delcourt, R. P., & Sharkey, G. (2019) · The Cognitive Behaviour Therapist · doi:10.1017/s1754470x18000284 · Cited by 14
“These negative effects can include worsening symptoms, dependency, stigma, hopelessness, a sense of failure, unpleasant memories and anxiety (Rozental et al. , 2016).”
BackgroundOlsen, M. H., Morthorst, B. R., Pagsberg, A. K., Heinrichsen, M., Møhl, B., Rubæk, L., Bjureberg, J., Simonsson, O., Lindschou, J., Gluud, C. N., et al. (2021) · Trials · doi:10.1186/s13063-021-05406-2 · Cited by 13
Watson, P., Leroux, E. J., Chowdhury, M., Mehra, K., Henderson, J. L., Szatmári, P., & Hawke, L. D. (2022) · Journal of Child and Family Studies · doi:10.1007/s10826-022-02431-w · Cited by 12
“The authors of the NEQ also conducted a factor analysis of its items, identifying six subscales of possible negative experiences associated with psychotherapy treatment (symptoms, quality, dependency, stigma, hopelessness, and failure (Rozental et al., 2016).”
BackgroundMethodIs psychotherapy for functional somatic syndromes harmful? A mixed methods study on negative effects
Holsting, A. F., Pedersen, H. F., Rask, M. T., Frostholm, L., & Schröder, A. (2017) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2017.05.010 · Cited by 13
Gaps in measures of adverse outcomes relating to psychological interventions
McIntosh, L. G., McMurran, M., Taylor, P. J., & Thomson, L. D. G. (2019) · Criminal Behaviour and Mental Health · doi:10.1002/cbm.2100 · Cited by 12
Morthorst, B. R., Olsen, M. H., Jakobsen, J. C., Lindschou, J., Gluud, C. N., Heinrichsen, M., Møhl, B., Rubæk, L., Ojala, O., Hellner, C., et al. (2022) · JCPP Advances · doi:10.1002/jcv2.12115 · Cited by 11
“…2009), parents' ability to cope with children's negative emotions rated by adolescents assessed with The Coping with Children's Negative Emotions Scale (CCNES‐APP) (Fabes et al., 2002), and finally adverse events of therapy assessed with Negative Effects Questionnaire (NEQ) (Rozental et al., 2016).”
MethodKalde, J., Atik, E., Stricker, J., Schückes, M., Neudeck, P., Pittig, A., & Pietrowsky, R. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2277866 · Cited by 14
“The Negative Effects Questionnaire (NEQ; Rozental et al., 2016) captures adverse events (negative effects) in psychotherapeutic treatment in three steps.”
Lindgren, A., & Rozental, A. (2021) · Ethics & Behavior · doi:10.1080/10508422.2021.1948855 · Cited by 12
“For example, the Negative Effects Questionnaire can aid patients in reporting novel symptoms or when they have no confidence in their healthcare provider (Rozental et al., 2016).”
BackgroundSelf-citationBerman, A. H., Topooco, N., Lindfors, P., Bendtsen, M., Lindner, P., Molander, O., Kraepelien, M., Sundström, C., Talebizadeh, N., Engström, K., et al. (2024) · Trials · doi:10.1186/s13063-024-07986-1 · Cited by 9
“Adverse events and harms during the RCT are measured with the Negative Experiences Questionnaire (NEQ), [61], and trial safety procedures include monitoring of symptom levels.”
MethodSelf-citationHooi, L. Y., Chen, P. L., Tan, K. W., de Vries, M., & Wong, H. K. (2025) · Scientific Reports · doi:10.1038/s41598-025-23727-z · Cited by 17
“The 20-item Negative Effects Questionnaire (NEQ) was used to measure the adverse effects of the intervention 126,127 .”
Harrison, G. J., & Pooe, D. R. I. (2022) · Acta Commercii · doi:10.4102/ac.v22i1.989 · Cited by 11
“Correlations between items should not be so high (above 0.8) as to present the problem of multicollinearity or so low (< 0.2) as to indicate that the items do not correlate sufficiently to group under the same construct (Rozental et al. 2016; Watson 2017).”
BackgroundMethodInfluentialWallsten, D., Norell‐Clarke, A., Anniko, M. K., Eriksson, O., Lamourín, V., Halldin, I., Kindbom, T., Hesser, H., Watkins, E. R., & Tillfors, M. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1196945 · Cited by 14
Side Effects in Psychodynamic and Cognitive Behavior Therapy
Balder, T., Linden, M., & Rose, M. (2024) · Journal of Contemporary Psychotherapy · doi:10.1007/s10879-023-09615-5 · Cited by 12
“Alternatively, there are more generalized lists that offer limited coverage and cannot encompass the entirety of potential side effects (e.g., Parker et al., 2013; Rozental et al., 2016).”
Shared reading as an intervention to improve health and well-being in adults: a scoping review
Järvholm, K., Ohlsson, A., Bernhardsson, K., Gustafsson, A. W., Johnsson, P., Malmström, M., Wistrand, J., & Forslid, T. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1484839 · Cited by 11
Klein, J. P., Hauer, A., Berger, T., Faßbinder, E., Schweiger, U., & Jacob, G. A. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00439 · Cited by 11
Wahlund, T., Jolstedt, M., Andersson, E., Vigerland, S., Perrin, S. G., Öst, L., Högström, J., & Serlachius, E. (2020) · mHealth · doi:10.21037/mhealth.2019.09.10 · Cited by 12
“Occurrence of adverse events was assessed using a 21-item version of the Negative Effects Questionnaire, adapted for adolescents and parents (NEQ-C/P) (22).”
MethodZeevenhooven, J., Timp, M. L., Singendonk, M. M. J., Benninga, M. A., & Tabbers, M. M. (2019) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2019.04.048 · Cited by 11
KONTAKT© for Australian adolescents on the autism spectrum: protocol of a randomized control trial
Afsharnejad, B., Falkmer, M., Black, M. H., Alach, T., Lenhard, F., Fridell, A., Coco, C., Milne, K., Chen, N. T. M., Bölte, S., et al. (2019) · Trials · doi:10.1186/s13063-019-3721-9 · Cited by 13
School-based social skills group training (SKOLKONTAKT™): a pilot randomized controlled trial
Fridell, A., Coco, C., Borg, A., & Bölte, S. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1128288 · Cited by 10
Taher, R., Bhanushali, P., Allan, S., Álvarez‐Jiménez, M., Bolton, H., Dennison, L., Wallace, B. E., Hadjistavropoulos, H. D., Hall, C. L., Hardy, A., et al. (2024) · BJPsych Open · doi:10.1192/bjo.2024.713 · Cited by 9
“10 Additional data collection to support this judgement could include onset, previous similar instances, and participant ’ s and clinician ’ s opinions on relatedness.”
Schulte, C. E., Sachser, C., Rosner, R., Ebert, D. D., & Zarski, A. (2024) · European Journal of Psychotraumatology · doi:10.1080/20008066.2024.2364469 · Cited by 9
“The Negative Effects Questionnaire (NEQ, Rozental et al., 2016) was used to assess the frequency (score range: 0–20), impact (score range: 0–80; α = .68), and attribution (to treatment or other circum-stances) of negative effects during the treatment period (T3 and T4).”
MethodSaur, S., Weisel, K. K., Lang, C., Fuhrmann, L. M., Steins‐Loeber, S., Enewoldsen, N., Reichl, D., Zink, M., Jakobi, F., Rudolph, M., et al. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100517 · Cited by 11
Distance supervision in the Aiglé Foundation's Latin American Psychotherapy Training Program
Fernández‐Álvarez, H., Grazioso, M. D. P., & Kirszman, D. (2018) · Journal of Clinical Psychology · doi:10.1002/jclp.22721 · Cited by 9
Rautio, D., Andrén, P., Gumpert, M., Jolstedt, M., Jassi, A. D., Krebs, G., Jansson‐Fröjmark, M., Lundgren, T., Serlachius, E., Mataix‐Cols, D., et al. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100688 · Cited by 9
“The modified version of the Negative Effects Questionnaire (Rozental et al., 2016) was filled in by the parents, with input from the adolescent, at mid-treatment, post-treatment, and at the 3-month follow-up.”
MethodMorthorst, B. R., Rubæk, L., Lindschou, J., Jakobsen, J. C., Gluud, C. N., Bjureberg, J., Hellner, C., Møhl, B., & Pagsberg, A. K. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00785-4 · Cited by 11
Beiner, E., Baumeister, D., Buhai, D.-V., Löffler, M., Löffler, A., Schick, A., Ader, L., Eich, W., Sirazitdinov, A., Malone, C., et al. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01199-6 · Cited by 9
“Treatment satisfaction, compliance and acceptability i. Negative treatment effects: Unwanted and negative effects of the treatment will be assessed via the Negative Effects Questionnaire (NEQ) [41] supplemented by the subscales for negative effects on working place, partnership and family/peers of the Inventory for As…”
MethodAxelsson, E., & Hedman‐Lagerlöf, E. (2023) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2023.2250915 · Cited by 9
“We aimed to advance knowledge in this area by conducting two studies: First, a systematic review and meta-analysis of unwanted effects in RCTs of CBT for pathological health anxiety.”
BackgroundMethodInfluentialShahidi, S., Zargar, F., Khajelangi, H. A., & Tarrahi, M. J. (2023) · PubMed Central · doi:10.30476/ijcbnm.2023.97271.2172 · Cited by 8
Geurtzen, N., Keijsers, G. P. J., Karremans, J. C., & Hutschemaekers, G. J. M. (2018) · Journal of Psychotherapy Integration · doi:10.1037/int0000150 · Cited by 8
Melvin, G. A., Finnin, L., Taffe, J. R., Dudley, A. L., Klimkeit, E., Gordon, M. S., & Tonge, B. J. (2019) · Clinical Child Psychology and Psychiatry · doi:10.1177/1359104518822681 · Cited by 9
Strålin, E. E., Thorell, L. B., Szybek, K., Lundgren, T., Bölte, S., & Bohman, B. (2022) · Nordic Psychology · doi:10.1080/19012276.2021.2020683 · Cited by 8
“Finally, adverse events were assessed using the Negative Effects of Psychological Treatments Questionnaire (NEQ; Rozental et al., 2016).”
Method
Metadata
- Authors
- Rozental, A., Kottorp, A., Boettcher, J., Andersson, G., & Carlbring, P.
- Affiliations
- Stockholm University (SE)
- Karolinska Institutet (SE)
- Freie Universität Berlin (DE)
- Linköping University (SE)
- Corresponding author
- Rozental, Alexander
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-06-22
- Volume
- 11
- Issue
- 6
- Pages
- e0157503
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4917117
- OpenAlex
- W2472395746
- Cited by
- 223
- References
- 94
- Citations by year
- 2026: 23 · 2025: 39 · 2024: 23 · 2023: 27 · 2022: 31 · 2021: 20 · 2020: 16 · 2019: 25 · 2018: 12 · 2017: 6 · 2016: 1
- FWCI
- 19.61
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DiVA (Linkoping University) (submittedVersion, other-oa)
- Zürcher Hochschule für Angewandte Wissenschaften digital collection (Zurich University of Applied Sciences) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Refubium (Universitätsbibliothek der Freien Universität Berlin) (submittedVersion, cc-by)
- ZHAW Digital Collection (cc-by)
- Funders
- FORTE (FORTE 2013-1107)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- adverse events, unwanted events, self-report instrument development, social anxiety disorder, symptom deterioration, treatment quality, therapeutic relationship, stigma, hopelessness, factor structure
- MeSH terms
- Adult, Mental Disorders, Demography, Factor Analysis, Statistical, Female, Humans, Male, Surveys and Questionnaires
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
