The DOI matches its Crossref record; 93 of the 94 references it cites were found in a registry or an index.
Negative Effects of Psychological Treatments: An Exploratory Factor Analysis of the Negative Effects Questionnaire for Monitoring and Reporting Adverse and Unwanted Events
Open accessAbstract
Research conducted during the last decades has provided increasing evidence for the use of psychological treatments for a number of psychiatric disorders and somatic complaints. However, by focusing only on the positive outcomes, less attention has been given to the potential of negative effects. Despite indications of deterioration and other adverse and unwanted events during treatment, little is known about their occurrence and characteristics. Hence, in order to facilitate research of negative effects, a new instrument for monitoring and reporting their incidence and impact was developed using a consensus among researchers, self-reports by patients, and a literature review: the Negative Effects Questionnaire. Participants were recruited via a smartphone-delivered self-help treatment for social anxiety disorder and through the media (N = 653). An exploratory factor analysis was performed, resulting in a six-factor solution with 32 items, accounting for 57.64% of the variance. The derived factors were: symptoms, quality, dependency, stigma, hopelessness, and failure. Items related to unpleasant memories, stress, and anxiety were experienced by more than one-third of the participants. Further, increased or novel symptoms, as well as lack of quality in the treatment and therapeutic relationship rendered the highest self-reported negative impact. In addition, the findings were discussed in relation to prior research and other similar instruments of adverse and unwanted events, giving credence to the items that are included. The instrument is presently available in eleven different languages and can be freely downloaded and used from www.neqscale.com.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“On the one hand, this result fits literature that shows, that psychological interventions, online and offline, can initially evoke negative effects and symptom deteriorations (Ladwig et al., 2014; Rozental et al., 2016).”
From Feasibility of a Software agent providing a brief Intervention for Self-help to Uplift psychological wellbeing (“SISU”). A single-group pretest-posttest trial investigating the potential of SISU to act as therapeutic agent (2021) · cites this paper as Rozental et al., 2016
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Adverse events and harms during the RCT are measured with the Negative Experiences Questionnaire (NEQ), [61]”
From Transdiagnostic and tailored internet intervention to improve mental health among university students: Research protocol for a randomized controlled trial (2024) · cites this paper as [61]
The cited source says what the claim attributes to it.
In the citing paper
“The internal consistency for the full instrument is excellent (Cronbach's α = 0.95), and is scored on several dimensions; 1) “Did you experience this?” (yes/no), 2) “If yes – here is how negatively it affected me”, from 0 (“Not at all”) to 4 (“Extremely”), and 3) “Probably caused by” (“The treatment I received” or “Other circumstances”).”
From A study protocol of a randomized controlled study of internet-based cognitive behavioral therapy for adult attention deficit hyperactivity disorder (2023) · cites this paper as Rozental et al., 2016
The cited source says what the claim attributes to it.
In the citing paper
“Internal Consistency of the NEQ is reported to be α = 0.95 (90).”
The cited source says what the claim attributes to it.
In the citing paper
“Others have also implied that non-response, dropout, and interpersonal difficulties may be perceived as negative effects (Dimidjian 2010; Rozental 2016).”
From Psychotherapeutic treatments for depression in older adults (2024) · cites this paper as Rozental 2016
In the citing paper
“and trials have already begun (Miloff et al. 2019) to include and report results from broader measures of negative effects used elsewhere in psychotherapy research (Rozental et al. 2016).”
From Better, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy (2021) · cites this paper as Rozental et al. 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ryan, R. M., & Deci, E. L. (2000) · American Psychologist · doi:10.1037//0003-066x.55.1.68 · Cited by 54,599
An inventory for measuring depression
BECK, A. T. (1961) · Archives of General Psychiatry · doi:10.1001/archpsyc.1961.01710120031004 · Cited by 38,032
(1998) · The Journal of Clinical Psychiatry · Cited by 19,139
The Scree Test For The Number Of Factors
Cattell, R. B. (1966) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr0102_10 · Cited by 13,554
The application of electronic computers to factor analysis
Kaiser, H. F. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000116 · Cited by 12,459
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195
(1998) · PubMed · Cited by 10,216
Evaluating the use of exploratory factor analysis in psychological research.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.3.272 · Cited by 9,587
A Rationale and Test for the Number of Factors in Factor Analysis
Horn, J. L. (1965) · Psychometrika · doi:10.1007/bf02289447 · Cited by 8,965
Selective Cortical Undercutting As a Means of Modifying and Studying Frontal Lobe Function in Man
Scoville, W. B. (1949) · Journal of neurosurgery · doi:10.3171/jns.1949.6.1.0065 · Cited by 7,374
Flagged first, then by citations
1 of the 94 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- I Ladwig, 2014, What are the Risks and Side Effects to Psychotherapy?—Development of an Inventory for the Assessment of Negative Effects of Psychotherapy (INEP), Verhaltenstherapie, 24, 252
- S Moritz, 2015, No pain, no gain? Adverse effects of psychotherapy in obsessive-compulsive disorder and its relationship to treatment gains, J Obsess-Compuls Rel, 5, 61
Cited by
Sorted by citations
Items 101–200 of 223
Unerwünschte Ereignisse durch unsachgemäß durchgeführte Psychotherapie
Gawlytta, R., Schwartze, D., Schönherr, D., Schleu, A., & Strauss, B. M. (2019) · Psychiatrische Praxis · doi:10.1055/a-1026-1577 · Cited by 8
“In diesem Zusammenhang wird beispielsweise neben einer Verschlechterung der Ausgangssymptomatik auch das Auftreten neuer Beschwerden und Probleme wie sozialer Rückzug, Angst, Misstrauen, Traurigkeit, Depression und Aggression genannt [5, 23].”
BackgroundEimontas, J., Medeišienė, J., Baranova, K., Biliūnaitė, I., Pakalniškienė, V., Zrumaitė, S., Hilbig, J., & Andersson, G. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.119406 · Cited by 8
Self-citationWorsening of Self-Reported Symptoms Through Suggestive Feedback
van Helvoort, D. H. J., Otgaar, H., & Merckelbach, H. L. G. J. (2019) · Clinical Psychological Science · doi:10.1177/2167702619869184 · Cited by 7
Olsen, M. H., Hagstrøm, J., Lønfeldt, N. N., Uhre, C. F., Uhre, V., Pretzmann, L., Christensen, S. H., Thoustrup, C. L., Korsbjerg, N. L. J., Mora‐Jensen, A. C., et al. (2022) · Trials · doi:10.1186/s13063-022-06799-4 · Cited by 7
“For the secondary outcome NEQ, we pragmati-cally chose a MIREDIF of 2.0 points corresponding to 10% of the maximum scale and expected an SD of 3.5 points [27], which results in 89% power [26].”
MethodInfluentialJeckel, J., Thiele, C., Hirschfeld, G. H. F., Teismann, T., Schneider, S., & von Brachel, R. (2024) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2968 · Cited by 10
Buhrman, M., Gelberg, O., Jovicic, F., Molin, K., Forsström, D., Andersson, G., Carlbring, P., Shafran, R., & Rozental, A. (2020) · Internet Interventions · doi:10.1016/j.invent.2020.100338 · Cited by 8
“The NEQ has demonstrated excellent internal consistency (.95) and is mainly used to de-scriptively determine the occurrence and nature of possible negative effects (Rozental et al., 2016).”
Self-citationBendig, E., Meißner, E., Erb, B., Weger, L., Küchler, A., Bauereiß, N., Ebert, D. D., & Baumeister, H. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-041573 · Cited by 7
Maki, S., Nagai, K., Ando, S., & Tamakoshi, K. (2021) · PLoS ONE · doi:10.1371/journal.pone.0250771 · Cited by 7
Hybelius, J., Gustavsson, A., Hammarberg, S. A. W., Toth-Pal, E., Johansson, R., Ljótsson, B., & Axelsson, E. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01105-0 · Cited by 7
Beschwerden über Psychotherapie und ihre Auswirkungen auf Patienten und Patientinnen
Scholten, S., Velten, J., Kintscher, M., Pataki, A., & Margraf, J. (2018) · PPmP - Psychotherapie · Psychosomatik · Medizinische Psychologie · doi:10.1055/a-0597-5073 · Cited by 7
“Eine negative Veränderung in der Symptomatik, die Störung der therapeutischen Beziehung und affektive Reaktionen werden besonders oft als negative Folgen berichtet.”
BackgroundInfluentialGlanaghy, E. M., Jackson, J.-L., Morris, P. G., Prentice, W. M., Dougall, N. J., & Hutton, P. (2021) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2648 · Cited by 7
Dülsen, P., Barck, K., Daubmann, A., Höller, A., Zeidler, J., Kilian, R., Wiegand‐Grefe, S., & Baumeister, H. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.816412 · Cited by 9
Forsström, D., Oscarsson, M., Buhrman, M., & Rozental, A. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100652 · Cited by 6
“Negative Effects Questionnaire (NEQ) The NEQ is a 32-item instrument that assesses unwanted and adverse events experienced by patients undergoing psychotherapy (Rozental et al., 2016; Rozental et al., 2019), e.g., “ I experienced more unpleasant feelings ” (Item 11).”
Self-citationLawson, Z., & Farquharson, L. (2025) · Psychology and Psychotherapy Theory Research and Practice · doi:10.1111/papt.12578 · Cited by 6
“The Negative Effects Questionnaire (NEQ; Rozental et al., 2016), for example, is a 32-item measure, which focuses more on client experiences of negative effects.”
Adverse experiences in psychological treatments: Where do we stand?
Strauss, B. M., Rosendahl, J., & Klatte, R. (2025) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2025.102161 · Cited by 6
Egan, S. J., Pauley-Gadd, S. B., Callaghan, T., O’Brien, A., Greene, D., Bills, E., Raghav, S., Payne, N. C., MYERS, B. J., Hall, C., et al. (2026) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2025.2603521 · Cited by 6
“Future research on the intervention should include a more comprehensive assessment of negative effects of the intervention that are likely relevant (e.g. hopelessness, stigma, failure) using, for instance, the Negative Effects Questionnaire (Rozental et al., 2016).”
Humour interventions for patients in palliative care—a randomized controlled trial
Linge-Dahl, L., Kreuz, R., Stoffelen, M., Heintz, S., Ruch, W., von Hirschhausen, E., & Radbruch, L. (2023) · Supportive Care in Cancer · doi:10.1007/s00520-023-07606-9 · Cited by 6
“This effect has been reported in literature in the past [30], and there has been even a questionnaire created to measure negative effects [31].”
BackgroundHoppe, J. M., Weise, C., Kleinstäuber, M., Skalkidou, A., Vegelius, J. R., Comasco, E., Gröndal, M., Kaltsouni, E., Sundström, F., Sampaio, F., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-091649 · Cited by 10
Self-citationHow to measure negative effects of psychological interventions
Rozental, A., & Řiháček, T. (2025) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2025.102084 · Cited by 8
Self-citationBrowne, C., Brown, G., & Smith, I. (2019) · Journal of Applied Research in Intellectual Disabilities · doi:10.1111/jar.12569 · Cited by 6
Guided Self-help Teletherapy for Behavioural Difficulties in Children with Epilepsy
Bennett, S. D., Heyman, I., Varadkar, S., Coughtrey, A. E., Walji, F., & Shafran, R. (2021) · Journal of Clinical Psychology in Medical Settings · doi:10.1007/s10880-021-09768-2 · Cited by 6
Kallestad, H., Saksvik, S. B., Vedaa, Ø., Langsrud, K., Morken, G., Lydersen, S., Simpson, M. R., Dørheim, S. K., Holmøy, B., Selvik, S. G., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-050661 · Cited by 7
Strategies for de-implementation of low-value care – a scoping reviewPreprint
Ingvarsson, S., Hasson, H., Schwarz, U. V. T., Nilsén, P., Powell, B. J., Lindberg, C., & Augustsson, H. (2022) · Research Square · doi:10.21203/rs.3.rs-1835750/v1 · Cited by 6
Eklund, M., Kiritsis, C., Livheim, F., & Ghaderi, A. (2023) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2023.01.003 · Cited by 5
Poulter, M. M. L., Wahl, T. D., Kiviharju, M. J., Campbell, T. S., & Gordon, J. L. (2025) · Human Reproduction · doi:10.1093/humrep/deaf237 · Cited by 5
“The NEQ demonstrates strong internal consistency, with a Cronbach’s alpha of .95 in a large, predominantly female sample from Sweden (Rozental et al. , 2016).”
Hlynsson, J. I., Mechler, J., Lindqvist, K., Andersson, G., & Carlbring, P. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.100960 · Cited by 6
“This improvement is particularly striking given that waitlists are typically associated with symptom stability or even deterioration (17.4% per Rozental et al., 2017), rather than the significant gains observed here.”
InfluentialSelf-citationSchulte, C. E., Sextl-Plötz, T., Baumeister, H., Titzler, I., Sander, L., Sachser, C., Steubl, L., & Zarski, A. (2024) · Internet Interventions · doi:10.1016/j.invent.2024.100710 · Cited by 5
“Internal consistency of the NEQ is reported to be α = 0.95 (Rozental et al., 2016).”
BackgroundInfluentialSeidler, K.-P., Grützmacher, S., Epner, A., & Schreiber‐Willnow, K. (2020) · Psychotherapeut · doi:10.1007/s00278-020-00432-y · Cited by 5
“Stigmabezogene negative Effekte schildern 8,7–13,5% der Patienten (Rozental et al. 2016).”
BackgroundInfluentialDahlin, M. (2020) · Linköping studies in behavioural science · doi:10.3384/diss.diva-170049 · Cited by 5
Westin, F., & Rozental, A. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2259064 · Cited by 5
“Several self-report measures probing for possible negative effects also exist and could help therapist address the issue with patients by administering them prior to and during the treatment period, e.g., the Negative Effects Questionnaire (Rozental et al., 2016, 2019).”
BackgroundSelf-citationAtenolol - Scopolamine Combination Drug Decreases Acute Anxiety Post Trauma Therapy Session
Benjamin, A. B. (2017) · Annals of Depression and Anxiety · doi:10.26420/anndepressanxiety.2017.1086 · Cited by 5
“One study suggests that this rate of increased anxiety and stress is as high as 33 percent [1].”
Background(Kunst)Fehler in der Psychotherapie
Frenzl, D., Gawlytta, R., Schleu, A., & Strauss, B. M. (2020) · Psychotherapeut · doi:10.1007/s00278-020-00462-6 · Cited by 7
“Da bisher allerdings nur einzelne, ausgewählte (Kunst)Fehler erfasst werden, wird eine separate Erfassung von Kunstfehlern getrennt von Nebenwirkungen imengerenSinneals sinnvollund für die differenzierte Betrachtung beider Konstrukte förderlich erachtet (Herzog et al. 2019; Rozental et al. 2016).”
BackgroundDandachi‐FitzGerald, B., Otgaar, H., & Merckelbach, H. L. G. J. (2022) · Springer eBooks · doi:10.1007/978-3-031-14332-8_16 · Cited by 5
Lindegaard, T., Persson, L., Thorängen, M., & Rozental, A. (2024) · Psychotherapy Research · doi:10.1080/10503307.2024.2420717 · Cited by 4
“This could for instance be characterized by mixed-methods studies that include both self-reports of adverse and unwanted events (e.g., the Negative Effects Questionnaire; Rozental et al., 2016), and interviews regarding how these incidents are perceived.”
InfluentialSelf-citationBar, M., Saad, A., Weiss‐Klayman, N., & Mendlovic, S. (2025) · Psychotherapy Research · doi:10.1080/10503307.2025.2455466 · Cited by 6
“…sessions without productive focus are not only ineffective, but can also exacerbate the patient’s distress (Linden, 2013) and worsen current symptoms (Rozental et al., 2016), leading to a belief that they cannot be helped and reducing the likelihood that the patient will seek therapy in the future.”
Sigrist, C., Bechdolf, A., Bertsch, K., Bullenkamp, R., Busse, M., Darrelmann, U. G., Dempfle, A., Drießen, M., Frodl, T., Kersting, J.-M., et al. (2025) · Trials · doi:10.1186/s13063-025-08985-6 · Cited by 4
“…for DSM-5 (PID-5-BF + M) [40]. o UCLA Loneliness Scale (UCLA-3) [41]. o Group Questionnaire (GQ-D) [42]. o Negative Effects Questionnaire (NEQ) [43]. o Difficulties in Emotion Regulation Scale (DERS-36) [44]. o Certainty About Mental State Questionnaire (CAMSQ) [45]. o Rejection Sensitivity…”
Rozental, A., Shafran, R., Johansson, F., Forsström, D., Jovicic, F., Gelberg, O., Molin, K., Carlbring, P., Andersson, G., & Buhrman, M. (2024) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2024.2327339 · Cited by 7
“….93 PSS-14 14 0–4 0–56 α = .84–.86 (Cohen et al., 1983) .84 CEQ 5 0–10 0–50 α = .86–.90 (Devilly & Borkovec, 2000) .82 NEQ 20 0–4 0–80 α = .95 (Rozental et al., 2016) .76 Furthermore, at PRE and POST participants provided a score on a 10-point Likert-scale regarding how negatively they were…”
BackgroundInfluentialSelf-citationSchulte, C. E., Zarski, A., Sachser, C., Rosner, R., & Ebert, D. D. (2022) · European Journal of Psychotraumatology · doi:10.1080/20008198.2022.2101345 · Cited by 3
“Internal consistency of the NEQ is reported to be excellent (α = .95) in Swedish adults (Rozental et al., 2016).”
BackgroundKommen Systemiker:innen (auch) aus Lake Wobegon?
Ochs, M., & Pfautsch, B. (2022) · Familiendynamik · doi:10.21706/fd-47-4-284 · Cited by 4
Bendig, E., SCHMITT, A. J., Wittenberg, A., Kulzer, B., Hermanns, N., Moshagen, M., & Baumeister, H. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059336 · Cited by 4
Geurtzen, N., Karremans, J. C., Keijsers, G. P. J., & Hutschemaekers, G. J. M. (2023) · Journal of Contemporary Psychotherapy · doi:10.1007/s10879-022-09578-z · Cited by 4
Rubel, J. A., Quest, J., Pruessner, L., Timm, C., Hartmann, S. F., Barnow, S., Rittmeyer, L., Rosenbaum, D., & Lalk, C. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100584 · Cited by 4
“Negative effects (NEQ) The Negative Effects Questionnaire (NEQ; Rozental et al., 2016) is used to capture possible adverse effects of the intervention.”
InfluentialSauer, K. S., Bailer, J., Jungmann, S. M., & Witthöft, M. (2023) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2022.08.001 · Cited by 3
Giguère, S., Potvin, S., Fortier, A., Phraxayavong, K., Lespérance, F., Do, A., Bahremand, A., & Dumais, A. (2026) · Psychiatry Research · doi:10.1016/j.psychres.2026.117080 · Cited by 4
Mejía-Castrejón, J., Sierra-Madero, J. G., Belaunzarán-Zamudio, P. F., Fresán, A., Molina-López, A., Álvarez-Mota, A. B., & Robles, R. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2239448 · Cited by 3
“…al., 2013; Parker et al., 2014), NEQ (Rozental et al., 2016), INEP (Ladwing et al., 2014), and SEPS (Moritz et al., 2015), are self-applicable; NEQ (Rozental et al., 2016), PANEPS (Moritz et al., 2018), INEP (Ladwing et al., 2014), and ETQ (Parker et al., 2013; Parker et al., 2014), do not…”
BackgroundMethodInfluentialVock, S., Hofmann, A., Lehnung, M., Böhm, K., Wieland, S., Seidler, G. H., Beiner, E., Hermes, M., Friederich, H., Eich, W., et al. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1449150 · Cited by 3
Berretta, S. A., Abaya, N. M., Parrish, E. M., McBride, L. E., Moore, R. C., Ackerman, R. A., Harvey, P. D., Pinkham, A. E., & Depp, C. A. (2025) · NPP—Digital Psychiatry and Neuroscience · doi:10.1038/s44277-024-00024-7 · Cited by 3
Simon, L., Terhorst, Y., Küchler, A., Riemann, D. W., Ebert, D. D., Rozental, A., Spiegelhalder, K., & Baumeister, H. (2025) · Sleep Medicine · doi:10.1016/j.sleep.2025.02.044 · Cited by 3
Self-citationDigital Psychological Intervention for Inflammatory Rheumatic Diseases
Knitza, J., Kraus, J., Krusche, M., Haase, I., Klemm, P., Hueber, A. J., Pfeil, A., Drott, U., Kühn, S., & Klein, J. P. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.29892 · Cited by 3
Knox, S., Callender, K. A., Mak, T. W., Skaistis, S. M., & Knowlton, G. (2020) · Counselling Psychology Quarterly · doi:10.1080/09515070.2020.1843004 · Cited by 3
Prado, A. D. S., Kohls, E., Baldofski, S., Rummel‐Kluge, C., & Freitas, J. D. L. (2023) · JMIR Formative Research · doi:10.2196/44887 · Cited by 3
“Accordingly, some adverse and unwanted events are expected to arise during psychological interventions [58,84] and may affect the outcome [85] and, therefore, should be assessed more frequently in such interventions.”
BackgroundMethodResultAdaptation of the negative effects questionnaire into the Czech context
Chvála, Ľ., Řiháček, T., Polakovská, L., Vybíral, Z., & Rozental, A. (2019) · Psychotherapy Research · doi:10.1080/10503307.2019.1622053 · Cited by 3
“The psychometric properties of the NEQ were examined in one study (Rozental et al., 2016) so far, and an exploratory factor analysis suggested the existence of six factors: symptoms, quality of the psychotherapy and the therapeutic relationship, clients’ dependency on psychotherapy, stigma, hopelessness, and a sense o…”
BackgroundMethodResultInfluentialSelf-citationZur Definition und Erfassung von Kunstfehlern und Kunstfehlerfolgen in der Psychotherapie
Schwartze, D., & Strauss, B. M. (2018) · Trauma und Gewalt · doi:10.21706/tg-12-4-296 · Cited by 3
Relaksasi dan Katarsis Singkat Daring bagi Peningkatan Kebersyukuran dan Penurunan Stres Pengasuhan
Rahmandani, A., & Kahija, Y. F. L. (2022) · Jurnal Psikologi Integratif · doi:10.14421/jpsi.v10i1.2258 · Cited by 3
Rubel, J. A., Väth, T., Hanraths, S., Pruessner, L., Timm, C., Hartmann, S. F., Barnow, S., & Lalk, C. (2023) · doi:10.31234/osf.io/kbhvy · Cited by 3
Arvidson, E., Hofstedt, A., & Gordh, A. S. (2025) · Trials · doi:10.1186/s13063-025-09147-4 · Cited by 3
“Scale used only during treatment (T2): Negative Effects Questionnaire (NEQ) The purpose of the NEQ is to measure negative or adverse effects of psychological treatment, as experienced by the patient [56].”
Astenvald, R., Isaksson, M., Meyer, J., Lööf, M., Frick, M. A., & Isaksson, J. (2025) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2025.2537812 · Cited by 3
“The original versions of the NEQ have demonstrated a solid factor structure (Rozental et al., 2016, 2019).”
McCabe, R., O’Keeffe, S., Ougrin, D., Priebe, S., Martin, P., Feng, Y., Temple, R., & Long, M. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-101015 · Cited by 2
“Based on a previous study of self-harm in adults, where the within-practitioner ICC was 0.01, we estimated the within-practitioner correlation as 0.01 (In Rozental et al ’s study, 31 a smaller ICC would result in higher statistical power).”
InfluentialAminoff, V., Bielinski, L. L., Berg, M., Berger, T., & Andersson, G. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100875 · Cited by 1
“Lastly, this is, to our knowledge, the first study to examine the relationship between undesirable therapist behaviors in ICBT (Hadjistavropoulos et al., 2019) and negative effects (Rozental et al., 2016).”
InfluentialSelf-citationChvála, Ľ., Řiháček, T., & Vybíral, Z. (2022) · Ceskoslovenska psychologie · doi:10.51561/cspsych.66.5.532 · Cited by 2
Pretzmann, L., Christensen, S. H., Christensen, A. B., Uhre, C. F., Uhre, V., Thoustrup, C. L., Clemmesen, I. T., Gudmandsen, T. A., Korsbjerg, N. L. J., Mora‐Jensen, A. C., et al. (2023) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2023.101173 · Cited by 2
“The NEQ is developed and validated for adult psychotherapy [53] meaning that certain adverse events occurring in psychotherapy for youths with OCD may not be included in the questionnaire.”
BackgroundMethodSelf-citationFisher, A. C., Rochesson, S. E. D., Bisby, M. A., Scott, A. J., Gandy, M., Heriseanu, A. I., Titov, N., & Dear, B. F. (2024) · Health Expectations · doi:10.1111/hex.13976 · Cited by 3
“45 It is noteworthy that the current sample's perceptions of decisional needs were low overall, with average DCS below the average scores reported in other mental health treatment samples ( M = 32.4 ± 11.4).”
BackgroundResultInfluentialGatley, D., Millar‐Sarahs, V., Brown, A., Brooks, C. P., & Matcham, F. (2024) · International Journal of Eating Disorders · doi:10.1002/eat.24350 · Cited by 2
“Experiencing poor outcomes in therapy causes stress and reduces self-esteem and self-efficacy (Rozental et al. 2016).”
Lalk, C., Väth, T., Hanraths, S., Pruessner, L., Timm, C., Hartmann, S. F., Barnow, S., & Rubel, J. A. (2025) · Journal of Medical Internet Research · doi:10.2196/54062 · Cited by 4
“Adverse treatment effects were assessed with the Negative Effects Questionnaire (NEQ; [43]).”
Sürig, S., Dale, R., Herzog, P., Glanert, S., Grave, U., Assmann, N., Zurowski, B., Borgwardt, S., Klein, J. P., & Probst, T. (2025) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2025.2464641 · Cited by 2
“When analyzing the NEQ, it is proposed to only analyze items when patients causally attribute the negative effect to therapy (Rozental et al., 2016).”
InfluentialHajabdollahi, Z., Loripoor, M., Mohseni, M., & Khalili, P. (2025) · Journal of Education and Health Promotion · doi:10.4103/jehp.jehp_1175_23 · Cited by 2
Conrad, C. E., Ziegler, S. M. T., Bilenberg, N., Christiansen, J. H., Fagerlund, B., Hermann, R., Jeppesen, P., Kamp, C. B., Thomsen, P. H., Jakobsen, J. C., et al. (2025) · PLoS ONE · doi:10.1371/journal.pone.0323798 · Cited by 3
“• For the Negative Effects Questionnaire [61], MCID was not calculated as it has not, to the authors knowledge, been applied to this population previously.”
Ottergård, M.-L. H., Pretzmann, L., Lundgren, T., McCracken, L. M., Buhrman, M., Molander, P., & Rozental, A. (2025) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2025.100914 · Cited by 2
“The primary outcome was the Negative Effects Questionnaire (NEQ; Rozental et al., 2016), consisting of 32 items probing for UE, e.g., “ I experienced more hopelessness ” (item 6) (download: https://osf. io/f4qma/).”
Self-citationAtik, E., Kalde, J., Stricker, J., Schückes, M., Neudeck, P., Pietrowsky, R., & Pittig, A. (2025) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2025.2503834 · Cited by 3
“The safety hypothesis was that patients in the bCBT group would not experience more adverse events or more serious adverse events (as assessed by patient-rated NEQ and therapist-rated electronic adverse event reports) during treatment.”
InfluentialFailures in cognitive behavior therapy: The state of the art
Emmelkamp, P. M. G. (2025) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2025.102122 · Cited by 2
Fenski, F., Heinrich, M., Schaeuffele, C., Zagorscak, P., Knaevelsrud, C., & Boettcher, J. (2025) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2025.2565674 · Cited by 2
Self-citationRubel, J., Quest, J., Pruessner, L., Timm, C., Hartmann, S., Barnow, S., Rittmeyer, L., Rosenbaum, D., & Lalk, C. (2023) · JMIR Research Protocols · doi:10.2196/41440 · Cited by 2
Shanks, E., Jonsson, U., Wiklund, S., & Rozental, A. (2019) · Child & Family Social Work · doi:10.1111/cfs.12661 · Cited by 2
Self-citationLalk, C., Väth, T., Hanraths, S., Pruessner, L., Timm, C., Hartmann, S. F., Barnow, S., & Rubel, J. A. (2023) · doi:10.31234/osf.io/w9f4s · Cited by 2
Maimone, S., Haubrock, L. S., Joleby, M., Seto, M. C., Hillert, J., Dekker, A., Letourneau, E. J., Carvalho, J., Azevedo, V. C., Andersson, G., et al. (2025) · doi:10.2196/preprints.90249 · Cited by 2
Self-citationCobbaert, L. (2026) · Clinical Psychologist · doi:10.1080/13284207.2026.2700189 · Cited by 2
Mathiesen, A. S., Zoffmann, V., Lindschou, J., Jakobsen, J. C., Gluud, C. N., Olsen, M. H., Rasmussen, B., Marqvorsen, E. H. S., & Rothmann, M. J. (2024) · Trials · doi:10.1186/s13063-024-08589-6 · Cited by 1
Psychometrische Validierung der deutschen Version des Care Dependency Questionnaire
Schwab, S., Geurtzen, N., Faßbinder, E., Glanert, S., Schaich, A., Assmann, N., Sürig, S., Borgwardt, S., & Klein, J. P. (2025) · Fortschritte der Neurologie · Psychiatrie · doi:10.1055/a-2600-3682 · Cited by 1
Hansen, N. S., Ries, E. S., Ward, T., Cardi, V., Christensen, A. B., Hjorthøj, C. R., Nordentoft, M., Micali, N., & Glenthøj, L. B. (2025) · PLoS ONE · doi:10.1371/journal.pone.0319875 · Cited by 1
A Risk Ontology for Evaluating AI-Powered Psychotherapy Virtual Agents
Steenstra, I., & Bickmore, T. (2025) · ACM International Conference on Intelligent Virtual Agents (IVA) · doi:10.1145/3717511.3749286 · Cited by 10
Dear, B. F., Fisher, A. C., Bisby, M. A., Scott, A. J., & Titov, N. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100887 · Cited by 1
“Consistent with recommendations (Rozental et al., 2016), participants were asked at post-treatment whether they had noticed “ a worsening in [their] mental health ” since starting the intervention.”
Exploring negative experiences in psychotherapy using an NLP approach on online forum data
Steinbrenner, T., Lalk, C., Kabjesz, A., Ferizaj, D., Loray, J. S. P., Iovoli, F., & Rubel, J. A. (2025) · npj Mental Health Research · doi:10.1038/s44184-025-00172-4 · Cited by 1
Virtual Reality–Based Avatar Intervention for Eating Disorders: Mixed Methods Feasibility Study
Hansen, N. S., Ries, E. S., Rasmussen, K., Cardi, V., Christensen, A. B., Hjorthøj, C. R., Micali, N., Ward, T., & Glenthøj, L. B. (2026) · JMIR Formative Research · doi:10.2196/88445 · Cited by 1
“The Negative Effects Questionnaire was applied as a participant-reported, exploratory measure of negative effects, including a broad scope of psychosocial, emotional, and cognitive effects attributed to the treatment [33].”
When Evidence-Based Interventions Become Destabilizing: The Role of Timing in Clinical Judgment
Foerster, K. (2026) · Journal of Contemporary Psychotherapy · doi:10.1007/s10879-026-09729-6 · Cited by 1
Salim, O., Jansson‐Fröjmark, M., Sandlund, C., & Norell‐Clarke, A. (2024) · BMJ Open · doi:10.1136/bmjopen-2024-086676 · Cited by 1
“Finally, the Negative Effects Questionnaire (NEQ), 40 a 20-item self-report measure assessing any negative effects experienced during treatment, including the client’s belief about whether these effects were treatment-related or due to external factors, and the extent of their negative impact will…”
BackgroundZwischen Heilung und Schaden: Die komplexe Natur von Nebenwirkungen in der Psychotherapie
Gawlytta, R., Bartholdy, S., Brakemeier, E., & Strauss, B. M. (2025) · Verhaltenstherapie · doi:10.1159/000542736 · Cited by 2
Steinbrenner, T., Lalk, C., Kabjesz, A., Loray, J. S. P., Iovoli, F., & Rubel, J. A. (2025) · Research Square · doi:10.21203/rs.3.rs-6766808/v1 · Cited by 1
Psychotherapeutic treatments for depression in older adults
Ang, L., Lee, M. S., Song, E., Lee, H. W., Cao, L., Zhang, J., Wang, Q., Jung, J., Jang, S., Gastaldon, C., et al. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd015976
Do your eyes protect your memory?
Houben, S. T. L. (2021) · doi:10.26481/dis.20210527sh
马, 前., & 杨, 晓. (2019) · Theory and Practice of Psychological Counseling · doi:10.35534/tppc/0105018
Conrad, C. E., Ziegler, S. M. T., Bilenberg, N., Christiansen, J. H., Fagerlund, B., Hermann, R., Jeppesen, P., Kamp, C. B., Thomsen, P. H., Jakobsen, J. C., et al. (2023) · Research Square · doi:10.21203/rs.3.rs-2404235/v1
Conrad, C. E., Ziegler, S. M. T., Bilenberg, N., Christiansen, J. H., Fagerlund, B., Hermann, R., Jeppesen, P., Kamp, C. B., Thomsen, P. H., Jakobsen, J. C., et al. (2023) · Research Square · doi:10.21203/rs.3.rs-2495096/v1 · Cited by 1
Sextl-Plötz, T., Schulte, C. E., Baumeister, H., Titzler, I., Sander, L., Sachser, C., Steubl, L., & Zarski, A. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4620592
Non-pharmacological Interventions for Mental Disorders
(2024) · Frontiers research topics · doi:10.3389/978-2-8325-4708-3
Mathiesen, A. S., Zoffmann, V., Lindschou, J., Jakobsen, J. C., Gluud, C. N., Olsen, M. H., Rasmussen, B., Marqvorsen, E. H. S., & Rothmann, M. J. (2024) · Research Square · doi:10.21203/rs.3.rs-3669654/v1
Hansen, N. S., Ries, E. S., Ward, T. T., Cardi, V., Christensen, A., Hjorthøj, C. R., Nordentoft, M., Micali, N., & Glenthøj, L. (2025) · medRxiv · doi:10.1101/2025.02.12.25322135
Hed, S., Berg, A. I., Tillfors, M., Ehnvall, A., Westrin, Å., Wiktorsson, S., Gysin-Maillart, A., & Wærn, M. (2025) · BMC Psychiatry · doi:10.1186/s12888-025-07016-7
“Further, the Negative Effects Questionnaire (NEQ) [50], will be applied after the last ASSIP-OA session to answer the research question on side effects.”
Chalker, S. A., Carter, J., Imai, Y., Depp, C. A., & Chinman, M. (2025) · JMIR Research Protocols · doi:10.2196/66182
“To assess potential iatrogenic effects of delivering the intervention, peer specialist participants will complete the Negative Effects Questionnaire [26,27] (See peer specialist participant measure section below) as part of their posttreatment assessment.”
Feasibility, utility, and acceptability of an online ACT-based rehabilitation for clinical burnout
van de Leur, J. C., Lundbäck, K., Forslund, S., Virtanen, N., & Buhrman, M. (2025) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2025.2530420
“It consists of 32 items rated on a 5-point scale ranging from “not at all” to “extremely so,” with an underlying factor structure of six factors: symptoms, quality, dependency, stigma, hopelessness, and failure (Rozental et al., 2016).”
Dülsen, P., Barck, K., Wiegand‐Grefe, S., Leidger, A., Paumen, T., & Baumeister, H. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1526995
Metadata
- Authors
- Rozental, A., Kottorp, A., Boettcher, J., Andersson, G., & Carlbring, P.
- Affiliations
- Stockholm University (SE)
- Karolinska Institutet (SE)
- Freie Universität Berlin (DE)
- Linköping University (SE)
- Corresponding author
- Rozental, Alexander
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-06-22
- Volume
- 11
- Issue
- 6
- Pages
- e0157503
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4917117
- OpenAlex
- W2472395746
- Cited by
- 223
- References
- 94
- Citations by year
- 2026: 23 · 2025: 39 · 2024: 23 · 2023: 27 · 2022: 31 · 2021: 20 · 2020: 16 · 2019: 25 · 2018: 12 · 2017: 6 · 2016: 1
- FWCI
- 19.61
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DiVA (Linkoping University) (submittedVersion, other-oa)
- Zürcher Hochschule für Angewandte Wissenschaften digital collection (Zurich University of Applied Sciences) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Refubium (Universitätsbibliothek der Freien Universität Berlin) (submittedVersion, cc-by)
- ZHAW Digital Collection (cc-by)
- Funders
- FORTE (FORTE 2013-1107)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- adverse events, unwanted events, self-report instrument development, social anxiety disorder, symptom deterioration, treatment quality, therapeutic relationship, stigma, hopelessness, factor structure
- MeSH terms
- Adult, Mental Disorders, Demography, Factor Analysis, Statistical, Female, Humans, Male, Surveys and Questionnaires
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
