The DOI matches its Crossref record; 45 of the 46 references it cites were found in a registry or an index.
The Negative Effects Questionnaire: psychometric properties of an instrument for assessing negative effects in psychological treatments
Open accessAbstract
Background: Psychological treatments provide many benefits for patients with psychiatric disorders, but research also suggests that negative effects might occur from the interventions involved. The Negative Effects Questionnaire (NEQ) has previously been developed as a way of determining the occurrence and characteristics of such incidents, consisting of 32 items and six factors. However, the NEQ has yet to be examined using modern test theory, which could help to improve the understanding of how well the instrument works psychometrically. Aims: The current study investigated the reliability and validity of the NEQ from both a person and item perspective, establishing goodness-of-fit, item bias, and scale precision. Method: The NEQ was distributed to 564 patients in five clinical trials at post-treatment. Data were analysed using Rasch analysis, i.e. a modern test theory application. Results: (1) the NEQ exhibits fairness in testing across sociodemographics, (2) shows comparable validity for a final and condensed scale of 20 instead of 32 items, (3) uses a rating scale that advances monotonically in steps of 0 to 4, and (4) is suitable for monitoring negative effects on an item-level. Conclusions: The NEQ is proposed as a useful instrument for investigating negative effects in psychological treatments, and its newer shorter format could facilitate its use in clinical and research settings. However, further research is needed to explore the relationship between negative effects and treatment outcome, as well as to test it in more diverse patient populations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The 20-item version has demonstrated satisfactory psychometric properties [52].”
From Internet-Delivered Therapy for Parents With Health Anxiety by Proxy: Protocol for a Single-Case Experimental Design Study (2023) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“The internal consistency for the full instrument is excellent (Cronbach's α = 0.95), and is scored on several dimensions; 1) “Did you experience this?” (yes/no), 2) “If yes – here is how negatively it affected me”, from 0 (“Not at all”) to 4 (“Extremely”), and 3) “Probably caused by” (“The treatment I received” or “Other circumstances”).”
From A study protocol of a randomized controlled study of internet-based cognitive behavioral therapy for adult attention deficit hyperactivity disorder (2023) · cites this paper as Rozental et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“The 20-item NEQ shows comparable validity to the original 32-item version.”
From Efficacy of an internet-based psychological intervention for problem gambling and gambling disorder: Study protocol for a randomized controlled trial (2021) · cites this paper as Rozental et al., 2019
In the citing paper
“Negative treatment effects are here assessed post-treatment using the negative effects questionnaire (NEQ), a tool with reliable and valid psychometrics.74”
From Development, evaluation and implementation of a digital behavioural health treatment for chronic pain: study protocol of the multiphase DAHLIA project (2022) · cites this paper as [74]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.10.1092 · Cited by 31,765 · cited 1×
“Several secondary outcome measures were also administered; the Patient Health Questionnaire – 9 items (PHQ-9; Löwe et al., 2004), the Generalized Anxiety Disorder – 7 items (GAD-7; Spitzer et al., 2006), and the Brunnsviken Brief Quality of Life Scale (BBQ; Lindner et al., 2016).”
MethodLovibond, P. F., & Lovibond, S. H. (1995) · Behaviour Research and Therapy · doi:10.1016/0005-7967(94)00075-u · Cited by 14,523 · cited 2×
“A similar study was conducted using Rasch analysis for the Depression, Anxiety and Stress Scales (Lovibond and Lovibond, 1995), suggesting that a number of items could be removed and that it was not supported as a general instrument for mental distress (Shea et al.”
MethodApplying the Rasch Model: Fundamental Measurement in the Human Sciences
Bond, T. G., & Fox, C. M. (2003) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.2003.tb01103.x · Cited by 4,490
The Efficacy of Cognitive Behavioral Therapy: A Review of Meta-analyses
Hofmann, S. G., Asnaani, A., Vonk, I. J. J., Sawyer, A. T., & Fang, A. (2012) · Cognitive Therapy and Research · doi:10.1007/s10608-012-9476-1 · Cited by 3,445 · cited 1×
“Research on the efficacy of evidencebased approaches, such as cognitive behaviour therapy (CBT), suggest that a large number of patients improve thanks to the interventions they receive (Hofmann et al., 2012).”
BackgroundA Rating Formulation for Ordered Response Categories
Andrich, D. (1978) · Psychometrika · doi:10.1007/bf02293814 · Cited by 2,984 · cited 1×
“This can be helpful for identifying abnormal responses that might warrant further development of the instrument (Andrich, 1978).”
BackgroundChi-Square Tests with One Degree of Freedom; Extensions of the Mantel- Haenszel Procedure
Mantel, N. (1963) · Journal of the American Statistical Association · doi:10.1080/01621459.1963.10500879 · Cited by 2,899 · cited 1×
“The magnitude of DIF was evaluated using the Mantel-Haenszel statistic for polytomous scales using log-odds estimators (Mantel, 1963).”
MethodSummated Rating Scale Construction
Spector, P. E. (1992) · Sage eBooks · doi:10.4135/9781412986038 · Cited by 2,173 · cited 1×
“This converted item measure is then applied to determine whether they are scored on a similar unidimensional construct, which is often viewed as crucial in terms of validity in both classical and modern test theory (Spector, 1992).”
MethodKrosnick, J. A. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.537 · Cited by 1,852 · cited 1×
“It is possible that the negative effects that were reported were affected by recall bias, primacy-recency effects, and social desirability (Krosnick, 1999), resulting in less valid responses.”
BackgroundLöwe, B., Kroenke, K., Herzog, W., & Gräfe, K. (2004) · Journal of Affective Disorders · doi:10.1016/s0165-0327(03)00198-8 · Cited by 1,558 · cited 2×
“Several secondary outcome measures were also administered; the Patient Health Questionnaire – 9 items (PHQ-9; Löwe et al., 2004), the Generalized Anxiety Disorder – 7 items (GAD-7; Spitzer et al., 2006), and the Brunnsviken Brief Quality of Life Scale (BBQ; Lindner et al., 2016).”
MethodThe psychotherapeutic process: A research handbook.
Greenberg, L. S., & Pinsof, W. M. (1986) · Cited by 1,386
Research works printed but not linked: 4
- 10.1037/0003-066X.32.3.187 · DOI
- Fisher, 1992, Reliability, separation, strata statistics, Rasch Measurement Transactions, 6, 238
- Linacre, 2005, Rasch dichotomous model vs. one-parameter logistic model, Rasch Measurement Transactions, 19, 1032
- Raîche, 2005, Critical eigenvalue sizes in standardized residual principal components analysis, Rasch Measurement Transactions, 19, 1012
Cited by
Sorted by citations
Defining and Measuring Meditation-Related Adverse Effects in Mindfulness-Based Programs
Britton, W. B., Lindahl, J. R., Cooper, D. J., Canby, N. K., & Palitsky, R. (2021) · Clinical Psychological Science · doi:10.1177/2167702621996340 · Cited by 278
“or worsening symptoms caused by psychotherapy when measured systematically with a questionnaire (42%– 51%; Moritz et al., 2015; Rozental et al., 2019; SchermulyHaupt et al., 2018).”
BackgroundInfluentialCurran, J., Parry, G. D., Hardy, G. E., Darling, J., Mason, A.-M., & Chambers, E. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00347 · Cited by 84
Rozental, A., Andersson, G., & Carlbring, P. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00589 · Cited by 74
Riva, G., Bernardelli, L., Browning, M. H. E. M., Castelnuovo, G., Cavedoni, S., Chirico, A., Cipresso, P., de Paula, D. M. B., Di Lernia, D., Fernández-Álvarez, J., et al. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.563319 · Cited by 72
“• Negative Effects Questionnaire (NEQ): The self-report measure consists of three parts for a total of 32 items (47).”
BackgroundThe Safety of Digital Mental Health Interventions: Systematic Review and Recommendations
Taher, R., Hsu, C.-W., Hampshire, C., Fialho, C., Heaysman, C., Stahl, D., Shergill, S., & Yiend, J. (2023) · JMIR Mental Health · doi:10.2196/47433 · Cited by 68
“Symptom deterioration was one of the main risks reported in this review, similar to face-to-face therapies [13,45].”
BackgroundNegative effects of psychotherapy: estimating the prevalence in a random national sample
Strauss, B. M., Gawlytta, R., Schleu, A., & Frenzl, D. (2021) · BJPsych Open · doi:10.1192/bjo.2021.1025 · Cited by 59
Fenlon, D., Maishman, T., Day, L., Nuttall, J., May, C. R., Ellis, M. R., Raftery, J. P., Turner, L., Fields, J., Griffiths, G. O., et al. (2020) · Psycho-Oncology · doi:10.1002/pon.5432 · Cited by 59
Hensler, I., Sveen, J., Cernvall, M., & Arnberg, F. K. (2022) · Journal of Medical Internet Research · doi:10.2196/31419 · Cited by 53
“Negative reactions were measured using the Negative Effects Questionnaire [36].”
Pellas, J., Renner, F., Ji, J. L., & Damberg, M. (2021) · International Journal of Geriatric Psychiatry · doi:10.1002/gps.5646 · Cited by 45
“NEQ Adverse effects were measured using the Negative Effects Scale (NEQ)—short form, a 20‐item questionnaire used to assess adverse and unwanted effects of psychological treatments.(49) In the analysis, only the items rated by the participant as probably caused by the”
MethodFoulkes, L., Andrews, J. L., Reardon, T. C., & Stringaris, A. K. (2024) · Nature Mental Health · doi:10.1038/s44220-024-00208-2 · Cited by 48
“This can include deterioration in the primary outcomes, deterioration in any other measured variables and specific adverse events (for example, hospitalization or suicide attempt of a participant) 2–4 .”
Systematic review and meta-analysis of adverse events in clinical trials of mental health apps
Linardon, J., Fuller-Tyszkiewicz, M., Firth, J., Goldberg, S. B., Anderson, C., McClure, Z., & Torous, J. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01388-y · Cited by 44
Client perspectives on psychotherapy failure
Knox, S., Miller, C., Twidwell, R. E., & Knowlton, G. (2022) · Psychotherapy Research · doi:10.1080/10503307.2022.2110020 · Cited by 44
“…measures, and clinical populations studied (Crawford et al., 2016; Moritz et al., 2015; Rheker et al., 2017; Rozental et al., 2019), studies show that negative treatment effects are not uncommon, and in some populations seem to be the norm (e.g., Rozental et al., 2019; Schneibel et al., 2017).”
BackgroundLätth, J., Landgren, V., McMahan, A., Sparre, C., Eriksson, J., Malki, K., Söderquist, E., Öberg, K. G., Rozental, A., Andersson, G., et al. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100590 · Cited by 38
Self-citationSpanhel, K., Hovestadt, E., Lehr, D., Spiegelhalder, K., Baumeister, H., Bengel, J., & Sander, L. B. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.832196 · Cited by 37
“Negative Effects of the Intervention At T3, negative effects of the intervention were assessed within the IG with the Negative Effects Questionnaire, self-adapted to digital interventions (89, 90) [NEQ; 18 items; 3 scales per item: I.”
MethodEffectiveness of a Web-Based Cognitive Behavioral Self-Help Intervention for Binge Eating Disorder
Pruessner, L., Timm, C., Barnow, S., Rubel, J. A., Lalk, C., & Hartmann, S. F. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.11127 · Cited by 43
Schaeuffele, C., Homeyer, S. L., Perea, L., Scharf, L., Schulz, A., Knaevelsrud, C., Renneberg, B., & Boettcher, J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0270178 · Cited by 39
“We assessed negative effects of treatment with the Negative Effects Questionnaire (NEQ) [57], a 20-item measure of the occurrence and characteristics of negative effects following treatment.”
MethodResultInfluentialSelf-citationMaroti, D., Ek, J., Widlund, R.-M., Schubiner, H. H., Lumley, M. A., Lilliengren, P., Bilevičiūtė-Ljungar, I., Ljótsson, B., & Johansson, R. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.620359 · Cited by 32
“2% of patients (51) rendering it difficult to set a specific criterion; nevertheless, we proposed that negative effects reported in each of the five NEQ domains by fewer than 10% of participants was acceptable.”
Lundström, L., Flygare, O., Ivanova, E., Mataix‐Cols, D., Enander, J., Djurfeldt, D. R., Chen, L. L., Andersson, E., & Rück, C. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100608 · Cited by 35
Andersson, R., Åhlén, J., Mataix‐Cols, D., Lenhard, F., Henje, E., Månsson, C., Sahlin, H., Beckman, M., Serlachius, E., & Vigerland, S. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-066357 · Cited by 34
Baumeister, H., Bauereiss, N., Zarski, A.-C., Braun, L., Buntrock, C., Hoherz, C., Idrees, A. R., Kraft, R., Meyer, P., Nguyen, T. B. D., et al. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.660534 · Cited by 30
“Additionally, possible negative effects of the psychotherapies are assessed by means of the 20-item version of the Negative Effects Questionnaire [NEQ; 20 items; (90, 91)] which measures the frequency (score range: 0–20) and impact (score range: 0–80) of several different negative effects during the treatment period.”
Methodvan de Leur, J. C., Buhrman, M., Åhs, F., Rozental, A., & Jansen, G. B. (2020) · BMC Psychiatry · doi:10.1186/s12888-020-02907-3 · Cited by 30
Self-citationDülsen, P., & Baumeister, H. (2024) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-024-02404-y · Cited by 32
“…events assessed through validated negative effects questionnaires Only one study [36] used a validated questionnaire which was developed especially for the assessment of negative effects during psychotherapeutic treatments, namely the symptom subscale of the negative effects questionnaire [50].”
Bartels, S. L., Johnsson, S. I., Boersma, K., Flink, I., McCracken, L. M., Petersson, S., Christie, H. L., Feldman, I., Simons, L. E., Onghena, P., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-059152 · Cited by 23
“74 Descriptive statistics of the retrospective questionnaires will summarise demographics and pre-treatment clinical characteristics of the sample.”
Gullickson, K. M., Hadjistavropoulos, H. D., Dear, B. F., & Titov, N. (2019) · Internet Interventions · doi:10.1016/j.invent.2019.100278 · Cited by 26
Wiklund, T., Molander, P., Lindner, P., Andersson, G., Gerdle, B. U. C., & Dragioti, E. (2022) · Journal of Medical Internet Research · doi:10.2196/29258 · Cited by 25
“Finally, the most reported negative effect in both treatment arms (item 2: I felt like I was under more stress) was also the most frequent in several other CBT studies on different diagnoses [58].”
InfluentialSelf-citationHuang, S., Wang, Y., Li, G., Hall, B. J., & Nyman, T. J. (2024) · JMIR Mental Health · doi:10.2196/56650 · Cited by 26
Mutter, A., Küchler, A., Idrees, A. R., Kählke, F., Terhorst, Y., & Baumeister, H. (2023) · BMC Psychology · doi:10.1186/s40359-023-01312-1 · Cited by 22
“The original version of the NEQ consists of 20 items [58].”
MethodMaking the leap: From experimental psychopathology to clinical trials
Blackwell, S. E., & Woud, M. L. (2022) · Journal of Experimental Psychopathology · doi:10.1177/20438087221080075 · Cited by 19
“…in relation to assessment of negative effects of psychological interventions has been the construction of standardized scales to measure these (e.g. Bieda et al., 2018; Ladwig et al., 2014; Rozental et al., 2019), analogous to the measures of side-effects long used within pharmacological trials.”
BackgroundWiberg, A.-C., Ghaderi, A., Danielsson, H. B., Safarzadeh, K., Parling, T., Carlbring, P., Jansson, M., & Welch, E. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100570 · Cited by 22
Self-citationBlackwell, S. E., Schönbrodt, F. D., Woud, M. L., Wannemüller, A., Bektas, B., Rodrigues, M. B., Hirdes, J., Stumpp, M. T., & Margraf, J. (2022) · Psychological Medicine · doi:10.1017/s0033291722003294 · Cited by 23
Finazzi, E., & MacBeth, A. M. (2021) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2650 · Cited by 19
Verkooyen, R., Broers, N. J., & Dandachi-FitzGerald, B. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1378456 · Cited by 18
Web-Based Cognitive Behavioral Treatment for Bulimia Nervosa
Hartmann, S. F., Timm, C., Barnow, S., Rubel, J. A., Lalk, C., & Pruessner, L. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.19019 · Cited by 21
Dietrichkeit, M., Hagemann-Goebel, M., Nestoriuc, Y., Moritz, S., & Jelinek, L. (2021) · Scientific Reports · doi:10.1038/s41598-021-87198-8 · Cited by 16
Spiegelhalder, K., Baumeister, H., Al-Kamaly, A., Bader, M., Bauereiß, N., Benz, F., Braun, L., Buntrock, C., Burkhardt, M., Cuijpers, P. J., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-058212 · Cited by 16
Dandachi‐FitzGerald, B., Houben, S. T. L., Broers, N. J., & Merckelbach, H. L. G. J. (2024) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.3014 · Cited by 17
Moriuchi, E., Berbary, C. M., & Easton, C. J. (2023) · Journal of Technology in Behavioral Science · doi:10.1007/s41347-022-00298-8 · Cited by 15
“However, there have been studies that found that ICBT has potential negative effects due to the lack of assessment of novel symptoms (Rozental et al., 2019).”
BackgroundJensen, K. H. R., Dam, V. H., Ganz, M., Fisher, P. M., Ip, C.-T., Sankar, A., Marstrand-Joergensen, M. R., Ozenne, B., Osler, M., Penninx, B. W. J. H., et al. (2023) · BMC Psychiatry · doi:10.1186/s12888-023-04618-x · Cited by 16
“We use the short 20-item form of the Negative Effects Questionnaire (NEQ) to assess adverse and unwanted events in psychological treatment, i.e., new symptoms, dependency, stigma, hopelessness, and the experienced quality of treatment [74, 75].”
MethodMathiesen, A. S., Zoffmann, V., Skytte, T. B., Jakobsen, J. C., Gluud, C. N., Lindschou, J., Rasmussen, B., Marqvorsen, E. H. S., Thomsen, T., & Rothmann, M. J. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-047037 · Cited by 13
Schaeuffele, C., Heinrich, M., Behr, S., Fenski, F., Hammelrath, L., Zagorscak, P., Jansen, A., Pohl, S., Boettcher, J., & Knaevelsrud, C. (2022) · Trials · doi:10.1186/s13063-022-06757-0 · Cited by 15
“We will assess satisfaction with treatment with an adapted version of the 8-item Client Satisfaction Questionnaire (CSQ-8) [40, 41] and negative effects with the 20-item Negative Effects Questionnaire (NEQ) rated both by patients as well as psychotherapists [42].”
MethodSelf-citationTaher, R., Hall, C. L., Bergin, A. D. G., Gupta, N., Heaysman, C., Jacobsen, P., Kabir, T., Kalnad, N., Keppens, J., Hsu, C.-W., et al. (2024) · Trials · doi:10.1186/s13063-024-08421-1 · Cited by 8
“It is used to collect data on the negative effects experienced by patients/users during treatment, their severity and whether they were related to the intervention or other circumstances [19].”
MethodInfluentialWatson, P., Leroux, E. J., Chowdhury, M., Mehra, K., Henderson, J. L., Szatmári, P., & Hawke, L. D. (2022) · Journal of Child and Family Studies · doi:10.1007/s10826-022-02431-w · Cited by 12
“quantitative data about the negative effects of psychotherapy (Rozental et al., 2016; Rozental et al., 2019).”
BackgroundHerzog, P., Häusler, S., Normann, C., & Brakemeier, E.-L. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.575837 · Cited by 13
“Over the past 10 years, several instruments for assessing negative effects of psychotherapy have been developed and partially validated, notably the Inventory for the Assessment of Negative Effects of Psychotherapy [INEP; (3)] and the Negative Effect Questionnaire [NEQ; (4)].”
BackgroundInternet-based self-help intervention for procrastination: randomized control group trial protocol
Ying, Z., & Wang, J. (2023) · Trials · doi:10.1186/s13063-023-07112-7 · Cited by 20
“(1) Attitude towards online psychological intervention (Attitudes Towards Psychological Online Interventions Scale, APOI) [60] The APOI questionnaire was used to determine the participants’ concerns regarding the adverse events, feasibility, professionalism, and effectiveness of online psychological treatments or inte…”
BackgroundWittmann, L., Blomert, E., & Linden, M. (2023) · Psychotherapy Research · doi:10.1080/10503307.2023.2290029 · Cited by 12
“Some self-rating instruments such as the Negative Effects Questionnaire (Rozental et al., 2019) ask the patient to attribute the negative events to either treat-ment or other circumstances.”
InfluentialAustin, S. F., Frøsig, A. J., Buus, N., Lincoln, T. M., von Malachowski, A., Schlier, B., Frost, M. M., & Simonsen, E. H. (2021) · Qualitative Health Research · doi:10.1177/1049732320986556 · Cited by 13
“The Negative Effects Questionnaire (NEQ; Rozental et al., 2019) was used to identify any negative events attributable to the mHealth solution.”
MethodBerman, A. H., Topooco, N., Lindfors, P., Bendtsen, M., Lindner, P., Molander, O., Kraepelien, M., Sundström, C., Talebizadeh, N., Engström, K., et al. (2024) · Trials · doi:10.1186/s13063-024-07986-1 · Cited by 9
Self-citationHooi, L. Y., Chen, P. L., Tan, K. W., de Vries, M., & Wong, H. K. (2025) · Scientific Reports · doi:10.1038/s41598-025-23727-z · Cited by 17
“The 20-item Negative Effects Questionnaire (NEQ) was used to measure the adverse effects of the intervention 126,127 .”
Adaptation and Feasibility of the Mandarin Version of PEERS® for Autistic Adolescents
Lao, U., Li, Y., Bai, W., Wang, Y., Li, Y., Xie, Y., Huang, X., Zhu, H., & Zou, X. (2023) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-023-06056-w · Cited by 11
“The NEQ is a 32-item instrument developed for monitoring and reporting the occurrences and magni - tudes of negative effects during psychological interventions (Rozental et al., 2019 ).”
Mide, M., Mattiasson, J., Norlin, D., Sehlin, H., Rasmusson, J., Ljung, S., Lindskog, A., Petersson, J., Saavedra, F., & Gordh, A. S. (2023) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2023.1243826 · Cited by 12
Side Effects in Psychodynamic and Cognitive Behavior Therapy
Balder, T., Linden, M., & Rose, M. (2024) · Journal of Contemporary Psychotherapy · doi:10.1007/s10879-023-09615-5 · Cited by 12
“Moreover, there is an increase in reported side effects when patients provide self-ratings, with rates such as 37.7% (Gerke et al., 2020), 50.9% (Rozental et al., 2019), 52.6% (Peth et al., 2018), 70.4% (Abeling et al., 2018), 71% (Pou-rová et al., 2022 ), 78.8% (Hoffmann et al., 2021), 84%…”
InfluentialThe preliminary development and psychometric properties of the Psychotherapy Side Effects Scale
Chen, F., Liu, L., Xudong, Z., Qiang, F., Congcong, G., & Yunhan, Z. (2023) · Brain and Behavior · doi:10.1002/brb3.2885 · Cited by 9
“…for the Assessment of Negative Effects of Psychotherapy (INEP, Ladwig et al., 2014), the Positive and Negative Effects of Psychotherapy Scale (PANEPS, Steffen et al., 2019), and the Negative Effects Questionnaire (NEQ, Rozental et al., 2019) focused on the negative effects experienced by patients.”
BackgroundSchool-based social skills group training (SKOLKONTAKT™): a pilot randomized controlled trial
Fridell, A., Coco, C., Borg, A., & Bölte, S. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1128288 · Cited by 10
Balci, S., Küchler, A., Ebert, D. D., & Baumeister, H. (2024) · BMC Digital Health · doi:10.1186/s44247-024-00074-z · Cited by 11
“NEQ consists of 20 questions and shows acceptable item fit and reliability [54].”
BackgroundRecommendations on recording harms in randomised controlled trials of behaviour change interventions
Papaioannou, D. E., Hamer‐Kiwacz, S., Mooney, C., Sprange, K., Cooper, C. L., & O’Cathain, A. (2024) · BMJ · doi:10.1136/bmj-2023-077418 · Cited by 10
Lassen, N. F., Hougaard, E., Arendt, K. B., & Thastum, M. (2019) · Trials · doi:10.1186/s13063-019-3885-3 · Cited by 10
Ekbäck, E., Granåsen, G., Svärling, R., Blomqvist, I., & Henje, E. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.674583 · Cited by 10
Thaysen‐Petersen, D., Hammerum, S. K., Vissing, A.-C., Arnfred, B. T., Nordahl, R., Adjorlu, A., Nordentoft, M., Oestrich, I. H., Düring, S. W., & Fink‐Jensen, A. (2023) · BMJ Open · doi:10.1136/bmjopen-2022-068658 · Cited by 12
Pedersen, S. S., Skovbakke, S. J., Skov, O., Carlbring, P., Burg, M. M., Habibović, M., & Ahm, R. (2023) · Current Cardiology Reports · doi:10.1007/s11886-023-01867-w · Cited by 12
Self-citationDiaz-Sanahuja, L., Campos, D., Mira, A., Castilla, D., García-Palacios, A., & Bretón-López, J. M. (2021) · Internet Interventions · doi:10.1016/j.invent.2021.100466 · Cited by 10
Lundin, J., Jansson‐Fröjmark, M., Gustafsson-Björverud, L., Grey, N., Santoft, F., Ehlers, A., Carlbring, P., Lundgren, T., Bragesjö, M., & Salomonsson, S. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1447651 · Cited by 9
Self-citationMolander, O., Berman, A. H., Jakobson, M., Gajecki, M., Hällström, H., Ramnerö, J., Bjureberg, J., Carlbring, P., & Lindner, P. (2024) · Journal of Medical Internet Research · doi:10.2196/54754 · Cited by 9
“This was an expected result, since the iCBT program actively encouraged patients with GD to confront gambling-related situations and manage them differently; similar negative effects have been reported for CBT in the context of anxiety [36].”
Self-citationAxelsson, E., & Hedman‐Lagerlöf, E. (2023) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2023.2250915 · Cited by 9
“For this reason, we recommend future trials to combine the use of open-ended questions to approximate what events patients deem problematic, and closed questions (e.g. the Negative Effects Questionnaire [85]) to ensure that rates are exhaustive and can be compared over treat-ment conditions.”
BackgroundBenz, F., Grolig, L., Hannibal, S., Buntrock, C., Cuijpers, P. J., Domschke, K., Ebert, D. D., Ell, J., Janneck, M., Jenkner, C., et al. (2024) · Trials · doi:10.1186/s13063-024-08214-6 · Cited by 10
“First, the occurrence and impact of possible negative effects during treatment will be measured using the 20-item Negative Effects Questionnaire (NEQ; [53]).”
MethodStark, R. M., Markert, C., Golder, S., Psarros, R., Discher, J. P., Khatib, S., Metzger, J., Palmer, S., Rechmann, J., Storz, F., et al. (2024) · Journal of Behavioral Addictions · doi:10.1556/2006.2024.00046 · Cited by 10
“Potential side effects of psycho-therapy are recorded with the Negative Effects Questionnaire (NEQ, Rozental et al., 2019) and evaluated exploratively.”
Yiend, J., Taher, R., Fialho, C., Hampshire, C., Hsu, C.-W., Kabir, T., Keppens, J., McGuire, P. K., Mouchlianitis, E. D., PETERS, E. R., et al. (2024) · Trials · doi:10.1186/s13063-024-08570-3 · Cited by 7
Seitz, K. I., Schouler, N., Hundertmark, J., Wilhelm, M., Franz, S., Bauer, S., Taubner, S., Korn, C. W., Haun, M. W., Ditzen, B., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-090476 · Cited by 9
Hentati, A., Forsell, E., Ljótsson, B., Lindefors, N., & Kraepelien, M. (2022) · BJPsych Open · doi:10.1192/bjo.2022.14 · Cited by 8
Pruessner, L., Hartmann, S. F., Rubel, J. A., Lalk, C., Barnow, S., & Timm, C. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100514 · Cited by 9
Hartmann, S. F., Pruessner, L., Rubel, J. A., Lalk, C., Barnow, S., & Timm, C. (2022) · Internet Interventions · doi:10.1016/j.invent.2022.100512 · Cited by 9
“…interventions such as group or dissonance-based therapies (e.g., Green et al., 2018; Zerwas et al., 2017), were restricted to a subgroup of patients (e.g., Sanchez-Ortiz et al., 2011; Wagner et al., 2013) or included a mixture of sub- and fullthreshold BN patients (e.g., Ruwaard et al., 2013).”
BackgroundMethodMolander, O., Lindner, P., Ramnerö, J., Bjureberg, J., Carlbring, P., & Berman, A. H. (2020) · Pilot and Feasibility Studies · doi:10.1186/s40814-020-00647-5 · Cited by 8
“Satisfaction with treatment and potential negative effects due to treatment will be measured with the Client Satisfaction Questionnaire—8-item version [31] and the Negative Effects Questionnaire—20-item version [32], respectively, both administered at post-treatment.”
MethodSelf-citationGholamrezaei, A., Magee, M. R., McNeilage, A. G., Dwyer, L., Sim, A., Ferreira, M. L., Darnall, B. D., Brake, T., Aggarwal, A., Craigie, M., et al. (2024) · PAIN Reports · doi:10.1097/pr9.0000000000001128 · Cited by 8
“Negative Effects Questionnaire is a self-report measure of common unwanted effects associated with psychological treatments 53 which has previously been used in clinical trials of digital health interventions 18,32,33 including an ongoing study on chronic pain management.”
BackgroundMethodSennerstam, V., Föyen, L. F., Kontio, E., Svärdman, F., Lekander, M., Lindsäter, E., & Hedman‐Lagerlöf, E. (2025) · Psychotherapy and Psychosomatics · doi:10.1159/000546221 · Cited by 7
Olsen, M. H., Hagstrøm, J., Lønfeldt, N. N., Uhre, C. F., Uhre, V., Pretzmann, L., Christensen, S. H., Thoustrup, C. L., Korsbjerg, N. L. J., Mora‐Jensen, A. C., et al. (2022) · Trials · doi:10.1186/s13063-022-06799-4 · Cited by 7
“For the secondary outcome NEQ, we pragmati-cally chose a MIREDIF of 2.0 points corresponding to 10% of the maximum scale and expected an SD of 3.5 points [27], which results in 89% power [26].”
BackgroundMethodInfluentialHybelius, J., Gustavsson, A., Hammarberg, S. A. W., Toth-Pal, E., Johansson, R., Ljótsson, B., & Axelsson, E. (2022) · Pilot and Feasibility Studies · doi:10.1186/s40814-022-01105-0 · Cited by 7
Dülsen, P., Barck, K., Daubmann, A., Höller, A., Zeidler, J., Kilian, R., Wiegand‐Grefe, S., & Baumeister, H. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.816412 · Cited by 9
Riva, G. (2021) · http://isrctn.com/ · doi:10.1186/isrctn63887521 · Cited by 6
Forsström, D., Oscarsson, M., Buhrman, M., & Rozental, A. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100652 · Cited by 6
“Negative Effects Questionnaire (NEQ) The NEQ is a 32-item instrument that assesses unwanted and adverse events experienced by patients undergoing psychotherapy (Rozental et al., 2016; Rozental et al., 2019), e.g., “ I experienced more unpleasant feelings ” (Item 11).”
Self-citationArendt, I. T. P., Gondan, M., Juul, S., Hastrup, L. H., Hjorthøj, C. R., Bach, B., Videbech, P., Jørgensen, M. B., & Møeller, S. B. (2024) · Trials · doi:10.1186/s13063-024-08079-9 · Cited by 9
“It is deemed suitable to monitor effects on an item-level and demonstrates an acceptable person goodness-of-fit [55].”
BackgroundDigital Emotion Regulation Interventions for Patients With Congenital Heart Disease
Pruessner, L., Hartmann, S., Ehmann, A.-L., Barnow, S., Bauer, U. M. M., & Helm, P. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.38813 · Cited by 7
Lafortune, D., Lapointe, V. A., Dubé, S., & Bonneau, J. (2024) · The Journal of Sexual Medicine · doi:10.1093/jsxmed/qdae154 · Cited by 7
How to measure negative effects of psychological interventions
Rozental, A., & Řiháček, T. (2025) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2025.102084 · Cited by 8
Self-citationStrauss, B. M. (2021) · International Journal of Group Psychotherapy · doi:10.1080/00207284.2021.1890089 · Cited by 6
“Following Lilienfeld’s (2007) seminal paper describing harmful treatments, several attempts were made to develop measures of negative effects, such as the Negative Effects Questionnaire (Rozental et al., 2019), and to study their prevalence.”
MethodHensler, I., Sveen, J., Cernvall, M., & Arnberg, F. K. (2023) · Internet Interventions · doi:10.1016/j.invent.2023.100618 · Cited by 6
“Negative effects related to using PTSD Coach were measured with a study-specific version of the Negative Effects Questionnaire (Rozental et al., 2019) which was adjusted for app interventions in contrast to psychological treatment.”
MethodCunningham, N. R., Miller, A. K., Ely, S. L., Reid, M. R., Danguecan, A. N., Mossad, S. I., Pereira, L. F., Abulaban, K. M., Kessler, E., Rosenwasser, N. L., et al. (2023) · Pediatric Rheumatology · doi:10.1186/s12969-023-00835-6 · Cited by 7
“Adverse events The Negative Effects Questionnaire, a 20-item questionnaire to examine adverse/unwanted events experienced during TEACH [39].”
BackgroundEklund, M., Kiritsis, C., Livheim, F., & Ghaderi, A. (2023) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2023.01.003 · Cited by 5
Balsom, A. A., Klest, B., Sander, B., & Gordon, J. L. (2024) · Reproductive Health · doi:10.1186/s12978-024-01766-5 · Cited by 6
Schaeuffele, C., Homeyer, S. L., Perea, L., Scharf, L., Schulz, A., Knaevelsrud, C., Renneberg, B., & Boettcher, J. (2020) · doi:10.31234/osf.io/528tw · Cited by 9
Self-citationPoulter, M. M. L., Wahl, T. D., Kiviharju, M. J., Campbell, T. S., & Gordon, J. L. (2025) · Human Reproduction · doi:10.1093/humrep/deaf237 · Cited by 5
“Participants could endorse up to 16 negative effects reflecting level of symptoms, dependency, stigma, perception of program quality, and feelings of hopelessness (Rozental et al. , 2019).”
Hlynsson, J. I., Mechler, J., Lindqvist, K., Andersson, G., & Carlbring, P. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.100960 · Cited by 6
Self-citationGordon, J. L., Poulter, M. M. L., Balsom, A. A., & Campbell, T. S. (2023) · JMIR Research Protocols · doi:10.2196/52662 · Cited by 5
“The Negative Effects Questionnaire [35] will assess any potential adverse events perceived to be related to the intervention.”
BackgroundIvanova, E. N., Fondberg, R., Flygare, O., Sannemalm, M., Asplund, S., Dahlén, S., Sampaio, F., Andersson, E., Mataix‐Cols, D., Ivanov, V. Z., et al. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-076361 · Cited by 5
Seidler, K.-P., Grützmacher, S., Epner, A., & Schreiber‐Willnow, K. (2020) · Psychotherapeut · doi:10.1007/s00278-020-00432-y · Cited by 5
“In gewisser Weise findet damit die klinische Erfahrung von KBT-Therapeuten ihre Bestätigung, wonach die körpertherapeutische Arbeit zeitweise mit starken, d.h. auch unangenehmen Affekten verbunden sein kann (Schreiber-Willnow 2006).”
BackgroundRemote blended treatment for individuals with suicidal ideation: A single‐arm proof‐of‐concept trial
Büscher, R., Teismann, T., Hartleitner, P., Klein, J. P., Baumeister, H., & Sander, L. (2023) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2905 · Cited by 5
Gholamrezaei, A., Magee, M. R., McNeilage, A. G., Dwyer, L., Jafari, H., Sim, A., Ferreira, M. L., Darnall, B. D., Glare, P. A., & Ashton-James, C. E. (2023) · BMJ Open · doi:10.1136/bmjopen-2023-073297 · Cited by 5
von Malachowski, A., Schlier, B., Austin, S. F., Frost, M. M., Frøsig, A. J., Heinzle, M., Holzapfel, H., Lipps, A., Simonsen, E. H., & Lincoln, T. M. (2022) · Schizophrenia Research · doi:10.1016/j.schres.2021.12.011 · Cited by 5
Dandachi‐FitzGerald, B., Otgaar, H., & Merckelbach, H. L. G. J. (2022) · Springer eBooks · doi:10.1007/978-3-031-14332-8_16 · Cited by 5
Lindegaard, T., Persson, L., Thorängen, M., & Rozental, A. (2024) · Psychotherapy Research · doi:10.1080/10503307.2024.2420717 · Cited by 4
“This correspond to a rate of 50.9% ( n = 281) in Rozental, Kottorp, et al. (2019) for patients receiving CBT for various conditions, and 45.2% (psychosomatic patients receiving CBT) and 58.7% (psychiatric patients receiving either CBT or PP) demonstrated by Rheker et al. (2017) ( n = 203).”
InfluentialSelf-citationBöge, K., Bergmann, N., Hahne, I., Emde, L., Cartner, S., Bell, I., Pillny, M., & Thomas, N. (2025) · Internet Interventions · doi:10.1016/j.invent.2025.100804 · Cited by 4
Loades, M. E., Perry, G., & Marshall, N. (2025) · JMIR Mental Health · doi:10.2196/72077 · Cited by 4
“Studies could include adverse events and harm measures such as the Negative Effects Questionnaire [71].”
Metadata
- Authors
- Rozental, A., Kottorp, A., Forsström, D., Månsson, K., Boettcher, J., Andersson, G., Furmark, T., & Carlbring, P.
- Affiliations
- Karolinska Institutet (SE)
- Malmö University (SE)
- Stockholm University (SE)
- Uppsala University (SE)
- Freie Universität Berlin (DE)
- Linköping University (SE)
- Corresponding author
- Rozental, Alexander
- ORCID
- Venue
- Behavioural and Cognitive Psychotherapy
- Publisher
- Cambridge University Press (CUP)
- Published
- 2019-03-15
- Volume
- 47
- Issue
- 5
- Pages
- 559-572
- ISSN
- 1352-4658 (ISSN-L)
- 1469-1833
- OpenAlex
- W2922529521
- Cited by
- 230
- References
- 46
- Citations by year
- 2026: 47 · 2025: 47 · 2024: 47 · 2023: 32 · 2022: 30 · 2021: 15 · 2020: 7 · 2019: 4
- FWCI
- 10.19
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Behavioural and Cognitive Psychotherapy (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Uppsala University Publications (Uppsala University) (submittedVersion, other-oa)
- Malmö University Publications (Malmö University) (submittedVersion)
- University of Southern Denmark Research Portal (University of Southern Denmark) (submittedVersion, cc-by)
- Funders
- University of Illinois at Urbana-Champaign
- University of Illinois at Chicago
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Keywords
- adverse effects, psychometric properties, Rasch analysis, differential item functioning, item response theory, short-form scale, treatment outcome monitoring, mental illness
- MeSH terms
- Negative Results, Adult, Mental Disorders, Female, Humans, Male, Psychology, Psychometrics, Surveys and Questionnaires, Reproducibility of Results, Treatment Failure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
