The DOI matches its Crossref record; 45 of the 46 references it cites were found in a registry or an index.
The Negative Effects Questionnaire: psychometric properties of an instrument for assessing negative effects in psychological treatments
Open accessAbstract
Background: Psychological treatments provide many benefits for patients with psychiatric disorders, but research also suggests that negative effects might occur from the interventions involved. The Negative Effects Questionnaire (NEQ) has previously been developed as a way of determining the occurrence and characteristics of such incidents, consisting of 32 items and six factors. However, the NEQ has yet to be examined using modern test theory, which could help to improve the understanding of how well the instrument works psychometrically. Aims: The current study investigated the reliability and validity of the NEQ from both a person and item perspective, establishing goodness-of-fit, item bias, and scale precision. Method: The NEQ was distributed to 564 patients in five clinical trials at post-treatment. Data were analysed using Rasch analysis, i.e. a modern test theory application. Results: (1) the NEQ exhibits fairness in testing across sociodemographics, (2) shows comparable validity for a final and condensed scale of 20 instead of 32 items, (3) uses a rating scale that advances monotonically in steps of 0 to 4, and (4) is suitable for monitoring negative effects on an item-level. Conclusions: The NEQ is proposed as a useful instrument for investigating negative effects in psychological treatments, and its newer shorter format could facilitate its use in clinical and research settings. However, further research is needed to explore the relationship between negative effects and treatment outcome, as well as to test it in more diverse patient populations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The 20-item version has demonstrated satisfactory psychometric properties [52].”
From Internet-Delivered Therapy for Parents With Health Anxiety by Proxy: Protocol for a Single-Case Experimental Design Study (2023) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“The internal consistency for the full instrument is excellent (Cronbach's α = 0.95), and is scored on several dimensions; 1) “Did you experience this?” (yes/no), 2) “If yes – here is how negatively it affected me”, from 0 (“Not at all”) to 4 (“Extremely”), and 3) “Probably caused by” (“The treatment I received” or “Other circumstances”).”
From A study protocol of a randomized controlled study of internet-based cognitive behavioral therapy for adult attention deficit hyperactivity disorder (2023) · cites this paper as Rozental et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“The 20-item NEQ shows comparable validity to the original 32-item version.”
From Efficacy of an internet-based psychological intervention for problem gambling and gambling disorder: Study protocol for a randomized controlled trial (2021) · cites this paper as Rozental et al., 2019
In the citing paper
“Negative treatment effects are here assessed post-treatment using the negative effects questionnaire (NEQ), a tool with reliable and valid psychometrics.74”
From Development, evaluation and implementation of a digital behavioural health treatment for chronic pain: study protocol of the multiphase DAHLIA project (2022) · cites this paper as [74]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.10.1092 · Cited by 31,765 · cited 1×
“Several secondary outcome measures were also administered; the Patient Health Questionnaire – 9 items (PHQ-9; Löwe et al., 2004), the Generalized Anxiety Disorder – 7 items (GAD-7; Spitzer et al., 2006), and the Brunnsviken Brief Quality of Life Scale (BBQ; Lindner et al., 2016).”
MethodLovibond, P. F., & Lovibond, S. H. (1995) · Behaviour Research and Therapy · doi:10.1016/0005-7967(94)00075-u · Cited by 14,523 · cited 2×
“A similar study was conducted using Rasch analysis for the Depression, Anxiety and Stress Scales (Lovibond and Lovibond, 1995), suggesting that a number of items could be removed and that it was not supported as a general instrument for mental distress (Shea et al.”
MethodApplying the Rasch Model: Fundamental Measurement in the Human Sciences
Bond, T. G., & Fox, C. M. (2003) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.2003.tb01103.x · Cited by 4,490
The Efficacy of Cognitive Behavioral Therapy: A Review of Meta-analyses
Hofmann, S. G., Asnaani, A., Vonk, I. J. J., Sawyer, A. T., & Fang, A. (2012) · Cognitive Therapy and Research · doi:10.1007/s10608-012-9476-1 · Cited by 3,445 · cited 1×
“Research on the efficacy of evidencebased approaches, such as cognitive behaviour therapy (CBT), suggest that a large number of patients improve thanks to the interventions they receive (Hofmann et al., 2012).”
BackgroundA Rating Formulation for Ordered Response Categories
Andrich, D. (1978) · Psychometrika · doi:10.1007/bf02293814 · Cited by 2,984 · cited 1×
“This can be helpful for identifying abnormal responses that might warrant further development of the instrument (Andrich, 1978).”
BackgroundChi-Square Tests with One Degree of Freedom; Extensions of the Mantel- Haenszel Procedure
Mantel, N. (1963) · Journal of the American Statistical Association · doi:10.1080/01621459.1963.10500879 · Cited by 2,899 · cited 1×
“The magnitude of DIF was evaluated using the Mantel-Haenszel statistic for polytomous scales using log-odds estimators (Mantel, 1963).”
MethodSummated Rating Scale Construction
Spector, P. E. (1992) · Sage eBooks · doi:10.4135/9781412986038 · Cited by 2,173 · cited 1×
“This converted item measure is then applied to determine whether they are scored on a similar unidimensional construct, which is often viewed as crucial in terms of validity in both classical and modern test theory (Spector, 1992).”
MethodKrosnick, J. A. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.537 · Cited by 1,852 · cited 1×
“It is possible that the negative effects that were reported were affected by recall bias, primacy-recency effects, and social desirability (Krosnick, 1999), resulting in less valid responses.”
BackgroundLöwe, B., Kroenke, K., Herzog, W., & Gräfe, K. (2004) · Journal of Affective Disorders · doi:10.1016/s0165-0327(03)00198-8 · Cited by 1,558 · cited 2×
“Several secondary outcome measures were also administered; the Patient Health Questionnaire – 9 items (PHQ-9; Löwe et al., 2004), the Generalized Anxiety Disorder – 7 items (GAD-7; Spitzer et al., 2006), and the Brunnsviken Brief Quality of Life Scale (BBQ; Lindner et al., 2016).”
MethodThe psychotherapeutic process: A research handbook.
Greenberg, L. S., & Pinsof, W. M. (1986) · Cited by 1,386
Research works printed but not linked: 4
- 10.1037/0003-066X.32.3.187 · DOI
- Fisher, 1992, Reliability, separation, strata statistics, Rasch Measurement Transactions, 6, 238
- Linacre, 2005, Rasch dichotomous model vs. one-parameter logistic model, Rasch Measurement Transactions, 19, 1032
- Raîche, 2005, Critical eigenvalue sizes in standardized residual principal components analysis, Rasch Measurement Transactions, 19, 1012
Cited by
Sorted by citations
Items 201–230 of 230
Mattelin, E., Weyler, H., Andersson, R., Paulsen, J., Tielman, S., Vikgren, A., Bondjers, K., Serlachius, E., Mataix‐Cols, D., & Bragesjö, M. (2026) · medRxiv · doi:10.64898/2026.01.18.26343775
Macit, B., Lipp, A., Zlomuzica, A., Engler, H., Blackwell, S. E., Würtz, F., Margraf, J., & Woud, M. L. (2026) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2026.103131
Faurholt‐Jepsen, M., Dyreholt, M. S., Kyster, N. B., Iversen, N., Kortsen, E. M., Amann, B. L., Hogg, B., Gatto, D. L., de Córdoba, C., Petrovic, M., et al. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-115773
Online self-help intervention for procrastination: a randomized controlled trial
Zhou, Y. F., An, Z., Cao, Y., Liu, X., & Peng, H. (2026) · Psychology Health & Medicine · doi:10.1080/13548506.2026.2632842
Heltne, A., Kenter, R. M. F., Gulseth, R., & Nordgreen, T. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.100922
“While this is encouraging, it is important to note that specific measures of patient reported negative effects (e.g., Rozental et al., 2019) were not actively assessed in the current study.”
Boalt, A., Ghaderi, A., & Sahlin, H. (2026) · International Journal of Eating Disorders · doi:10.1002/eat.70065
Kanaan, R., Macdonell, R., Long, M., Richardson, R. J., Rogers, C., McManus, M. A., Braat, S., Zaloumis, S. G., Mihalopoulos, C., Chatterton, M. L., et al. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-112627
Client perspectives of internet-based treatment for depression in Arabic-speaking countries
El‐Haj‐Mohamad, R., Nohr, L., Stammel, N., Salloum, Z., Böhm, M., Nesterko, Y., Wagner, B., Böttche, M., & Knaevelsrud, C. (2026) · BMC Psychology · doi:10.1186/s40359-026-04463-z
“This underlines the importance of considering adverse effects during but also after treatment, for example, using check-lists for adverse effects and monitoring them during and after treatment (e.g., [48]).”
Niemi, M., Michalak, J., Velana, M., Barnhofer, T., & Harrer, M. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-106362
Janssen, W. C., Mérelle, S. Y. M., Gilissen, R., & Bockting, C. (2026) · Trials · doi:10.1186/s13063-026-09725-0
Córdoba-Delgado, M. A., Sierra, V. M., López, J. G. R., Ferreira, N. S. M., Meza, M. G. C., & González-Ballesteros, L. M. (2026) · doi:10.2196/preprints.99756
Bielinski, L. L., Arnold, S., Kojan, N., Staeger, L., Berger, T., Bisby, M. A., & Dear, B. F. (2026) · JMIR Research Protocols · doi:10.2196/preprints.102297
Mattelin, E., Weyler, H., Andersson, R., Paulsen, J., Tielman, S., Vikgren, A., Bondjers, K., Serlachius, E., Mataix‐Cols, D., & Bragesjö, M. (2026) · BMJ Open · doi:10.1136/bmjopen-2026-117024
Skoko, A., Hlynsson, J. I., Čihařová, M., Berger, T., Carlbring, P., Krieger, T., Bergström, J., Daele, T. V., van Klaveren, C., Cornelisz, I., et al. (2026) · doi:10.2196/preprints.103262
Self-citationMathiesen, A. S., Zoffmann, V., Lindschou, J., Jakobsen, J. C., Gluud, C., Olsen, M. H., Rasmussen, B., Marqvorsen, E. H. S., & Rothmann, M. J. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-111164
“…consequently deemed to be low, which may indicate that previous trials potentially overestimated the beneficial effects of GSD. 17 In addition, the included trials did not report any occurrence of adverse events or potential negative effects 18 or applied an attention control group as a comparison.”
Bei Risiken und Nebenwirkungen fragen Sie Kinder und Eltern
Pflug, V., & Bartnick, C. (2026) · KJP up2date · doi:10.1055/a-2589-6023
“Indikationsprü-fung vermindern das Risiko, Behandlungen einzuset-zen, die für bestimmte Kinder ungeeignet oder poten-ziell belastend sind.”
Beaudoin, M., Potvin, S., Giguère, S., Giguère, C., Aardema, F., Abdel‐Baki, A., Benedictis, L. D., Lipp, O., Lalonde, P., Stip, É., et al. (2026) · JMIR Mental Health · doi:10.2196/106791
“It includes 6 domains: symptoms, quality of the treatment, dependency on the treatment or therapist, stigma (ie, being perceived negatively by others because of the treatment), hopelessness, and lowered self-esteem [38,39].”
Jacobsen, C. F., Hjorthøj, C. R., Østergård, O. K., & Poulsen, S. (2026) · Trials · doi:10.1186/s13063-026-09901-2
“The NEQ has demonstrated reliability and acceptable item fit amongst a large sample of Swedish outpatients [58].”
Andersson, R., Åhlén, J., Lenhard, F., Ohlis, A., Wachtmeister, V., Karemyr, M., Högström, J., Bottai, M., Mataix‐Cols, D., Vigerland, S., et al. (2026) · BMJ Mental Health · doi:10.1136/bmjment-2026-302619
“Negative effects were assessed with the 20-item Negative Effects Questionnaire 37 with therapist-specific items omitted for self-guided participants.”
Lipinska, G., Boland, J., Seef, S. E., & Rink, J. (2026) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2026.1651928
“ANOVA yielded a non-signi fi cant result for differences in overall NEQ scores between groups, as shown in Table 1.”
InfluentialHolas, P., & Piasecka, A. (2026) · Psychiatria i Psychologia Kliniczna · doi:10.15557/pipk.2026.0034
Lindegaard, T., Berg, M., Käll, A., Lockman, J., Schönberg, E., Björklund, M., Frederick, R. J., & Andersson, G. (2026) · Psychodynamic Psychiatry · doi:10.1521/pdps.2026.54.3.511
Self-citationUnintended Consequences: A Study on the Adverse Effects of Psychological Counselling
Kansara, P. H. (2025) · Vidhyayana · doi:10.58213/2erm6f09
Lukka, L., Juvonen, J. J., Palva, S., Isometsä, E. T., & Palva, J. M. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.101000
“Treatment dependency is a recognized concern in face-to-face treatments (Rozental et al., 2019), but such problems have not typically been considered for DMHIs either with or without gaming elements (Fenski et al., 2021; Linardon et al., 2024).”
Skoko, A., Hlynsson, J. I., Čihařová, M., Berger, T., Carlbring, P., Krieger, T., Bergström, J., Daele, T. V., van Klaveren, C., Cornelisz, I., et al. (2026) · JMIR Research Protocols · doi:10.2196/103262
Self-citationBrüggmann, L., Kunas, S. L., Knaevelsrud, C., & Asselmann, E. (2026) · Mental Health & Prevention · doi:10.1016/j.mhp.2026.200576
Mapping negative experiences in psychotherapy: A network analysis
Bobei, T., Flückiger, C., & Podina, I. R. (2026) · Psychotherapy Research · doi:10.1080/10503307.2026.2743645
Hlynsson, J. I., Andersson, G., & Carlbring, P. (2026) · Internet Interventions · doi:10.1016/j.invent.2026.101012
Self-citationMason, L., Yarmey, V., Li, S., Muzamil, M., Kumar, S., Linardon, J., Sherry, S., Lacroix, É., Mills, J. S., Keshen, A., et al. (2026) · Body Image · doi:10.1016/j.bodyim.2026.102211
Ertl, R., & von Wirth, E. (2026) · Zeitschrift für Kinder- und Jugendpsychiatrie und Psychotherapie · doi:10.1024/1422-4917/a001104
Metadata
- Authors
- Rozental, A., Kottorp, A., Forsström, D., Månsson, K., Boettcher, J., Andersson, G., Furmark, T., & Carlbring, P.
- Affiliations
- Karolinska Institutet (SE)
- Malmö University (SE)
- Stockholm University (SE)
- Uppsala University (SE)
- Freie Universität Berlin (DE)
- Linköping University (SE)
- Corresponding author
- Rozental, Alexander
- ORCID
- Venue
- Behavioural and Cognitive Psychotherapy
- Publisher
- Cambridge University Press (CUP)
- Published
- 2019-03-15
- Volume
- 47
- Issue
- 5
- Pages
- 559-572
- ISSN
- 1352-4658 (ISSN-L)
- 1469-1833
- OpenAlex
- W2922529521
- Cited by
- 230
- References
- 46
- Citations by year
- 2026: 47 · 2025: 47 · 2024: 47 · 2023: 32 · 2022: 30 · 2021: 15 · 2020: 7 · 2019: 4
- FWCI
- 10.19
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Behavioural and Cognitive Psychotherapy (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Uppsala University Publications (Uppsala University) (submittedVersion, other-oa)
- Malmö University Publications (Malmö University) (submittedVersion)
- University of Southern Denmark Research Portal (University of Southern Denmark) (submittedVersion, cc-by)
- Funders
- University of Illinois at Urbana-Champaign
- University of Illinois at Chicago
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Keywords
- adverse effects, psychometric properties, Rasch analysis, differential item functioning, item response theory, short-form scale, treatment outcome monitoring, mental illness
- MeSH terms
- Negative Results, Adult, Mental Disorders, Female, Humans, Male, Psychology, Psychometrics, Surveys and Questionnaires, Reproducibility of Results, Treatment Failure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
