The DOI matches its Crossref record; 45 of the 46 references it cites were found in a registry or an index.
The Negative Effects Questionnaire: psychometric properties of an instrument for assessing negative effects in psychological treatments
Open accessAbstract
Background: Psychological treatments provide many benefits for patients with psychiatric disorders, but research also suggests that negative effects might occur from the interventions involved. The Negative Effects Questionnaire (NEQ) has previously been developed as a way of determining the occurrence and characteristics of such incidents, consisting of 32 items and six factors. However, the NEQ has yet to be examined using modern test theory, which could help to improve the understanding of how well the instrument works psychometrically. Aims: The current study investigated the reliability and validity of the NEQ from both a person and item perspective, establishing goodness-of-fit, item bias, and scale precision. Method: The NEQ was distributed to 564 patients in five clinical trials at post-treatment. Data were analysed using Rasch analysis, i.e. a modern test theory application. Results: (1) the NEQ exhibits fairness in testing across sociodemographics, (2) shows comparable validity for a final and condensed scale of 20 instead of 32 items, (3) uses a rating scale that advances monotonically in steps of 0 to 4, and (4) is suitable for monitoring negative effects on an item-level. Conclusions: The NEQ is proposed as a useful instrument for investigating negative effects in psychological treatments, and its newer shorter format could facilitate its use in clinical and research settings. However, further research is needed to explore the relationship between negative effects and treatment outcome, as well as to test it in more diverse patient populations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The 20-item version has demonstrated satisfactory psychometric properties [52].”
From Internet-Delivered Therapy for Parents With Health Anxiety by Proxy: Protocol for a Single-Case Experimental Design Study (2023) · cites this paper as [52]
The cited source says what the claim attributes to it.
In the citing paper
“The internal consistency for the full instrument is excellent (Cronbach's α = 0.95), and is scored on several dimensions; 1) “Did you experience this?” (yes/no), 2) “If yes – here is how negatively it affected me”, from 0 (“Not at all”) to 4 (“Extremely”), and 3) “Probably caused by” (“The treatment I received” or “Other circumstances”).”
From A study protocol of a randomized controlled study of internet-based cognitive behavioral therapy for adult attention deficit hyperactivity disorder (2023) · cites this paper as Rozental et al., 2019
The cited source says what the claim attributes to it.
In the citing paper
“The 20-item NEQ shows comparable validity to the original 32-item version.”
From Efficacy of an internet-based psychological intervention for problem gambling and gambling disorder: Study protocol for a randomized controlled trial (2021) · cites this paper as Rozental et al., 2019
In the citing paper
“Negative treatment effects are here assessed post-treatment using the negative effects questionnaire (NEQ), a tool with reliable and valid psychometrics.74”
From Development, evaluation and implementation of a digital behavioural health treatment for chronic pain: study protocol of the multiphase DAHLIA project (2022) · cites this paper as [74]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.10.1092 · Cited by 31,765 · cited 1×
“Several secondary outcome measures were also administered; the Patient Health Questionnaire – 9 items (PHQ-9; Löwe et al., 2004), the Generalized Anxiety Disorder – 7 items (GAD-7; Spitzer et al., 2006), and the Brunnsviken Brief Quality of Life Scale (BBQ; Lindner et al., 2016).”
MethodLovibond, P. F., & Lovibond, S. H. (1995) · Behaviour Research and Therapy · doi:10.1016/0005-7967(94)00075-u · Cited by 14,523 · cited 2×
“A similar study was conducted using Rasch analysis for the Depression, Anxiety and Stress Scales (Lovibond and Lovibond, 1995), suggesting that a number of items could be removed and that it was not supported as a general instrument for mental distress (Shea et al.”
MethodApplying the Rasch Model: Fundamental Measurement in the Human Sciences
Bond, T. G., & Fox, C. M. (2003) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.2003.tb01103.x · Cited by 4,490
The Efficacy of Cognitive Behavioral Therapy: A Review of Meta-analyses
Hofmann, S. G., Asnaani, A., Vonk, I. J. J., Sawyer, A. T., & Fang, A. (2012) · Cognitive Therapy and Research · doi:10.1007/s10608-012-9476-1 · Cited by 3,445 · cited 1×
“Research on the efficacy of evidencebased approaches, such as cognitive behaviour therapy (CBT), suggest that a large number of patients improve thanks to the interventions they receive (Hofmann et al., 2012).”
BackgroundA Rating Formulation for Ordered Response Categories
Andrich, D. (1978) · Psychometrika · doi:10.1007/bf02293814 · Cited by 2,984 · cited 1×
“This can be helpful for identifying abnormal responses that might warrant further development of the instrument (Andrich, 1978).”
BackgroundChi-Square Tests with One Degree of Freedom; Extensions of the Mantel- Haenszel Procedure
Mantel, N. (1963) · Journal of the American Statistical Association · doi:10.1080/01621459.1963.10500879 · Cited by 2,899 · cited 1×
“The magnitude of DIF was evaluated using the Mantel-Haenszel statistic for polytomous scales using log-odds estimators (Mantel, 1963).”
MethodSummated Rating Scale Construction
Spector, P. E. (1992) · Sage eBooks · doi:10.4135/9781412986038 · Cited by 2,173 · cited 1×
“This converted item measure is then applied to determine whether they are scored on a similar unidimensional construct, which is often viewed as crucial in terms of validity in both classical and modern test theory (Spector, 1992).”
MethodKrosnick, J. A. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.537 · Cited by 1,852 · cited 1×
“It is possible that the negative effects that were reported were affected by recall bias, primacy-recency effects, and social desirability (Krosnick, 1999), resulting in less valid responses.”
BackgroundLöwe, B., Kroenke, K., Herzog, W., & Gräfe, K. (2004) · Journal of Affective Disorders · doi:10.1016/s0165-0327(03)00198-8 · Cited by 1,558 · cited 2×
“Several secondary outcome measures were also administered; the Patient Health Questionnaire – 9 items (PHQ-9; Löwe et al., 2004), the Generalized Anxiety Disorder – 7 items (GAD-7; Spitzer et al., 2006), and the Brunnsviken Brief Quality of Life Scale (BBQ; Lindner et al., 2016).”
MethodThe psychotherapeutic process: A research handbook.
Greenberg, L. S., & Pinsof, W. M. (1986) · Cited by 1,386
Research works printed but not linked: 4
- 10.1037/0003-066X.32.3.187 · DOI
- Fisher, 1992, Reliability, separation, strata statistics, Rasch Measurement Transactions, 6, 238
- Linacre, 2005, Rasch dichotomous model vs. one-parameter logistic model, Rasch Measurement Transactions, 19, 1032
- Raîche, 2005, Critical eigenvalue sizes in standardized residual principal components analysis, Rasch Measurement Transactions, 19, 1012
Cited by
Sorted by citations
Defining and Measuring Meditation-Related Adverse Effects in Mindfulness-Based Programs
Britton, W. B., Lindahl, J. R., Cooper, D. J., Canby, N. K., & Palitsky, R. (2021) · Clinical Psychological Science · doi:10.1177/2167702621996340 · Cited by 278
“or worsening symptoms caused by psychotherapy when measured systematically with a questionnaire (42%– 51%; Moritz et al., 2015; Rozental et al., 2019; SchermulyHaupt et al., 2018).”
BackgroundInfluentialCurran, J., Parry, G. D., Hardy, G. E., Darling, J., Mason, A.-M., & Chambers, E. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00347 · Cited by 84
Rozental, A., Andersson, G., & Carlbring, P. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00589 · Cited by 74
Riva, G., Bernardelli, L., Browning, M. H. E. M., Castelnuovo, G., Cavedoni, S., Chirico, A., Cipresso, P., de Paula, D. M. B., Di Lernia, D., Fernández-Álvarez, J., et al. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.563319 · Cited by 72
“• Negative Effects Questionnaire (NEQ): The self-report measure consists of three parts for a total of 32 items (47).”
BackgroundThe Safety of Digital Mental Health Interventions: Systematic Review and Recommendations
Taher, R., Hsu, C.-W., Hampshire, C., Fialho, C., Heaysman, C., Stahl, D., Shergill, S., & Yiend, J. (2023) · JMIR Mental Health · doi:10.2196/47433 · Cited by 68
“Symptom deterioration was one of the main risks reported in this review, similar to face-to-face therapies [13,45].”
Background
Metadata
- Authors
- Rozental, A., Kottorp, A., Forsström, D., Månsson, K., Boettcher, J., Andersson, G., Furmark, T., & Carlbring, P.
- Affiliations
- Karolinska Institutet (SE)
- Malmö University (SE)
- Stockholm University (SE)
- Uppsala University (SE)
- Freie Universität Berlin (DE)
- Linköping University (SE)
- Corresponding author
- Rozental, Alexander
- ORCID
- Venue
- Behavioural and Cognitive Psychotherapy
- Publisher
- Cambridge University Press (CUP)
- Published
- 2019-03-15
- Volume
- 47
- Issue
- 5
- Pages
- 559-572
- ISSN
- 1352-4658 (ISSN-L)
- 1469-1833
- OpenAlex
- W2922529521
- Cited by
- 230
- References
- 46
- Citations by year
- 2026: 47 · 2025: 47 · 2024: 47 · 2023: 32 · 2022: 30 · 2021: 15 · 2020: 7 · 2019: 4
- FWCI
- 10.19
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Behavioural and Cognitive Psychotherapy (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Uppsala University Publications (Uppsala University) (submittedVersion, other-oa)
- Malmö University Publications (Malmö University) (submittedVersion)
- University of Southern Denmark Research Portal (University of Southern Denmark) (submittedVersion, cc-by)
- Funders
- University of Illinois at Urbana-Champaign
- University of Illinois at Chicago
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Keywords
- adverse effects, psychometric properties, Rasch analysis, differential item functioning, item response theory, short-form scale, treatment outcome monitoring, mental illness
- MeSH terms
- Negative Results, Adult, Mental Disorders, Female, Humans, Male, Psychology, Psychometrics, Surveys and Questionnaires, Reproducibility of Results, Treatment Failure
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
