The DOI matches its Crossref record; 93 of the 94 references it cites were found in a registry or an index.
Negative Effects of Psychological Treatments: An Exploratory Factor Analysis of the Negative Effects Questionnaire for Monitoring and Reporting Adverse and Unwanted Events
Open accessAbstract
Research conducted during the last decades has provided increasing evidence for the use of psychological treatments for a number of psychiatric disorders and somatic complaints. However, by focusing only on the positive outcomes, less attention has been given to the potential of negative effects. Despite indications of deterioration and other adverse and unwanted events during treatment, little is known about their occurrence and characteristics. Hence, in order to facilitate research of negative effects, a new instrument for monitoring and reporting their incidence and impact was developed using a consensus among researchers, self-reports by patients, and a literature review: the Negative Effects Questionnaire. Participants were recruited via a smartphone-delivered self-help treatment for social anxiety disorder and through the media (N = 653). An exploratory factor analysis was performed, resulting in a six-factor solution with 32 items, accounting for 57.64% of the variance. The derived factors were: symptoms, quality, dependency, stigma, hopelessness, and failure. Items related to unpleasant memories, stress, and anxiety were experienced by more than one-third of the participants. Further, increased or novel symptoms, as well as lack of quality in the treatment and therapeutic relationship rendered the highest self-reported negative impact. In addition, the findings were discussed in relation to prior research and other similar instruments of adverse and unwanted events, giving credence to the items that are included. The instrument is presently available in eleven different languages and can be freely downloaded and used from www.neqscale.com.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“On the one hand, this result fits literature that shows, that psychological interventions, online and offline, can initially evoke negative effects and symptom deteriorations (Ladwig et al., 2014; Rozental et al., 2016).”
From Feasibility of a Software agent providing a brief Intervention for Self-help to Uplift psychological wellbeing (“SISU”). A single-group pretest-posttest trial investigating the potential of SISU to act as therapeutic agent (2021) · cites this paper as Rozental et al., 2016
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Adverse events and harms during the RCT are measured with the Negative Experiences Questionnaire (NEQ), [61]”
From Transdiagnostic and tailored internet intervention to improve mental health among university students: Research protocol for a randomized controlled trial (2024) · cites this paper as [61]
The cited source says what the claim attributes to it.
In the citing paper
“The internal consistency for the full instrument is excellent (Cronbach's α = 0.95), and is scored on several dimensions; 1) “Did you experience this?” (yes/no), 2) “If yes – here is how negatively it affected me”, from 0 (“Not at all”) to 4 (“Extremely”), and 3) “Probably caused by” (“The treatment I received” or “Other circumstances”).”
From A study protocol of a randomized controlled study of internet-based cognitive behavioral therapy for adult attention deficit hyperactivity disorder (2023) · cites this paper as Rozental et al., 2016
The cited source says what the claim attributes to it.
In the citing paper
“Internal Consistency of the NEQ is reported to be α = 0.95 (90).”
The cited source says what the claim attributes to it.
In the citing paper
“Others have also implied that non-response, dropout, and interpersonal difficulties may be perceived as negative effects (Dimidjian 2010; Rozental 2016).”
From Psychotherapeutic treatments for depression in older adults (2024) · cites this paper as Rozental 2016
In the citing paper
“and trials have already begun (Miloff et al. 2019) to include and report results from broader measures of negative effects used elsewhere in psychotherapy research (Rozental et al. 2016).”
From Better, Virtually: the Past, Present, and Future of Virtual Reality Cognitive Behavior Therapy (2021) · cites this paper as Rozental et al. 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ryan, R. M., & Deci, E. L. (2000) · American Psychologist · doi:10.1037//0003-066x.55.1.68 · Cited by 54,599
An inventory for measuring depression
BECK, A. T. (1961) · Archives of General Psychiatry · doi:10.1001/archpsyc.1961.01710120031004 · Cited by 38,032
(1998) · The Journal of Clinical Psychiatry · Cited by 19,139
The Scree Test For The Number Of Factors
Cattell, R. B. (1966) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr0102_10 · Cited by 13,554
The application of electronic computers to factor analysis
Kaiser, H. F. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000116 · Cited by 12,459
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195
(1998) · PubMed · Cited by 10,216
Evaluating the use of exploratory factor analysis in psychological research.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.3.272 · Cited by 9,587
A Rationale and Test for the Number of Factors in Factor Analysis
Horn, J. L. (1965) · Psychometrika · doi:10.1007/bf02289447 · Cited by 8,965
Selective Cortical Undercutting As a Means of Modifying and Studying Frontal Lobe Function in Man
Scoville, W. B. (1949) · Journal of neurosurgery · doi:10.3171/jns.1949.6.1.0065 · Cited by 7,374
Research works printed but not linked: 2
- I Ladwig, 2014, What are the Risks and Side Effects to Psychotherapy?—Development of an Inventory for the Assessment of Negative Effects of Psychotherapy (INEP), Verhaltenstherapie, 24, 252
- S Moritz, 2015, No pain, no gain? Adverse effects of psychotherapy in obsessive-compulsive disorder and its relationship to treatment gains, J Obsess-Compuls Rel, 5, 61
Cited by
Sorted by citations
The Lancet Psychiatry Commission on psychological treatments research in tomorrow's science
Holmes, E. A., Ghaderi, A., Harmer, C. J., Ramchandani, P. G., Cuijpers, P., Morrison, A. P., Roiser, J. P., Bockting, C. L. H., O'Connor, R. C., Shafran, R., et al. (2018) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30513-8 · Cited by 624
Rozental, A., Kottorp, A., Forsström, D., Månsson, K., Boettcher, J., Andersson, G., Furmark, T., & Carlbring, P. (2019) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465819000018 · Cited by 230
“In relation to the factors obtained by Rozental et al. (2016), the first component is associated with symptoms, while the other is linked to four separate factors.”
BackgroundMethodResultInfluentialSelf-citationThe Return of the Repressed: The Persistent and Problematic Claims of Long-Forgotten Trauma
Otgaar, H., Howe, M. L., Patihis, L., Merckelbach, H. L. G. J., Lynn, S. J., Lilienfeld, S. O., & Loftus, E. F. (2019) · Perspectives on Psychological Science · doi:10.1177/1745691619862306 · Cited by 211
“, & Rubin, D. C. (2003). Confidence, not consistency, characterizes flashbulb memories.”
Lindner, P., Miloff, A., Hamilton, W., Reuterskiöld, L., Andersson, G., Powers, M. B., & Carlbring, P. (2017) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2017.1280843 · Cited by 187
“If VRET is to be considered for integration into regular health care settings alongside other psychological treatments, RCT quality needs to be improved, including the systematic reporting of negative effects (Rozental, Kottorp, Boettcher, Andersson, & Carlbring, 2016).”
BackgroundSelf-citationLindner, P., Miloff, A., Fagernäs, S., Andersen, J., Sigeman, M., Andersson, G., Furmark, T., & Carlbring, P. (2019) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.07.003 · Cited by 175
Self-citation
Metadata
- Authors
- Rozental, A., Kottorp, A., Boettcher, J., Andersson, G., & Carlbring, P.
- Affiliations
- Stockholm University (SE)
- Karolinska Institutet (SE)
- Freie Universität Berlin (DE)
- Linköping University (SE)
- Corresponding author
- Rozental, Alexander
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-06-22
- Volume
- 11
- Issue
- 6
- Pages
- e0157503
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4917117
- OpenAlex
- W2472395746
- Cited by
- 223
- References
- 94
- Citations by year
- 2026: 23 · 2025: 39 · 2024: 23 · 2023: 27 · 2022: 31 · 2021: 20 · 2020: 16 · 2019: 25 · 2018: 12 · 2017: 6 · 2016: 1
- FWCI
- 19.61
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DiVA (Linkoping University) (submittedVersion, other-oa)
- Zürcher Hochschule für Angewandte Wissenschaften digital collection (Zurich University of Applied Sciences) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Refubium (Universitätsbibliothek der Freien Universität Berlin) (submittedVersion, cc-by)
- ZHAW Digital Collection (cc-by)
- Funders
- FORTE (FORTE 2013-1107)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- adverse events, unwanted events, self-report instrument development, social anxiety disorder, symptom deterioration, treatment quality, therapeutic relationship, stigma, hopelessness, factor structure
- MeSH terms
- Adult, Mental Disorders, Demography, Factor Analysis, Statistical, Female, Humans, Male, Surveys and Questionnaires
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
