The value of suppressor effects in explicating the construct validity of symptom measures.
Open accessAbstract
Suppressor effects are operating when the addition of a predictor increases the predictive power of another variable. We argue that suppressor effects can play a valuable role in explicating the construct validity of symptom measures by bringing into clearer focus opposing elements that are inherent--but largely hidden--in the measure's overall score. We illustrate this point using theoretically grounded, replicated suppressor effects that have emerged in analyses of the original Inventory of Depression and Anxiety Symptoms (IDAS; Watson et al., 2007) and its expanded 2nd version (IDAS-II; Watson et al., 2012). In Study 1, we demonstrate that the IDAS-II Appetite Gain and Appetite Loss scales contain both (a) a shared distress component that creates a positive correlation between them and (b) a specific symptom component that produces a natural negative association between them (i.e., people who recently have experienced decreased interest in food/loss of appetite are less likely to report a concomitant increase in appetite/weight). In Study 2, we establish that mania scales also contain 2 distinct elements-namely, high energy/positive emotionality and general distress/dysfunction-that oppose each another in many instances. In both studies, we obtained evidence of suppression effects that were highly robust across different types of respondents (e.g., clinical outpatients, community adults, college students) and using both self-report and interview-based measures. These replicable suppressor effects establish that many homogeneous, unidimensional symptom scales actually contain distinguishable components with distinct--at times, even antagonistic--properties.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The negative association may be explained by the presence of a suppression effect.
“This may be explained by the presence of a suppression effect [82], which occurs when an independent variable contributes significantly to the tested model despite weak initial association with the dependant variable, i.e. despite being a weak predictor on its own.”
From “It’s not you, it’s me”: identity disturbance as the main contributor to interpersonal problems in pathological narcissism (2023) · cites this paper as [82]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Detection of Postnatal Depression: Development of the 10-item Edinburgh Postnatal Depression Scale
Cox, J. L., Holden, J. M., & Sagovsky, R. (1987) · British Journal of Psychiatry · doi:10.1192/bjp.150.6.782 · Cited by 14,183 · see citing works
Kessler, R. C., & Üstün, T. B. (2004) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.168 · Cited by 4,771 · see citing works
Equivalence of the Mediation, Confounding and Suppression Effect
MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000) · Prevention Science · doi:10.1023/a:1026595011371 · Cited by 4,188 · see citing works
Comorbidity of anxiety and unipolar mood disorders
Mineka, S., Watson, D., & Clark, L. A. (1998) · Annual Review of Psychology · doi:10.1146/annurev.psych.49.1.377 · Cited by 1,573 · see citing works
Self-citationRethinking the mood and anxiety disorders: A quantitative hierarchical model for DSM-V.
Watson, D. (2005) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.114.4.522 · Cited by 1,332 · cited 3× · see citing works
“On the other hand, both scales also are indicators of psychopathology and, consequently, contain a shared component of general distress/negative affect (Ben-Porath & Tellegen, 2008; Lahey et al., 2012; Watson, 2005, 2009) that produces a positive correlation between them.”
BackgroundSelf-citationTwo replicable suppressor situations in personality research
Paulhus, D. L., Robins, R. W., Trzesniewski, K. H., & Tracy, J. L. (2004) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3902_7 · Cited by 631 · see citing works
Differentiating the mood and anxiety disorders: A quadripartite model
Watson, D. (2009) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.032408.153510 · Cited by 498 · see citing works
Self-citationWatson, D., O’Hara, M. W., Naragon-Gainey, K., Koffel, E., Chmielewski, M., Kotov, R., Stasik, S. M., & Ruggero, C. J. (2012) · Assessment · doi:10.1177/1073191112449857 · Cited by 470 · see citing works
Self-citationWatson, D., & Naragon-Gainey, K. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2009.11.002 · Cited by 349 · see citing works
Self-citationEvidence for response bias as a source of error variance in applied assessment.
McGrath, R. E., Mitchell, M., Kim, B. H., & Hough, L. (2010) · Psychological Bulletin · doi:10.1037/a0019216 · Cited by 282 · see citing works
Flagged first, then by citations
Research works printed but not linked: 27
- Barthlow DL Graham JR Ben-Porath YS Tellegen A McNulty JL 2002 The appropriateness of the MMPI-2 K correction Assessment 9 219 229 12216779
- Ben-Porath YS Tellegen A 2008 Minnesota Multiphasic Personality Inventory-2 Restructured Form (MMPI-2-RF): Manual for administration, scoring, and interpretation Minneapolis, MN University of Minnesota Press
- Blonigen DM Patrick CJ Douglas KS Poythress NG Skeem JL Lilienfeld SO Krueger RF 2010 Multimethod assessment of psychopathy in relation to factors of internalizing and externalizing from the Personality Assessment Inventory: The impact of method variance and suppressor effects Psychological Assessment 22 96 107 20230156 PMC2845919
- Borkenau P Ostendorf F 1992 Social desirability scales as moderator and suppressor variables European Journal of Personality 6 199 214
- Cohen J Cohen P 1975 Applied multiple correlation/regression analysis for the social sciences New York Wiley
- Eberhard-Gran M Eskild A Tambs K Opjordsmoen S Samuelsen SO 2001 Review of validation studies of the Edinburgh Postnatal Depression Scale Acta Psychiatrica Scandinavica 104 243 249 11722298
- Gamez W Kotov R Watson D 2010 The validity of self-report assessment of avoidance and distress Anxiety, Stress & Coping: An International Journal 23 87 99 19322706
- Gaylord-Harden NK Cunningham JA Holmbeck GN Grant KE 2010 Suppressor effects in coping research with African American adolescents from low-income communities Journal of Consulting and Clinical Psychology 78 843 855 20873890
- Ghiselli EE 1972 Comment on the use of moderator variables Journal of Applied Psychology 56 270
- Gruber J Mauss IB Tamir M 2011 A dark side of happiness? How, when, and why happiness is not always good Perspectives on Psychological Science 6 222 233 26168514
- Harmon-Jones E Harmon-Jones C 2010 On the relationship of trait PANAS positive activation and trait anger: Evidence of a suppressor relationship Journal of Research in Personality 44 120 123
- Hicks BM Patrick CJ 2006 Psychopathy and negative emotionality: Analyses of suppressor effects reveal distinct relations with emotional distress, fearfulness, and anger-hostility Journal of Abnormal Psychology 115 276 287 16737392 PMC2276566
- Hopwood CJ Morey LC 2008 Emotional problems suppress disorder/performance associations in adult ADHD assessment Journal of Psychopathology and Behavior Assessment 30 204 210
- Joiner TE Jr Walker RL Pettit JW Perez M Cukrowicz KC 2005 Evidence-based assessment of depression in adults Psychological Assessment 17 267 277 16262453
- Kotov R Gamez W Schmidt F Watson D 2010 Linking “big” personality traits to anxiety, depressive, and substance use disorders: A meta-analysis Psychological Bulletin, 2010 136 768 821 20804236
- Krueger RF Eaton NR Derringer J Markon KE Watson D Skodol AE 2011 Personality in DSM-5: Helping delineate personality disorder content and framing the metastructure Journal of Personality Assessment 93 325 331 22804671
- Lahey BB Applegate B Hakes JK Zald DH Hariri AR Rathouz PJ 2012 Is there a general factor of prevalent psychopathology during adulthood? Journal of Abnormal Psychology 121 971 977 22845652 PMC4134439
- Lynam DR Hoyle RH Newman JP 2006 The perils of partialling: Cautionary tales from aggression and psychopathy Assessment 13 328 341 16880283 PMC3152746
- McNemar Q 1945 The mode of operation of suppressant variables American Journal of Psychology 58 554 555
- Meehl PE Hathaway SR 1946 The K factor as a suppressor variable in the Minnesota Multiphasic Personality Inventory Journal of Applied Psychology 30 525 564 20282179
- Miller GA Chapman JP 2001 Misunderstanding analysis of covariance Journal of Abnormal Psychology 110 40 48 11261398
- Piedmont RL McCrae RR Riemann R Angleitner A 2000 On the invalidity of validity scales: Evidence from self-reports and observer ratings in volunteer samples Journal of Personality and Social Psychology 78 582 593 10743882
- Smith GT McCarthy DM Zapolski TCB 2009 On the value of homogeneous constructs for construct validation, theory testing, and the description of psychopathology Psychological Assessment 21 272 284 19719340 PMC2854033
- Watson D Clark LA 1984 Negative affectivity: The disposition to experience aversive emotional states Psychological Bulletin 96 465 490 6393179
- Watson D O’Hara MW Chmielewski M McDade-Montez EA Koffel E Naragon K Stuart S 2008 Further validation of the IDAS: Evidence of convergent, discriminant, criterion, and incremental validity Psychological Assessment 20 248 259 18778161
- Watson D O’Hara MW Simms LJ Kotov R Chmielewski M McDade-Montez E Stuart S 2007 Development and validation of the Inventory of Depression and Anxiety Symptoms (IDAS) Psychological Assessment 19 253 268 17845118
- Wiggins JS 1973 Personality and prediction: Principles of personality assessment New York Addison-Wesley
Cited by
Sorted by citations
Constructing validity: New developments in creating objective measuring instruments.
Clark, L. A., & Watson, D. (2019) · Psychological Assessment · doi:10.1037/pas0000626 · Cited by 1,470 · see citing works
Self-citationPersonality, Emotions, and the Emotional Disorders
Watson, D., & Naragon-Gainey, K. (2014) · Clinical Psychological Science · doi:10.1177/2167702614536162 · Cited by 242 · see citing works
Self-citationDevelopment and Validation of the Faceted Inventory of the Five-Factor Model (FI-FFM)
Watson, D., Nus, E., & Wu, K. D. (2019) · Assessment · doi:10.1177/1073191117711022 · Cited by 83 · see citing works
Self-citationBowes, S. M., Costello, T. H., Lee, C., McElroy-Heltzel, S., Davis, D. E., & Lilienfeld, S. O. (2022) · Personality and Social Psychology Bulletin · doi:10.1177/0146167221997619 · Cited by 87 · see citing works
Intolerance of uncertainty and insula activation during uncertain reward
Gorka, S. M., Nelson, B. D., Phan, K. L., & Shankman, S. A. (2016) · Cognitive, Affective, & Behavioral Neuroscience · doi:10.3758/s13415-016-0443-2 · Cited by 55 · see citing works
Wens, V., Bourguignon, M., Vander Ghinst, M., Mary, A., Marty, B., Coquelet, N., Naeije, G., Peigneux, P., Goldman, S., & De Tiège, X. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.05.081 · Cited by 51 · see citing works
Affective models of depression and anxiety: Extension to within-person processes in daily life
Naragon-Gainey, K. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.09.061 · Cited by 37 · see citing works
Nelson, B. D., Liu, H., Sarapas, C., & Shankman, S. A. (2016) · Journal of Experimental Psychopathology · doi:10.5127/jep.048115 · Cited by 32 · see citing works
Lieberman, L., Gorka, S. M., Funkhouser, C. J., Shankman, S. A., & Phan, K. L. (2017) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2017.04.002 · Cited by 30 · see citing works
Biberdzic, M., Tan, J., & Day, N. J. S. (2023) · Borderline Personality Disorder and Emotion Dysregulation · doi:10.1186/s40479-022-00209-6 · Cited by 12 · see citing works
“This may be explained by the presence of a suppression effect [82], which occurs when an independent variable contributes significantly to the tested model despite weak initial association with the dependant variable, i.”
Facet-Level Personality Relations of the Symptom Dimensions of the Tripartite Model
Khoo, S., Stanton, K., Clark, L. A., & Watson, D. (2020) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-019-09763-w · Cited by 7 · see citing works
Self-citation
- Authors
- Watson, D., Clark, L. A., Chmielewski, M., & Kotov, R.
- Affiliations
- University of Notre Dame (US)
- Southern Methodist University (US)
- Stony Brook University (US)
- Corresponding author
- Watson, David
- Venue
- Psychological Assessment
- Published
- 2013-09
- Volume
- 25
- Issue
- 3
- Pages
- 929-941
- ISSN
- 1040-3590 (ISSN-L)
- 1939-134X
- PMC
- PMC3780394
- OpenAlex
- W2332426242
- Cited by
- 176
- References
- 10
- Citations by year
- 2026: 26 · 2025: 14 · 2024: 18 · 2023: 13 · 2022: 22 · 2021: 26 · 2020: 14 · 2019: 11 · 2018: 8 · 2017: 7 · 2016: 5 · 2015: 6 · 2014: 5
- FWCI
- 7.29
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Psychological Assessment (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute of Mental Health (R01-MH068472)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- suppressor effect, construct validity, symptom measures, appetite loss, scale structure, unidimensional scale, depression, anxiety, bipolar disorder
- MeSH terms
- Adolescent, Adult, Aged, Aged, 80 and over, Anxiety, Feeding and Eating Disorders, Bipolar Disorder, Depression, Female, Humans, Interviews as Topic, Male, Middle Aged, Models, Theoretical, Psychiatric Status Rating Scales, Reproducibility of Results, Young Adult, Self Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
