Further Investigating Method Effects Associated With Negatively Worded Items on Self-Report Surveys
The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Abstract
This article used multitrait-multimethod methodology and covariance modeling for an investigation of the presence and correlates of method effects associated with negatively worded items on the Rosenberg Self-Esteem (RSE) scale (Rosenberg, 1989) using a sample of 757 adults. Results showed that method effects associated with negative item phrasing on the RSE scale were present. Method effects associated with negative item wording were similarly observed with the Social Physique Anxiety Scale (SPAS; Hart, Leary, & Rejeski, 1989) and method effects were present and significantly correlated in analyses that included both the RSE scale and SPAS simultaneously. Path analysis modeling that incorporated personality measures identified factors that correlated with the presence of method effects. These findings further suggest that method effects associated with negatively worded items may be considered a response style.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
(1991) · Cited by 15,554
A new scale of social desirability independent of psychopathology.
Crowne, D. P., & Marlowe, D. (1960) · Journal of Consulting Psychology · doi:10.1037/h0047358 · Cited by 8,686
Scheier, M. F., Carver, C. S., & Bridges, M. W. (1994) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.67.6.1063 · Cited by 7,456
The standards for educational and psychological testing.
Eignor, D. R. (2013) · APA handbook of testing and assessment in psychology, Vol. 1: Test theory and testing and assessment in industrial and organizational psychology. · doi:10.1037/14047-013 · Cited by 6,988
Journal of social behavior and personality
(1986) · Personality and Individual Differences · doi:10.1016/0191-8869(86)90025-5 · Cited by 6,706
Reliability and Validity Assessment
Carmines, E., & Zeller, R. (1979) · doi:10.4135/9781412985642 · Cited by 5,884
Reliability and Validity Assessment
(2011) · The Encyclopedia of Political Science · doi:10.4135/9781608712434.n1341 · Cited by 5,040
Public and private self-consciousness: Assessment and theory.
Fenigstein, A., Scheier, M. F., & Buss, A. H. (1975) · Journal of Consulting and Clinical Psychology · doi:10.1037/h0076760 · Cited by 3,703
Cited by
The Self-Compassion Scale is a Valid and Theoretically Coherent Measure of Self-Compassion
Neff, K. D. (2016) · Mindfulness · doi:10.1007/s12671-015-0479-3 · Cited by 805
Sources of self-efficacy in mathematics: A validation study
Usher, E. L., & Pajares, F. (2009) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2008.09.002 · Cited by 664
Francis, A. W., Dawson, D. L., & Golijani-Moghaddam, N. (2016) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2016.05.003 · Cited by 496
Some Guidelines Concerning the Modeling of Traits and Abilities in Test Construction
Schweizer, K. (2010) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000001 · Cited by 448
School-Related Social Support and Students' Perceived Life Satisfaction
Danielsen, A. G., Samdal, O., Hetland, J., & Wold, B. (2009) · The Journal of Educational Research · doi:10.3200/joer.102.4.303-320 · Cited by 446
Method Effects, Measurement Error, and Substantive Conclusions
Lance, C. E., Dawson, B., Birkelbach, D., & Hoffman, B. J. (2010) · Organizational Research Methods · doi:10.1177/1094428109352528 · Cited by 355
Marsh, H. W., Scalas, L. F., & Nagengast, B. (2010) · Psychological Assessment · doi:10.1037/a0019225 · Cited by 348
An Empirical Comparison of Seven Populist Attitudes Scales
Castanho Silva, B., Jungkunz, S., Helbling, M., & Littvay, L. (2020) · Political Research Quarterly · doi:10.1177/1065912919833176 · Cited by 333
Research into Questionnaire Design: A Summary of the Literature
Lietz, P. (2010) · International Journal of Market Research · doi:10.2501/s147078530920120x · Cited by 313
Tourism students’ entrepreneurial intentions
Gurel, E., Altinay, L., & Daniele, R. (2010) · Annals of Tourism Research · doi:10.1016/j.annals.2009.12.003 · Cited by 290
Neff, K. D., Whittaker, T. A., & Karl, A. (2017) · Journal of Personality Assessment · doi:10.1080/00223891.2016.1269334 · Cited by 266
“Of course, fi ndings could have been due to a method effect, given that EFA is highly sensitive to positive versus negative affect (DiStefano & Motl, 2006).”
MethodBasic Psychological Need Satisfaction in Leisure Activities and Adolescents’ Life Satisfaction
Leversen, I., Danielsen, A. G., Birkeland, M. S., & Samdal, O. (2012) · Journal of Youth and Adolescence · doi:10.1007/s10964-012-9776-5 · Cited by 254
The connectedness to nature scale: A measure of emotional connection to nature?
Perrin, J. L., & Benassi, V. A. (2009) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2009.03.003 · Cited by 253
Guo, J., Marsh, H. W., Parker, P. D., Morin, A. J. S., & Yeung, A. S. (2015) · Learning and Individual Differences · doi:10.1016/j.lindif.2015.01.008 · Cited by 235
“Method effects associated with negative item wording have been reported in many studies (DiStefano & Motl, 2006; Marsh, 1986; Marsh & O'Mara, 2008;Marsh, Scalas, & Nagengast, 2010).”
BackgroundGawke, J. C., Gorgievski, M. J., & Bakker, A. B. (2019) · European Management Journal · doi:10.1016/j.emj.2019.03.001 · Cited by 202
Student Conceptions of Feedback: Impact on Self-Regulation, Self-Efficacy, and Academic Achievement
Brown, G. T. L., Peterson, E. R., & Yao, E. S. (2016) · British Journal of Educational Psychology · doi:10.1111/bjep.12126 · Cited by 198
Measuring the dark core of personality.
Moshagen, M., Zettler, I., & Hilbig, B. E. (2020) · Psychological Assessment · doi:10.1037/pas0000778 · Cited by 187
The Emotion Regulation Questionnaire: Validation of the ERQ-9 in two community samples.
Spaapen, D. L., Waters, F., Brummer, L., Stopa, L., & Bucks, R. S. (2014) · Psychological Assessment · doi:10.1037/a0034474 · Cited by 158
“The results showed that the original model obtained by Gross and John (2003) did not present an adequate fit for either community sample.”
ResultRogge, R. D., Daks, J. S., Dubler, B. A., & Saint, K. J. (2019) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2019.10.001 · Cited by 109
Dispositional contempt: a first look at the contemptuous person
Schriber, R. A., Chung, J. M., Sorensen, K. S., & Robins, R. W. (2017) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000101 · Cited by 96
“Yielding two factors that consist of positive versus negative items is a common finding in psychometric analyses of self-report measures (DiStefano & Motl, 2006; Marsh, 1996).”
BackgroundVowles, K. E., Sowden, G., & Ashworth, J. (2014) · Behavior Therapy · doi:10.1016/j.beth.2013.12.009 · Cited by 92
“This raises the possibility of a method effect within the analyses (cf. DiStefano & Motl, 2006), at least with regard to these subscales as the Pain Willingness subscale of the CPAQ is also negatively worded and did not load on this factor.”
MethodSchroeders, U., Kubera, F., & Gnambs, T. (2022) · Assessment · doi:10.1177/10731911211033894 · Cited by 76
“Such method effects for negatively keyed items are not specific for the TAS, but have repeatedly been reported for many self-report scales (e.g., Gnambs & Schroeders, 2020) and seem to capture a general test-taker’s response style (e.g., DiStefano & Motl, 2006).”
MethodValidation of the chronic pain acceptance questionnaire-8 in an Australian pain clinic sample
Baranoff, J., Hanrahan, S. J., Kapur, D., & Connor, J. P. (2014) · International Journal of Behavioral Medicine · doi:10.1007/s12529-012-9278-6 · Cited by 46
“For example, previous research has identified that an acquiescence bias may influence the emergence of separate factors of negatively keyed items during factor analysis [25].”
The job perception inventory: Considering human factors and needs in the design of human-AI work
Berretta, S., Tausch, A., Peifer, C., & Kluge, A. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1128945 · Cited by 35
“Thus, this factor is only a method factor (DiStefano and Motl, 2006) and has no significance for the content of the scale.”
MethodCharacteristics of clinically anxious versus non-anxious regular, heavy marijuana users
Van Dam, N. T., Bedi, G., & Earleywine, M. (2012) · Addictive Behaviors · doi:10.1016/j.addbeh.2012.05.021 · Cited by 26
Ventromedial prefrontal volume in adolescence predicts hyperactive/inattentive symptoms in adulthood
Albaugh, M. D., Ivanova, M., Chaarani, B., Orr, C., Allgaier, N., Althoff, R. R., D’ Alberto, N., Hudson, K., Mackey, S., Spechler, P. A., et al. (2019) · Cerebral Cortex · doi:10.1093/cercor/bhy066 · Cited by 24
Walach, H., Sauer, S., Kohls, N., Rose, N., & Schmidt, S. (2025) · BMC Psychology · doi:10.1186/s40359-025-03671-3 · Cited by 6
Metadata
- Authors
- DiStefano, C., & Motl, R. W.
- Venue
- Structural Equation Modeling: A Multidisciplinary Journal
- Publisher
- Informa UK Limited
- Published
- 2006-06-28
- Volume
- 13
- Issue
- 3
- Pages
- 440-464
- ISSN
- 1070-5511 (ISSN-L)
- 1532-8007
- OpenAlex
- W2077821296
- Cited by
- 467
- References
- 47
- FWCI
- 9.59
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.