The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Reversed item bias: An integrative model.
Open accessAbstract
In the recent methodological literature, various models have been proposed to account for the phenomenon that reversed items (defined as items for which respondents' scores have to be recoded in order to make the direction of keying consistent across all items) tend to lead to problematic responses. In this article we propose an integrative conceptualization of three important sources of reversed item method bias (acquiescence, careless responding, and confirmation bias) and specify a multisample confirmatory factor analysis model with 2 method factors to empirically test the hypothesized mechanisms, using explicit measures of acquiescence and carelessness and experimentally manipulated versions of a questionnaire that varies 3 item arrangements and the keying direction of the first item measuring the focal construct. We explain the mechanisms, review prior attempts to model reversed item bias, present our new model, and apply it to responses to a 4-item self-esteem scale (N = 306) and the 6-item Revised Life Orientation Test (N = 595). Based on the literature review and the empirical results, we formulate recommendations on how to use reversed items in questionnaires.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.5.879 · Cited by 82,194
Society and the Adolescent Self-Image
Rosenberg, M. (1965) · doi:10.1515/9781400876136 · Cited by 30,126
Review of Psychometric theory (2nd ed.).
SITGREAVES, R. (1979) · Contemporary Psychology: A Journal of Reviews · doi:10.1037/018882 · Cited by 11,425
Scheier, M. F., Carver, C. S., & Bridges, M. W. (1994) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.67.6.1063 · Cited by 7,456
Optimism, coping, and health: Assessment and implications of generalized outcome expectancies.
Scheier, M. F., & Carver, C. S. (1985) · Health Psychology · doi:10.1037/0278-6133.4.3.219 · Cited by 6,690 · cited 1×
“For the grouped-massed condition, we used two additional items (both from the original LOT scale; Scheier & Carver, 1985), either ‘I always look on the bright side of things’ for the regular item first condition (p4) or ‘Things never work out the way I want them to’ for the reversed item first…”
MethodThe Psychology of Survey Response
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000) · doi:10.1017/cbo9780511819322 · Cited by 4,734
Identifying careless responses in survey data.
Meade, A. W., & Craig, S. B. (2012) · Psychological Methods · doi:10.1037/a0028085 · Cited by 3,865
Measurement and Control of Response Bias
Paulhus, D. L. (1991) · Measures of Personality and Social Psychological Attitudes · doi:10.1016/b978-0-12-590241-0.50006-x · Cited by 3,598
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Response strategies for coping with the cognitive demands of attitude measures in surveys
Krosnick, J. A. (1991) · Applied Cognitive Psychology · doi:10.1002/acp.2350050305 · Cited by 2,460
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242
Questions and answers in attitude surveys: Experiments on question form, wording, and context
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90054-w · Cited by 1,713
Response Styles in Marketing Research: A Cross-National Investigation
Baumgartner, H., & Steenkamp, J.-B. E. M. (2001) · Journal of Marketing Research · doi:10.1509/jmkr.38.2.143.18840 · Cited by 1,255
Self-citationPositive and negative global self-esteem: A substantively meaningful distinction or artifactors?
Marsh, H. W. (1996) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.70.4.810 · Cited by 811
Weijters, B., Cabooter, E., & Schillewaert, N. (2010) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2010.02.004 · Cited by 801
Self-citationDo We Really Need Multiple-Item Measures in Service Research?
Drolet, A. L., & Morrison, D. G. (2001) · Journal of Service Research · doi:10.1177/109467050133001 · Cited by 736
Lifestyle of the Tight and Frugal: Theory and Measurement
Lastovicka, J. L., Bettencourt, L. A., Hughner, R. S., & Kuntze, R. J. (1999) · Journal of Consumer Research · doi:10.1086/209552 · Cited by 645
Factors Defined by Negatively Keyed Items: The Result of Careless Respondents?
Schmitt, N., & Stuits, D. M. (1985) · Applied Psychological Measurement · doi:10.1177/014662168500900405 · Cited by 535
Cole, D. A., Ciesla, J. A., & Steiger, J. H. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.4.381 · Cited by 448
Misresponse to Reversed and Negated Items in Surveys: A Review
Weijters, B., & Baumgartner, H. (2012) · Journal of Marketing Research · doi:10.1509/jmr.11.0368 · Cited by 421
Self-citationCareless Responding to Reverse-Worded Items: Implications for Confirmatory Factor Analysis
Woods, C. M. (2006) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-005-9004-7 · Cited by 400
Assessing Three Sources of Misresponse to Reversed Likert Items
Swain, S. D., Weathers, D., & Niedrich, R. W. (2008) · Journal of Marketing Research · doi:10.1509/jmkr.45.1.116 · Cited by 386
Measuring Extreme Response Style
Greenleaf, E. A. (1992) · Public Opinion Quarterly · doi:10.1086/269326 · Cited by 366
Modeling Acquiescence in Measurement Models for Two Balanced Sets of Items
Billiet, J. B., & McClendon, M. J. (2000) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0704_5 · Cited by 351
Marsh, H. W. (1986) · Developmental Psychology · doi:10.1037//0012-1649.22.1.37 · Cited by 320
Rosenberg's self‐esteem scale: Two factors or method effects
Tomas, J. M., & Oliver, A. (1999) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705519909540120 · Cited by 310
Linear Causal Modeling with Structural Equations
Mulaik, S. A. (2009) · Chapman & Hall/CRC eBooks · doi:10.1201/9781439800393 · Cited by 278
Random intercept item factor analysis.
Maydeu-Olivares, A., & Coffman, D. L. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.4.344 · Cited by 260
Identification of content and style: A two-dimensional interpretation of acquiescence.
Bentler, P. M., Jackson, D. N., & Messick, S. (1971) · Psychological Bulletin · doi:10.1037/h0031474 · Cited by 211
Schriesheim, C. A., Eisenbach, R. J., & Hill, K. D. (1991) · Educational and Psychological Measurement · doi:10.1177/0013164491511005 · Cited by 195
Wording Effects in Self-Esteem Scales: Methodological Artifact or Response Style?
Horan, P. M., DiStefano, C., & Motl, R. W. (2003) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem1003_6 · Cited by 184
WATSON, D. (1992) · Sociological Methods & Research · doi:10.1177/0049124192021001003 · Cited by 176
A comparison of four approaches to account for method effects in latent state–trait analyses.
Geiser, C., & Lockhart, G. (2012) · Psychological Methods · doi:10.1037/a0026977 · Cited by 167
Welkenhuysen-Gybels, J., Billiet, J., & Cambré, B. (2003) · Journal of Cross-Cultural Psychology · doi:10.1177/0022022103257070 · Cited by 158
Longitudinal Invariance of Self-Esteem and Method Effects Associated With Negatively Worded Items
Motl, R. W., & DiStefano, C. (2002) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1207/s15328007sem0904_6 · Cited by 147
DiStefano, C., & Motl, R. W. (2008) · Personality and Individual Differences · doi:10.1016/j.paid.2008.10.020 · Cited by 122
The Stability of Individual Response StylesPreprint
Weijters, B., Geuens, M., & Schillewaert, N. (2008) · RePEc: Research Papers in Economics · Cited by 122
Self-citationGeiser, C., Eid, M., & Nußbeck, F. W. (2008) · Psychological Methods · doi:10.1037/1082-989x.13.1.49 · Cited by 118
Directional Questions Direct Self-Conceptions
Kunda, Z., Fong, G. T., Sanitioso, R. B., & Reber, E. (1993) · Journal of Experimental Social Psychology · doi:10.1006/jesp.1993.1004 · Cited by 107
The Instability of Response Sets
Hui, C.-C. H., & Triandis, H. C. (1985) · Public Opinion Quarterly · doi:10.1086/268918 · Cited by 103
McPherson, J., & Mohr, P. B. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.1.120 · Cited by 70
Acquiescence—Measurement and Theory*
Martín, J. F. (1964) · British Journal of Social and Clinical Psychology · doi:10.1111/j.2044-8260.1964.tb00430.x · Cited by 51
Alessandri, G., Vecchione, M., Fagnani, C., Bentler, P. M., Barbaranelli, C., Medda, E., Nisticò, L., Stazi, M. A., & Caprara, G. V. (2010) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2010.510064 · Cited by 46
Harris, M. M., & Bladen, A. (1994) · Journal of Management · doi:10.1177/014920639402000409 · Cited by 33
Davies, M. F. (2003) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.85.4.736 · Cited by 29
The bipolarity of the Comrey personality scales: A confirmatory factor analysis
Levin, J., & Montag, I. (1989) · Personality and Individual Differences · doi:10.1016/0191-8869(89)90074-3 · Cited by 4
Cited by
Hulland, J., Baumgartner, H., & Smith, K. M. (2018) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-017-0532-y · Cited by 1,067
Self-citationValidity and Mechanical Turk: An assessment of exclusion methods and interactive experiments
Thomas, K. A., & Clifford, S. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.08.038 · Cited by 625
Ineffectiveness of Reverse Wording of Questionnaire Items: Let’s Learn from Cows in the Rain
Sonderen, E. V., Sanderman, R., & Coyne, J. C. (2013) · PLoS ONE · doi:10.1371/journal.pone.0068967 · Cited by 544
Body image, physical activity, and sport: A scoping review
Sabiston, C. M., Pila, E., Vani, M., & Thogersen-Ntoumani, C. (2019) · Psychology of Sport and Exercise · doi:10.1016/j.psychsport.2018.12.010 · Cited by 318
Kreitchmann, R. S., Abad, F. J., Ponsoda, V., Nieto, M. D., & Morillo, D. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02309 · Cited by 269
Metadata
- Authors
- Weijters, B., Baumgartner, H., & Schillewaert, N.
- Venue
- Psychological Methods
- Published
- 2013
- Volume
- 18
- Issue
- 3
- Pages
- 320-334
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2153894404
- Cited by
- 386
- References
- 46
- FWCI
- 16.77
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.