Ineffectiveness of Reverse Wording of Questionnaire Items: Let’s Learn from Cows in the Rain
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: We examined the effectiveness of reverse worded items as a means of reducing or preventing response bias. We first distinguished between several types of response bias that are often confused in literature. We next developed arguments why reversing items is probably never a good way to address response bias. We proposed testing whether reverse wording affects response bias with item-level data from the Multidimensional Fatigue Inventory (MFI-20), an instrument that contains reversed worded items. METHODS: With data from 700 respondents, we compared scores on items that were similar with respect either to content or to direction of wording. Psychometric properties of sets of these items worded in the same direction were compared with sets consisting of both straightforward and reversed worded items. RESULTS: We did not find evidence that ten reverse-worded items prevented response bias. Instead, the data suggest scores were contaminated by respondent inattention and confusion. CONCLUSIONS: Using twenty items, balanced for scoring direction, to assess fatigue did not prevent respondents from inattentive or acquiescent answering. Rather, fewer mistakes are made with a 10-item instrument with items posed in the same direction. Such a format is preferable for both epidemiological and clinical studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In this context, a study by Sonderen et al. (2013) showed that reverse-worded items may lead to inattention when responding to a questionnaire.”
From Development and Validation of a Theory-Based Questionnaire to Measure Different Types of Cognitive Load (2023) · cites this paper as Sonderen et al., 2013
The cited source says what the claim attributes to it.
In the citing paper
“The latter could indicate a measurement issue related to including both positively and negatively phrased items in a scale [111].”
From Validation of the compassionate engagement and action scales, compassion scale, and Sussex-Oxford compassion scales in a French-Canadian sample (2024) · cites this paper as [111]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116 · cited 2×
“We consider this a variety of the Multi-Trait-Multi-Method matrix discussed by Campbell & Fiske [29].”
MethodA three-component conceptualization of organizational commitment
Meyer, J. P., & Allen, N. J. (1991) · Human Resource Management Review · doi:10.1016/1053-4822(91)90011-z · Cited by 11,049 · cited 2×
“1) Meyer and Allen’s [11] scale to assess affective commitment, of which Merritt [12] demonstrated that the answers on reversed items were prone to careless responding and cognitive fatigue.”
BackgroundDevelopment and validation of the Penn State Worry Questionnaire
Meyer, T. J., Miller, M. L., Metzger, R. L., & Borkovec, T. D. (1990) · Behaviour Research and Therapy · doi:10.1016/0005-7967(90)90135-6 · Cited by 5,099 · cited 1×
“’s [14] scale to assess worry contains 11 positively and five reversed worded items, leading Fresco et al.”
BackgroundThe Psychology of Survey Response
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000) · doi:10.1017/cbo9780511819322 · Cited by 4,734 · cited 2×
“The process of obtaining survey data is complex, with many possibilities of discrepancies arising between the state or opinion the researcher wants to elicit and the answer given by the respondent [3].”
BackgroundSmets, E. M. A., Garssen, B., Bonke, B., & De Haes, J. C. J. M. (1995) · Journal of Psychosomatic Research · doi:10.1016/0022-3999(94)00125-o · Cited by 3,613 · cited 2×
“[28] constructed a version with seven categories, but in accordance with their adaptation in 1996 [27], a version is used with five answer categories, ranging from 1 (‘yes, that is true’) to 5 (‘no, that is not true’).”
BackgroundMethodMeasurement and Control of Response Bias
Paulhus, D. L. (1991) · Measures of Personality and Social Psychological Attitudes · doi:10.1016/b978-0-12-590241-0.50006-x · Cited by 3,598
Krosnick, J. A. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.537 · Cited by 1,852 · cited 1×
“Krosnick [7] mentions a satisficing response style, whereby the respondent deliberately makes less effort to understand all subtleties of the question.”
On the bipolarity of positive and negative affect.
Russell, J. A., & Carroll, J. M. (1999) · Psychological Bulletin · doi:10.1037/0033-2909.125.1.3 · Cited by 1,481 · cited 1×
“More researchers attribute the claim of multidimensionality that is often made in scales with reversed worded items, not to conceptual differences, but to artificial factors resulting from response bias [20–25].”
BackgroundResponse Styles in Marketing Research: A Cross-National Investigation
Baumgartner, H., & Steenkamp, J.-B. E. M. (2001) · Journal of Marketing Research · doi:10.1509/jmkr.38.2.143.18840 · Cited by 1,255
Do We Really Need Multiple-Item Measures in Service Research?
Drolet, A. L., & Morrison, D. G. (2001) · Journal of Service Research · doi:10.1177/109467050133001 · Cited by 736 · cited 2×
“As Drolet and Morrison [9] demonstrate, respondents sometimes just answer the first item and assume that this answer also holds for subsequent (considered similar or even identical) items.”
Background
Cited by
Items 301–400 of 543
Effectiveness of the Concept Map in Nursing Education; Developing a Tool for Student Opinions
Tarım, S. L., Boy, Y., & ŞANLITÜRK, D. (2022) · Hospital Practices and Research · doi:10.34172/hpr.2022.14 · Cited by 9
Silva, A., Antunes, R., Monteiro, D., Jacinto, M., Matos, R. M. N., & Rodrigues, F. F. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20043352 · Cited by 7
“While the original version has 10 items, we only evaluated five items that were positively coded (item example, “I believe I have several good traits”) because reversecoded items can contaminate results [51].”
MethodEvaluating the psychometric properties of the Chronic Time Pressure Inventory using Rasch analysis
Denovan, A., Dagnall, N., Drinkwater, K. G., & Escolà‐Gascón, Á. (2023) · PeerJ · doi:10.7717/peerj.15218 · Cited by 6
Gaming on the Go? Translation and Validation of the Video Game Demand Scale to Korean
Lee, H., Lee, Y. E., Bowman, N. D., Yao, S., & Chen, S.-C. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2260980 · Cited by 6
“…two items (both opposite-valence items also problematic in prior samples of Chinese-and Turkish-speak-ing respondents) were removed, further demonstrating documented concerns with the reverse wording of items in survey research (see Boley et al., 2021; Kulas et al., 2019; Sonderen et al., 2013).”
Hsu, J. L., Clark, N., Hill, K., & Rowland‐Goldsmith, M. (2023) · CBE—Life Sciences Education · doi:10.1187/cbe.22-12-0249 · Cited by 5
“For instance, negatively worded multiple choice questions tend to reduce student performance and can lead to more confusion on both assessments and surveys (Johnson et al. , 2004; Roszkowski and Soven, 2010; Sonderen et al. , 2013; Chiavaroli, 2019).”
BackgroundGilbert, J. B., Zhang, L., Ulitzsch, E., & Domingue, B. W. (2025) · Behavior Research Methods · doi:10.3758/s13428-025-02625-2 · Cited by 12
“With the exception of three items that include explicit negation (i.e., use of the word “not”), all negatively framed items are reverse worded such that a higher item score indicates a lower level of the latent trait, analogous to the preceding example (Sonderen, Sanderman, & Coyne, 2013).”
Dillman's Graphic Language and Negative Worded Statement in Work Related Attitude Measurement
Jamadin, N., & Noordin, F. (2016) · Procedia - Social and Behavioral Sciences · doi:10.1016/j.sbspro.2016.05.046 · Cited by 5
Kim, S., & Yip, W. C. (2018) · IJASS(International Journal of Applied Sports Sciences) · doi:10.24985/ijass.2018.30.2.160 · Cited by 8
Organizational Identification at a Multinational Company
Visscher, I., & Beerkens, R. (2020) · Journal of Intercultural Management · doi:10.2478/joim-2020-0045 · Cited by 5
Porro, B., Petit, A., Bourbouloux, E., Colombat, P., Onfroy, M. L. B., Fassier, J., & Roquelaure, Y. (2021) · Bulletin du Cancer · doi:10.1016/j.bulcan.2021.06.016 · Cited by 5
Dupont, K., Gervai, J., Danis, I., Tóth, I., Koren, R., & Török, S. (2022) · Journal of Personality Assessment · doi:10.1080/00223891.2021.2019054 · Cited by 10
“In most cases, the use of reversed items on Likert-type scales is recommended to avoid acquiescence, response, or agreement bias (Horan et al., 2003; Sonderen et al., 2013; Zhang et al., 2016).”
BackgroundAre the Items of the Starkstein Apathy Scale Fit for the Purpose of Measuring Apathy Post-stroke?
Hum, S., Fellows, L. K., Lourenço, C. B., & Mayo, N. E. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.754103 · Cited by 7
“Using words with opposite meaning has been discussed as a means to control for acquiescence and social desirability bias(33) but also may have the effect of artificially increasing the reliability of the SAS.”
BackgroundMaternal Evaluation of a Team-Based Maternity Care Model for Women of Low Obstetric Risk
Perrella, S. L., Miraudo, J., Rea, A., Geddes, D. T., & Prosser, S. A. (2022) · Journal of Patient Experience · doi:10.1177/23743735221092606 · Cited by 8
“Reverse-scored questions in the original survey were reworded to positively worded questions, as pilot feedback suggested these were confusing, and the reverse-scoring method is ineffective in reducing response bias (18).”
MethodInstrument for evaluating digital educational comic
Zaibon, S. B., Azman, F. N., & Shiratuddin, N. (2019) · International Journal of Advanced Computer Research · doi:10.19101/ijacr.pid107 · Cited by 5
“The instrument design does not employ reverseworded statements based on the evidences that negatively-worded items do not prevent acquiescence response bias [26, 27].”
BackgroundKampf, P. H., Hernández, A., & González‐Romá, V. (2020) · Revista Psicologia Organizações e Trabalho · doi:10.17652/rpot/2020.4.10 · Cited by 9
Bayley, S. H., Wole, D., Ramchandani, P. G., Rose, P. M., Woldehanna, T., & Yorke, L. (2021) · doi:10.35489/bsg-rise-wp_2021/082 · Cited by 8
Kumar, A., Dutta, S. K., Samara, G., & Deep, S. (2025) · Equality Diversity and Inclusion An International Journal · doi:10.1108/edi-05-2025-0321 · Cited by 5
Beyond Risk Reduction: Vigilant Trust in Artificial Intelligence Based on Evidence from China
Ding, W., Wu, Y., & Wang, J. (2026) · Behavioral Sciences · doi:10.3390/bs16010095 · Cited by 6
“This pattern is consistent with prior research indicating that reverse-coded items often introduce methodological noise and attenuate internal consistency, particularly at short scales (Podsakoff et al., 2003; Schmitt & Stults, 1985; van Sonderen et al., 2013).”
Lin, X., Cui, H., & Zhu, X. (2026) · Journal of Special Education Technology · doi:10.1177/01626434261435251 · Cited by 5
“Although the original TAM includes reverse-worded statements, negatively phrased items were intentionally excluded to reduce potential method effects associated with reverse wording in translated survey instruments (Sonderen et al., 2013).”
A Comparison of Four Commonly Used Scales for Measuring Adolescent Life Satisfaction
Jovanović, V. (2024) · International Journal of Applied Positive Psychology · doi:10.1007/s41042-024-00162-5 · Cited by 5
“Negatively worded items are likely to cause difficulties in under - standing (van Sonderen et al., 2013) and the exclusive use of positively worded items is known to increase reliability and improve the factor structure (Dodeen, 2023).”
Melnikov, S., Fridman, S., Aboav, A., Moore, F., & Cohen, Y. (2022) · Journal of Nursing Management · doi:10.1111/jonm.13629 · Cited by 3
“According to van Sonderen et al. (2013) and Zhang et al. (2016), the reversed worded items may not introduce the conceptual differences, at the same time, thereby introducing a response bias that poses a threat to the validity of the scale (van Sonderen et al., 2013; Zhang et al., 2016).”
BackgroundRivaldo, I., Sutrisno, H., & Manik, A. C. (2022) · Jurnal Pendidikan Kimia Indonesia · doi:10.23887/jpk.v6i1.45338 · Cited by 6
“It may be due to the tendency of respondents to fill in inconsistently when the instrument consists of negative and positive items, and respondents are less careful in understanding the difference in meaning between the two types of statement items (Colosi, 2005; Roszkowski & Soven, 2010; Sonderen et al., 2013).”
BackgroundRyland, H., Cook, J. A., Ciobanasu, O., Oluwabamise, O., Cornish, R., Al-Taiar, H., Chris-Okoro, E., Hoggart, C., Vallakalil, A., Fitzpatrick, R. M., et al. (2022) · Psychology Crime and Law · doi:10.1080/1068316x.2022.2076855 · Cited by 4
“Patients ’ comments were mostly supportive of the tool ’ s usefulness and highlighted the positive framing of the questions as appealing (Sonderen et al., 2013).”
BackgroundViolence Prevention Climate and Health-Oriented Leadership in German Emergency Departments
Guliani, M., Reißmann, S., Westenhöfer, J., Harth, V., & Mache, S. (2023) · Healthcare · doi:10.3390/healthcare11162234 · Cited by 6
Pinning down state authenticity: defining and validating a state authenticity measure
Horton, C. B., Bailey, E. R., & Iyengar, S. S. (2024) · Self and Identity · doi:10.1080/15298868.2024.2434750 · Cited by 7
“…composite reliability score shared by the factors ( CR =.86), we recommend the factors be aggregated by future authenticity researchers (Brown, 2006); nevertheless, given the reverse-coding, it is important to present the two factors to participants in separate matrices (Sonderen et al., 2013).”
Mulyani, S., Mitchell, G., Carter, G., Woods, B. T., Warsini, S., Saifullah, A. D., Ulayya, A. I., & Wilson, C. B. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07332-2 · Cited by 9
“Regarding reversed items, confusion is a response style that varies based on the difficulties of the items and the cognitive skills required to respond accurately to the situation [84].”
Pavan, G. N., Roselli, L. F., & D’Souza-Li, L. F. R. (2020) · Jornal de Pediatria · doi:10.1016/j.jped.2019.11.008 · Cited by 4
Rodríguez‐Meirinhos, A., & Suárez, L. A. (2020) · Universitas Psychologica · doi:10.11144/javeriana.upsy19.esem · Cited by 11
Turkish Adaptation Study of the Game User Experience Satisfaction Scale: GUESS-TR
Berkman, M. İ., Bostan, B., & Şenyer, S. (2021) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2021.1987679 · Cited by 7
“Therefore, researchers suggest avoiding negative wording as much as possible (DeVellis, 2016, p. 128; Van Sonderen et al., 2013).”
BackgroundPaek, H., Oh, H. J., & Hove, T. B. (2024) · Risk Analysis · doi:10.1111/risa.14270 · Cited by 7
“Although reverse coding has been recommended as a way to prevent response bias, at least one study has raised doubts about its efficacy in doing so (van Sonderen et al., 2013).”
The Effect of Unpreparedness for Immigration Court on Psychopathology
Bailey, C. A., Venta, A. C., Crosby, J. W., Varela, J. G., & Boccaccini, M. T. (2018) · Journal of International Migration and Integration / Revue de l integration et de la migration internationale · doi:10.1007/s12134-018-0614-9 · Cited by 4
Prices, Engel Curves, and Time-Space Deflation: Impacts on Poverty and Inequality in Vietnam
Gibson, J. K., Le, T., & Kim, B. (2017) · World Bank, Washington, DC eBooks · doi:10.1596/1813-9450-8097 · Cited by 7
Winston, B. E. (2020) · Preprints.org · doi:10.20944/preprints202003.0222.v1 · Cited by 4
“Although van Sonderen, Sanderman, and Coyne (2013) argue that scales should not use reversed wording, I used the eight-item short form in this current study due to Dawley et al.'s success with the scale.”
BackgroundMethodLaw…What is it Good For? How People see the Law, Lawyers and Courts in Australia
Balmer, N. J., Pleasence, P. T., Hagland, T., & McRae, C. (2019) · UCL Discovery (University College London) · Cited by 4
Posis, A. I. B., Yarish, N. M., Ryu, R. H., Tindle, H. A., Michael, Y. L., Wactawski‐Wende, J., & Bellettiere, J. (2023) · Journal of Women s Health · doi:10.1089/jwh.2023.0056 · Cited by 3
Psychosocial correlates of mental health of university students in Hong Kong under COVID-19
Shek, D. T. L., Chai, W., Dou, D., Li, X., & Chan, C. H. M. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1294026 · Cited by 6
Kitamura, N., Kitade, N., Yokoyama, R., Ojima, T., Shimoyama, K., Tanabe, K., Shimada, Y., Akemoto, Y., Doki, Y., & Tsuchiya, T. (2025) · Surgery Today · doi:10.1007/s00595-025-03023-z · Cited by 3
“Although questions were formulated neutrally and clearly, this study did not examine the appropriateness of the question wording [22].”
Geng, X., & Han, J. (2025) · International Journal of Educational Research · doi:10.1016/j.ijer.2025.102777 · Cited by 4
Ertuna, L., Uyanık, G. K., & Gençaslan, D. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1815225 · Cited by 4
Yeoh, T. T., Kumwenda, B., & Chew, L. S. T. (2023) · International Journal of Pharmacy Practice · doi:10.1093/ijpp/riac105 · Cited by 4
Katsanos, C., Melissourgos, G., & Tselios, N. K. (2023) · Pan-Hellenic Conference on Progress in Computing and Informatics (PCI) · doi:10.1145/3635059.3635075 · Cited by 3
Carpentier, S. T., Deng, W., Blackwood, J., & de Winckel, A. V. (2024) · BMC Musculoskeletal Disorders · doi:10.1186/s12891-024-07893-1 · Cited by 8
“Combining positively and negatively worded items in the same scale may result in measurement inaccuracies, which could have been why those items were misfitting [28, 29].”
BackgroundMelson, E., Bridle, C., & Markham, W. A. (2017) · Health Education · doi:10.1108/he-02-2017-0013 · Cited by 4
User Engagement Using an Etextbook: A Descriptive Study
Verkuyl, M. A., Atack, L., Lapum, J. L., Hughes, M. C., St-Amant, O., & Petrie, P. (2021) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000000720 · Cited by 3
“We modified three negatively worded items to make them positive statements, as the team has previously found students became confused with negative-worded items, and the literature supports this modification.(17) We modified the wording of one item, “The application X was aesthetically appealing” to “I”
Examining the Psychometric Properties of the Chemistry Self-Concept Inventory Using Rasch Modeling
Werner, S. M., Chen, Y., & Stieff, M. (2021) · Journal of Chemical Education · doi:10.1021/acs.jchemed.1c00436 · Cited by 6
Chang, C.-T., Shaw, B. R., Solin, J., Gorby, T. A., & Robinson, P. (2023) · Journal of Forestry · doi:10.1093/jofore/fvad027 · Cited by 3
Measuring organizational play in small businesses
Mukerjee, J., Thurik, R., Torrès, O., & Wismans, A. (2023) · Management Review Quarterly · doi:10.1007/s11301-023-00363-3 · Cited by 4
“Past research shows that respondents’ difficulty in interpreting items increase with reverse items, especially so if the described state is not in accordance with the respondent’s actual state (Swain et al. 2008; Sonderen et al. 2013).”
Measuring Self-Esteem with Expanded Format in a Fraction of Time: ESE-S and ESE-US
Kam, C. C. S., Cheng, E. H., & Cui, T. (2023) · Journal of Personality Assessment · doi:10.1080/00223891.2023.2259990 · Cited by 4
“Second, the presentation order of regular-and reverse-keyed items may puzzle respondents, causing them to misunderstand the intended meaning of statements and provide incorrect replies (Sonderen et al., 2013; X. Zhang et al., 2016).”
BackgroundResultInfluentialBlötner, C., & Grüning, D. J. (2023) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000805 · Cited by 9
Portuguese adaptation of the Behavioral Inhibition Questionnaire (BIQ)
Fernandes, C., Martins, A. T., & Faísca, L. (2024) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-023-10113-0 · Cited by 4
“To account for the unexplained variability in participants’ responses, an orthogonal method factor was introduced to capture the impact of semantically reversed items in the BIQ (van Sonderen et al., 2013).”
Contribution of the Pilot Study for Methodological Enrichment of the Main Study
Shrestha, T., Bhandari, N., & Siluwal, N. (2024) · Journal of Health and Allied Sciences · doi:10.37107/jhas398 · Cited by 3
Murphy, M., Rio, E., Scholes, M. J., Jones, D., Pazzinatto, M. F., Johnston, R. T., Coburn, S., & Kemp, J. L. (2024) · Musculoskeletal Science and Practice · doi:10.1016/j.msksp.2024.103200 · Cited by 3
Lau, S. T., López, V., Liaw, S. Y., & Lau, Y. (2019) · Health & Social Care in the Community · doi:10.1111/hsc.12809 · Cited by 3
Teams of rural physicians matter: Testing a framework of team effectiveness
Orrantia, E., Kline, T. J. B., & Nutbrown, L. (2022) · Canadian Family Physician · doi:10.46747/cfp.6804280 · Cited by 5
Burke, A. (2015) · OhioLink ETD Center (Ohio Library and Information Network) · Cited by 8
The “About Me” Questionnaire: Factorial Structure and Measurement Invariance
Maras, P., Thompson, T., Gridley, N., & Moon, A. (2016) · Journal of Psychoeducational Assessment · doi:10.1177/0734282916682909 · Cited by 6
Pengaruh E-Servicescape terhadap Trust dan Dampaknya pada Repurchase Intention
Oebit, Z. (2018) · Jurnal Sistem Informasi · doi:10.21609/jsi.v14i2.698 · Cited by 20
Casanova, M. P., Nelson, M. C., & Baker, R. T. (2020) · Cogent Social Sciences · doi:10.1080/23311886.2020.1761509 · Cited by 4
“1002/jts.22203 Van Sonderen, E., Sanderman, R., Coyne, J. C., & Baradaran, H. R. (2013).”
BackgroundRitchwood, T. D., Massa, C., Kamanga, G., Pettifor, A. E., Hoffman, I., & Corneli, A. L. (2020) · AIDS Education and Prevention · doi:10.1521/aeap.2020.32.3.260 · Cited by 3
Vilca, L. W., Echabaudes-Ilizarbe, R. I., Aquino-Hidalgo, J. M., Ventura‐León, J., Martinez‐Munive, R., & White, M. (2020) · Psychological Reports · doi:10.1177/0033294120981930 · Cited by 4
Tanaś, Ł. (2021) · Psychological Test Adaptation and Development · doi:10.1027/2698-1866/a000007 · Cited by 5
“The scale includes only positively worded items, which prevents some methodological artifacts related to wording ambiguity (van Sonderen et al., 2013).”
BackgroundIt takes two to tango : an inquiry into healthcare professional education environments
Palmgren, P. J. (2016) · Figshare · Cited by 13
Pollet, T. V., Saxton, T. K., & mitchell, M. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/eyknz · Cited by 3
Development and Validation of the New Helping Attitude ScalePreprint
Trzeciak, S., Mazzarelli, A. J., Roberts, M., & Roberts, B. W. (2022) · doi:10.31234/osf.io/ra78j · Cited by 3
Obeïd, S. (2023) · Research Square · doi:10.21203/rs.3.rs-3425029/v1 · Cited by 3
Wilmore, T., Kriletic, A., Svyantek, D. J., & Donnelly, L. I. (2024) · Management Decision · doi:10.1108/md-03-2023-0370 · Cited by 2
Ibasco, G. C., Ahmed, S., Cai, M., & Chib, A. I. (2024) · International Journal of Intercultural Relations · doi:10.1016/j.ijintrel.2024.102004 · Cited by 3
Zhao, J.-L., Jia, R., Shields, J., Wu, Y.-J., & Huang, W.-W. (2025) · Archives of Sexual Behavior · doi:10.1007/s10508-025-03143-0 · Cited by 5
Improving the Measurement of the Big Five via Alternative Formats for the BFI-2
Zhang, X., Huang, M., Sun, J., & Savalei, V. (2025) · Journal of Personality Assessment · doi:10.1080/00223891.2025.2531187 · Cited by 2
“The low reliabilities—especially for some negatively worded items—for the Likert formats are consistent with past research that scales that use the Likert format often have a few negatively worded items with low correlations with other items in the same scale (e.g., Sonderen et al., 2013).”
Gentès, M., Ramachandran, R. P., Mupondwa, E., Ross, K. A., Koutchma, T., & Shi, J. (2025) · Innovative Food Science & Emerging Technologies · doi:10.1016/j.ifset.2025.104207 · Cited by 6
Niknampour, H. (2025) · Journal of Research in Music Education · doi:10.1177/00224294251374529 · Cited by 2
“Second, on value-laden topics, such as DEI policy, reverse-coded statements can feel manipulative, depress trust, and provoke inconsistent answers, ultimately lowering data quality (Swain et al., 2008; van Sonderen et al., 2013).”
Harper, A., Gray, T. L., & Sahlberg, P. (2025) · Journal of Outdoor and Environmental Education · doi:10.1007/s42322-025-00226-3 · Cited by 4
“This aligned with evidence that reverse − worded items may negatively impact unidimensionality and introduce unwanted variance (Suárez-Álvarez, et al., 2018), and may be ineffective in mitigating response bias caused by inattention or confusion (van Sonderen, et al., 2013).”
Ingeman, K., Carstensen, T. B. W., Smith, A. B., Simard, S., Frostholm, L., Ørnbøl, E., Wellnitz, K. B., & Lyhne, J. D. (2026) · Journal of Patient-Reported Outcomes · doi:10.1186/s41687-026-01051-y · Cited by 2
“Psychometric literature suggests such items can reduce model fit rather than reduce bias and that they increase the likelihood of participants making mistakes [35].”
Liebherr, M., Yankouskaya, A., Alshakhsi, S. A. A., Montag, C., & Ali, R. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01781-4 · Cited by 2
“The reverse-coded item RD5 ( I use it solely as a tool , not to express my feelings or expect it to understand me ) consistently showed weaker factor loadings across all three validations, a pattern well-documented in the psychometric literature on reverse-coded items [45, 46].”
Examining the questionnaire on attitudes consistent with sexual offending
Gray, E. (2019) · Journal of Applied Research in Intellectual Disabilities · doi:10.1111/jar.12663 · Cited by 3
A Critique of Likert-Response Items in Social Science Research
Winston, B. E. (2021) · Advances in human resources management and organizational development book series · doi:10.4018/978-1-7998-7665-6.ch003 · Cited by 2
Psychometric Properties of the Revised Trolling Questionnaire in Argentinean Adults
Resett, S. A., & Caino, P. C. G. (2023) · Acta Colombiana de Psicología · doi:10.14718/acp.2023.26.2.9 · Cited by 3
“Computers in Human Behavior, 114, Article e106551. https://doi.org/10.1016/j.chb.2020.106551 Psychometric Properties of Trolling Questionnaire 112 Nitschinsk, L, Tobin, S.J., & Vanman, E.J. (2022).”
Influentialvan der Kolk, L., Smit, E., Bloemer, J. M. M., & van Wijk, L. M. (2023) · Patient Related Outcome Measures · doi:10.2147/prom.s416182 · Cited by 2
Mansolf, M. A., Blackwell, C. K., Lai, J., & Cella, D. F. (2023) · Psychological Assessment · doi:10.1037/pas0001256 · Cited by 2
Karababa, A. (2024) · International Journal of Intercultural Relations · doi:10.1016/j.ijintrel.2024.101966 · Cited by 3
Franco-Jimenez, R. A. (2024) · Propósitos y Representaciones · doi:10.20511/pyr2024.v12.1992 · Cited by 1
Examining Predictors of Implementation Quality in an Emerging International Extension Context
Gagnon, R. J., & Pettigrew, J. (2018) · Journal of Extension · doi:10.34068/joe.56.04.12 · Cited by 2
“Although self-report measures could introduce bias on the part of study respondents (e.g., Sonderen, Sanderman, & Coyne, 2013), the reduction in resources necessary may balance out the limitations of this approach as compared to more resource-intensive observational techniques.”
Greaves, L. M., Houkamau, C. A., & Sibley, C. G. (2016) · Personality and Individual Differences · doi:10.1016/j.paid.2016.09.026 · Cited by 2
Refining the Measure of Impression-Relevant Involvement
Parcha, J. M. (2020) · Western Journal of Communication · doi:10.1080/10570314.2020.1731590 · Cited by 3
“Building on ideas from Zimbardo (1960), Johnson and Eagly (1989) specifically contend that IRI is the “concern [one has] with holding an opinion that is socially acceptable to potential evaluators” (p.”
BackgroundCoping with Multiple Sclerosis Scale
Parkerson, H. A., Kehler, M. D., Sharpe, D., & Hadjistavropoulos, H. D. (2016) · International Journal of MS Care · doi:10.7224/1537-2073.2015-031 · Cited by 3
Propiedades psicométricas de una Escala de Phubbing en una muestra argentina
Resett, S. A., & Caino, P. C. G. (2020) · Revista Evaluar · doi:10.35670/1667-4545.v20.n2.30111 · Cited by 6
Aluh, D. O., Lopes, S. A., Pedrosa, B., Silva, M., Grigaitė, U., Martins, A., Mousinho, M. F. D. A., Cardoso, G., & Caldas‐de‐Almeida, J. M. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e24114 · Cited by 2
“The reversal of wording has been found ineffective in addressing bias and can even be counterproductive in some cases [72].”
BackgroundCornejo, B., Vela, B., Vilca, L. W., Vallejos, M. A., & Huancahuire‐Vega, S. (2021) · Journal of Primary Care & Community Health · doi:10.1177/21501327211051935 · Cited by 3
Outcomes of Nurse Practitioner Led-Care in Adult Patients with Atrial Fibrillation
Smigorowsky, M. J. (2019) · ERA: Education and Research Archive (University of Alberta) · doi:10.7939/r3-79n2-a216 · Cited by 2
Utopian Impulse: An Individual-differences Approach to Transformative Social ChangePreprint
Bassó, F., & Krpan, D. (2020) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/nvm2j · Cited by 4
Blötner, C., & Grüning, D. J. (2023) · doi:10.31234/osf.io/euh9f · Cited by 2
Sirlopú, D., Pérez-Salas, C., Villarroel, V., García, C., Barrera, K., Fierro, E., & Rabanal, I. F. C. (2019) · Universitas Psychologica · doi:10.11144/javeriana.upsy18-1.avve · Cited by 1
Assessing the Relationship Between Brand Image, Market Orientation, and Competitive Advantage
Rua, O. L., & Santos, C. (2020) · Advances in business strategy and competitive advantage book series · doi:10.4018/978-1-7998-4387-0.ch010 · Cited by 2
Kleve, L., Skarstein, L., & Elgen, I. B. (2020) · Health Education and Care · doi:10.15761/hec.1000184 · Cited by 1
Development and psychometric testing of the actualisation of evidence‐based nursing instrument
Parisod, H., Stolt, M., Holopainen, A., Siltanen, H., Pasanen, M., & Suhonen, R. (2024) · Journal of Clinical Nursing · doi:10.1111/jocn.17024 · Cited by 3
Mohammed, A., & Alonso‐Arbiol, I. (2024) · Journal of Adolescence · doi:10.1002/jad.12304 · Cited by 2
Pedersen, M. M., Carstensen, T. B. W., Ørnbøl, E., Fink, P., Jørgensen, T., Dantoft, T. M., & Frostholm, L. (2024) · Scandinavian Journal of Pain · doi:10.1515/sjpain-2024-0022 · Cited by 1
Uncovering the Internal Structure of the German Version of the CORE‐OM: A Network Analysis
Fuchshuber, J., Zeldovich, M., Aranyi, G., Winter, L., Kuška, M., Dumont, D., Humer, E. C., & Unterrainer, H. (2025) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.70063 · Cited by 2
Collective intelligence in work teams: a new multilevel construct and scale
Aubouin-Bonnaventure, J., Chevalier, S., Morel, N., Gillet, N., & Fouquereau, E. (2025) · Current Psychology · doi:10.1007/s12144-025-07818-8 · Cited by 3
Metadata
- Authors
- Sonderen, E. V., Sanderman, R., & Coyne, J. C.
- Affiliations
- University Medical Center Groningen (NL)
- Corresponding author
- Sonderen, Eric van
- ORCID
- Sanderman, Robbert
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-07-31
- Volume
- 8
- Issue
- 7
- Pages
- e68967
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3729568
- OpenAlex
- W2141632279
- Cited by
- 544
- References
- 29
- Citations by year
- 2026: 44 · 2025: 50 · 2024: 49 · 2023: 48 · 2022: 54 · 2021: 59 · 2020: 70 · 2019: 51 · 2018: 44 · 2017: 27 · 2016: 29 · 2015: 16 · 2014: 3
- FWCI
- 49.55
- Type
- journal article
- Language
- en
- Update notices
- correction (2013-09-13)
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (publishedVersion, cc-by)
- University of Twente Research Information (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, cc-by)
- University of Twente Research Information (submittedVersion, cc-by)
- Funders
- Universiteit Maastricht
- Maastricht Universitair Medisch Centrum
- Universitair Medisch Centrum Groningen
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Keywords
- response bias, acquiescence bias, questionnaire design, Multidimensional Fatigue Inventory, fatigue measurement, psychometric properties
- MeSH terms
- Animals, Cattle, Fatigue, Humans, Psychometrics, Surveys and Questionnaires, Reproducibility of Results, Inflammatory Bowel Diseases, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.