Ineffectiveness of Reverse Wording of Questionnaire Items: Let’s Learn from Cows in the Rain
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: We examined the effectiveness of reverse worded items as a means of reducing or preventing response bias. We first distinguished between several types of response bias that are often confused in literature. We next developed arguments why reversing items is probably never a good way to address response bias. We proposed testing whether reverse wording affects response bias with item-level data from the Multidimensional Fatigue Inventory (MFI-20), an instrument that contains reversed worded items. METHODS: With data from 700 respondents, we compared scores on items that were similar with respect either to content or to direction of wording. Psychometric properties of sets of these items worded in the same direction were compared with sets consisting of both straightforward and reversed worded items. RESULTS: We did not find evidence that ten reverse-worded items prevented response bias. Instead, the data suggest scores were contaminated by respondent inattention and confusion. CONCLUSIONS: Using twenty items, balanced for scoring direction, to assess fatigue did not prevent respondents from inattentive or acquiescent answering. Rather, fewer mistakes are made with a 10-item instrument with items posed in the same direction. Such a format is preferable for both epidemiological and clinical studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In this context, a study by Sonderen et al. (2013) showed that reverse-worded items may lead to inattention when responding to a questionnaire.”
From Development and Validation of a Theory-Based Questionnaire to Measure Different Types of Cognitive Load (2023) · cites this paper as Sonderen et al., 2013
The cited source says what the claim attributes to it.
In the citing paper
“The latter could indicate a measurement issue related to including both positively and negatively phrased items in a scale [111].”
From Validation of the compassionate engagement and action scales, compassion scale, and Sussex-Oxford compassion scales in a French-Canadian sample (2024) · cites this paper as [111]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116 · cited 2×
“We consider this a variety of the Multi-Trait-Multi-Method matrix discussed by Campbell & Fiske [29].”
MethodA three-component conceptualization of organizational commitment
Meyer, J. P., & Allen, N. J. (1991) · Human Resource Management Review · doi:10.1016/1053-4822(91)90011-z · Cited by 11,049 · cited 2×
“1) Meyer and Allen’s [11] scale to assess affective commitment, of which Merritt [12] demonstrated that the answers on reversed items were prone to careless responding and cognitive fatigue.”
BackgroundDevelopment and validation of the Penn State Worry Questionnaire
Meyer, T. J., Miller, M. L., Metzger, R. L., & Borkovec, T. D. (1990) · Behaviour Research and Therapy · doi:10.1016/0005-7967(90)90135-6 · Cited by 5,099 · cited 1×
“’s [14] scale to assess worry contains 11 positively and five reversed worded items, leading Fresco et al.”
BackgroundThe Psychology of Survey Response
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000) · doi:10.1017/cbo9780511819322 · Cited by 4,734 · cited 2×
“The process of obtaining survey data is complex, with many possibilities of discrepancies arising between the state or opinion the researcher wants to elicit and the answer given by the respondent [3].”
BackgroundSmets, E. M. A., Garssen, B., Bonke, B., & De Haes, J. C. J. M. (1995) · Journal of Psychosomatic Research · doi:10.1016/0022-3999(94)00125-o · Cited by 3,613 · cited 2×
“[28] constructed a version with seven categories, but in accordance with their adaptation in 1996 [27], a version is used with five answer categories, ranging from 1 (‘yes, that is true’) to 5 (‘no, that is not true’).”
BackgroundMethodMeasurement and Control of Response Bias
Paulhus, D. L. (1991) · Measures of Personality and Social Psychological Attitudes · doi:10.1016/b978-0-12-590241-0.50006-x · Cited by 3,598
Krosnick, J. A. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.537 · Cited by 1,852 · cited 1×
“Krosnick [7] mentions a satisficing response style, whereby the respondent deliberately makes less effort to understand all subtleties of the question.”
On the bipolarity of positive and negative affect.
Russell, J. A., & Carroll, J. M. (1999) · Psychological Bulletin · doi:10.1037/0033-2909.125.1.3 · Cited by 1,481 · cited 1×
“More researchers attribute the claim of multidimensionality that is often made in scales with reversed worded items, not to conceptual differences, but to artificial factors resulting from response bias [20–25].”
BackgroundResponse Styles in Marketing Research: A Cross-National Investigation
Baumgartner, H., & Steenkamp, J.-B. E. M. (2001) · Journal of Marketing Research · doi:10.1509/jmkr.38.2.143.18840 · Cited by 1,255
Do We Really Need Multiple-Item Measures in Service Research?
Drolet, A. L., & Morrison, D. G. (2001) · Journal of Service Research · doi:10.1177/109467050133001 · Cited by 736 · cited 2×
“As Drolet and Morrison [9] demonstrate, respondents sometimes just answer the first item and assume that this answer also holds for subsequent (considered similar or even identical) items.”
Background
Cited by
Items 101–200 of 543
Lim, Z. X., Chua, W. L., Lim, W. S., Lim, A. Q., Chua, K. C., & Chan, E. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19084639 · Cited by 31
Hinz, A., Benzing, C., Brähler, E., Zenger, M., Herzberg, P. Y., Finck, C., Schmalbach, B., & Petrowski, K. (2019) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2019.12.005 · Cited by 41
Nishimura, T., Bradshaw, E. L., Deci, E. L., & Ryan, R. M. (2020) · Social Development · doi:10.1111/sode.12473 · Cited by 35
To be Direct or not: Reversing Likert Response Format Items
García-Fernández, J., Postigo, Á., Cuesta, M., González‐Nuevo, C., Menéndez-Aller, Á., & García-Cueto, E. (2022) · The Spanish Journal of Psychology · doi:10.1017/sjp.2022.20 · Cited by 35
Shen, X., Chick, G., & Pitas, N. A. (2017) · International Journal of Play · doi:10.1080/21594937.2017.1382983 · Cited by 29
Heffernan, E., Coulson, N., & Ferguson, M. (2018) · International Journal of Audiology · doi:10.1080/14992027.2018.1483585 · Cited by 30
“Many questionnaires include reverse-worded items (i.e. items that are worded in the opposite direction to the other items) as a means of preventing response biases, especially inattention and acquiescence (Rattray and Jones 2007; van Sonderen et al. 2013).”
BackgroundAge Appropriateness of the Self-Report Strengths and Difficulties Questionnaire
Black, L., Mansfield, R., & Panayiotou, M. (2020) · Assessment · doi:10.1177/1073191120903382 · Cited by 34
“First, items should ideally consist of single statements (Irwing & Hughes, 2018; Saris, 2014; Terwee et al., 2007), and avoid reverse wording to reduce confusion (Irwing & Hughes, 2018; van Sonderen et al., 2013).”
BackgroundRomppel, M., Hinz, A., Finck, C., Young, J. C., Brähler, E., & Glaesmer, H. (2017) · International Journal of Methods in Psychiatric Research · doi:10.1002/mpr.1532 · Cited by 32
Jensen, C. G., Krogh, S. C., Westphael, G., & Hjordt, L. V. (2019) · Psychological Assessment · doi:10.1037/pas0000667 · Cited by 34
Investigating the Reliability of Self-report Data in the Wild: The Quest for Ground Truth
Gao, N., Rahaman, M. S., Shao, W., & Salim, F. D. (2021) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3460418.3479338 · Cited by 42
Park, K., Yoon, S., Cho, S., Choi, Y., Lee, S., & Choi, K. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.992068 · Cited by 32
Veronese, G., & Pepe, A. (2018) · Current Psychology · doi:10.1007/s12144-018-9891-x · Cited by 29
“In line with recent recommendations in questionnaire development, reverse-worded items should be avoided, because they do not compensate for response biases and are sometimes difficult to understand (Van Sonderen et al. 2013).”
BackgroundChen, S., Liao, F., Murphy, D., & Joseph, S. (2021) · Current Psychology · doi:10.1007/s12144-021-02348-5 · Cited by 33
Chen, S., Liao, F., Murphy, D., & Joseph, S. (2021) · Measurement and Evaluation in Counseling and Development · doi:10.1080/07481756.2021.1955212 · Cited by 26
Keesler, J. M., & Fukui, S. (2020) · Journal of Intellectual Disability Research · doi:10.1111/jir.12766 · Cited by 35
Laus, M. F., do Vales, L. D. M. F., Oliveira, N. G., Costa, T. M. B., & Almeida, S. S. (2019) · Eating and Weight Disorders - Studies on Anorexia Bulimia and Obesity · doi:10.1007/s40519-019-00758-w · Cited by 39
Validation of the psychometric properties of the Self‐Directed Learning Readiness Scale
Chen, S., & Fan, J. (2022) · Nursing Open · doi:10.1002/nop2.1418 · Cited by 25
“Adding reverse worded items on a scale is a common strategy to reduce or eliminate acquiescence bias, a respondent tendency, whereby participants provide a specific answer regardless of its content (van Sonderen et al., 2013; Vigil- Colet et al., 2020).”
BackgroundThe Penn State Worry Questionnaire for Children: Age, Gender and Clinical Invariance
Păsărelu, C. R., Dobrean, A., Balázsi, R., Predescu, E., Șipoș, R., & Lupu, V. (2016) · Child Psychiatry & Human Development · doi:10.1007/s10578-016-0663-2 · Cited by 32
Høeg, E. R., Bruun‐Pedersen, J. R., & Serafin, S. (2021) · IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · doi:10.1109/vrw52623.2021.00052 · Cited by 29
“This is further substantiated by [61] who concludes that reversed questions do not effectively reduce response bias, but tend to lead to confusion instead.”
Nikiforova, T., Carter, A. E., Yecies, E. B., & Spagnoletti, C. L. (2021) · Southern Medical Journal · doi:10.14423/smj.0000000000001292 · Cited by 25
Fatigue screening in breast cancer patients: identifying likely cases of cancer‐related fatigue
Goedendorp, M. M., Jacobsen, P. B., & Andrykowski, M. A. (2015) · Psycho-Oncology · doi:10.1002/pon.3907 · Cited by 28
“These mistakes happen more often in questionnaires with reverse-worded items than with questionnaires containing items posed in the same direction [33].”
BackgroundIntercultural workplace relationships in the hospitality industry: Beyond the tip of the iceberg
Vassou, C., Zopiatis, A., & Theocharous, A. L. (2016) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2016.11.001 · Cited by 33
How Do Reverse-keyed Items in Inventories Affect Measurement Quality and Information Processing?
Menold, N. (2019) · Field Methods · doi:10.1177/1525822x19890827 · Cited by 28
“…artificially corresponds to the item form in balanced scales (i.e., an equal number of regular and reversekeyed items) or scales with at least two reverse-keyed items (e.g., Schotte et al. 1996; Schriesheim and Eisenbach 1995; Swain et al. 2008; van Sonderen et al. 2013; Wong et al. 2003).”
BackgroundPsychometric validation of the French version of the Suicidal Ideation Attributes Scale (SIDAS-FR)
Gauvin, G., Bardon, C., & Côté, L. (2021) · Death Studies · doi:10.1080/07481187.2021.1951395 · Cited by 27
“Combining regular and reversed items typically decreases the fit of the data, affecting the unidimensionality of instruments (Su arez- Alvarez et al., 2018; van Sonderen et al., 2013).”
BackgroundDevelopment and Initial Validation of a Measure to Assess Career Goal Feedback
Hu, S. M., Creed, P. A., & Hood, M. (2016) · Journal of Psychoeducational Assessment · doi:10.1177/0734282916654645 · Cited by 30
There’s More to Groove than Bass in Electronic Dance Music: Why Some People Won’t Dance to Techno
Wesolowski, B. C., & Hofmann, A. (2016) · PLoS ONE · doi:10.1371/journal.pone.0163938 · Cited by 36
Tuliao, A. P., & Holyoak, D. (2019) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2019.1658198 · Cited by 28
Measuring Educational Neuromyths: Lessons for Future Research
Sullivan, K. A., Hughes, B., & Gilmore, L. (2021) · Mind Brain and Education · doi:10.1111/mbe.12294 · Cited by 25
Acquiescence Response Bias — Yeasaying and Higher Education
Costello, S., & Roodenburg, J. (2015) · The Australian Educational and Developmental Psychologist · doi:10.1017/edp.2015.11 · Cited by 23
Baussard, L., Carayol, M., Porro, B., Baguet, F., & Cousson‐Gélie, F. (2018) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2018.07.005 · Cited by 23
El‐Wahsh, S., Heard, R. C., & Bogaardt, H. (2018) · International Journal of Speech-Language Pathology · doi:10.1080/17549507.2018.1499802 · Cited by 24
Agu, A. G., Onwuka, O. O., Okocha, E. R., & Madichie, N. O. (2021) · African Journal of Economic and Management Studies · doi:10.1108/ajems-05-2021-0227 · Cited by 27
Customer Service Chatbots: Anthropomorphism, Adoption and Word of Mouth
Sheehan, B. T. (2018) · Queensland University of Technology · doi:10.5204/thesis.eprints.121188 · Cited by 23
“Therefore, anthropomorphism as an independent variable may enhance the predictive value of the four dominant theoretical models, namely the technology acceptance model (TAM) (Davis, 1989), the theory of planned behavior (TPB) (Ajzen, 1991), the unified theory of acceptance and use (UTAUT) (Venkatesh et al., 2003) or t…”
BackgroundMethodInfluentialAre trust and satisfaction similar in dental care settings?
Song, Y., Luzzi, L., Chrisopoulos, S., & Brennan, D. S. (2020) · Community Dentistry And Oral Epidemiology · doi:10.1111/cdoe.12559 · Cited by 26
DuBay, M. F., Watson, L. R., Baranek, G. T., Lee, H. Y., Rojevic, C., Brinson, W., Smith, D., & Sideris, J. H. (2021) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-020-04837-1 · Cited by 25
Developing an empirical measure of everyday information mastering
Heinström, J., Sormunen, E., Savolainen, R., & Ek, S. (2019) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24305 · Cited by 22
Álvarez, E. F., Cuevas, R., López, J. C., & Spray, C. M. (2021) · Journal of Teaching in Physical Education · doi:10.1123/jtpe.2020-0070 · Cited by 29
Development and initial validation of the Generalized Tracking Questionnaire
Ruiz, F. J., Garcia-Martin, M., Suárez-Falcón, J. C., Bedoya-Valderrama, L., Segura‐Vargas, M. A., Peña-Vargas, A., Henao, Á. M., & Ávila-Campos, J. E. (2020) · PLoS ONE · doi:10.1371/journal.pone.0234393 · Cited by 22
“Participants were not asked to design items containing negative statements that would serve as reverse-scored items because they usually generate problems when conducting factor analyses [45,46].”
BackgroundLower‐Hoppe, L. M., Anderson‐Butcher, D., Lo, M., & Turner, B. A. (2020) · Leisure/Loisir · doi:10.1080/14927713.2020.1815564 · Cited by 24
Public–private partnerships in Tanzanian affordable housing schemes
Kavishe, N., Chileshe, N., & Jefferson, I. (2018) · Built Environment Project and Asset Management · doi:10.1108/bepam-01-2018-0010 · Cited by 20
Aramovich, N. P., & Blankenship, J. (2020) · The Leadership Quarterly · doi:10.1016/j.leaqua.2020.101387 · Cited by 23
The DREEM, part 2: psychometric properties in an osteopathic student population
Vaughan, B., Mulcahy, J., & McLaughlin, P. A. (2014) · BMC Medical Education · doi:10.1186/1472-6920-14-100 · Cited by 19
Validation of revised patient measures of safety: PMOS-30 and PMOS-10
Louch, G., Reynolds, C., Moore, S. G., Marsh, C., Heyhoe, J., Albutt, A., & Lawton, R. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-031355 · Cited by 20
Preliminary Development and Validation of the Mindful Student Questionnaire
Renshaw, T. L. (2016) · Assessment for Effective Intervention · doi:10.1177/1534508416678971 · Cited by 26
Jahya, A., Azlin, S. N. I., Othman, R., & Romaiha, N. R. (2020) · International Journal of Academic Research in Business and Social Sciences · doi:10.6007/ijarbss/v10-i10/7975 · Cited by 34
“Sonderen, Sanderman, and Coyne (2013) proposed that items with negative particles such as ‘not’ resulted in high complication, thus, increase bias.”
BackgroundXiao, X., Borah, P., Lee, D. K. L., Su, Y., & Kim, S. C. (2023) · American Journal of Health Promotion · doi:10.1177/08901171231184075 · Cited by 20
“This may be partly due to the inattention and confusion associated with reverse wording.(47) In part, since prior research suggests that multiple dimensions exist within the original 34-item NFC scale,(48) whether the shortened scale taps into various dimensions merits further exploration.”
BackgroundAnti-Brand Activism in the Collectivist Orientation: The Case of Online-Consumers-Run Communities
Do, H., Bich, N., Nguyen, T. M., & Nguyen, H. M. T. (2024) · Journal of Promotion Management · doi:10.1080/10496491.2024.2318665 · Cited by 29
“In the final questionnaire, all the measurement items were mixed up to avoid the possible response bias when respondents filled answers (van Sonderen et al., 2013).”
BackgroundManville, C. (2014) · Revue de gestion des ressources humaines · doi:10.3917/grhu.092.0052 · Cited by 22
Moore, H. E., Rutherfurd, I. D., & Peel, M. (2018) · Journal of Rural Studies · doi:10.1016/j.jrurstud.2018.07.012 · Cited by 18
Campbell, J. W., & Im, T. (2019) · Public Personnel Management · doi:10.1177/0091026019832632 · Cited by 22
Khalaf, M. A., & Said, T. T. A. (2021) · The Open Psychology Journal · doi:10.2174/1874350102114010083 · Cited by 18
The Psychometric Validation Of The Principal Practices Questionnaire Based On Item Response Theory
Perera, C. J., Sumintono, B., & Jiang, N. (2018) · International Online Journal of Educational Leadership · doi:10.22452/iojel.vol2no1.3 · Cited by 23
Bolt, D. M., Wang, Y. C., Meyer, R. H., & Pier, L. (2020) · Applied Measurement in Education · doi:10.1080/08957347.2020.1789140 · Cited by 18
Lombardo, C., Novara, C., Mallia, L., Pastore, M., & Vacca, M. (2021) · Journal of Personality Assessment · doi:10.1080/00223891.2021.1905651 · Cited by 20
“In fact, as evidenced by Van Sonderen et al. (2013), developing an instrument with a portion of items formulated in a different direction can be a counterproductive strategy, although many researchers support the use of reversed items in order to prevent response bias (for a comprehensive review,…”
ResultBlack, L., Humphrey, N., Panayiotou, M., & Marquez, J. (2023) · Assessment · doi:10.1177/10731911231158623 · Cited by 25
“It is known that reverse wording can create multidimensionality and confusion (Irwing & Hughes, 2018; van Sonderen et al., 2013), and it has been recommended that this is avoided in questionnaires with adolescents (Omrani et al., 2018).”
BackgroundNg, S., Leng, L., & Wang, Q. (2017) · Journal of Applied Gerontology · doi:10.1177/0733464816687448 · Cited by 22
Lau, J. T. F., Dang, L., Cheung, R. Y.-H., Zhang, M. X., Chen, J. H., & Wu, A. M. S. (2020) · Journal of Behavioral Addictions · doi:10.1556/2006.2020.00043 · Cited by 17
Brauer, K., & Proyer, R. T. (2020) · Journal of Personality Assessment · doi:10.1080/00223891.2020.1720699 · Cited by 17
Li, S., Xu, L., Yu, F., & Peng, K. (2020) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3319502.3374777 · Cited by 34
“…traits) combined both positive and negative traits, which might influence each other on people’s scoring as well as their acquiescence bias [55, 56, 62], we adjusted the two series into four separate 7-point Likert scales as follows (1: not at all characteristic of the robot; 7: entirely…”
MethodSchreurs, L., Lee, A. Y., Liu, X. “., & Hancock, J. T. (2024) · Communication Research · doi:10.1177/00936502241233787 · Cited by 16
“Yet, as both item 4 and 5 are negatively-worded, the emergence of this second factor may stem from a methodological artifact rather than a meaningful conceptual difference (van Sonderen et al., 2013).”
Park, H.-H., Ebesutani, C. K., Chung, K., & Stanick, C. (2016) · Assessment · doi:10.1177/1073191116634202 · Cited by 16
“For example, van Sonderen, Sanderman, and Coyne (2013) found that the inclusion of both non–reversedworded and reversed-worded items can contribute to respondent confusion while not adequately preventing response acquiescence.”
BackgroundThe Impact of Probation and Parole Officers’ Attitudes about Offenders on Professional Practices
Schaefer, L., & Williams, G. C. (2018) · Corrections · doi:10.1080/23774657.2018.1538710 · Cited by 18
The development of a brief and practical work safety climate measure
Summers, D., Sarris, A., Harries, J., & Kirby, N. (2021) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2021.103255 · Cited by 16
Dominance and Prestige as Self-Concept Facets
Körner, R., Heydasch, T., & Schütz, A. (2022) · Journal of Personality Assessment · doi:10.1080/00223891.2022.2137028 · Cited by 21
“In addition, reverse-coded items have been criticized for being cognitively too demanding while not being effective at controlling response biases (Sonderen et al., 2013).”
BackgroundPedersen, H. S., Christensen, K. S., Prior, A., & Christensen, K. B. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.10.109 · Cited by 23
Palaniappan, K., & Kum, I. Y. S. (2017) · Current Psychology · doi:10.1007/s12144-017-9733-2 · Cited by 23
“(van Sonderen et al. 2013) • Acquiescence (uncritical agreement with items) • Inattention (missing the intended meaning of an item) • Confusion (too difficult to comprehend) The MFI-20, designed to prevent response bias by using reverse wording on half the items, seems to increase the risk of…”
BackgroundZhang, X., Tse, W. W.-Y., & Savalei, V. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01286 · Cited by 21
Kieffer, J. M., Starreveld, D. E. J., Boekhout, A. H., & Bleiker, E. M. A. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.005 · Cited by 18
Masterton, W., Park, K. J., Carver, H., & Parkes, T. (2021) · Health & Place · doi:10.1016/j.healthplace.2021.102669 · Cited by 16
Reducing Affective Responses to Surgical Images and Videos Through Stylization
Besançon, L., Semmo, A., Biau, D. J., Frachet, B., Pineau, V., Sariali, E. H., Soubeyrand, M., Taouachi, R., Isenberg, T., & Dragicevic, P. (2019) · Computer Graphics Forum · doi:10.1111/cgf.13886 · Cited by 22
Liu, J., Gong, X., Weal, M. J., Dai, W., Hou, S., & Ma, J. (2023) · Digital Health · doi:10.1177/20552076231174101 · Cited by 23
Magallares, A., Apodaca, M.-J. F.-R. D., & Morales, J.-F. (2016) · International Journal of Social Psychology Revista de Psicología Social · doi:10.1080/02134748.2016.1248025 · Cited by 16
Kantor‐Martynuska, J., & Kenny, D. T. (2018) · Polish Psychological Bulletin · doi:10.24425/119500 · Cited by 18
I Can Also Make Robots! Inspiring Rural Indian Children to Learn Robotics
Manikutty, G., Frey, L. M., Natarajan, A., Chilakapati, U., Vennila, V., & Bhavani Rao, R. (2019) · IEEE International Conference on Technology for Education (T4E) · doi:10.1109/t4e.2019.00022 · Cited by 15
Rawy, M., Look, K. A., Amin, M. E. K., & Chewning, B. A. (2020) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2020.06.026 · Cited by 17
Teresi, J. A., Ocepek‐Welikson, K., Toner, J. A., Kleinman, M., Ramirez, M. M., Eimicke, J. P., Gurland, B. J., & Siu, A. L. (2017) · Applied Research in Quality of Life · doi:10.1007/s11482-017-9516-9 · Cited by 19
“However, evidence for their effectiveness is relatively sparse (van Sonderen et al. 2013).”
BackgroundNurse Comfort With Palliative and End-of-Life Communication
Isaacson, M. J., Minton, M. E., DaRosa, P., & Harming, S. (2019) · Journal of Hospice and Palliative Nursing · doi:10.1097/njh.0000000000000483 · Cited by 15
Bergeron, C., Moran, C., Coroiu, A., & Körner, A. (2019) · European Journal of Oncology Nursing · doi:10.1016/j.ejon.2019.03.006 · Cited by 16
Four Years in Review: Statistical Practices of Likert Scales in Human-Robot Interaction StudiesPreprint
Schrum, M., Johnson, M., Ghuy, M., & Gombolay, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.03231 · Cited by 15
Confirmation and Validation of the Equanimity Scale-16 (ES-16)
Cheever, J., Cayoun, B. A., Elphinstone, B., & Shires, A. G. (2023) · Mindfulness · doi:10.1007/s12671-022-02029-9 · Cited by 14
Hallford, D. J., & Austin, D. (2021) · Assessment · doi:10.1177/1073191121998767 · Cited by 15
“Item 7 is a reversecoded item, which typically show weaker correlations with other scale items and poorer loadings with latent variables (Swain et al., 2008; Van Sonderen et al., 2013; Woods, 2006).”
Chan, A. H. Y., Horne, R., Lycett, H., Raebel, E. M., Guitart, J., Wildman, E. K., & Ang, K. (2021) · Frontiers in Pharmacology · doi:10.3389/fphar.2021.608971 · Cited by 14
The Bidimensionality of Grit: Conceptual Dissociation or Methodological Artifact?
Postigo, Á., García-Fernández, J., Cuesta, M., González‐Nuevo, C., Menéndez-Aller, Á., & García-Cueto, E. (2023) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000766 · Cited by 16
Nima, A. A., Garcia, D., Sikström, S., & Cloninger, K. M. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e24386 · Cited by 26
Cipriani, E., Gemignani, A., & Menicucci, D. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e38461 · Cited by 20
“To keep our instrument simple and parsimonious, we opted not to use reverse-coded items, since the benefits of their utilization have been called into question [36].”
MethodLi, S., Yu, F., & Peng, K. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1631/1/012024 · Cited by 19
Lee, P., Son, M., & Jia, Z. (2025) · Journal of Business and Psychology · doi:10.1007/s10869-025-10067-y · Cited by 27
“Many studies show the detrimental effects of negative or reverse-coded items (DiStefano & Motl, 2006; Schriesheim & Eisenbach, 1995; Sliter & Zickar, 2014; Sonderen et al., 2013; Woods, 2006).”
Zhang, X., Zhou, L., & Savalei, V. (2022) · Educational and Psychological Measurement · doi:10.1177/00131644221111402 · Cited by 16
“…out when the item scores are added together, so that the participant’s total score will be unaffected by the acquiescence bias.1 However, many researchers have raised doubts about the wisdom of including NW items (e.g., Lindwall et al., 2012; Savalei & Falk, 2014; Van Sonderen et al., 2013).”
BackgroundBarriers instructors experience in adopting active learning: Instrument development
Carroll, L. J., Reeping, D. P., Finelli, C., Prince, M. J., Husman, J. E., Graham, M., & Borrego, M. J. (2023) · Journal of Engineering Education · doi:10.1002/jee.20557 · Cited by 29
“Typically, researchers suggest making items all positive and avoiding the inclusion of negatively worded items (Sauro & Lewis, 2011; Sonderen et al., 2013).”
BackgroundTsaousis, I., Sideridis, G. D., & Al-Saawi, F. (2017) · International Journal of Testing · doi:10.1080/15305058.2017.1345914 · Cited by 12
Groarke, J. M., & Hogan, M. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.566296 · Cited by 20
Mental health literacy: A focus on daily life context for population health measurement
Chua, K., Hahn, J. S., Farrell, S., Jolly, A., Khangura, R. K., & Henderson, C. (2022) · SSM - Mental Health · doi:10.1016/j.ssmmh.2022.100118 · Cited by 20
Leyva‐Moral, J. M., Terradas, R., Feijoo-Cid, M., Sánchez, R. D. D. I., Camps, L. M., Castaño, A. L., & Serrano, M. C. (2016) · Collegian Journal of the Royal College of Nursing Australia · doi:10.1016/j.colegn.2016.10.006 · Cited by 13
Feliu‐Soler, A., Pérez‐Aranda, A., Andrés‐Rodríguez, L., Butjosa, A., Díaz, N. S., Trujols, J., Núñez, C., Stephan‐Otto, C., Rozadilla‐Sacanell, A., Serrano‐Blanco, A., et al. (2018) · Journal of Applied Biobehavioral Research · doi:10.1111/jabr.12134 · Cited by 12
Aasvik, O., Ulleberg, P., & Hagenzieker, M. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1531386 · Cited by 14
“While reversed items is a common way to mitigate straightlining in surveys, it may also introduce some noise and confusion (Van Sonderen et al., 2013).”
In good company? Development and validation of the Family-Supportive Coworker Behavior Scale
Tortez, L. M., & Mills, M. J. (2022) · Journal of Vocational Behavior · doi:10.1016/j.jvb.2022.103724 · Cited by 14
PAY SATISFACTION AND ORGANISATIONAL TRUST: AN IMPORTANCE-PERFORMANCE MAP ANALYSIS
Gim, G. C. W., & Cheah, W.-S. (2020) · Journal of Applied Structural Equation Modeling · doi:10.47263/jasem.4(1)01 · Cited by 20
“Van Sonderen, Sanderman, and Coyne (2013) advocated that all questionnaire items should be worded in the same direction because they found that even having negatively worded items did not prevent acquiescence bias.”
BackgroundPerceptions of Vietnamese EFL High School Teachers and Students towards English as a Lingua Franca
Nguyen, T. B., & Lo, Y.-H. G. (2022) · International Journal of TESOL & Education · doi:10.54855/ijte.222120 · Cited by 12
Alharbi, F. S., Aljemaiah, A. I., & Osman, M. (2022) · BMC Psychology · doi:10.1186/s40359-022-00826-4 · Cited by 20
“This is known to be confusing to the respondents and could have affected our participants’ interpretation of the item verbatim [35].”
BackgroundNgah, H., Hairon, S. M., Hamzah, N. A., Noordin, S., & Shafei, M. N. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19127416 · Cited by 17
“6%) for the wrong answer on item K5 could be participants’ confusion and lack of attention despite indicating that a reverse statement was introduced to prevent response bias [32].”
Background
Metadata
- Authors
- Sonderen, E. V., Sanderman, R., & Coyne, J. C.
- Affiliations
- University Medical Center Groningen (NL)
- Corresponding author
- Sonderen, Eric van
- ORCID
- Sanderman, Robbert
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-07-31
- Volume
- 8
- Issue
- 7
- Pages
- e68967
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3729568
- OpenAlex
- W2141632279
- Cited by
- 544
- References
- 29
- Citations by year
- 2026: 44 · 2025: 50 · 2024: 49 · 2023: 48 · 2022: 54 · 2021: 59 · 2020: 70 · 2019: 51 · 2018: 44 · 2017: 27 · 2016: 29 · 2015: 16 · 2014: 3
- FWCI
- 49.55
- Type
- journal article
- Language
- en
- Update notices
- correction (2013-09-13)
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (publishedVersion, cc-by)
- University of Twente Research Information (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, cc-by)
- University of Twente Research Information (submittedVersion, cc-by)
- Funders
- Universiteit Maastricht
- Maastricht Universitair Medisch Centrum
- Universitair Medisch Centrum Groningen
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Keywords
- response bias, acquiescence bias, questionnaire design, Multidimensional Fatigue Inventory, fatigue measurement, psychometric properties
- MeSH terms
- Animals, Cattle, Fatigue, Humans, Psychometrics, Surveys and Questionnaires, Reproducibility of Results, Inflammatory Bowel Diseases, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.