The DOI matches its Crossref record; 23 of the 24 references it cites were found in a registry or an index.
Increasing Validity with Forced-Choice Criterion Measurement Formats
Abstract
The relative validities of forced‐choice (ipsative) and Likert rating‐scale item formats as criterion measures are examined. While there has been much debate about the relative technical and psychometric merits and demerits of ipsative instruments, the present research focused on the crucial question of whether the use of this format has any practical benefit – in terms of improved validity. An analysis is reported from a meta‐analysis data set. This demonstrates that higher operational validity coefficients (prediction of line‐manager ratings of competencies) are associated with the use of forced‐choice (r=.38) rather than rating scale (r=.25) item formats for the criterion measurement instrument when performance is rated by the same line managers on both formats and where the predictor is held constant. Thus the apparent criterion‐related validity of a predictor can increase by 50% simply by changing the format of the criterion measurement instrument. The implications of this for practice are discussed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The use of forced-choice, rank orders was motivated by research showing increased validity of these formats compared to rating scales [32].”
From What format of treatment do patients with emotional disorders prefer and why? Implications for public mental health settings and policies (2019) · cites this paper as [32]
Citations
Cites
Schmidt, F. L., & Hunter, J. E. (1998) · Psychological Bulletin · doi:10.1037//0033-2909.124.2.262 · Cited by 4,180
Hunter, J., & Schmidt, F. (2004) · doi:10.4135/9781412985031 · Cited by 3,023
The five factor model of personality and job performance in the European Community.
Salgado, J. F. (1997) · Journal of Applied Psychology · doi:10.1037/0021-9010.82.1.30 · Cited by 1,582
The Great Eight Competencies: A Criterion-Centric Approach to Validation.
Bartram, D. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.6.1185 · Cited by 745
Self-citationMeasurement error in psychological research: Lessons from 26 research scenarios.
Schmidt, F. L., & Hunter, J. E. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.199 · Cited by 567
Comparative analysis of the reliability of job performance ratings.
Viswesvaran, C., Ones, D. S., & Schmidt, F. L. (1996) · Journal of Applied Psychology · doi:10.1037/0021-9010.81.5.557 · Cited by 505
Salgado, J. F., Anderson, N. R., Moscoso, S., Bertua, C., Fruyt, F. D., & Rolland, J. P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.6.1068 · Cited by 364
Strengths and limitations of ipsative measurement
Baron, H. (1996) · Journal of Occupational and Organizational Psychology · doi:10.1111/j.2044-8325.1996.tb00599.x · Cited by 321
On the Reliability of a Weighted Composite
Mosier, C. I. (1943) · Psychometrika · doi:10.1007/bf02288700 · Cited by 261
Reconsidering forced-choice item formats for applicant personality assessment
Christiansen, N. D., Burns, G. N., & Montgomery, G. E. (2005) · Human Performance · doi:10.1207/s15327043hup1803_4 · Cited by 250
The Impact of Faking on Employment Tests: Does Forced Choice Offer a Solution?
Jackson, D. N., Wroblewski, V. R., & Ashton, M. C. (2000) · Human Performance · doi:10.1207/s15327043hup1304_3 · Cited by 233
Psychometric problems and issues involved with creating and using ipsative measures for selection
Meade, A. W. (2004) · Journal of Occupational and Organizational Psychology · doi:10.1348/0963179042596504 · Cited by 168
Competency and Individual Performance: Modelling the World of Work
Kurz, R. H., & Bartram, D. (2002) · Wiley eBooks · doi:10.1002/9780470696736.ch10 · Cited by 167
Self-citationThe reliability and validity of normative and ipsative approaches in the measurement of personality
Saville, P., & Willson, E. (1991) · Journal of Occupational Psychology · doi:10.1111/j.2044-8325.1991.tb00556.x · Cited by 154
The relationship between ipsatized and normative measures of personality
Bartram, D. (1996) · Journal of Occupational and Organizational Psychology · doi:10.1111/j.2044-8325.1996.tb00597.x · Cited by 152
Self-citationHalo in a multidimensional forced-choice performance evaluation scale.
King, L. M., Hunter, J. E., & Schmidt, F. L. (1980) · Journal of Applied Psychology · doi:10.1037/0021-9010.65.5.507 · Cited by 143
How effective are people at faking on personality questionnaires?
Martin, B. A., Bowen, C.-C., & Hunt, S. T. (2002) · Personality and Individual Differences · doi:10.1016/s0191-8869(01)00021-6 · Cited by 117
Forced Choice—The New Army Rating1
Sisson, E. D. (1948) · Personnel Psychology · doi:10.1111/j.1744-6570.1948.tb01316.x · Cited by 103
On the factoring and interpretation of ipsative data
Closs, S. J. (1996) · Journal of Occupational and Organizational Psychology · doi:10.1111/j.2044-8325.1996.tb00598.x · Cited by 96
Introduction: A Framework for Examining Organizational Effectiveness
Bartram, D., Robertson, I. T., & Callinan, M. (2002) · doi:10.1002/9780470696736.ch · Cited by 86
Self-citationOrganizational effectiveness : the role of psychology
Robertson, I. T., Callinan, M., & Bartram, D. (2002) · Medical Entomology and Zoology · Cited by 72
Self-citationBartram, D., Brown, A., Fleck, S., Inceoglu, I., & Ward, K. (2006) · Kent Academic Repository (University of Kent) · Cited by 33
Self-citationA critique and standardization of meta‐analytic validity coefficients in personnel selection
Hermelin, E., & Robertson, I. T. (2001) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317901167352 · Cited by 30
Research works printed but not linked: 5
- Baron H.(2002). Working with Ipsative Measures.Paper Presented at the 17th Annual Conference of the Society for Industrial and Organizational Psychology April 2002 Toronto Canada.
- 10.1037/0033-2909.124.2.262 · DOI · Paper page
- SHL, 1993, OPQ Concept Model: Manual and user's guide
- SHL, 1993, Inventory of Management Competencies: Manual and user's guide
- SHL, 1999, OPQ32: Manual and user's guide
Cited by
Sorted by citations
Cruchinho, P., López-Franco, M. D., Capelas, M. L., Almeida, S., Bennett, P. M., Miranda da Silva, M., Teixeira, G., Nunes, E., Lucas, P., & Gaspar, F. (2024) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s419714 · Cited by 469
OECD (2014) · Programme for international student assessment/Internationale Schulleistungsstudie · doi:10.1787/6341a959-en · Cited by 351
Item Response Modeling of Forced-Choice Questionnaires
Brown, A., & Maydeu‐Olivares, A. (2011) · Educational and Psychological Measurement · doi:10.1177/0013164410375112 · Cited by 333
How IRT can solve problems of ipsative data in forced-choice questionnaires.
Brown, A., & Maydeu‐Olivares, A. (2012) · Psychological Methods · doi:10.1037/a0030641 · Cited by 217
Salgado, J. F., & Táuriz, G. (2014) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2012.716198 · Cited by 205
Metadata
- Authors
- Bartram, D.
- Corresponding author
- Bartram, Dave
- ORCID
- Bartram, Dave
- Venue
- International Journal of Selection and Assessment
- Publisher
- Wiley
- Published
- 2007-08-14
- Volume
- 15
- Issue
- 3
- Pages
- 263-272
- ISSN
- 0965-075X (ISSN-L)
- 1468-2389
- OpenAlex
- W2131404783
- Cited by
- 132
- References
- 24
- Citations by year
- 2026: 3 · 2025: 9 · 2024: 9 · 2023: 8 · 2022: 9 · 2021: 12 · 2020: 7 · 2019: 13 · 2018: 8 · 2017: 7 · 2016: 8 · 2015: 6 · 2014: 7 · 2013: 4 · 2012: 6
- FWCI
- 5.26
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Selection and Assessment (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Employer Branding and e-HRM
- Keywords
- forced-choice format, criterion validity, operational validity, competency assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review