The DOI matches its Crossref record; 23 of the 24 references it cites were found in a registry or an index.
Increasing Validity with Forced-Choice Criterion Measurement Formats
Abstract
The relative validities of forced‐choice (ipsative) and Likert rating‐scale item formats as criterion measures are examined. While there has been much debate about the relative technical and psychometric merits and demerits of ipsative instruments, the present research focused on the crucial question of whether the use of this format has any practical benefit – in terms of improved validity. An analysis is reported from a meta‐analysis data set. This demonstrates that higher operational validity coefficients (prediction of line‐manager ratings of competencies) are associated with the use of forced‐choice (r=.38) rather than rating scale (r=.25) item formats for the criterion measurement instrument when performance is rated by the same line managers on both formats and where the predictor is held constant. Thus the apparent criterion‐related validity of a predictor can increase by 50% simply by changing the format of the criterion measurement instrument. The implications of this for practice are discussed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The use of forced-choice, rank orders was motivated by research showing increased validity of these formats compared to rating scales [32].”
From What format of treatment do patients with emotional disorders prefer and why? Implications for public mental health settings and policies (2019) · cites this paper as [32]
Citations
Cites
Schmidt, F. L., & Hunter, J. E. (1998) · Psychological Bulletin · doi:10.1037//0033-2909.124.2.262 · Cited by 4,180
Hunter, J., & Schmidt, F. (2004) · doi:10.4135/9781412985031 · Cited by 3,023
The five factor model of personality and job performance in the European Community.
Salgado, J. F. (1997) · Journal of Applied Psychology · doi:10.1037/0021-9010.82.1.30 · Cited by 1,582
The Great Eight Competencies: A Criterion-Centric Approach to Validation.
Bartram, D. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.6.1185 · Cited by 745
Self-citationMeasurement error in psychological research: Lessons from 26 research scenarios.
Schmidt, F. L., & Hunter, J. E. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.199 · Cited by 567
Comparative analysis of the reliability of job performance ratings.
Viswesvaran, C., Ones, D. S., & Schmidt, F. L. (1996) · Journal of Applied Psychology · doi:10.1037/0021-9010.81.5.557 · Cited by 505
Salgado, J. F., Anderson, N. R., Moscoso, S., Bertua, C., Fruyt, F. D., & Rolland, J. P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.6.1068 · Cited by 364
Strengths and limitations of ipsative measurement
Baron, H. (1996) · Journal of Occupational and Organizational Psychology · doi:10.1111/j.2044-8325.1996.tb00599.x · Cited by 321
On the Reliability of a Weighted Composite
Mosier, C. I. (1943) · Psychometrika · doi:10.1007/bf02288700 · Cited by 261
Reconsidering forced-choice item formats for applicant personality assessment
Christiansen, N. D., Burns, G. N., & Montgomery, G. E. (2005) · Human Performance · doi:10.1207/s15327043hup1803_4 · Cited by 250
Research works printed but not linked: 5
- Baron H.(2002). Working with Ipsative Measures.Paper Presented at the 17th Annual Conference of the Society for Industrial and Organizational Psychology April 2002 Toronto Canada.
- 10.1037/0033-2909.124.2.262 · DOI · Paper page
- SHL, 1993, OPQ Concept Model: Manual and user's guide
- SHL, 1993, Inventory of Management Competencies: Manual and user's guide
- SHL, 1999, OPQ32: Manual and user's guide
Cited by
Sorted by citations
Cruchinho, P., López-Franco, M. D., Capelas, M. L., Almeida, S., Bennett, P. M., Miranda da Silva, M., Teixeira, G., Nunes, E., Lucas, P., & Gaspar, F. (2024) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s419714 · Cited by 469
OECD (2014) · Programme for international student assessment/Internationale Schulleistungsstudie · doi:10.1787/6341a959-en · Cited by 351
Item Response Modeling of Forced-Choice Questionnaires
Brown, A., & Maydeu‐Olivares, A. (2011) · Educational and Psychological Measurement · doi:10.1177/0013164410375112 · Cited by 333
How IRT can solve problems of ipsative data in forced-choice questionnaires.
Brown, A., & Maydeu‐Olivares, A. (2012) · Psychological Methods · doi:10.1037/a0030641 · Cited by 217
Salgado, J. F., & Táuriz, G. (2014) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2012.716198 · Cited by 205
Metadata
- Authors
- Bartram, D.
- Corresponding author
- Bartram, Dave
- ORCID
- Bartram, Dave
- Venue
- International Journal of Selection and Assessment
- Publisher
- Wiley
- Published
- 2007-08-14
- Volume
- 15
- Issue
- 3
- Pages
- 263-272
- ISSN
- 0965-075X (ISSN-L)
- 1468-2389
- OpenAlex
- W2131404783
- Cited by
- 132
- References
- 24
- Citations by year
- 2026: 3 · 2025: 9 · 2024: 9 · 2023: 8 · 2022: 9 · 2021: 12 · 2020: 7 · 2019: 13 · 2018: 8 · 2017: 7 · 2016: 8 · 2015: 6 · 2014: 7 · 2013: 4 · 2012: 6
- FWCI
- 5.26
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Selection and Assessment (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Employer Branding and e-HRM
- Keywords
- forced-choice format, criterion validity, operational validity, competency assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review