The DOI matches its Crossref record; 23 of the 24 references it cites were found in a registry or an index.
Increasing Validity with Forced-Choice Criterion Measurement Formats
Abstract
The relative validities of forced‐choice (ipsative) and Likert rating‐scale item formats as criterion measures are examined. While there has been much debate about the relative technical and psychometric merits and demerits of ipsative instruments, the present research focused on the crucial question of whether the use of this format has any practical benefit – in terms of improved validity. An analysis is reported from a meta‐analysis data set. This demonstrates that higher operational validity coefficients (prediction of line‐manager ratings of competencies) are associated with the use of forced‐choice (r=.38) rather than rating scale (r=.25) item formats for the criterion measurement instrument when performance is rated by the same line managers on both formats and where the predictor is held constant. Thus the apparent criterion‐related validity of a predictor can increase by 50% simply by changing the format of the criterion measurement instrument. The implications of this for practice are discussed.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The use of forced-choice, rank orders was motivated by research showing increased validity of these formats compared to rating scales [32].”
From What format of treatment do patients with emotional disorders prefer and why? Implications for public mental health settings and policies (2019) · cites this paper as [32]
Citations
Cites
Schmidt, F. L., & Hunter, J. E. (1998) · Psychological Bulletin · doi:10.1037//0033-2909.124.2.262 · Cited by 4,180
Hunter, J., & Schmidt, F. (2004) · doi:10.4135/9781412985031 · Cited by 3,023
The five factor model of personality and job performance in the European Community.
Salgado, J. F. (1997) · Journal of Applied Psychology · doi:10.1037/0021-9010.82.1.30 · Cited by 1,582
The Great Eight Competencies: A Criterion-Centric Approach to Validation.
Bartram, D. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.6.1185 · Cited by 745
Self-citationMeasurement error in psychological research: Lessons from 26 research scenarios.
Schmidt, F. L., & Hunter, J. E. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.199 · Cited by 567
Comparative analysis of the reliability of job performance ratings.
Viswesvaran, C., Ones, D. S., & Schmidt, F. L. (1996) · Journal of Applied Psychology · doi:10.1037/0021-9010.81.5.557 · Cited by 505
Salgado, J. F., Anderson, N. R., Moscoso, S., Bertua, C., Fruyt, F. D., & Rolland, J. P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.6.1068 · Cited by 364
Strengths and limitations of ipsative measurement
Baron, H. (1996) · Journal of Occupational and Organizational Psychology · doi:10.1111/j.2044-8325.1996.tb00599.x · Cited by 321
On the Reliability of a Weighted Composite
Mosier, C. I. (1943) · Psychometrika · doi:10.1007/bf02288700 · Cited by 261
Reconsidering forced-choice item formats for applicant personality assessment
Christiansen, N. D., Burns, G. N., & Montgomery, G. E. (2005) · Human Performance · doi:10.1207/s15327043hup1803_4 · Cited by 250
Research works printed but not linked: 5
- Baron H.(2002). Working with Ipsative Measures.Paper Presented at the 17th Annual Conference of the Society for Industrial and Organizational Psychology April 2002 Toronto Canada.
- 10.1037/0033-2909.124.2.262 · DOI · Paper page
- SHL, 1993, OPQ Concept Model: Manual and user's guide
- SHL, 1993, Inventory of Management Competencies: Manual and user's guide
- SHL, 1999, OPQ32: Manual and user's guide
Cited by
Sorted by citations
Items 101–136 of 136
Hanging in the Balance: Assessing Goal Prioritization Among Street-Level Bureaucrats
Rutherford, A., & Khurana, S. (2025) · Public Performance & Management Review · doi:10.1080/15309576.2025.2572102 · Cited by 1
Young, M., & Dulewicz, V. (2025) · Military Psychology · doi:10.1080/08995605.2025.2493392 · Cited by 1
Job performance predictors in a group of information and communication technology specialists
Berga, L., & Austers, I. (2022) · BALTIC JOURNAL OF PSYCHOLOGY · doi:10.22364/bjp.23.01 · Cited by 1
Saksvik‐Lehouillier, I., Eriksen, I. B., & Langvik, E. (2020) · Cogent Psychology · doi:10.1080/23311908.2020.1772631 · Cited by 3
Ипсативный подход к оценке личности: возможности и ограничения
Аксенова, Е. (2012) · Психологические исследования · doi:10.54359/ps.v5i24.756 · Cited by 1
Personality and Job Performance: Using Innovative Techniques to Improve Predictive Validities
Owens, C. E., Paul Irwing, B., & Clarke, S. (2019) · Academy of Management Proceedings · doi:10.5465/ambpp.2019.13292abstract · Cited by 1
(2018) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-78743-785-220181022 · Cited by 1
On the information obtainable from comparative judgmentsPreprint
Bürkner, P. (2021) · doi:10.31234/osf.io/j5efn · Cited by 1
Comparison between Wave and OPQ32 – a comment on McDowall and Kurz
Bartram, D. (2008) · The Coaching Psychologist · doi:10.53841/bpstcp.2008.4.3.134 · Cited by 1
Self-citationFarrés, M. G. (2013) · Dipòsit Digital de la Universitat de Barcelona (Universitat de Barcelona)
Competencies as a predictor of work performance for branch managers in a banking institution
Pema-Mistry, D. (2014) · Unisa Institutional Repository (University of South Africa)
Earnings Manipulation and Relative Deprivation - A Meaning-Making EndeavourPreprint
Mohapatra, S. (2011) · SSRN Electronic Journal · doi:10.2139/ssrn.1923191
The Forced-Choice Certainty Method: A Review of an Underutilized Survey Approach
Seifried, C. S. (2013) · Kinesiology Review · doi:10.1123/krj.2.2.130
Development of the Older Adult Prescription Drug Assessment Questionnaire for Case Workers
Pope, N. D., Slovak, K. L., & Giger, J. T. (2016) · Journal of Applied Gerontology · doi:10.1177/0733464816655437
Seleksjonssituasjonens betydning på personlighetsskårer
Pedersen, M. M. (2015) · BIBSYS Brage (BIBSYS (Norway))
Critical competencies of self-employed workers:development and validation of a survey instrument
Bongers, T. T. (2011) · TU/e Research Portal
Waldron, C. (2018) · Massey Research Online (Massey University)
Comparison of a Ranking and Rating Format of the 5Plus5: A Personality Measure
Jungemann, K. L. (2011) · Cornerstone (Minnesota State University, Mankato)
Hughes, A. W., Dunlop, P. D., Holtrop, D., & Wee, S. (2021) · doi:10.31234/osf.io/r78mb
Cook, M. (2008) · Wiley eBooks · doi:10.1002/9780470742723.refs
IRT-based scoring methods for multidimensional forced choice tests
Juan, L. I. U., ZHENG, C., Yunchuan, L. I., & Xu, L. I. A. N. (2022) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2022.01410
Nguyen, P. L. L. (2022) · doi:10.31234/osf.io/qx82g
Gabalou, P. F., Younesi, J., Delāvar, A., Asgari, M., & zamanpour, E. (2022) · The Journal Of Psychological Science · doi:10.52547/jps.21.113.885
Baltic Journal of Psychology, Vol. 23 (1/2), 2022
University of Latvia (2022) · BALTIC JOURNAL OF PSYCHOLOGY · doi:10.22364/bjp.23
Optimizing the Value of Assessments Through Myth Busting and Stakeholder Communications
Seaton, G. A., & Church, A. H. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780197611050.003.0030
Lavoie-Trudeau, É., Beaudoin, M., & Carrier, A. (2023) · Mesure et évaluation en éducation · doi:10.7202/1111100ar
Transforming Depression to Healing: Design Strategy for Community Mental Health
Liu, Z., & Pan, L. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-94162-7_14
The Dominant Trait Profile Method of Scoring Multidimensional Forced-Choice Questionnaires
Dimitrov, D. M. (2025) · Educational and Psychological Measurement · doi:10.1177/00131644251360386
社会的望ましさバイアスに頑健な比較型尺度の構築法――総説とチュートリアルPreprint
Tan, Y., Bunji, K., Nomura, K., & Okada, K. (2025) · doi:10.31234/osf.io/7ugez_v1
Ayuni, R. P., & Hakim, M. A. (2025) · Psympathic Jurnal ilmiah Psikologi · doi:10.15575/psy.v12i2.44353
Lee, P., & Ivany, R. (2025) · OSF Preprints (OSF Preprints)
Henríquez-Rivas, C., Gómez, A. S. V., & Campos, J. G. (2026) · Eurasia Journal of Mathematics Science and Technology Education · doi:10.29333/ejmste/17910
Wang, J., (65477), Y. W., & Wei, Q. (2026) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2026.103852
Grobler, S. (2026) · SA Journal of Industrial Psychology · doi:10.4102/sajip.v52i0.2447
Kinahan, M. (2014) · Dublin City University Open Access Institutional Repository (Dublin City University)
Metadata
- Authors
- Bartram, D.
- Corresponding author
- Bartram, Dave
- ORCID
- Bartram, Dave
- Venue
- International Journal of Selection and Assessment
- Publisher
- Wiley
- Published
- 2007-08-14
- Volume
- 15
- Issue
- 3
- Pages
- 263-272
- ISSN
- 0965-075X (ISSN-L)
- 1468-2389
- OpenAlex
- W2131404783
- Cited by
- 132
- References
- 24
- Citations by year
- 2026: 3 · 2025: 9 · 2024: 9 · 2023: 8 · 2022: 9 · 2021: 12 · 2020: 7 · 2019: 13 · 2018: 8 · 2017: 7 · 2016: 8 · 2015: 6 · 2014: 7 · 2013: 4 · 2012: 6
- FWCI
- 5.26
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Selection and Assessment (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Employer Branding and e-HRM
- Keywords
- forced-choice format, criterion validity, operational validity, competency assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review