The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Situational judgment tests: a review of recent research
Abstract
Purpose: The purpose of this paper is to give an empirically‐based review of the strengths and weaknesses of situational judgment tests (SJTs). Design/methodology/approach: The features, history, and development of SJTs are discussed. Next, a computerized search (from 1990‐2007) is conducted to retrieve empirical studies related to SJTs. The effectiveness of SJTs is discussed in terms of reliability, criterion‐related validity, incremental validity, construct‐related validity, utility, adverse impact, applicant perceptions, fakability, and susceptibility to practice and coaching effects. Findings: Strengths of SJTs are that they show criterion‐related validity and incremental validity above cognitive ability and personality tests. SJTs have also less adverse impact towards minorities (especially if the cognitive loading of the SJT is low). Furthermore, applicant reactions towards SJTs are positive and SJTs enable to test large applicant groups at once (through the Internet). In terms of weaknesses, SJTs might be prone to faking, practice, and coaching effects. There is also debate about what constructs are measured by SJTs. Research limitations/implications: Five avenues for future research are discussed: construct‐related validity of SJTs, utility of SJTs vis‐à‐vis other predictors, impact of SJT features on validity and adverse impact, examination of alternative stimulus and response formats, and cross‐cultural transportability of SJTs. Practical implications: Practitioners receive evidence‐based information about the features, development, and strengths and weaknesses of SJTs. Originality/value: Apart from the USA, SJTs have not made strong inroads in selection practice in Europe and other parts of the world. This evidence‐based paper might highlight the value of SJTs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Schmidt, F. L., & Hunter, J. E. (1998) · Psychological Bulletin · doi:10.1037//0033-2909.124.2.262 · Cited by 4,180 · cited 1×
“Typically referred to as incremental validity, the use of additional predictors is of value from a utility viewpoint only when they add additional variance explained in the criterion, beyond that which is accounted for by other, less expensive predictors (Schmidt and Hunter, 1998).”
BackgroundGender differences in personality traits across cultures: Robust and surprising findings.
Costa, P. T., Terracciano, A., & McCrae, R. R. (2001) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.81.2.322 · Cited by 2,986
The g factor: the science of mental ability
(1998) · Choice Reviews Online · doi:10.5860/choice.36-2443 · Cited by 2,746
Applicant Reactions to Selection Procedures: An Updated Model and Meta‐Analysis
Hausknecht, J. P., Day, D. V., & Thomas, S. C. (2004) · Personnel Psychology · doi:10.1111/j.1744-6570.2004.00003.x · Cited by 816 · cited 1×
“That is one of the reasons why work samples and AC exercises typically receive favourable ratings (Hausknecht et al., 2004).”
BackgroundPractical intelligence in real-world pursuits: The role of tacit knowledge.
Wagner, R. K., & Sternberg, R. J. (1985) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.49.2.436 · Cited by 755
Psychological Tests and Personnel Decisions.
Ashton, G. C., Cronbach, L. J., & Gleser, C. (1968) · Biometrics · doi:10.2307/2528052 · Cited by 718
MORGESON, F. P., REIDER, M. H., & CAMPION, M. A. (2005) · Personnel Psychology · doi:10.1111/j.1744-6570.2005.655.x · Cited by 538
An alternative selection procedure: The low-fidelity simulation.
Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990) · Journal of Applied Psychology · doi:10.1037//0021-9010.75.6.640 · Cited by 533
Staffing in the 21st Century: New Challenges and Strategic Opportunities
Ployhart, R. E. (2006) · Journal of Management · doi:10.1177/0149206306293625 · Cited by 505
Use of situational judgment tests to predict job performance: A clarification of the literature.
McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.4.730 · Cited by 470
SITUATIONAL JUDGMENT TESTS, RESPONSE INSTRUCTIONS, AND VALIDITY: A META‐ANALYSIS
McDANIEL, M. A., HARTMAN, N. S., WHETZEL, D. L., & GRUBB, W. L. (2007) · Personnel Psychology · doi:10.1111/j.1744-6570.2007.00065.x · Cited by 468
Psychological Tests and Personnel Decisions.
Hyman, R., Cronbach, L. J., & Gleser, G. C. (1958) · Journal of the American Statistical Association · doi:10.2307/2282612 · Cited by 426
Chan, D. K., & Schmitt, N. (1997) · Journal of Applied Psychology · doi:10.1037//0021-9010.82.1.143 · Cited by 411
RYAN, A. M., McFARLAND, L., & SHL, H. B. (1999) · Personnel Psychology · doi:10.1111/j.1744-6570.1999.tb00165.x · Cited by 377
Staffing work teams: Development and validation of a selection test for teamwork settings
Stevens, M. (1999) · Journal of Management · doi:10.1016/s0149-2063(99)80010-5 · Cited by 361
Oswald, F. L., Schmitt, N., Kim, B. H., Ramsay, L. J., & Gillespie, M. A. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.2.187 · Cited by 305
Situational Judgment Tests: A Review of Practice and Constructs Assessed
Cabrera, M. A. M., & Nguyen, N. T. (2001) · International Journal of Selection and Assessment · doi:10.1111/1468-2389.00167 · Cited by 294
Situational Judgment and Job Performance
Chan, D., & Schmitt, N. (2002) · Human Performance · doi:10.1207/s15327043hup1503_01 · Cited by 230
Incremental validity of situational judgment tests.
Clevenger, J., Pereira, G. M., Wiechmann, D., Schmitt, N., & Harvey, V. S. (2001) · Journal of Applied Psychology · doi:10.1037//0021-9010.86.3.410 · Cited by 184
Subgroup Differences in Situational Judgment Test Performance: A Meta-Analysis
Whetzel, D. L., McDaniel, M. A., & Nguyen, N. T. (2008) · Human Performance · doi:10.1080/08959280802137820 · Cited by 183
Video-based versus written situational judgment tests: A comparison in terms of predictive validity.
Lievens, F., & Sackett, P. R. (2006) · Journal of Applied Psychology · doi:10.1037/0021-9010.91.5.1181 · Cited by 176
Self-citationLievens, F., Buyse, T., & Sackett, P. R. (2005) · Journal of Applied Psychology · doi:10.1037/0021-9010.90.3.442 · Cited by 167
Self-citationVIDEO‐BASED SITUATIONAL TESTING
WEEKLEY, J. A., & JONES, C. (1997) · Personnel Psychology · doi:10.1111/j.1744-6570.1997.tb00899.x · Cited by 166
PLOYHART, R. E., WEEKLEY, J. A., HOLTZ, B. C., & KEMP, C. (2003) · Personnel Psychology · doi:10.1111/j.1744-6570.2003.tb00757.x · Cited by 163
Scoring Situational Judgment Tests: Once You Get the Data, Your Troubles Begin
Bergman, M. E., Drasgow, F., Donovan, M. A., Henning, J. B., & Juraska, S. E. (2006) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2006.00345.x · Cited by 156
Motowidlo, S. J., Hooper, A. C., & Jackson, H. L. (2006) · Journal of Applied Psychology · doi:10.1037/0021-9010.91.4.749 · Cited by 155
Selection in Teams: An Exploration of the Teamwork Knowledge, Skills, and Ability Test
McClough, A. C., & Rogelberg, S. G. (2003) · International Journal of Selection and Assessment · doi:10.1111/1468-2389.00226 · Cited by 155
FURTHER STUDIES OF SITUATIONAL TESTS
Weekley, J. A., & Jones, C. (1999) · Personnel Psychology · doi:10.1111/j.1744-6570.1999.tb00176.x · Cited by 153
RETEST EFFECTS IN OPERATIONAL SELECTION SETTINGS: DEVELOPMENT AND TEST OF A FRAMEWORK
Lievens, F., Buyse, T., & Sackett, P. R. (2005) · Personnel Psychology · doi:10.1111/j.1744-6570.2005.00713.x · Cited by 143
Self-citationPloyhart, R. E., & Ehrhart, M. G. (2003) · International Journal of Selection and Assessment · doi:10.1111/1468-2389.00222 · Cited by 137
Effects of Response Instructions on Faking a Situational Judgment Test
Nguyen, N. T. H., Biderman, M. D., & McDaniel, M. A. (2005) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2005.00322.x · Cited by 132
Pulakos, E. D., & Schmitt, N. (1996) · Human Performance · doi:10.1207/s15327043hup0903_4 · Cited by 130
Examining the impact of administration medium on examinee perceptions and attitudes.
Richman, W. L., Olson‐Buchanan, J. B., & Drasgow, F. (2000) · Journal of Applied Psychology · doi:10.1037/0021-9010.85.6.880 · Cited by 116
Peeters, H., & Lievens, F. (2004) · Educational and Psychological Measurement · doi:10.1177/0013164404268672 · Cited by 109
Self-citationSituational judgment tests theory, measurement, and application
Weekley, J. A., & Ployhart, R. E. (2006) · Cited by 107
Incremental Validity of Situational Judgment Tests for Task and Contextual Job Performance
O’Connell, M. S., Hartman, N. S., McDaniel, M. A., Grubb, W. L., & Lawrence, A. D. (2007) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2007.00364.x · Cited by 101
INTERACTIVE VIDEO ASSESSMENT OF CONFLICT RESOLUTION SKILLS
Olson‐Buchanan, J. B., Drasgow, F., Moberg, P. J., Mead, A. D., Keenan, P. A., & Donovan, M. A. (1998) · Personnel Psychology · doi:10.1111/j.1744-6570.1998.tb00714.x · Cited by 99
Situational judgment test research: Informing the debate on practical intelligence theory
McDaniel, M. A., & Whetzel, D. L. (2005) · Intelligence · doi:10.1016/j.intell.2005.02.001 · Cited by 88
Threats to the Operational Use of Situational Judgment Tests in the College Admission Process
Cullen, M. J., Sackett, P. R., & Lievens, F. (2006) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2006.00340.x · Cited by 87
Self-citationSelection Testing via the Internet: Practical Considerations and Exploratory Empirical Findings*
Potosky, D., & Bobko, P. (2004) · Personnel Psychology · doi:10.1111/j.1744-6570.2004.00013.x · Cited by 86
Lievens, F., & Sackett, P. R. (2007) · Journal of Applied Psychology · doi:10.1037/0021-9010.92.4.1043 · Cited by 85
Self-citationDevelopment and Validation of a Situational Judgment Test of Employee Integrity
Becker, T. E. (2005) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2005.00319.x · Cited by 73
Chan, D. K., & Schmitt, N. (2017) · Wiley eBooks · doi:10.1002/9781405164221.ch10 · Cited by 72
Situational Judgment Tests: Method or Construct?
Schmitt, N., & Chan, D. K. (2006) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 72
Applicant Perspectives in Selection: Going beyond preference reactions
Hulsheger, U. R., & Anderson, N. R. (2009) · International Journal of Selection and Assessment · doi:10.1111/j.1468-2389.2009.00477.x · Cited by 58
From the Subjects' Point of View
Kanning, U. P., Grewe, K., Hollenberg, S., & Hadouch, M. (2006) · European Journal of Psychological Assessment · doi:10.1027/1015-5759.22.3.168 · Cited by 58
PARALLEL TEST FORM DEVELOPMENT: A PROCEDURE FOR ALTERNATE PREDICTORS AND AN EXAMPLE
Clause, C. S., Mullins, M. E., NEE, M. T., Pulakos, E. D., & Schmitt, N. (1998) · Personnel Psychology · doi:10.1111/j.1744-6570.1998.tb00722.x · Cited by 54
predicting insurance agent turnover using a video-based situational judgment test
Dalessio, A. T. (1994) · Journal of Business and Psychology · doi:10.1007/bf02230984 · Cited by 34
Web‐based Assessment of Call Center Agents: Development and Validation of a Computerized Instrument
Konradt, U., Hertel, G., & Joder, K. (2003) · International Journal of Selection and Assessment · doi:10.1111/1468-2389.00242 · Cited by 31
A Supervisory Practices Test *
Bruce, M. M., & LEARNER, D. B. (1958) · Personnel Psychology · doi:10.1111/j.1744-6570.1958.tb00015.x · Cited by 26
Extending a Practical Method for Developing Alternate Test Forms Using Independent Sets of Items
Oswald, F. L., Friede, A., Schmitt, N., Kim, B. H., & Ramsay, L. J. (2005) · Organizational Research Methods · doi:10.1177/1094428105275365 · Cited by 25
International Situational Judgment Tests
Lievens, F. (2006) · Ghent University Academic Bibliography (Ghent University) · Cited by 22
Self-citationEvaluation of video-based assessment in transit operator selection
Smiderle, D., Perry, B. A., & Cronshaw, S. F. (1994) · Journal of Business and Psychology · doi:10.1007/bf02230983 · Cited by 19
Research works printed but not linked: 5
- Cook, M. (2003), Personnel Selection: Adding Value through People, Wiley, Chichester.
- Funke, U. and Schuler, H. (1998), “Validity of stimulus and response components in a video test of social competence”, International Journal of Selection and Assessment, Vol. 6, pp. 115‐23. · doi:10.1111/1468-2389.00080
- Hunter, D.R. (2003), “Measuring general aviation pilot judgment using a situational judgment technique”, International Journal of Aviation Psychology, Vol. 13 No. 4, pp. 373‐86. · doi:10.1207/s15327108ijap1304_03
- McHenry, J.J. and Schmitt, N. (1994), “Multimedia testing”, in Rumsey, M.G. and Walker, C.B. (Eds), Personnel Selection and Classification, Lawrence Erlbaum Associates, Hillsdale, NJ, pp. 193‐232.
- Such, M.J. and Schmidt, D.B. (2004), “Examining the effectiveness of empirical keying: a cross‐cultural perspective”, paper presented at the 19th Annual Conference of the Society for Industrial and Organizational Psychology, Chicago, IL, April.
Cited by
How effective are selection methods in medical education? A systematic review
Patterson, F., Knight, A., Dowell, J., Nicholson, S., Cousans, F., & Cleland, J. (2016) · Medical Education · doi:10.1111/medu.12817 · Cited by 481
Assessing Intercultural Competence in Higher Education: Existing Research and Future Directions
Griffith, R. L., Wolfeld, L., Armon, B. K., Rios, J., & Liu, O. L. (2016) · ETS Research Report Series · doi:10.1002/ets2.12112 · Cited by 211
Empowering Leadership and Effective Collaboration in Geographically Dispersed Teams
Hill, N. S., & Bartol, K. M. (2016) · Personnel Psychology · doi:10.1111/peps.12108 · Cited by 170
Armstrong, M. B., Landers, R. N., & Collmus, A. B. (2016) · Advances in Multimedia and Interactive Technologies · doi:10.4018/978-1-4666-8651-9.ch007 · Cited by 85
Correcting Misconceptions About Gamification of Assessment: More Than SJTs and Badges
Armstrong, M. B., Ferrell, J. Z., Collmus, A. B., & Landers, R. N. (2016) · Industrial and Organizational Psychology · doi:10.1017/iop.2016.69 · Cited by 38
Metadata
- Authors
- Lievens, F., Peeters, H., & Schollaert, E.
- Venue
- Personnel Review
- Publisher
- Emerald
- Published
- 2008-06-06
- Volume
- 37
- Issue
- 4
- Pages
- 426-441
- ISSN
- 0048-3486 (ISSN-L)
- OpenAlex
- W1968956327
- Cited by
- 301
- References
- 53
- FWCI
- 5.58
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.emerald.com/insight/site-policies (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.