Developing and evaluating language‐based machine learning algorithms for inferring applicant personality in video interviews
Published version of a preprint
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Organisations are increasingly relying on people analytics to aid human resources decision‐making. One application involves using machine learning to automatically infer applicant characteristics from employment interview responses. However, management research has provided scant validity evidence to guide organisations' decisions about whether and how best to implement these algorithmic approaches. To address this gap, we use closed vocabulary text mining on mock video interviews to train and test machine learning algorithms for predicting interviewee's self‐reported ( automatic personality recognition ) and interviewer‐rated personality traits ( automatic personality perception ). We use 10‐fold cross‐validation to test the algorithms' accuracy for predicting Big Five personality traits across both rating sources. The cross‐validated accuracy for predicting self‐reports was lower than large‐scale investigations using language in social media posts as predictors. The cross‐validated accuracy for predicting interviewer ratings of personality was more than double that found for predicting self‐reports. We discuss implications for future research and practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods
Tausczik, Y. R., & Pennebaker, J. W. (2010) · Journal of Language and Social Psychology · doi:10.1177/0261927x09351676 · Cited by 5,774 · cited 3×
“These approaches are based on the idea that personality affects both what people talk about and how they talk about it, which results in between-person differences in patterns of language use (Tausczik & Pennebaker, 2010).”
BackgroundInfluentialBayesian calibration of computer models
Kennedy, M. C., & O'Hagan, A. (2001) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00294 · Cited by 4,222
Mischel, W., & Shoda, Y. (1995) · Psychological Review · doi:10.1037/0033-295x.102.2.246 · Cited by 3,889
The Mini-IPIP Scales: Tiny-yet-effective measures of the Big Five Factors of Personality.
Donnellan, M. B., Oswald, F. L., Baird, B. M., & Lucas, R. E. (2006) · Psychological Assessment · doi:10.1037/1040-3590.18.2.192 · Cited by 2,442 · cited 15×
“In the present study, the strongest correlation between any Big Five personality trait score and LIWC predictor was between interviewer-reported extraversion and word count (i.e., number of words used by interviewee, r = .45; Table 3).”
InfluentialLinguistic styles: language use as an individual difference
Pennebaker, J. W., & King, L. A. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.77.6.1296 · Cited by 1,961
Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., Shah, A., Kosinski, M., Stillwell, D., Seligman, M. E. P., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0073791 · Cited by 1,700
Who knows what about a person? The self–other knowledge asymmetry (SOKA) model.
Vazire, S. (2010) · Journal of Personality and Social Psychology · doi:10.1037/a0017908 · Cited by 1,080 · cited 3×
“…it is also important to note that even with longer interviews and trained raters (e.g., professional recruiters), self-and interviewer-ratings may not perfectly converge due to the unique perspectives that interviewee and interviewer bring (Connelly & Ones, 2010; Funder & West, 1993; Vazire, 2010).”
BackgroundMethodConnelly, B. S., & Ones, D. S. (2010) · Psychological Bulletin · doi:10.1037/a0021212 · Cited by 1,059 · cited 6×
“…it is also important to note that even with longer interviews and trained raters (e.g., professional recruiters), self-and interviewer-ratings may not perfectly converge due to the unique perspectives that interviewee and interviewer bring (Connelly & Ones, 2010; Funder & West, 1993; Vazire, 2010).”
BackgroundResultInfluentialAutomatic personality assessment through social media language.
Park, G., Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Kosinski, M., Stillwell, D. J., Ungar, L. H., & Seligman, M. E. P. (2015) · Journal of Personality and Social Psychology · doi:10.1037/pspp0000020 · Cited by 802 · cited 1×
“Data-driven approaches allow exploring all potential interviewee language (e.g., Park et al., 2015) rather than limiting automatic personality recognition and perception to traditional conceptualizations of personality.”
BackgroundMehl, M. R., Gosling, S. D., & Pennebaker, J. W. (2006) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.90.5.862 · Cited by 764 · cited 1×
“…given context, personality traits should relate to language use, and this is what researchers have found in a variety of contexts, including everyday conversations (Mehl et al., 2006), personal essays (Pennebaker & King, 1999), blogs (Yarkoni, 2010), and social media posts (Schwartz et al., 2013).”
Background
Research works printed but not linked: 43
- 10.1016/j.paid.2017.12.018 · DOI · Paper page
- 10.1111/j.1744‐6570.2000.tb02424.x · DOI
- 10.1177/1088868318772990 · DOI · Paper page
- 10.1111/ijsa.12174 · DOI
- 10.1037/apl0000108 · DOI · Paper page
- 10.1016/j.cobeha.2017.04.007 · DOI
- 10.1037/met0000088 · DOI
- 10.1111/jopy.12161 · DOI
- 10.1111/j.1467‐6494.1993.tb00778.x · DOI · Paper page
- 10.1037/1040-3590.4.1.26 · DOI · Paper page
- Goldberg L. R., 1999, A broad‐bandwidth, public domain, personality inventory measuring the lower‐level facets of several five‐factor models, Personality Psychology in Europe, 7, 7
- 10.1016/S0092‐6566(03)00046‐1 · DOI · Paper page
- Harris K. D., 2018, Letter to U.S. Equal Employment OpportunityCommission regarding risks of facial recognition technology
- 10.25035/pad.2019.03.003 · DOI
- 10.1177/1094428120971683 · DOI · Paper page
- Hogan R. T.(1991).Personality and personality measurement. InM. D.Dunnette&L. M.Hough(Eds.) Handbook of industrial and organizational psychology(2nded.) Vol 2 pp.873–919.Palo Alto CA:Consulting Psychologists Press Inc.
- 10.1037//0003‐066X.51.5.469 · doi:10.1037//0003-066x.51.5.469 · Paper page
- 10.1016/j.hrmr.2011.05.003 · DOI
- IBM, 2019, IBM Watson Speech to Text
- 10.1016/j.hrmr.2016.08.003 · DOI · Paper page
- 10.1177/1073191113514104 · DOI · Paper page
- 10.1053/j.sodo.2009.03.002 · DOI
- Kutik B., 2015, HireVue: From video to predictive analytics
- 10.1111/peps.12052 · DOI · Paper page
- 10.1037/a0025741 · DOI · Paper page
- 10.1177/0149206316657594 · DOI · Paper page
- Mondragon N., 2019, The next generation of assessments
- 10.1146/annurev‐orgpsych‐032117‐104553 · DOI · Paper page
- Pennebaker J. W., 2015, The development and psychometric properties of LIWC2015
- 10.1037/apl0000081 · DOI · Paper page
- 10.1177/1094428117697041 · DOI
- 10.1016/j.jrp.2012.08.008 · DOI · Paper page
- 10.1017/iop.2018.6 · DOI
- 10.3389/fpsyg.2019.02227 · DOI
- 10.1111/j.1744‐6570.2012.01250.x · DOI · Paper page
- 10.1111/peps.12263 · DOI
- 10.1002/per.2290 · DOI
- 10.1177/0149206307305563 · DOI
- 10.1037/0021‐9010.90.3.536 · DOI · Paper page
- 10.1037/a0024343 · DOI · Paper page
- 10.1109/TAFFC.2014.2330816 · DOI · Paper page
- 10.1037/0000193-000 · doi:10.1037/0000193-000
- 10.1111/j.1467‐9868.2005.00527.x · DOI · Paper page
Cited by
Edwards, M. R., Charlwood, A., Guenole, N., & Marler, J. (2024) · Human Resource Management Journal · doi:10.1111/1748-8583.12435 · Cited by 66
Murugeah, M. K. (2024) · International Journal for Multidimensional Research Perspectives · doi:10.61877/ijmrp.v2i7.162 · Cited by 20
Ding, N., Chen, M., & Hu, L. (2025) · Acta Psychologica · doi:10.1016/j.actpsy.2025.105709 · Cited by 7
McCartney, S., Minbaeva, D., Crean, S., & Fahy, A. (2026) · Human Resource Management Review · doi:10.1016/j.hrmr.2026.101147 · Cited by 2
Metadata
- Authors
- Hickman, L., Saef, R., Ng, V., Woo, S. E., Tay, L., & Bosch, N.
- Venue
- Human Resource Management Journal
- Publisher
- Wiley
- Published
- 2021-05-07
- Volume
- 34
- Issue
- 2
- Pages
- 255-274
- ISSN
- 0954-5395 (ISSN-L)
- 1748-8583
- OpenAlex
- W3162383644
- Cited by
- 58
- References
- 17
- FWCI
- 7.95
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Developing and evaluating language‐based machine learning algorithms for inferring applicant personality in video interviews. https://app.cochat.ai/papers/10.1111/1748-8583.12356
- The paper
- Hickman, L., Saef, R., Ng, V., Woo, S. E., Tay, L., & Bosch, N. (2024). Developing and evaluating language‐based machine learning algorithms for inferring applicant personality in video interviews. Human Resource Management Journal. https://doi.org/10.1111/1748-8583.12356
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.