The DOI matches its Crossref record; 25 of the 27 references it cites were found in a registry or an index, and 1 are not checked yet.
Manipulating the Perceived Personality Traits of Language Models
Open accessAbstract
Psychology research has long explored aspects of human personality like extroversion, agreeableness and emotional stability, three of the personality traits that make up the 'Big Five'.Categorizations like the 'Big Five' are commonly used to assess and diagnose personality types.In this work, we explore whether text generated from large language models exhibits consistency in it's perceived 'Big Five' personality traits.For example, is a language model such as GPT2 likely to respond in a consistent way if asked to go out to a party?We also show that when exposed to different types of contexts (such as personality descriptions, or answers to diagnostic questions about personality traits), language models such as BERT and GPT2 consistently identify and mirror personality markers in those contexts.This behavior illustrates an ability to be manipulated in a predictable way (with correlations up to 0.84 between intended and realized changes in personality traits), and frames them as tools for controlling personas in applications such as dialog systems.We contribute two data-sets of personality descriptions of humans subjects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Today's state-of-the-art language models are shaped by human training data and exhibit many human abilities, such as reasoning by analogy, theory of mind, and personality, which makes them compelling proxies for studying human behavior and attitude change.
“On the one hand, these models are shaped by human training data and exhibit many human abilities, such as reasoning by analogy21, theory of mind22, and personality23, which makes them compelling proxies for studying human behavior and attitude change24,25.”
From Generative language models exhibit social identity biases (2024) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
(1999) · Cited by 7,995
Self-citationAn alternative "description of personality": The Big-Five factor structure.
Goldberg, L. R. (1990) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.59.6.1216 · Cited by 6,652
The structure of phenotypic personality traits.
Goldberg, L. R. (1993) · American Psychologist · doi:10.1037/0003-066x.48.1.26 · Cited by 4,816
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
Bender, E. M., & Koller, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.463 · Cited by 1,082 · cited 1×
“Thus, even though language models encode probability distributions over text and the tendency to assign cognitive abilities to them has been criticized (Bender and Koller, 2020), the way users perceive these systems can have significant consequences.”
BackgroundBig Five personality predictors of post-secondary academic performance
O’Connor, M. C., & Paunonen, S. V. (2007) · Personality and Individual Differences · doi:10.1016/j.paid.2007.03.017 · Cited by 1,049
Out of One, Many: Using Language Models to Simulate Human Samples
Argyle, L. P., Busby, E. C., Fulda, N., Gubler, J. R., Rytting, C., & Wingate, D. (2023) · Political Analysis · doi:10.1017/pan.2023.2 · Cited by 783
A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk
Hara, K., Adams, A., Milland, K., Savage, S., Callison-Burch, C., & Bigham, J. P. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174023 · Cited by 497
Smith, S., Patwary, M., Norick, B., LeGresley, P., Rajbhandari, S., Casper, J., Liu, Z., Prabhumoye, S., Zerveas, G., Korthikanti, V., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.11990 · Cited by 286
Big five personality variables and relationship constructs
White, J. K., Hendrick, S. S., & Hendrick, C. (2004) · Personality and Individual Differences · doi:10.1016/j.paid.2004.02.019 · Cited by 284
Identifying and Reducing Gender Bias in Word-Level Language Models
Bordia, S., & Bowman, S. R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop · doi:10.18653/v1/n19-3002 · Cited by 219
Large language models associate Muslims with violence
Abid, A., Farooqi, M., & Zou, J. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00359-2 · Cited by 176
Christian, H., Suhartono, D., Chowanda, A., & Zamli, K. Z. (2021) · Journal Of Big Data · doi:10.1186/s40537-021-00459-1 · Cited by 169
Who is GPT-3? An exploration of personality, values and demographics
Miotto, M., Rossberg, N., & Kleinberg, B. (2022) · Workshop on Natural Language Processing and Computational Social Science (NLP+CSS) · doi:10.18653/v1/2022.nlpcss-1.24 · Cited by 159
Bottom-Up and Top-Down: Predicting Personality with Psycholinguistic and Language Model Features
Mehta, Y., Fatehi, S., Kazameini, A., Stachl, C., Cambria, E., & Eetemadi, S. (2020) · Proceedings · doi:10.1109/icdm50108.2020.00146 · Cited by 146
Evaluating and Inducing Personality in Pre-trained Language Models
Jiang, G., Xu, M., Zhu, S. C., Zhang, C., Zhang, C., & Zhu, Y. (2023) · neural information processing systems · doi:10.52202/075280-0466 · Cited by 138
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation
Huang, P.-S., Zhang, H., Jiang, R., Stanforth, R., Welbl, J., Rae, J., Maini, V., Yogatama, D., & Kohli, P. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.7 · Cited by 124
Do personality characteristics predict longevity? Findings from the Tokyo Centenarian Study
Masui, Y., Gondo, Y., Inagaki, H., & Hirose, N. (2006) · AGE · doi:10.1007/s11357-006-9024-6 · Cited by 85
Estimating the Personality of White-Box Language ModelsPreprint
Karra, S. R., Nguyen, S. T., & Tulabandhula, T. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4598766 · Cited by 77
The myers-briggs type indicator: A critical review and practical guide
Corr, P. J. (1996) · Personality and Individual Differences · doi:10.1016/0191-8869(96)83453-2 · Cited by 60
What Context Features Can Transformer Language Models Use?
O’Connor, J., & Andreas, J. (2021) · doi:10.18653/v1/2021.acl-long.70 · Cited by 59
Cognitive Anthropomorphism of AI: How Humans and Computers Classify Images
Mueller, S. T. (2020) · Ergonomics in Design The Quarterly of Human Factors Applications · doi:10.1177/1064804620920870 · Cited by 32
Learning to Answer Psychological Questionnaire for Personality Detection
Yang, F., Yang, T., Quan, X., & Su, Q. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.98 · Cited by 19
A negation detection assessment of GPTs: analysis with the xNot360 datasetPreprint
Nguyen, H.-T., Goebel, R. G., Toni, F., Stathis, K., & Satoh, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.16638 · Cited by 16
Comparative evaluation of medical vs. Social treatment of alcohol withdrawal syndrome
McGovern, M. P. (1983) · Journal of Clinical Psychology · doi:10.1002/1097-4679(198309)39:5 · Cited by 14
Flagged first, then by citations
1 of the 27 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- Oliver P. John, Laura P. Naumann, and Christopher J. Soto. 2008. Paradigm shift to the integrative big-five trait taxonomy: History, measurement, and concep- tual issues. In Oliver P. John, Richard W. Robins, and Lawrence A. Pervin, editors, Handbook of per- sonality: Theory and research, pages 114-158. The Guilford Press, New York, New York.
- Saketh Reddy Karra, Son Nguyen, and Theja Tula- bandhula. 2022. AI Personification: Estimating the personality of language models. arXiv preprint arXiv:2204.12000. · arXiv
- Anastasia Kuzminykh, Jenny Sun, Nivetha Govindaraju, Jeff Avery, and Edward Lank. 2020. Genie in the bottle: Anthropomorphized perceptions of conversa- tional agents. In Proceedings of the 2020 CHI Con- ference on Human Factors in Computing Systems, pages 1-13.
- Arleen Salles, Kathinka Evers, and Michele Farisco. 2020. Anthropomorphism in ai. AJOB neuroscience, 11(2):88-95.
- Jerome P Wagner and Ronald E Walker. 1983. Relia- bility and validity study of a sufi personality typol- ogy: The enneagram. Journal of clinical psychology, 39(5):712-717.
- Marty J Wolf, Keith W Miller, and Frances S Grodzin- sky. 2017. Why we should have seen that com- ing: comments on microsoft's tay "experiment," and wider implications. The ORBIT Journal, 1(2):1-12.
Cited by
Sorted by citations
Sarstedt, M., Adler, S. J., Rau, L., & Schmitt, B. (2024) · Psychology & Marketing · doi:10.1002/mar.21982 · Cited by 149
Language Model Behavior: A Comprehensive Survey
Chang, T. A., & Bergen, B. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00492 · Cited by 105
Generative language models exhibit social identity biases
Hu, T., Kyrychenko, Y., Rathje, S., Collier, N., van der Linden, S., & Roozenbeek, J. (2024) · Nature Computational Science · doi:10.1038/s43588-024-00741-1 · Cited by 100
“GPT-2-Large Positive .5106 .9375 2 Results”
Milička, J., Marklová, A., VanSlambrouck, K., Pospíšilová, E., Šimsová, J., Harvan, S., & Drobil, O. (2024) · PLoS ONE · doi:10.1371/journal.pone.0298522 · Cited by 34
Can AI with High Reasoning Ability Replicate Human-like Decision Making in Economic Experiments?
Kitadai, A., Lugo, S. D. R., Tsurusaki, Y., Fukasawa, Y., & Nishino, N. (2025) · Group Decision and Negotiation · doi:10.1007/s10726-025-09946-9 · Cited by 21
Editing Personality For Large Language Models
Mao, S., Wang, X., Wang, M., Jiang, Y., Xie, P., Huang, F., & Zhang, N. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-9434-8_19 · Cited by 58
PsychAdapter: adapting LLMs to reflect traits, personality, and mental health
Vu, H., Nguyen, H. A., Ganesan, A. V., Juhng, S., Kjell, O., Sedoc, J., Kern, M. L., Boyd, R. L., Ungar, L., Schwartz, H. A., et al. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00071-9 · Cited by 21
LMLPA: Language Model Linguistic Personality Assessment
Zheng, J., Wang, X., Hosio, S. J., Xu, X., & Lee, L. (2025) · Computational Linguistics · doi:10.1162/coli_a_00550 · Cited by 7
Barbera, D. L., Lunardi, R., Zhuang, M., & Roitero, K. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744581 · Cited by 5
Collectivism and individualism political bias in large language models: A two-step approach
Xiao-bo, S. H. A. N., Teng, Y., Wang, Y., Zhao, H., & Wang, Y. (2025) · Big Data & Society · doi:10.1177/20539517251343861 · Cited by 3
Challenging the Validity of Personality Tests for Large Language Models
Sühr, T., Dorner, F. E., Samadi, S., & Kelava, A. (2025) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3757887.3763016 · Cited by 2
Communication experiment with moral-based chatbot: toward solution of divisions in democracy
Miwa, K., Yamakawa, M., Nakahara, A., & Shimada, A. (2025) · AI & Society · doi:10.1007/s00146-025-02377-9 · Cited by 2
APEE: Assessing the Personality Expressions of LLM-Driven Role Play Agent Beyond Self-Perception
Li, K., Dai, C., Zhou, W., & Hu, S. (2025) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd64889.2025.11033568
Experimentos en apertura y consciencia: modificando aspectos de personalidad con MML y prompting
Meza, I., García, I. V., & Castro, O. R. G. (2025) · TIES Revista de Tecnología e Innovación en Educación Superior · doi:10.22201/dgtic.26832968e.2025.12.74
An Empathetic Dialogue Generation Method Incorporating Personality Traits
Yan, C., Ding, B., Pei, Z., Chen, H., Guo, X., & Yao, Y. (2025) · International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI) · doi:10.1109/ichci67614.2025.11292076
Extralinguistic factors of language personality formation
Искакова, К., Kushkarova, G. K., Sarkulova, Z., Turlan, Z., & Kurmanova, Z. B. (2026) · Lodz Papers in Pragmatics · doi:10.1515/lpp-2024-0041
Sasaki, K., Kawaguchi, S., Sakura, M., & Nishide, S. (2026) · Electronics · doi:10.3390/electronics15102030
Rehumanizer: Enhancing LLM Recommendations for Humanlike Engagement and Intriguing Interactions
Galitsky, B. (2026) · Springer eBooks · doi:10.1007/978-3-032-16524-4_5
Kim, J. (2026) · Journal of Smart Tourism · doi:10.1177/27652157261488111
Who fights best? Discovering winning personality pairings for LLM debate agents
Wang, J., Liu, K., Xiang, L., Zhang, W., Qian, T., & Lu, W. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114990
Metadata
- Authors
- Caron, G., & Srivastava, S.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- ORCID
- Srivastava, Shashank
- Venue
- Findings of the Association for Computational Linguistics: EMNLP 2023
- Published
- 2023
- Pages
- 2370-2386
- OpenAlex
- W4389519968
- Cited by
- 20
- References
- 27
- Citations by year
- 2026: 6 · 2025: 9 · 2024: 4 · 2023: 1
- FWCI
- 3.54
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by)
- Funders
- National Science Foundation (2112635)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Psychology › Clinical Psychology › Personality Traits and Psychology
- Keywords
- Big Five personality traits, large language models, personality consistency, GPT-2, BERT, personality description, dialogue systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
