CoChat
Sign Up
PreprintCited by 8Emotion and Mood RecognitionEmotional Intelligence and PerformancePsychiatry, Mental Health, Neuroscience

Emotional Intelligence of Large Language Models

Open access
Wang, Xuena, Li, Xueting, Yin, Zi, Wu, Yue, Jia, Liu·2023·arXiv
DOI 10.48550/arxiv.2307.09042arXiv 2307.09042
Open this paper in CoChat — free accountOpen PDF

Abstract

Large Language Models (LLMs) have demonstrated remarkable abilities across numerous disciplines, primarily assessed through tasks in language generation, knowledge utilization, and complex reasoning. However, their alignment with human emotions and values, which is critical for real-world applications, has not been systematically evaluated. Here, we assessed LLMs' Emotional Intelligence (EI), encompassing emotion recognition, interpretation, and understanding, which is necessary for effective communication and social interactions. Specifically, we first developed a novel psychometric assessment focusing on Emotion Understanding (EU), a core component of EI, suitable for both humans and LLMs. This test requires evaluating complex emotions (e.g., surprised, joyful, puzzled, proud) in realistic scenarios (e.g., despite feeling underperformed, John surprisingly achieved a top score). With a reference frame constructed from over 500 adults, we tested a variety of mainstream LLMs. Most achieved above-average EQ scores, with GPT-4 exceeding 89% of human participants with an EQ of 117. Interestingly, a multivariate pattern analysis revealed that some LLMs apparently did not reply on the human-like mechanism to achieve human-level performance, as their representational patterns were qualitatively distinct from humans. In addition, we discussed the impact of factors such as model size, training method, and architecture on LLMs' EQ. In summary, our study presents one of the first psychometric evaluations of the human-like characteristics of LLMs, which may shed light on the future development of LLMs aiming for both high intellectual and emotional intelligence. Project website: https://emotional-intelligence.github.io/

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2307.09042

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 39

  1. Amin, M. M., Cambria, E., & Schuller, B. W. (2023). Will Affective Computing Emerge from Foundation Models and General AI? A First Evaluation on ChatGPT. https://doi.org/10.48550/ARXIV.2303.03186 · DOI
  2. Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., Chen, C., Olsson, C., Olah, C., Hernandez, D., Drain, D., Ganguli, D., Li, D., Tran-Johnson, E., Perez, E., … Kaplan, J. (2022). Constitutional AI: Harmlessness from AI Feedback (arXiv:2212.08073). arXiv. http://arxiv.org/abs/2212.08073 · arXiv
  3. Baron-Cohen, S., Leslie, A. M., & Frith, U. (1985). Does the autistic child have a "theory of mind" ? Cognition, 21(1), 37-46. https://doi.org/10.1016/0010- 0277(85)90022-8 · DOI · Paper page
  4. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., & others. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877-1901.
  5. Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., Nori, H., Palangi, H., Ribeiro, M. T., & Zhang, Y. (2023). Sparks of Artificial General Intelligence: Early experiments with GPT-4. https://doi.org/10.48550/ARXIV.2303.12712 · DOI · Paper page
  6. de Leeuw, J. R., Gilbert, R. A., & Luchterhandt, B. (2023). jsPsych: Enabling an open- source collaborative ecosystem of behavioral experiments. Journal of Open Source Software, 8(85), 5351. https://doi.org/10.21105/joss.05351 · DOI
  7. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (arXiv:1810.04805). arXiv. http://arxiv.org/abs/1810.04805 · arXiv
  8. Dulewicz, V., & Higgs, M. (2000). Emotional intelligence -A review and evaluation study. Journal of Managerial Psychology, 15(4), 341-372. https://doi.org/10.1108/02683940010330993 · DOI
  9. Hanafi, Z., & Noor, F. (2016). Relationship between Emotional Intelligence and Academic Achievement in Emerging Adults: A Systematic Review. International Journal of Academic Research in Business and Social Sciences, 6(6), Pages 268-290. https://doi.org/10.6007/IJARBSS/v6-i6/2197 · DOI
  10. Hebenstreit, K., Praas, R., Kiesewetter, L. P., & Samwald, M. (2023). An automatically discovered chain-of-thought prompt generalizes to novel models and datasets (arXiv:2305.02897). arXiv. http://arxiv.org/abs/2305.02897 · arXiv
  11. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., & Steinhardt, J. (2021). Measuring Massive Multitask Language Understanding (arXiv:2009.03300). arXiv. http://arxiv.org/abs/2009.03300 · arXiv
  12. Izard, V., & Spelke, E. S. (2009). Development of Sensitivity to Geometry in Visual Forms. Human Evolution, 23(3), 213-248.
  13. King, M. (2023). Administration of the text-based portions of a general IQ test to five different large language models [Preprint]. https://doi.org/10.36227/techrxiv.22645561.v1 · DOI
  14. Köpf, A., Kilcher, Y., von Rütte, D., Anagnostidis, S., Tam, Z.-R., Stevens, K., Barhoum, A., Duc, N. M., Stanley, O., Nagyfi, R., & others. (2023). OpenAssistant Conversations-Democratizing Large Language Model Alignment. ArXiv Preprint ArXiv:2304.07327. · arXiv
  15. Kosinski, M. (2023). Theory of Mind May Have Spontaneously Emerged in Large Language Models (arXiv:2302.02083). arXiv. http://arxiv.org/abs/2302.02083 · arXiv
  16. Lange, K., Kühn, S., & Filevich, E. (2015). "Just Another Tool for Online Studies" (JATOS): An Easy Solution for Setup and Management of Web Servers Supporting Online Studies. PLOS ONE, 10(6), e0130834. https://doi.org/10.1371/journal.pone.0130834 · DOI
  17. Lea, R. G., Davis, S. K., Mahoney, B., & Qualter, P. (2019). Does Emotional Intelligence Buffer the Effects of Acute Stress? A Systematic Review. Frontiers in Psychology, 10, 810. https://doi.org/10.3389/fpsyg.2019.00810 · DOI
  18. Mayer, J. D., Caruso, D. R., & Salovey, P. (2016). The Ability Model of Emotional Intelligence: Principles and Updates. Emotion Review, 8(4), 290-300. https://doi.org/10.1177/1754073916639667 · DOI
  19. Mayer, J. D., Perkins, D. M., Caruso, D. R., & Salovey, P. (2001). Emotional intelligence and giftedness. Roeper Review, 23(3), 131-137. https://doi.org/10.1080/02783190109554084 · DOI
  20. Mayer, J. D., & Salovey, P. (1995). Emotional intelligence and the construction and regulation of feelings. Applied and Preventive Psychology, 4(3), 197-208. https://doi.org/10.1016/S0962-1849(05)80058-7 · DOI
  21. Mayer, J. D., Salovey, P., Caruso, D. R., & Sitarenios, G. (2001). Emotional intelligence as a standard intelligence. Emotion, 1(3), 232-242. https://doi.org/10.1037/1528-3542.1.3.232 · DOI
  22. Mayer, J. D., Salovey, P., Caruso, D. R., & Sitarenios, G. (2003). Measuring emotional intelligence with the MSCEIT V2.0. Emotion, 3(1), 97-105. https://doi.org/10.1037/1528-3542.3.1.97 · DOI
  23. McCleskey, J. (2014). Emotional intelligence and leadership: A review of the progress, controversy, and criticism. International Journal of Organizational Analysis, 22(1), 76-93. https://doi.org/10.1108/IJOA-03-2012-0568 · DOI
  24. Moghaddam, S. R., & Honey, C. J. (2023). Boosting Theory-of-Mind Performance in Large Language Models via Prompting. https://doi.org/10.48550/ARXIV.2304.11490 · DOI · Paper page
  25. Nair, V., Schumacher, E., Tso, G., & Kannan, A. (2023). DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents (arXiv:2303.17071). arXiv. http://arxiv.org/abs/2303.17071 · arXiv
  26. OpenAI. (2023). GPT-4 Technical Report (arXiv:2303.08774). arXiv. http://arxiv.org/abs/2303.08774 · arXiv
  27. Palmer, B. R., Gignac, G., Manocha, R., & Stough, C. (2005). A psychometric evaluation of the Mayer-Salovey-Caruso Emotional Intelligence Test Version 2.0. Intelligence, 33(3), 285-305. https://doi.org/10.1016/j.intell.2004.11.003 · DOI
  28. Salovey, P., & Mayer, J. D. (1990). Emotional Intelligence. Imagination, Cognition and Personality, 9(3), 185-211. https://doi.org/10.2190/DUGG-P24E-52WK- 6CDG · DOI · Paper page
  29. Sap, M., LeBras, R., Fried, D., & Choi, Y. (2023). Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs (arXiv:2210.13312). arXiv. http://arxiv.org/abs/2210.13312 · arXiv
  30. Shinn, N., Cassano, F., Labash, B., Gopinath, A., Narasimhan, K., & Yao, S. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. https://doi.org/10.48550/ARXIV.2303.11366 · DOI · Paper page
  31. Sperber, D., & Wilson, D. (2002). Pragmatics, modularity and mind-reading. Mind & Language, 17(1-2), 3-23. https://doi.org/10.1111/1468-0017.00186 · DOI
  32. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., & Hashimoto, T. B. (2023a). Alpaca: A strong, replicable instruction-following model. Stanford Center for Research on Foundation Models. Https://Crfm. Stanford. Edu/2023/03/13/Alpaca. Html, 3(6), 7.
  33. Tian, X., Wang, R., Zhao, Y., Zhen, Z., Song, Y., & Liu, J. (2020). Multi-Item Discriminability Pattern to Faces in Developmental Prosopagnosia Reveals Distinct Mechanisms of Face Processing. Cerebral Cortex, 30(5), 2986-2996. https://doi.org/10.1093/cercor/bhz289 · DOI
  34. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., & Lample, G. (2023). LLaMA: Open and Efficient Foundation Language Models (arXiv:2302.13971). arXiv. http://arxiv.org/abs/2302.13971 · arXiv
  35. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, \Lukasz, & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.
  36. Warwick, J., & Nettelbeck, T. (2004). Emotional intelligence is…? Personality and Individual Differences, 37(5), 1091-1100. https://doi.org/10.1016/j.paid.2003.12.003 · DOI
  37. Zeng, A., Liu, X., Du, Z., Wang, Z., Lai, H., Ding, M., Yang, Z., Xu, Y., Zheng, W., Xia, X., Tam, W. L., Ma, Z., Xue, Y., Zhai, J., Chen, W., Zhang, P., Dong, Y., & Tang, J. (2022). GLM-130B: An Open Bilingual Pre-trained Model (arXiv:2210.02414). arXiv. http://arxiv.org/abs/2210.02414 · arXiv
  38. Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., Du, Y., Yang, C., Chen, Y., Chen, Z., Jiang, J., Ren, R., Li, Y., Tang, X., Liu, Z., … Wen, J.-R. (2023). A Survey of Large Language Models (arXiv:2303.18223). arXiv. http://arxiv.org/abs/2303.18223 · arXiv
  39. Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., & Stoica, I. (2023). Judging LLM- as-a-judge with MT-Bench and Chatbot Arena. Table S2 Footnote: Table S2 shows the SECEU scores, EQ scores, pattern similarity, and properties of OpenAI GPT series with the assistance of prompt. Failed: The LLMs cannot complete the test. %: The percent of humans whose performance was below that of an LLM in the test. Pattern Similarity: The degree of similarity is indexed Pearson correlation coefficient (r). *: p < 0.05; **: p < 0.01.

Cited by

Sorted by citations

Authors
Wang, X., Li, X., Yin, Z., Wu, Y., & Jia, L.
Affiliations
  • Tsinghua University (CN)
  • Renmin University of China (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2307.09042
arXiv
2307.09042
OpenAlex
W4384807978
Cited by
8
References
0
Citations by year
2026: 2 · 2025: 4 · 2024: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Tsinghua University
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
  • Social Sciences › Psychology › Social Psychology › Emotional Intelligence and Performance
  • Physical Sciences › Computer Science › Artificial Intelligence › Psychiatry, Mental Health, Neuroscience
Keywords
large language models, emotional intelligence, emotional understanding, emotion recognition, psychometric assessment, GPT-4, human-AI comparison, multivariate pattern analysis, model size, training methods, complex emotions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review