Will Affective Computing Emerge From Foundation Models and General Artificial Intelligence? A First Evaluation of ChatGPT
Open accessAbstract
ChatGPT has shown the potential of emerging general artificial intelligence capabilities, as it has demonstrated competent performance across many natural language processing tasks. In this work, we evaluate the capabilities of ChatGPT to perform text classification on three affective computing problems, namely, big-five personality prediction, sentiment analysis, and suicide tendency detection. We utilize three baselines, a robust language model (RoBERTa-base), a legacy word model with pretrained embeddings (Word2Vec), and a simple bag-of-words (BoW) baseline. Results show that the RoBERTa model trained for a specific downstream task generally has a superior performance. On the other hand, ChatGPT provides decent results and is relatively comparable to the Word2Vec and BoW baselines. ChatGPT further shows robustness against noisy data, where the Word2Vec model achieves worse results due to noise. Results indicate that ChatGPT is a good generalist model that is capable of achieving good results across various problems without any specialized training; however, it is not as good as a specialized model for a downstream task.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs such as Mental-Alpaca, Med-PaLM 2, and ChatGPT have been shown to screen for possible depressive symptoms and suicide risk, with varying degrees of accuracy.
“LLMs such as Mental-Alpaca, a mental health domain–specific LLM, Med-PaLM 2, a medical domain–specific LLM, and ChatGPT, which is general-purpose, have also been shown to screen for possible depressive symptoms and suicide risk, with varying degrees of accuracy [17-20].”
From The Opportunities and Risks of Large Language Models in Mental Health (2024) · cites this paper as [18]
Researchers have noted GPT's ability to help with computational social science tasks, detect misinformation, infer politicians' ideologies, write persuasive political arguments, respond to patient questions, simulate human research participants, and model collective behavior.
“Researchers have also recently noted GPT’s ability to help with computational social science tasks (49–54), detect misinformation (55), infer politicians’ ideologies (56), write persuasive political arguments (57), respond to patient questions (58), simulate human research participants (59–61), and model collective behavior (56).”
From GPT is an effective tool for multilingual psychological text analysis (2024) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Pattern Recognition and Machine Learning
(2006) · doi:10.1007/978-0-387-45528-0 · Cited by 9,824 · see citing works
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Schuller, B. W., Steidl, S., Batliner, A. M., Vinciarelli, A., Scherer, K. R., Ringeval, F., Chétouani, M., Weninger, F. J., Eyben, F., Marchi, E., et al. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-56 · Cited by 732 · see citing works
Self-citationA comprehensive survey on pretrained foundation models: a history from BERT to ChatGPT
Zhou, C., Li, Q., Li, C., Yu, J., Liu, Y., Wang, G., Zhang, K., Ji, C., Yan, Q., He, L., et al. (2024) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-024-02443-6 · Cited by 704 · see citing works
Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
Qin, C., Zhang, A., Zhang, Z., Chen, J., Yasunaga, M., & Yang, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.85 · Cited by 519 · see citing works
A Categorical Archive of ChatGPT Failures
Borji, A. (2023) · doi:10.21203/rs.3.rs-2895792/v1 · Cited by 514 · see citing works
Good, P. (1994) · Springer series in statistics · doi:10.1007/978-1-4757-2346-5 · Cited by 391 · see citing works
Mao, R., Liu, Q., He, K., Li, W., & Cambria, E. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3204972 · Cited by 247 · see citing works
Self-citationSentiment and Sarcasm Classification With Multitask Learning
Majumder, N., Poria, S., Peng, H., et al. (2019) · IEEE Intelligent Systems · doi:10.1109/mis.2019.2904691 · Cited by 236 · see citing works
ChaLearn LAP 2016: First Round Challenge on First Impressions - Dataset and Results
Ponce-López, V., Chen, B., Oliu, M., Corneanu, C. A., Clapés, A., Guyon, I., Baró, X., Escalante, H. J., & Escalera, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-49409-8_32 · Cited by 215 · see citing works
How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation
Hendy, A., Abdelrehim, M., Sharaf, A., Raunak, V., Gabr, M., Matsushita, H., Kim, Y. J., Afify, M., & Awadalla, H. H. (2023) · arXiv · doi:10.48550/arxiv.2302.09210 · Cited by 202 · cited 2× · see citing works
“Hendy et al. 8 explore the capabilities of GPT language models (including ChatGPT) in machine translation.”
BackgroundSMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization
Lindauer, M., Eggensperger, K., Feurer, M., Biedenkapp, A., Deng, D., Benjamins, C., Ruhopf, T., Sass, R., & Hutter, F. (2021) · arXiv · doi:10.48550/arxiv.2109.09831 · Cited by 195 · see citing works
Explainable Metaphor Identification Inspired by Conceptual Metaphor Theory
Ge, M., Mao, R., & Cambria, E. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21313 · Cited by 73 · see citing works
Self-citationSKIER: A Symbolic Knowledge Integrated Model for Conversational Emotion Recognition
Li, W., Zhu, L., Mao, R., & Cambria, E. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26541 · Cited by 73 · see citing works
Self-citationKaya, H., Gürpınar, F., & Salah, A. A. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.210 · Cited by 67 · see citing works
Design of an explainable machine learning challenge for video interviews
Escalante, H. J., Guyon, I., Escalera, S., Jacques, J. C. S., Madadi, M., Baró, X., Ayache, S., Viegas, E., Güçlütürk, Y., Güçlü, U., et al. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966320 · Cited by 66 · see citing works
Scaling Laws for Reward Model Overoptimization
Gao, L., Schulman, J., & Hilton, J. (2022) · arXiv · doi:10.48550/arxiv.2210.10760 · Cited by 62 · see citing works
Suicide and Depression Detection in Social Media Forums
Desu, V., Komati, N., Lingamaneni, S., et al. (2022) · Smart innovation, systems and technologies · doi:10.1007/978-981-16-9705-0_26 · Cited by 8 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- T. Mikolov et al., "Distributed representations of words and phrases and their compositionality," in Proc. Adv. Neural Inf. Process. Syst., Lake Tahoe, NV, USA: Curran Associates, Inc., 2013, pp. 3111-3119.
- E. Cambria et al., "SenticNet 7: A commonsense- based neurosymbolic AI framework for explainable sentiment analysis," in Proc. 13th Lang. Resour. Eval. Conf. (LREC), 2022, pp. 3829-3839.
Cited by
Sorted by citations
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4
Kalyan, K. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100048 · Cited by 335 · see citing works
GPT is an effective tool for multilingual psychological text analysis
Rathje, S., Mirea, D.-M., Sucholutsky, I., Marjieh, R., Robertson, C. E., & Van Bavel, J. J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2308950121 · Cited by 297 · see citing works
Sohail, S. S., Farhat, F., Himeur, Y., Nadeem, M., Madsen, D. Ø., Singh, Y., Atalla, S., & Mansoor, W. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.101675 · Cited by 192 · see citing works
Emotional intelligence of Large Language Models
Wang, X., Li, X., Yin, Z., Wu, Y., & Liu, J. (2023) · Journal of Pacific Rim Psychology · doi:10.1177/18344909231213958 · Cited by 128 · see citing works
He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., & Cambria, E. (2024) · doi:10.2139/ssrn.4809363 · Cited by 69 · see citing works
Self-citationPan, S., Xu, G. J. W., Guo, K., Park, S. H., & Ding, H. (2024) · IEEE Transactions on Games · doi:10.1109/tg.2024.3366239 · Cited by 38 · see citing works
Zhou, X., Liang, P., Zhang, B., Li, Z., Ahmad, A., Shahin, M., & Waseem, M. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112204 · Cited by 30 · see citing works
- Authors
- Amin, M. M., Cambria, E., & Schuller, B. W.
- Affiliations
- University of Augsburg (DE)
- Nanyang Technological University (SG)
- Venue
- IEEE Intelligent Systems
- Published
- 2023-03
- Volume
- 38
- Issue
- 2
- Pages
- 15-23
- ISSN
- 1541-1672 (ISSN-L)
- 1941-1294
- OpenAlex
- W4367353919
- Cited by
- 162
- References
- 22
- Citations by year
- 2026: 22 · 2025: 55 · 2024: 61 · 2023: 24
- FWCI
- 15.25
- Type
- journal article
- Language
- en
- Open access
- Licence
- Locations
- IEEE Intelligent Systems (publishedVersion)
- OPUS (Augsburg University) (submittedVersion)
- DR-NTU (Nanyang Technological University) (submittedVersion)
- Funders
- Nanyang Technological University
- Albert-Ludwigs-Universität Freiburg
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Keywords
- affective computing, ChatGPT, artificial general intelligence, foundation models, text classification, sentiment analysis, RoBERTa, Word2Vec, bag-of-words, noisy data robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
