GoEmotions: A Dataset of Fine-Grained Emotions
Open accessAbstract
Understanding emotion expressed in language has a wide range of applications, from building empathetic chatbots to detecting harmful online behavior. Advancement in this area can be improved using large-scale datasets with a fine-grained typology, adaptable to multiple downstream tasks. We introduce GoEmotions, the largest manually annotated dataset of 58k English Reddit comments, labeled for 27 emotion categories or Neutral. We demonstrate the high quality of the annotations via Principal Preserved Component Analysis. We conduct transfer learning experiments with existing emotion benchmarks to show that our dataset generalizes well to other domains and different emotion taxonomies. Our BERT-based model achieves an average F1-score of .46 across our proposed taxonomy, leaving much room for improvement.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Results on the GoEmotions dataset show improved performance of transformer-based models when integrated with EmoVis_lin.
“results on the GoEmotions dataset showing improved performance of transformer-based models when integrated with EmoVis lin [30], [49]”
From Emotion Classification With Visibility Graphs (2025) · cites this paper as [30]
The paper introduces GoEmotions and reports that its BERT model outperforms a biLSTM model (Section 2.3, p. 2). It does not report integrating a transformer-based model with EmoVis_lin or attribute any performance improvement to that integration. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works
An argument for basic emotions
Ekman, P. (1992) · Cognition and Emotion · doi:10.1080/02699939208411068 · Cited by 9,646 · see citing works
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382 · see citing works
Core affect and the psychological construction of emotion.
Russell, J. A. (2003) · Psychological Review · doi:10.1037/0033-295x.110.1.145 · Cited by 6,113 · see citing works
Monroe, B. L., Colaresi, M. P., & Quinn, K. M. (2008) · Political Analysis · doi:10.1093/pan/mpn018 · Cited by 542 · see citing works
Why Cohen’s Kappa should be avoided as performance measure in classification
Delgado, R., & Tibau, X.-A. (2019) · PLOS ONE · doi:10.1371/journal.pone.0222916 · Cited by 324 · see citing works
Flagged first, then by citations
Research works printed but not linked: 31
- Muhammad Abdul-Mageed and Lyle Ungar. 2017. EmoNet: Fine-Grained Emotion Detection with Gated Recurrent Neural Networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 718-728.
- Cecilia Ovesdotter Alm, Dan Roth, and Richard Sproat. 2005. Emotions from text: Machine learning for text-based emotion prediction. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing , pages 579-586, Vancouver, British Columbia, Canada. Association for Computational Linguistics.
- Laura-Ana-Maria Bostan and Roman Klinger. 2018. An analysis of annotated corpora for emotion classification in text. In Proceedings of the 27th International Conference on Computational Linguistics , pages 2104-2119.
- Luke Breitfeller, Emily Ahn, David Jurgens, and Yulia Tsvetkov. 2019. Finding microaggressions in the wild: A case for locating elusive phenomena in social media posts. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP) , pages 1664-1674.
- Sven Buechel and Udo Hahn. 2017. Emobank: Studying the impact of annotation perspective and representation format on dimensional emotion analysis. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers , pages 578585.
- Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement , 20(1):37-46.
- Alan Cowen, Disa Sauter, Jessica L Tracy, and Dacher Keltner. 2019a. Mapping the passions: Toward a high-dimensional taxonomy of emotional experience and expression. Psychological Science in the Public Interest , 20(1):69-90.
- Alan S Cowen, Hillary Anger Elfenbein, Petri Laukka, and Dacher Keltner. 2018. Mapping 24 emotions conveyed by brief human vocalization. American Psychologist , 74(6):698-712.
- Alan S Cowen, Xia Fang, Disa Sauter, and Dacher Keltner. in press. What music makes us feel: At least thirteen dimensions organize subjective experiences associated with music across cultures. Proceedings of the National Academy of Sciences .
- Alan S Cowen and Dacher Keltner. 2017. Self-report captures 27 distinct categories of emotion bridged by continuous gradients. Proceedings of the National Academy of Sciences , 114(38):E7900-E7909.
- Alan S Cowen, Petri Laukka, Hillary Anger Elfenbein, Runjing Liu, and Dacher Keltner. 2019b. The primacy of categories in the recognition of 12 emotions in speech prosody across two cultures. Nature Human Behaviour , 3(4):369.
- Paul Ekman. 1992a. Are there basic emotions? Psychological Review , 99(3):550-553.
- Diman Ghazi, Diana Inkpen, and Stan Szpakowicz. 2015. Detecting Emotion Stimuli in EmotionBearing Sentences. In International Conference on Intelligent Text Processing and Computational Linguistics , pages 152-165. Springer.
- Chao-Chun Hsu and Lun-Wei Ku. 2018. SocialNLP 2018 EmotionX challenge overview: Recognizing emotions in dialogues. In Proceedings of the Sixth International Workshop on Natural Language Processing for Social Media , pages 27-31, Melbourne, Australia. Association for Computational Linguistics.
- Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. Dailydialog: A manually labelled multi-turn dialogue dataset. arXiv preprint arXiv:1710.03957 . · arXiv
- Chen Liu, Muhammad Osama, and Anderson De Andrade. 2019. Dens: A dataset for multi-class emotion analysis. arXiv preprint arXiv:1910.11769 . · arXiv
- Saif Mohammad. 2018. Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 english words. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 174-184.
- Saif Mohammad, Felipe Bravo-Marquez, Mohammad Salameh, and Svetlana Kiritchenko. 2018. SemEval2018 task 1: Affect in tweets. In Proceedings of The 12th International Workshop on Semantic Evaluation , pages 1-17, New Orleans, Louisiana. Association for Computational Linguistics.
- Saif M Mohammad. 2012. # emotional tweets. In Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation , pages 246-255. Association for Computational Linguistics.
- Saif M Mohammad, Xiaodan Zhu, Svetlana Kiritchenko, and Joel Martin. 2015. Sentiment, emotion, purpose, and style in electoral tweets. Information Processing & Management , 51(4):480-499.
- Shruthi Mohan, Apala Guha, Michael Harris, Fred Popowich, Ashley Schuster, and Chris Priebe. 2017. The impact of toxic language on the health of reddit communities. In Canadian Conference on Artificial Intelligence , pages 51-56. Springer.
- Emily ¨ Ohman, Kaisla Kajava, J¨ org Tiedemann, and Timo Honkela. 2018. Creating a dataset for multilingual fine-grained emotion-detection using gamification-based annotation. In Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis , pages 24-30.
- F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research , 12:2825-2830.
- Inna Pirina and C ¸ a˘ grı C ¸ ¨ oltekin. 2018. Identifying depression on reddit: The effect of training data. In Proceedings of the 2018 EMNLP Workshop SMM4H: The 3rd Social Media Mining for Health Applications Workshop & Shared Task , pages 9-12.
- Robert Plutchik. 1980. A general psychoevolutionary theory of emotion. In Theories of emotion , pages 3-33. Elsevier.
- Klaus R Scherer and Harald G Wallbott. 1994. Evidence for Universality and Cultural Variation of Differential Emotion Response Patterning. Journal of personality and social psychology , 66(2):310.
- Hendrik Schuff, Jeremy Barnes, Julian Mohme, Sebastian Pad´ o, and Roman Klinger. 2017. Annotation, modelling and analysis of fine-grained emotions on a stance and sentiment detection corpus. In Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis , pages 13-23.
- Carlo Strapparava and Rada Mihalcea. 2007. SemEval2007 task 14: Affective text. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007) , pages 70-74, Prague, Czech Republic. Association for Computational Linguistics.
- Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. BERT Rediscovers the Classical NLP Pipeline. In Association for Computational Linguistics .
- Henry Tsai, Jason Riesa, Melvin Johnson, Naveen Arivazhagan, Xin Li, and Amelia Archer. 2019. Small and Practical BERT Models for Sequence Labeling. In EMNLP 2019 .
- Wenbo Wang, Lu Chen, Krishnaprasad Thirunarayan, and Amit P Sheth. 2012. Harnessing twitter 'big data' for automatic emotion identification. In 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing , pages 587-592. IEEE.
Cited by
Sorted by citations
Large Language Models for Mental Health Applications: Systematic Review
Guo, Z., Lai, A., Thygesen, J. H., Farrington, J., Keen, T., & Li, K. (2024) · JMIR Mental Health · doi:10.2196/57400 · Cited by 269 · see citing works
Sentiment Analysis in the Age of Generative AI
Krugmann, J. O., & Hartmann, J. (2024) · Customer Needs and Solutions · doi:10.1007/s40547-024-00143-4 · Cited by 189 · see citing works
“Additionally, in a seven-class sentiment classification study on the Google Emotions Dataset [15], the accuracy gap between a fine-tuned transfer learning model (Emotion English DistilRoBERTa-base) [20] and the lowest-perform-ing LLM (Llama 2) was 16.4 percentage points, compared to 8.1 percentage…”
InfluentialLarge Language Models in Mental Health Care: A Scoping Review
Hua, Y., Liu, F., Yang, K., Li, Z., Na, H., Sheu, Y.-H., Zhou, P., Moran, L. V., Ananiadou, S., Clifton, D. A., et al. (2025) · Current Treatment Options in Psychiatry · doi:10.1007/s40501-025-00363-y · Cited by 92 · see citing works
SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization
Kim, H., Hessel, J., Jiang, L., West, P., Lu, X., Yu, Y., Zhou, P., Bras, R., Alikhani, M., Kim, G., et al. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.799 · Cited by 63 · see citing works
Large Language Models in Mental Health Care: A Systematic Scoping Review (Preprint)
hua, Y., Liu, F., Yang, K., Li, Z., Sheu, Y.-H., Zhou, P., Moran, L. V., Ananiadou, S., Clifton, D. A., & Beam, A. (2024) · doi:10.2196/preprints.64088 · Cited by 46 · see citing works
ModelShield: Adaptive and Robust Watermark Against Model Extraction Attack
Pang, K., Qi, T., Wu, C., Bai, M., Jiang, M., & Huang, Y. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3530691 · Cited by 29 · see citing works
RWKV: Reinventing RNNs for the Transformer Era
Peng, B., Alcaide, E., Anthony, Q., Albalak, A., Arcadinho, S., Biderman, S., Cao, H., Cheng, X., Chung, M., Grella, M., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.13048 · Cited by 17 · see citing works
Learning Interpretable Style Embeddings via Prompting LLMs
Patel, A., Rao, D., Kothary, A., McKeown, K., & Callison-Burch, C. (2023) · arXiv · doi:10.48550/arxiv.2305.12696 · Cited by 10 · see citing works
Meta-Harness: End-to-End Optimization of Model Harnesses
Lee, Y., Nair, R., Zhang, Q., Lee, K., Khattab, O., & Finn, C. (2026) · arXiv · doi:10.48550/arxiv.2603.28052 · Cited by 3 · see citing works
“• GoEmotions is a fine-grained emotion classification benchmark introduced by Demszky et al. [15].”
Emotionally based strategic communications as a new tool in defensive cognitive warfare
Ćosić, K., Popović, S., & Wiederhold, B. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1751406 · Cited by 4 · see citing works
Emotion Classification With Visibility Graphs
Şimşek, E., Topcu, A., Koç, E., Saritas, E. U., & Koç, A. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3567033 · Cited by 2 · see citing works
A Simple and Interactive Transformer for Fine-Grained Emotion Detection
Sui, D., Li, B., Yang, H., Chen, Y., Liu, K., Chu, D., & Zhao, J. (2025) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/taslp.2024.3487418 · Cited by 2 · see citing works
Caring Without Feeling: Affective Dynamics as the Control Layer of Human-AI Agent Collaboration
Xu, J., Wu, X., Zhang, Z., Xu, Y., Yang, Y., Zhu, J., Xiao, L., Wu, W., & He, L. (2026) · arXiv · doi:10.48550/arxiv.2606.18259
“…requires a multi-layer design that connects input conditions to behavioural consequences: controlled variation in user affect at the input layer [26, 102]; assessment of affective calibration, boundary setting and refusal quality at the response layer [100]; and measurement of changes in user…”
Instruction Following by Principled Boosting Attention of Large Language Models
Guardieiro, V., Khare, A., Stein, A., & Wong, E. (2025) · arXiv · doi:10.48550/arxiv.2506.13734
- Authors
- Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., & Ravi, S.
- Affiliations
- Stanford University (US)
- University of California, Berkeley (US)
- Google (United States) (US)
- Venue
- Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
- Published
- 2020
- Pages
- 4040-4054
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W3022113899
- Cited by
- 873
- References
- 7
- Citations by year
- 2026: 192 · 2025: 200 · 2024: 138 · 2023: 172 · 2022: 105 · 2021: 54 · 2020: 12
- FWCI
- 32.76
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Keywords
- emotion recognition, GoEmotions dataset, Reddit comments, manual annotation, transfer learning, BERT, annotation quality evaluation, domain generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
