MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations
Open accessAbstract
Emotion recognition in conversations (ERC) is a challenging task that has recently gained popularity due to its potential applications.Until now, however, there has been no largescale multimodal multi-party emotional conversational database containing more than two speakers per dialogue.To address this gap, we propose the Multimodal EmotionLines Dataset (MELD), an extension and enhancement of EmotionLines.MELD contains about 13,000 utterances from 1,433 dialogues from the TV-series Friends.Each utterance is annotated with emotion and sentiment labels, and encompasses audio, visual, and textual modalities.We propose several strong multimodal baselines and show the importance of contextual and multimodal information for emotion recognition in conversations.The full dataset is available for use at http:// affective-meld.github.io.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
MELD contains six leading roles occurring frequently in the dataset.
“Because the benchmark dataset for the MERMCtask, i.e., MELD (Poria et al., 2019), contains six leading roles occurring frequently in the dataset”
From A Facial Expression-Aware Multimodal Multi-task Learning Framework for Emotion Recognition in Multi-party Conversations (2023) · cites this paper as Poria et al., 2019
The Character Distribution section says MELD has primary characters and groups speakers with fewer than 1% of utterances as “Others,” supporting the point that some roles occur frequently. The text in our copy does not state that there are exactly six such roles. (LLM, conf=0.78)
Experimental — accuracy not yet measuredMELD contains 13,707 video clips extracted from Friends, with utterance, audio, video, speaker identity, and emotion annotations with seven classes.
“MELD contains 13,707 video clips extracted from the sitcom Friends , which contain information such as utterance, audio, video, and speaker identity. It also provides emotion annotations on each utterance with seven classes, including neutral , surprise , fear , sadness , joy , disgust , and anger .”
From A Facial Expression-Aware Multimodal Multi-task Learning Framework for Emotion Recognition in Multi-party Conversations (2023) · cites this paper as Poria et al., 2019
The Abstract identifies Friends as the source, Section 3 describes extracting an audiovisual clip for each utterance, and Section 3.2 lists the seven stated emotion classes. Table 1 lists 9,989 training, 1,109 development, and 2,610 test utterances—13,708 in total, not the claimed 13,707 clips. Table 6 also shows utterance, speaker, and emotion fields. (LLM, conf=0.89)
Experimental — accuracy not yet measuredMost previous MERMC studies primarily focus on the text and audio modalities.
“In the literature, most previous MERMC studies primarily focus on the text and audio modalities (Poria et al., 2019; Liang et al., 2020; Mao et al., 2021; Chen et al., 2021)”
From A Facial Expression-Aware Multimodal Multi-task Learning Framework for Emotion Recognition in Multi-party Conversations (2023) · cites this paper as Poria et al., 2019
The Introduction discusses earlier conversational emotion-recognition work but does not say that most studies focused on text and audio. The Baseline Models and Future Directions sections show that MELD's own baselines use text and audio while leaving visual features for future work; that does not establish the claimed pattern across previous studies. I searched for a comparison of prior studies' modality choices and found none. (LLM, conf=0.70)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 30 · Cited by 1,416
Research works printed but not linked: 18
- Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jean- nette N Chang, Sungbok Lee, and Shrikanth S Narayanan. 2008. Iemocap: Interactive emotional dyadic motion capture database. Language re- sources and evaluation, 42(4):335-359.
- Sheng-Yeh Chen, Chao-Chun Hsu, Chuan-Chun Kuo, Lun-Wei Ku, et al. 2018. Emotionlines: An emotion corpus of multi-party conversations. arXiv preprint arXiv:1802.08379. · arXiv
- Junyoung Chung, C ¸aglar Gülc ¸ehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. CoRR, abs/1412.3555. · arXiv
- Florian Eyben, Martin Wöllmer, and Björn Schuller. 2010. Opensmile: the munich versatile and fast open-source audio feature extractor. In Proceedings of the 18th ACM international conference on Multi- media, pages 1459-1462. ACM.
- Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Zimmermann. 2018. Conversational memory net- work for emotion recognition in dyadic dialogue videos. In NAACL, volume 1, pages 2122-2132.
- Yoon Kim. 2014. Convolutional neural networks for sentence classification. In EMNLP, pages 1746- 1751.
- Jiwei Li, Michel Galley, Chris Brockett, Georgios Sp- ithourakis, Jianfeng Gao, and Bill Dolan. 2016. A persona-based neural conversation model. In ACL, volume 1, pages 994-1003.
- Navonil Majumder, Soujanya Poria, Devamanyu Haz- arika, Rada Mihalcea, Alexander Gelbukh, and Erik Cambria. 2019. DialogueRNN: An attentive RNN for emotion detection in conversations. Thirty-Third AAAI Conference on Artificial Intelligence.
- Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word rep- resentation. In EMNLP, pages 1532-1543.
- Verónica Pérez-Rosas, Rada Mihalcea, and Louis- Philippe Morency. 2013. Utterance-level multi- modal sentiment analysis. In ACL (1), pages 973- 982.
- Soujanya Poria, Erik Cambria, Devamanyu Haz- arika, Navonil Mazumder, Amir Zadeh, and Louis- Philippe Morency. 2017. Context-dependent sen- timent analysis in user-generated videos. In ACL, pages 873-883.
- Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard Hovy. 2019. Emotion recognition in conversation: Research challenges, datasets, and re- cent advances. arXiv preprint arXiv:1905.02947. · arXiv
- Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2018. I know the feeling: Learning to converse with empathy. arXiv preprint arXiv:1811.00207. · arXiv
- Björn Schuller, Michel Valster, Florian Eyben, Roddy Cowie, and Maja Pantic. 2012. Avec 2012: the con- tinuous audio/visual emotion challenge. In Proceed- ings of the 14th ACM international conference on Multimodal interaction, pages 449-456. ACM.
- Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C Courville, and Yoshua Bengio. 2017. A hierarchical latent variable encoder-decoder model for generating dia- logues. In AAAI, pages 3295-3301.
- Martin Wollmer, Felix Weninger, Timo Knaup, Bjorn Schuller, Congkai Sun, Kenji Sagae, and Louis- Philippe Morency. 2013. Youtube movie reviews: Sentiment analysis in an audio-visual context. IEEE Intelligent Systems, 28(3):46-53.
- Amir Zadeh, Tadas Baltrušaitis, and Louis-Philippe Morency. 2016a. Deep constrained local mod- els for facial landmark detection. arXiv preprint arXiv:1611.08657. · arXiv
- Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2017. Emotional chatting machine: Emotional conversation generation with internal and external memory. arXiv preprint arXiv:1704.01074. · arXiv
- Authors
- Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., & Mihalcea, R.
- Affiliations
- Singapore University of Technology and Design (SG)
- National University of Singapore (SG)
- Nanyang Technological University (SG)
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 527-536
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2895615590
- Cited by
- 1,416
- References
- 30
- Citations by year
- 2026: 257 · 2025: 313 · 2024: 243 · 2023: 243 · 2022: 143 · 2021: 135 · 2020: 62 · 2019: 20
- FWCI
- 113.47
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Greater South Information System
- Funders
- National Science Foundation (1815291)
- John Templeton Foundation (61156)
- Defense Advanced Research Projects Agency (HR001117S0026-AIDA-FP-045)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- emotion recognition in conversation, multimodal emotion recognition, multi-party conversation, MELD dataset, Friends (television series), emotion annotation, sentiment annotation, contextual information
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
