The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Conversational Memory Network for Emotion Recognition in Dyadic Dialogue Videos
Open accessAbstract
Emotion recognition in conversations is crucial for the development of empathetic machines. Present methods mostly ignore the role of inter-speaker dependency relations while classifying emotions in conversations. In this paper, we address recognizing utterance-level emotions in dyadic conversational videos. We propose a deep neural framework, termed conversational memory network, which leverages contextual information from the conversation history. The framework takes a multimodal approach comprising audio, visual and textual features with gated recurrent units to model past utterances of each speaker into memories. Such memories are then merged using attention-based hops to capture inter-speaker dependencies. Experiments show an accuracy improvement of 3-4% over the state of the art.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
(2012) · Cited by 8,009
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601
Ekman, P. (1993) · American Psychologist · doi:10.1037/0003-066x.48.4.384 · Cited by 3,441
Opensmile: the munich versatile and fast open-source audio feature extractor
Eyben, F., Wöllmer, M., & Schuller, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874246 · Cited by 2,642 · cited 1×
“To extract audio features we use openSMILE (Eyben et al., 2010).”
MethodTensor Fusion Network for Multimodal Sentiment Analysis
Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1115 · Cited by 2,102
Self-citationA review of affective computing: From unimodal analysis to multimodal fusion
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644 · cited 9×
“Previous research in affective computing has looked at dialogues as an essential basis to learn emotional dynamics (Sidnell and Stivers, 2012; Poria et al., 2017a; Zhou et al., 2017).”
BackgroundMethodInfluentialSelf-citationThe Handbook of Conversation Analysis
Buchholz, M. B. (2013) · Language and Psychoanalysis · doi:10.7565/landp.2013.003 · Cited by 1,611
Predicting Depression via Social Media
De Choudhury, M., Gamon, M., Counts, S., & Horvitz, E. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14432 · Cited by 1,579
Sukhbaatar, S., Szlam, A., Weston, J., & Fergus, R. (2015) · arXiv · doi:10.48550/arxiv.1503.08895 · Cited by 1,345
Multimodal fusion for multimedia analysis: a survey
Atrey, P. K., Hossain, M. A., El Saddik, A., & Kankanhalli, M. S. (2010) · Multimedia Systems · doi:10.1007/s00530-010-0182-0 · Cited by 1,279
Recent Trends in Deep Learning Based Natural Language Processing
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02709 · Cited by 1,192
Self-citationGraves, A., Wayne, G., & Danihelka, I. (2014) · arXiv · doi:10.48550/arxiv.1410.5401 · Cited by 1,058 · cited 2×
“It improves speaker-based emotion modeling by using memory networks which are efficient in capturing long-term dependencies and summarizing task-specific details using attention models (Weston et al., 2014; Graves et al., 2014; Young et al., 2017).”
BackgroundAsk Me Anything: Dynamic Memory Networks for Natural Language Processing
Kumar, A., Irsoy, O., Ondruska, P., Iyyer, M., Bradbury, J., Gulrajani, I., Zhong, V., Paulus, R., & Socher, R. (2015) · arXiv · doi:10.48550/arxiv.1506.07285 · Cited by 1,020
Context-Dependent Sentiment Analysis in User-Generated Videos
Poria, S., Cambria, E., Hazarika, D., Majumder, N., Zadeh, A., & Morency, L.-P. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1081 · Cited by 939
Self-citationAnalysis of emotion recognition using facial expressions, speech and multimodal information
Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C. M., Kazemzadeh, A., Lee, S., Neumann, U., & Narayanan, S. (2004) · Proceedings of the 6th international conference on Multimodal interfaces · doi:10.1145/1027933.1027968 · Cited by 840 · cited 1×
“Hence, many works used visual and audio features together for detecting affect (Busso et al., 2004; Castellano et al., 2008; Ranganathan et al., 2016).”
BackgroundEmotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory
Zhou, H., Huang, M., Zhang, T., Zhu, X., & Liu, B. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11325 · Cited by 763
End-to-End Multimodal Emotion Recognition using Deep Neural Networks
Tzirakis, P., Trigeorgis, G., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2017) · arXiv · doi:10.48550/arxiv.1704.08619 · Cited by 752
A Review and Meta-Analysis of Multimodal Affect Detection Systems
D'mello, S. K., & Kory, J. (2015) · ACM Computing Surveys · doi:10.1145/2682899 · Cited by 628
How Emotions Work: The Social Functions of Emotional Expression in Negotiations
Morris, M. W., & Keltner, D. (2000) · Research in Organizational Behavior · doi:10.1016/s0191-3085(00)22002-9 · Cited by 447
Sentiment Analysis Is a Big Suitcase
Cambria, E., Poria, S., Gelbukh, A., & Thelwall, M. (2017) · IEEE Intelligent Systems · doi:10.1109/mis.2017.4531228 · Cited by 439
Self-citationEmotion cycles: On the social influence of emotion in organizations
Hareli, S., & Rafaeli, A. (2008) · Research in Organizational Behavior · doi:10.1016/j.riob.2008.04.007 · Cited by 427
SenticNet 5: Discovering Conceptual Primitives for Sentiment Analysis by Means of Context Embeddings
Cambria, E., Poria, S., Hazarika, D., & Kwok, K. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11559 · Cited by 411
Self-citationNatural language based financial forecasting: a survey
Xing, F. Z., Cambria, E., & Welsch, R. E. (2018) · Artificial Intelligence Review · doi:10.1007/s10462-017-9588-9 · Cited by 398
Self-citationEmotion Regulation in Romantic Relationships: The Cognitive Consequences of Concealing Feelings
Richards, J. M., Butler, E. A., & Gross, J. J. (2003) · Journal of Social and Personal Relationships · doi:10.1177/02654075030205002 · Cited by 323
Liu, F., & Maitlis, S. (2014) · Journal of Management Studies · doi:10.1111/j.1467-6486.2012.01087.x · Cited by 305
A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks
Poria, S., Cambria, E., Hazarika, D., & Vij, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.08815 · Cited by 292
Self-citationAugmenting End-to-End Dialogue Systems With Commonsense Knowledge
Young, T., Cambria, E., Chaturvedi, I., Zhou, H., Biswas, S., & Huang, M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11923 · Cited by 266
Self-citationAffective Computing: From Laughter to IEEE
Picard, R. W. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.10 · Cited by 265
Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech
Castellano, G., Kessous, L., & Caridakis, G. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-85099-1_8 · Cited by 259
Multimodal emotion recognition using deep learning architectures
Ranganathan, H., Chakraborty, S., & Panchanathan, S. (2016) · 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2016.7477679 · Cited by 249
Select-additive learning: Improving generalization in multimodal sentiment analysis
Wang, H., Meghawat, A., Morency, L.-P., & Xing, E. P. (2017) · 2017 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2017.8019301 · Cited by 203
Self-citationWöllmer, M., Metallinou, A., Eyben, F., Schuller, B., & Narayanan, S. S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-646 · Cited by 201 · cited 2×
“Our work, which performs context-sensitive recognition (Wöllmer et al., 2010) uses three modalities: audio, visual and text.”
Multi-level Multiple Attentions for Contextual Multimodal Sentiment Analysis
Poria, S., Cambria, E., Hazarika, D., Mazumder, N., Zadeh, A., & Morency, L.-P. (2017) · 2017 IEEE International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2017.134 · Cited by 193
Self-citationKoval, P., & Kuppens, P. (2012) · Emotion · doi:10.1037/a0024756 · Cited by 163
Dellaert, F., Polzin, T., & Waibel, A. (1996) · 4th International Conference on Spoken Language Processing (ICSLP 1996) · doi:10.21437/icslp.1996-462 · Cited by 158
Real-Time Speech Emotion and Sentiment Recognition for Interactive Dialogue Systems
Bertero, D., Siddique, F. B., Wu, C.-S., Wan, Y., Chan, R. H. Y., & Fung, P. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1110 · Cited by 148
Emotion, Affect and Conversation
Ruusuvuori, J. (2012) · Wiley eBooks · doi:10.1002/9781118325001.ch16 · Cited by 147
Ensemble of SVM trees for multimodal emotion recognition
Rozgić, V., Ananthakrishnan, S., Saleem, S., Kumar, R., & Prasad, R. (2012) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · Cited by 145
Audio-visual based emotion recognition-a new approach
Song, M., Bu, J., Chen, C., & Li, N. (2004) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2004.1315276 · Cited by 139
Semantic Audiovisual Data Fusion for Automatic Emotion Recognition
Datcu, D., & Rothkrantz, L. J. M. (2015) · Wiley eBooks · doi:10.1002/9781118910566.ch16 · Cited by 97
Classifying skewed data: importance weighting to optimize average recall
Rosenberg, A. (2012) · Interspeech (USB) · doi:10.21437/interspeech.2012-131 · Cited by 79
Towards an automatic classification of emotions in speech
Amir, N., & Ron, S. (1998) · International Conference on Spoken Language Processing (ICSLP) · doi:10.21437/icslp.1998-105 · Cited by 69
Pooling acoustic and lexical features for the prediction of valence
Aldeneh, Z., Khorram, S., Dimitriadis, D., & Provost, E. M. (2017) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3136755.3136760 · Cited by 46
Emotion recognition using bimodal data fusion
Datcu, D., & Rothkrantz, L. J. M. (2011) · International Conference on Computer Systems and Technologies (CompSysTech) · doi:10.1145/2023607.2023629 · Cited by 37
Mirroring Facial Expressions and Emotions in Dyadic Conversations
Navarretta, C. (2016) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2ayot5sgfyek · Cited by 34
Augmenting End-to-End Dialog Systems with Commonsense KnowledgePreprint
Young, T., Cambria, E., Chaturvedi, I., Huang, M., Zhou, H., & Biswas, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.05453 · Cited by 30
Self-citation
Cited by
MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations
Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., & Mihalcea, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1050 · Cited by 1,416
“Recently, Hazarika et al. (2018) proposed a multi-modal memory network that can recognize emotion in dyadic dialogues.”
BackgroundMethodInfluentialSelf-citationDialogueRNN: An Attentive RNN for Emotion Detection in Conversations
Majumder, N., Poria, S., Hazarika, D., Mihalcea, R., Gelbukh, A., & Cambria, E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016818 · Cited by 860
Self-citationDialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation
Ghosal, D., Majumder, N., Poria, S., Chhaya, N., & Gelbukh, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1015 · Cited by 717
Self-citationIntegrating Multimodal Information in Large Pretrained Transformers
Rahman, W., Hasan, M. K., Lee, S., Bagher Zadeh, A., Mao, C., Morency, L.-P., & Hoque, E. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.214 · Cited by 673
Self-citationEmotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances
Poria, S., Majumder, N., Mihalcea, R., & Hovy, E. (2019) · IEEE Access · doi:10.1109/access.2019.2929050 · Cited by 496
Self-citation
Metadata
- Authors
- Hazarika, D., Poria, S., Zadeh, A., Cambria, E., Morency, L.-P., & Zimmermann, R.
- Venue
- Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)
- Published
- 2018
- Volume
- 2018
- Pages
- 2122-2132
- OpenAlex
- W2805662932
- Cited by
- 463
- References
- 52
- FWCI
- 54.25
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.