The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Memory Fusion Network for Multi-view Sequential Learning
Open accessAbstract
Multi-view sequential learning is a fundamental problem in machine learning dealing with multi-view sequences. In a multi-view sequence, there exists two forms of interactions between different views: view-specific interactions and cross-view interactions. In this paper, we present a new neural architecture for multi-view sequential learning called the Memory Fusion Network (MFN) that explicitly accounts for both interactions in a neural architecture and continuously models them through time. The first component of the MFN is called the System of LSTMs, where view-specific interactions are learned in isolation through assigning an LSTM function to each view. The cross-view interactions are then identified using a special attention mechanism called the Delta-memory Attention Network (DMAN) and summarized through time with a Multi-view Gated Memory. Through extensive experimentation, MFN is compared to various proposed approaches for multi-view sequential learning on multiple publicly available benchmark datasets. MFN outperforms all the multi-view approaches. Furthermore, MFN outperforms all current state-of-the-art models, setting new state-of-the-art results for all three multi-view datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
An argument for basic emotions
Ekman, P. (1992) · Cognition and Emotion · doi:10.1080/02699939208411068 · Cited by 9,646
The random subspace method for constructing decision forests
Ho, T. K. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.709601 · Cited by 7,005
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
Statistical Inference for Probabilistic Functions of Finite State Markov Chains
Baum, L. E., & Petrie, T. (1966) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177699147 · Cited by 2,775
Tensor Fusion Network for Multimodal Sentiment Analysis
Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1115 · Cited by 2,102
Self-citationMultiple kernel learning, conic duality, and the SMO algorithm
Bach, F. R., Lanckriet, G. R. G., & Jordan, M. I. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015424 · Cited by 1,453
Fast Human Detection Using a Cascade of Histograms of Oriented Gradients
Zhu, Q., Yeh, M.-C., Cheng, K.-T., & Avidan, S. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.119 · Cited by 1,399
A Survey on Multi-view LearningPreprint
Xu, C., Tao, D., & Xu, C. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1304.5634 · Cited by 1,252
Research works printed but not linked: 3
- Alku, P.; Strik, H.; and Vilkman, E. 1997. Parabolic spectral parametera new method for quantification of the glottal flow. Speech Communication 22(1):67-79.
- Degottex, G.; Kane, J.; Drugman, T.; Raitio, T.; and Scherer, S. 2014. Covarepa collaborative voice analysis repository for speech technologies. In Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on , 960-964. IEEE.
- Perez-Rosas, V.; Mihalcea, R.; and Morency, L.-P. 2013. UtteranceLevel Multimodal Sentiment Analysis. In Association for Computational Linguistics (ACL) .
Cited by
Sorted by citations
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Bagher Zadeh, A., Liang, P. P., Poria, S., Cambria, E., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1208 · Cited by 1,433
Self-citationEfficient Low-rank Multimodal Fusion With Modality-Specific Factors
Liu, Z., Shen, Y., Lakshminarasimhan, V. B., Liang, P. P., Bagher Zadeh, A., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1209 · Cited by 1,170
Self-citationMISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis
Hazarika, D., Zimmermann, R., & Poria, S. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413678 · Cited by 1,073
Self-citationYu, W., Xu, H., Yuan, Z., & Wu, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17289 · Cited by 816
Gandhi, A., Adhvaryu, K., Poria, S., Cambria, E., & Hussain, A. (2023) · Information Fusion · doi:10.1016/j.inffus.2022.09.025 · Cited by 783
Self-citationMultimodal deep learning for biomedical data fusion: a review
Stahlschmidt, S. R., Ulfenborg, B., & Synnergren, J. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbab569 · Cited by 766
Integrating Multimodal Information in Large Pretrained Transformers
Rahman, W., Hasan, M. K., Lee, S., Zadeh, A. B., Mao, C., Morency, L., & Hoque, M. E. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.214 · Cited by 778
Self-citationHarnessing multimodal data integration to advance precision oncology
Boehm, K. M., Khosravi, P., Vanguri, R., Gao, J., & Shah, S. P. (2022) · Nature Reviews Cancer · doi:10.1038/s41568-021-00408-3 · Cited by 662
Deep Multimodal Representation Learning: A Survey
Guo, W., Wang, J., & Wang, S. (2019) · IEEE Access · doi:10.1109/access.2019.2916887 · Cited by 535
Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
Wang, Y., Shen, Y., Liu, Z., Liang, P. P., Zadeh, A., & Morency, L. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017216 · Cited by 501
Self-citationICON: Interactive Conversational Memory Network for Multimodal Emotion Detection
Hazarika, D., Poria, S., Mihalcea, R. F., Cambria, E., & Zimmermann, R. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1280 · Cited by 471
Self-citationLiu, W., Qiu, J., Zheng, W., & Lu, B. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3071170 · Cited by 488
CH-SIMS: A Chinese Multimodal Sentiment Analysis Dataset with Fine-grained Annotation of Modality
Yu, W., Xu, H., Meng, F., Zhu, Y., Ma, Y., Wu, J., Zou, J., & Yang, K. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.343 · Cited by 482
Sun, Z., Sarma, P., Sethares, W., & Liang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6431 · Cited by 410
Multimodal sentiment analysis based on fusion methods: A survey
Zhu, L., Zhu, Z., Zhang, C., Xu, Y., & Kong, X. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.02.028 · Cited by 375
MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation
Hu, J., Liu, Y., Zhao, J., & Jin, Q. (2021) · doi:10.18653/v1/2021.acl-long.440 · Cited by 368
TETFN: A text enhanced transformer fusion network for multimodal sentiment analysis
Wang, D., Guo, X., Tian, Y., Liu, J., He, L., & Luo, X. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.109259 · Cited by 349
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research
Poria, S., Hazarika, D., Majumder, N., & Mihalcea, R. F. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3038167 · Cited by 358
Self-citationImage–text sentiment analysis via deep multimodal attentive fusion
Huang, F., Zhang, X., Zhao, Z., Xu, J. J., & Li, Z. (2019) · Knowledge-Based Systems · doi:10.1016/j.knosys.2019.01.019 · Cited by 310
CTNet: Conversational Transformer Network for Emotion Recognition
Lian, Z., Liu, B., & Tao, J. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3049898 · Cited by 309
Attention, please! A survey of neural attention models in deep learning
Correia, A. D. S., & Colombini, E. L. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10148-x · Cited by 358
Adapting BERT for Target-Oriented Multimodal Sentiment Classification
Yu, J., & Jiang, J. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/751 · Cited by 302
Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis
Mai, S., Zeng, Y., Zheng, S., & Hu, H. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3172360 · Cited by 307
Sun, L., Lian, Z., Liu, B., & Tao, J. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3274829 · Cited by 288
MM-DFN: Multimodal Dynamic Fusion Network for Emotion Recognition in Conversations
Hu, D., Hou, X., Wei, L., Jiang, L., & Mo, Y. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747397 · Cited by 278
M3ER: Multiplicative Multimodal Emotion Recognition using Facial, Textual, and Speech Cues
Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., & Manocha, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i02.5492 · Cited by 23
Decoupled Multimodal Distilling for Emotion Recognition
Li, Y., Wang, Y., & Cui, Z. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00641 · Cited by 249
Mai, S., Hu, H., & Xing, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5347 · Cited by 237
Entity-Sensitive Attention and Fusion Network for Entity-Level Multimodal Sentiment Classification
Yu, J., Jiang, J., & Xia, R. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2957872 · Cited by 235
Multimodal Language Analysis with Recurrent Multistage Fusion
Liang, P. P., Liu, Z., Bagher Zadeh, A., & Morency, L.-P. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1014 · Cited by 230
Self-citationMulti-task Learning for Multi-modal Emotion Recognition and Sentiment Analysis
Akhtar, M. S., Chauhan, D. S., Ghosal, D., Poria, S., Ekbal, A., & Bhattacharyya, P. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1034 · Cited by 238
Self-citationZhang, D., Wu, L., Sun, C., Li, S., Zhu, Q., & Zhou, G. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/752 · Cited by 230
Lv, F., Chen, X., Huang, Y., Duan, L., & Lin, G. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00258 · Cited by 231
Transformer-based Feature Reconstruction Network for Robust Multimodal Sentiment Analysis
Yuan, Z., Li, W., Xu, H., & Yu, W. (2021) · Proceedings of the 29th ACM International Conference on Multimedia · doi:10.1145/3474085.3475585 · Cited by 211
A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face
Lian, H., Lu, C., Li, S., Zhao, Y., Tang, C., & Zong, Y. (2023) · Entropy · doi:10.3390/e25101440 · Cited by 224
A Review on Explainability in Multimodal Deep Neural Nets
Joshi, G., Walambe, R., & Kotecha, K. (2021) · IEEE Access · doi:10.1109/access.2021.3070212 · Cited by 204
Foundations & Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions
Liang, P. P., Zadeh, A., & Morency, L. (2024) · ACM Computing Surveys · doi:10.1145/3656580 · Cited by 438
Self-citationConFEDE: Contrastive Feature Decomposition for Multimodal Sentiment Analysis
Yang, J., Yu, Y., Niu, D., Guo, W., & Xu, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.421 · Cited by 201
Real-Time Video Emotion Recognition Based on Reinforcement Learning and Domain Knowledge
Zhang, K., Li, Y., Wang, J., Cambria, E., & Li, X. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3072412 · Cited by 184
Self-citationA Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations
Ma, H., Wang, J., Lin, H., Zhang, B., Zhang, Y., & Xu, B. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3271019 · Cited by 196
Mai, S., Zeng, Y., & Hu, H. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3171679 · Cited by 182
State of the art: a review of sentiment analysis based on sequential transfer learning
Chan, J. Y.-L., Bea, K. T., Leow, S. M. H., Phoong, S. W., & Cheng, W. K. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10183-8 · Cited by 196
CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation
Sun, H., Wang, H., Liu, J., Chen, Y., & Lin, L. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548025 · Cited by 191
UR-FUNNY: A Multimodal Language Dataset for Understanding Humor
Hasan, M. K., Rahman, W., Bagher Zadeh, A., Zhong, J., Tanveer, M. I., Morency, L.-P., & Hoque, M. (. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1211 · Cited by 174
Self-citationTMBL: Transformer-based multimodal binding learning model for multimodal sentiment analysis
Huang, J., Zhou, J. W., Tang, Z., Lin, J., & Chen, C. Y. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111346 · Cited by 173
COGMEN: COntextualized GNN based Multimodal Emotion recognitioN
Joshi, A., Bhat, A., Jain, A., Singh, A., & Modi, A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.306 · Cited by 161
Khoo, L. S., McNaney, R., Lim, M. K., & McNaney, R. (2024) · Sensors · doi:10.3390/s24020348 · Cited by 194
Mai, S., Hu, H., & Xing, S. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1046 · Cited by 153
AOBERT: All-modalities-in-One BERT for multimodal sentiment analysis
Kim, K.-H., & Park, S. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.11.022 · Cited by 149
Multimodal Video Sentiment Analysis Using Deep Learning Approaches, a Survey
Abdu, S. A., Yousef, A. H., & Salem, A. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.06.003 · Cited by 154
Quantum-inspired multimodal fusion for video sentiment analysis
Li, Q., Gkoumas, D., Lioma, C., & Melucci, M. (2021) · Information Fusion · doi:10.1016/j.inffus.2020.08.006 · Cited by 143
Memory-Guided Multi-View Multi-Domain Fake News Detection
Zhu, Y., Sheng, Q., Cao, J., Nan, Q., Shu, K., Wu, M., Wang, J., & Zhuang, F. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3185151 · Cited by 146
Transformer Encoder With Multi-Modal Multi-Head Attention for Continuous Affect Recognition
Chen, H., Jiang, D., & Hichem Sahli, B. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3037496 · Cited by 146
Video sentiment analysis with bimodal information-augmented multi-head attention
Wu, T., Peng, J., Zhang, W., Zhang, H., Tan, S., Yi, F., Ma, C., & Huang, Y. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107676 · Cited by 138
A Review of Key Technologies for Emotion Analysis Using Multimodal Information
Zhu, X., Guo, C., Feng, H., Huang, Y., Feng, Y., Wang, X., & Wang, R. (2024) · Cognitive Computation · doi:10.1007/s12559-024-10287-z · Cited by 143
Shenoy, A., & Sardana, A. (2020) · doi:10.18653/v1/2020.challengehml-1.3 · Cited by 124
Gated Mechanism for Attention Based Multi Modal Sentiment Analysis
Kumar, A., & Vepa, J. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053012 · Cited by 123
GraphMFT: A graph network based multimodal fusion technique for emotion recognition in conversation
Li, J., Wang, X., Lv, G., & Zeng, Z. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126427 · Cited by 127
Wu, Y., Lin, Z., Zhao, Y., Qin, B., & Zhu, L.-N. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.417 · Cited by 129
Li, J., Wang, X., Lv, G., & Zeng, Z. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3261279 · Cited by 120
A cross modal hierarchical fusion multimodal sentiment analysis method based on multi-task learning
Wang, L., Peng, J., Zheng, C., Zhao, T., & Zhu, L. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103675 · Cited by 111
Emotion recognition based on brain-like multimodal hierarchical perception
Zhu, X., Huang, Y., Wang, X., & Wang, R. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17347-w · Cited by 120
Transformer-based Multimodal Information Fusion for Facial Expression Analysis
Zhang, W., Qiu, F., Wang, S., Zeng, H., Zhang, Z., An, R., Ma, B., & Ding, Y. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00271 · Cited by 114
HGMF: Heterogeneous Graph-based Fusion for Multimodal Data with Incompleteness
Chen, J., & Zhang, A. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403182 · Cited by 107
Dynamic interactive multiview memory network for emotion recognition in conversation
Wen, J., Jiang, D., Tu, G., Liu, C., & Cambria, E. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.10.009 · Cited by 102
Self-citationMultimodal sentiment analysis with unidirectional modality translation
Yang, B., Shao, B., Wu, L., & Lin, X. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.09.041 · Cited by 101
M3Care: Learning with Missing Modalities in Multimodal Healthcare Data
Zhang, C., Chu, X., Ma, L., Zhu, Y., Wang, Y., Wang, J., & Zhao, J. (2022) · Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3534678.3539388 · Cited by 102
Sentiment analysis in textual, visual and multimodal inputs using recurrent neural networks
Tembhurne, J. V., & Diwan, T. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10037-x · Cited by 108
Context-aware Interactive Attention for Multi-modal Sentiment and Emotion Analysis
Chauhan, D. S., Akhtar, M. S., Ekbal, A., & Bhattacharyya, P. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1566 · Cited by 97
Social-IQ: A Question Answering Benchmark for Artificial Social Intelligence
Zadeh, A., Chan, M., Liang, P. P., Tong, E., & Morency, L. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00901 · Cited by 107
Self-citationAF: An Association-Based Fusion Method for Multi-Modal Classification
Liang, X., Qian, Y., Guo, Q. J., Cheng, H., & Liang, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3125995 · Cited by 98
M-SENA: An Integrated Platform for Multimodal Sentiment Analysis
Mao, H., Yuan, Z., Xu, H., Yu, W., Liu, Y., & Gao, K. (2022) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2022.acl-demo.20 · Cited by 116
A Survey of Cutting-edge Multimodal Sentiment Analysis
Singh, U., Abhishek, K., & Azad, H. K. (2024) · ACM Computing Surveys · doi:10.1145/3652149 · Cited by 103
M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis
Wang, X., He, J., Jin, Z., Yang, M., Wang, Y., & Qu, H. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3114794 · Cited by 7
Multimodal End-to-End Sparse Model for Emotion Recognition
Dai, W., Cahyawijaya, S., Liu, Z., & Fung, P. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.417 · Cited by 91
Ma, J., Xiong, G., Xu, J., & Chen, X. (2023) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2023.3313635 · Cited by 91
Gkoumas, D., Li, Q., Lioma, C., Yu, Y., & Song, D. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.09.005 · Cited by 96
SMIN: Semi-Supervised Multi-Modal Interaction Network for Conversational Emotion Recognition
Lian, Z., Liu, B., & Tao, J. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3141237 · Cited by 92
Mai, S., Xing, S., & Hu, H. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2925966 · Cited by 89
Mai, S., Xing, S., & Hu, H. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3068598 · Cited by 89
Multimodal Emotion Recognition Fusion Analysis Adapting BERT With Heterogeneous Feature Unification
Lee, S., Han, D. K., & Ko, H. (2021) · IEEE Access · doi:10.1109/access.2021.3092735 · Cited by 99
BAFN: Bi-Direction Attention Based Fusion Network for Multimodal Sentiment Analysis
Tang, J., Liu, D., Jin, X., Peng, Y. K., Zhao, Q., Ding, Y., & Kong, W. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3218018 · Cited by 90
TeFNA: Text-centered fusion network with crossmodal attention for multimodal sentiment analysis
Huang, C., Zhang, J., Wu, X., Wang, Y., Li, M., & Huang, X. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110502 · Cited by 89
Adapted Dynamic Memory Network for Emotion Recognition in Conversation
Xing, S., Mai, S., & Hu, H. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3005660 · Cited by 87
DialogueRNN: An Attentive RNN for Emotion Detection in Conversations
Majumder, N., Poria, S., Hazarika, D., Mihalcea, R. F., Gelbukh, A. F., & Cambria, E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016818 · Cited by 103
Self-citationMulti-Modal Sarcasm Detection and Humor Classification in Code-Mixed Conversations
Bedi, M., Kumar, S., Akhtar, M. S., & Chakraborty, T. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3083522 · Cited by 96
MissModal: Increasing Robustness to Missing Modality in Multimodal Sentiment Analysis
Lin, R., & Hu, H. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00628 · Cited by 98
Tang, J., Li, K., Jin, X., Cichocki, A. S., Zhao, Q., & Kong, W. (2021) · doi:10.18653/v1/2021.acl-long.412 · Cited by 87
PS-Mixer: A Polar-Vector and Strength-Vector Mixer Model for Multimodal Sentiment Analysis
Lin, H., Zhang, P., Ling, J., Yang, Z., Lee, L., & Liu, W. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103229 · Cited by 85
Deep Emotional Arousal Network for Multimodal Sentiment Analysis and Emotion Recognition
Zhang, F., Li, X., Lim, C. P., Hua, Q., Dong, C.-R., & Zhai, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.07.006 · Cited by 81
Efficient Low-rank Multimodal Fusion with Modality-Specific FactorsPreprint
Liu, Z., Shen, Y., Lakshminarasimhan, V., Liang, P. P., Zadeh, A., & Morency, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00064 · Cited by 80
Self-citationTailor Versatile Multi-Modal Learning for Multi-Label Emotion Recognition
Zhang, Y., Chen, M., Shen, J., & Wang, C. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20895 · Cited by 85
GPT-4V with emotion: A zero-shot benchmark for Generalized Emotion Recognition
Lian, Z., Sun, L., Sun, H., Chen, K., Wen, Z., Gu, H., Liu, B., & Tao, J. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102367 · Cited by 97
Multi-Modal Sequence Fusion via Recursive Attention for Emotion Recognition
Beard, R., Das, R., Ng, R. W. M., Gopalakrishnan, P. G. K., Eerens, L., Świętojański, P., & Mikšík, O. (2018) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k18-1025 · Cited by 77
Multimodal Sentiment Analysis With Two-Phase Multi-Task Learning
Yang, B., Wu, L., Zhu, J., Shao, B., Lin, X., & Liu, T. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3178204 · Cited by 82
Multimodal emotion recognition based on audio and text by using hybrid attention networks
Zhang, S., Yang, Y., Chen, C., Liu, R., Tao, X., Guo, W., Xu, Y., & Zhao, X. (2023) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105052 · Cited by 81
Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis
Pham, H., Manzini, T., Liang, P. P., & Póczos, B. (2018) · doi:10.18653/v1/w18-3308 · Cited by 82
Self-citationMultimodal Learning using Optimal Transport for Sarcasm and Humor Detection
Pramanick, S., Roy, A., & Johns, V. M. P. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00062 · Cited by 82
Sun, H., Chen, Y., & Lin, L. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3233070 · Cited by 81
Attention-based Audio-Visual Fusion for Robust Automatic Speech Recognition
Sterpu, G., Saam, C., & Harte, N. (2018) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3242969.3243014 · Cited by 74
Metadata
- Authors
- Zadeh, A., Liang, P. P., Mazumder, N., Poria, S., Cambria, E., & Morency, L.-P.
- Affiliations
- Carnegie Mellon University (US)
- Instituto Politécnico Nacional (MX)
- Nanyang Technological University (SG)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2018-04-27
- Volume
- 32
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2787581402
- Cited by
- 855
- References
- 36
- Citations by year
- 2026: 100 · 2025: 169 · 2024: 163 · 2023: 125 · 2022: 118 · 2021: 87 · 2020: 48 · 2019: 29 · 2018: 16
- FWCI
- 13.57
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- LSTM, attention mechanism, temporal modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
