Cited by 231Multimodal Machine Learning ApplicationsSentiment Analysis and Opinion MiningEmotion and Mood Recognition

The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.

Multimodal Language Analysis with Recurrent Multistage Fusion

Open access

Liang, Paul Pu, Liu, Ziyin, Bagher Zadeh, AmirAli, Morency, Louis-Philippe·2018·Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing

DOI 10.18653/v1/d18-1014

Open this paper in CoChat — free account Open PDF

Abstract

Computational modeling of human multimodal language is an emerging research area in natural language processing spanning the language, visual and acoustic modalities.Comprehending multimodal language requires modeling not only the interactions within each modality (intra-modal interactions) but more importantly the interactions between modalities (cross-modal interactions).In this paper, we propose the Recurrent Multistage Fusion Network (RMFN) which decomposes the fusion problem into multiple stages, each of them focused on a subset of multimodal signals for specialized, effective fusion.Crossmodal interactions are modeled using this multistage fusion approach which builds upon intermediate representations of previous stages.Temporal and intra-modal interactions are modeled by integrating our proposed fusion approach with a system of recurrent neural networks.The RMFN displays state-of-the-art performance in modeling human multimodal language across three public datasets relating to multimodal sentiment analysis, emotion recognition, and speaker traits recognition.We provide visualizations to show that each stage of fusion focuses on a different subset of multimodal signals, learning increasingly discriminative multimodal representations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.18653/v1/d18-1014

Cites

72 of 72 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Random Forests
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Support-vector networks
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,043
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Finding structure in time
Elman, J. L. (1990) · Cognitive Science · doi:10.1207/s15516709cog1402_1 · Cited by 11,212
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172

Flagged first, then by citations

Show all 69 references

Research works printed but not linked: 7

Gilles Degottex, John Kane, Thomas Drugman, Tuomo Raitio, and Stefan Scherer. 2014. Covarepa collabo- rative voice analysis repository for speech technolo- gies. In Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on, pages 960-964. IEEE.
Sepp Hochreiter and Jürgen Schmidhuber. 1997b. Long short-term memory. Neural Comput., 9(8):1735-1780.
Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirec- tional lstm-crf models for sequence tagging. CoRR, abs/1508.01991. · arXiv
Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y. Ng. 2011. Multi- modal deep learning. In ICML, pages 689-696.
ACL.
In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Informa- tion Processing Systems 25, pages 2222-2230.
Julian Georg Zilly, Rupesh Kumar Srivastava, Jan Koutník, and Jürgen Schmidhuber. 2016. Re- current Highway Networks. arXiv preprint arXiv:1607.03474. · arXiv

Sorted by citations

Items 101–200 of 231

Sentiment Knowledge Enhanced Self-supervised Learning for Multimodal Sentiment Analysis
Fan, Q., Han, J., He, Y., Zheng, T., & Zheng, G. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.821 · Cited by 23
Multimodal Sentiment Analysis with Multi-perspective Fusion Network Focusing on Sense Attentive Language
Li, X., & Chen, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-63031-7_26 · Cited by 22
CCIN-SA:Composite cross modal interaction network with attention enhancement for multimodal sentiment analysis
Yang, L., Zhong, J., Wen, T., & Liao, Y. J. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103230 · Cited by 21
Multimodal Embeddings From Language Models for Emotion Recognition in the Wild
Tseng, S.-Y., Narayanan, S. S., & Georgiou, P. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3065598 · Cited by 23
Hybrid Multimodal Fusion for Humor Detection
Xu, H., Liu, W., Liu, J., Li, M., Feng, Y., Peng, Y., Shi, Y., Sun, X. N., & Wang, M. (2022) · doi:10.1145/3551876.3554802 · Cited by 23
Injecting Multimodal Information Into Pre-Trained Language Model for Multimodal Sentiment Analysis
Mai, S., Zeng, Y., Xiong, A., & Hu, H. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3553149 · Cited by 21
Feature Extraction Network with Attention Mechanism for Data Enhancement and Recombination Fusion for Multimodal Sentiment Analysis
Qi, Q., Lin, L., & Zhang, R. (2021) · Information · doi:10.3390/info12090342 · Cited by 19
Interpretable Multimodal Capsule Fusion
Wu, J., Mai, S., & Hu, H. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3178236 · Cited by 16
MMT: Multi-way Multi-modal Transformer for Multimodal Learning
Tang, J., Li, K., Hou, M., Jin, X., Kong, W., Ding, Y., & Zhao, Q. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/480 · Cited by 16
Multimodal GRU with directed pairwise cross-modal attention for sentiment analysis
Qin, Z., Luo, Q., Zang, Z., & Fu, H. (2025) · Scientific Reports · doi:10.1038/s41598-025-93023-3 · Cited by 20
Multimodal consistency-specificity fusion based on information bottleneck for sentiment analysis
Liu, W., Cao, S., & Zhang, S. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.101943 · Cited by 14
Multimodal Hierarchical Attention Neural Network: Looking for Candidates Behaviour Which Impact Recruiter's Decision
Hemamou, L., Guillon, A., Martin, J., & Clavel, C. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3113159 · Cited by 14
Improving Multimodal Accuracy Through Modality Pre-training and AttentionPreprint
Ismail, A. A., Hasan, M. M., & Ishtiaq, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.06102 · Cited by 28
Predict and Use: Harnessing Predicted Gaze to Improve Multimodal Sarcasm Detection
Tiwari, D., Kanojia, D., Ray, A., Nunna, A., & Bhattacharyya, P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.988 · Cited by 16
Asynchronous Multimodal Video Sequence Fusion via Learning Modality-Exclusive and -Agnostic Representations
Yang, D., Li, M., Qu, L., Yang, K., Zhai, P., Wang, S., & Zhang, L. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3435561 · Cited by 16
A deep spatiotemporal interaction network for multimodal sentimental analysis and emotion recognition
Li, X., Zhang, F., Hua, Q., & Dong, C.-R. (2024) · Information Sciences · doi:10.1016/j.ins.2024.121515 · Cited by 13
A Two-Stage Stacked Transformer Framework for Multimodal Sentiment Analysis
Yi, G., Fan, C., Tao, J., Lv, Z., Wen, Z., Pei, G., & Li, T. (2024) · Intelligent Computing · doi:10.34133/icomputing.0081 · Cited by 12
Reconstructing representations using diffusion models for multimodal sentiment analysis through reading comprehension
Zhang, H., Yan, Y., Cai, Z., Zhan, P., Chen, B. Y., Jiang, B., & Xie, B. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112346 · Cited by 14
Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data
Liang, P. P., Liu, T., Cai, A., Muszyński, M., Ishii, R., Allen, N., Auerbach, R. P., Brent, D. A., Salakhutdinov, R., & Morency, L. (2021) · doi:10.18653/v1/2021.acl-long.322 · Cited by 13
Self-citation
Temporal Graph Convolutional Network for Multimodal Sentiment Analysis
Huang, J., Lin, Z., Yang, Z., & Liu, W. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3479939 · Cited by 12
Causal Inference for Modality Debiasing in Multimodal Emotion Recognition
Kim, J., Hong, J., & Choi, Y. (2024) · Applied Sciences · doi:10.3390/app142311397 · Cited by 12
AB-GRU: An attention-based bidirectional GRU model for multimodal sentiment fusion and analysis
Wu, J., Zheng, X., Wang, J., Wu, J., & Wang, J. (2023) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2023822 · Cited by 11
Dense Fusion Network with Multimodal Residual for Sentiment Classification
Deng, H., Kang, P., Yang, Z., Hao, T., Li, Q., & Liu, W. (2021) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428321 · Cited by 11
Human Emotion Recognition by Integrating Facial and Speech Features: An Implementation of Multimodal Framework using CNN
Srinivas, P. V. V. S., & Mishra, P. (2022) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2022.0130172 · Cited by 11
Multi-Modal Sentiment Analysis Based on Interactive Attention Mechanism
Wu, J., Zhu, T., Zheng, X., & Wang, C. (2022) · Applied Sciences · doi:10.3390/app12168174 · Cited by 13
Integrating Cross-modal Interactions via Latent Representation Shift for Multi-modal Humor Detection
Chen, C., & Zhang, P. (2022) · doi:10.1145/3551876.3554805 · Cited by 11
Multi-scale Cooperative Multimodal Transformers for Multimodal Sentiment Analysis in Videos
Ma, L., Yu, Y., Liang, T., & Liu, T. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0351-0_21 · Cited by 11
On the Use of Modality-Specific Large-Scale Pre-Trained Encoders for Multimodal Sentiment Analysis
Ando, A., Masumura, R., Takashima, A., Suzuki, S., Makishima, N., Suzuki, K., Moriya, T., Ashihara, T., & Satō, H. (2023) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt54892.2023.10022548 · Cited by 12
TAC-Trimodal Affective Computing: Principles, integration process, affective detection, challenges, and solutions
Al-Saadawi, H. F. T., Daş, B., & Daş, R. (2024) · Displays · doi:10.1016/j.displa.2024.102731 · Cited by 10
Tripartite interaction representation learning for multi-modal sentiment analysis
Wang, B., Dong, G., Zhao, Y., Li, R., Yin, W., & Lu, L. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126279 · Cited by 10
MMATERIC: Multi-Task Learning and Multi-Fusion for AudioText Emotion Recognition in Conversation
Liang, X., Zou, Y., Zhuang, X., Yang, J., Niu, T., & Xu, R. (2023) · Electronics · doi:10.3390/electronics12071534 · Cited by 10
Text-image semantic relevance identification for aspect-based multimodal sentiment analysis
Zhang, T., Zhou, G., Lu, J., Li, Z., Wu, H., & Liu, S. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1904 · Cited by 12
Collaborative Attention Mechanism for Multi-View Action RecognitionPreprint
Bai, Y., Tao, Z., Wang, L., Li, S., Yin, Y., & Fu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.06599 · Cited by 15
Unsupervised Multimodal Language Representations using Convolutional AutoencodersPreprint
Koromilas, P., & Γιαννακόπουλος, Θ. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03007 · Cited by 10
Automatic movie genre classification & emotion recognition via a BiProjection Multimodal Transformer
Moreno-Galván, D. A., López, R., González, L. C., Montes-y-Gómez, M., Sánchez-Vega, F., & López-Monroy, A. P. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102641 · Cited by 14
A Multimodal Approach for Hate and Offensive Content Detection in Tamil: From Corpus Creation to Model Development
Mohan, J., Reddy, M. S., Premjith, B., Jyothish Lal, G., & Chakravarthi, B. R. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3712260 · Cited by 10
MIF: Multi-Shot Interactive Fusion Model for Cancer Survival Prediction Using Pathological Image and Genomic Data
Shi, Y., Wang, M., Liu, H., Zhao, F., Li, A., & Chen, X. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3363161 · Cited by 10
Improving Dimensional Emotion Recognition via Feature-wise Fusion
Liu, Y., Sun, W., Zhang, X., & Qin, Y. (2022) · doi:10.1145/3551876.3554804 · Cited by 9
Gate-Fusion Transformer for Multimodal Sentiment Analysis
Xie, L., & Zhang, X.-Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59830-3_3 · Cited by 9
Integrating Multimodal Information in Large Pretrained TransformersPreprint
Rahman, W., Hasan, M. K., Lee, S., Zadeh, A., Mao, C., Morency, L., & Hoque, M. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.05787 · Cited by 39
Self-citation
An Investigation of how Label Smoothing Affects GeneralizationPreprint
Chen, B., Ziyin, L., Wang, Z., & Liang, P. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12648 · Cited by 18
Self-citation
ScaleVLAD: Improving Multimodal Sentiment Analysis via Multi-Scale Fusion of Locally DescriptorsPreprint
Luo, H., Ji, L., Huang, Y., Wang, B., Ji, S., & Li, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.01368 · Cited by 20
From Extraction to Generation: Multimodal Emotion-Cause Pair Generation in Conversations
Ma, H., Yu, J., Wang, F., Cao, H., & Xia, R. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3446646 · Cited by 9
TriSAT: Trimodal Representation Learning for Multimodal Sentiment Analysis
Huan, R., Zhong, G., Chen, P., & Liang, R. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3458812 · Cited by 9
Dynamic Weighted Gating for Enhanced Cross-Modal Interaction in Multimodal Sentiment Analysis
Wang, N., & Wang, Q. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3702996 · Cited by 9
Multilayer interactive attention bottleneck transformer for aspect-based multimodal sentiment analysis
Sun, J., & Zhu, F. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01601-8 · Cited by 8
Multimodal emotion recognition method in complex dynamic scenes
Liu, L., Luo, Q., Zhang, W., Zhang, M., & Zhai, B. (2025) · Journal of Information and Intelligence · doi:10.1016/j.jiixd.2025.02.004 · Cited by 12
Cross-Modal Sentiment Analysis of Text and Video Based on Bi-GRU Cyclic Network and Correlation Enhancement
He, P., Qi, H., Wang, S., & Cang, J. (2023) · Applied Sciences · doi:10.3390/app13137489 · Cited by 8
Multimodal Fusion Interactions: A Study of Human and Automatic Quantification
Liang, P. P., Cheng, Y., Salakhutdinov, R., & Morency, L. (2023) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3577190.3614151 · Cited by 8
Self-citation
MORSE: MultimOdal sentiment analysis for Real-life SEttings
Yao, Y., Pérez‐Rosas, V., Abouelenien, M., & Burzo, M. G. (2020) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3382507.3418821 · Cited by 7
Af-CAN: Multimodal Emotion Recognition Method Based on Situational Attention Mechanism
Zhang, X., Wang, M., Zeng, X., & Zhuang, X. (2024) · IEEE Access · doi:10.1109/access.2024.3471613 · Cited by 7
MulDeF: A Model-Agnostic Debiasing Framework for Robust Multimodal Sentiment Analysis
Huan, R., Zhong, G., Chen, P., & Liang, R. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3521836 · Cited by 8
基于跨模块融合的多模态情感分析
天津师范大学, 张, 敬., 李, 彦., & 天津师范大学 (2023) · 计算机科学 · doi:10.55375/cps.2023.3.2 · Cited by 7
FgbCNN: A unified bilinear architecture for learning a fine-grained feature representation in facial expression recognition
Shabbir, N., & Rout, R. K. (2023) · Image and Vision Computing · doi:10.1016/j.imavis.2023.104770 · Cited by 7
CCDA: A Novel Method to Explore the Cross-Correlation in Dual-Attention for Multimodal Sentiment Analysis
Wang, P., Liu, S., & Chen, J. (2024) · Applied Sciences · doi:10.3390/app14051934 · Cited by 8
Towards Debiasing Sentence RepresentationsPreprint
Liang, P. P., Li, I., Zheng, E., Lim, Y. C., Salakhutdinov, R., & Morency, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08100 · Cited by 65
Self-citation
Self-HCL: Self-Supervised Multitask Learning with Hybrid Contrastive Learning Strategy for Multimodal Sentiment Analysis
Fu, Y., Fu, J., Xue, H., & Xu, Z. (2024) · Electronics · doi:10.3390/electronics13142835 · Cited by 6
MECG: modality-enhanced convolutional graph for unbalanced multimodal representations
Tang, J., Ni, B., Yang, Y., Ding, Y., & Kong, W. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06729-y · Cited by 6
When Language Evolution Meets Multimodality: Current Status and Challenges Toward Multimodal Computational Models
Grifoni, P., D’Ulizia, A., & Ferri, F. (2021) · IEEE Access · doi:10.1109/access.2021.3061756 · Cited by 6
Deep Learning Approaches on Multimodal Sentiment Analysis
Cai, Z., Gao, H., Li, J., & Wang, X. (2022) · IEEE International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA) · doi:10.1109/eebda53927.2022.9745018 · Cited by 6
Multimodel Sentiment Analysis with Self-attention
Cao, R., Ye, C., & Zhou, H. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-63128-4_2 · Cited by 7
On Emergent Communication in Competitive Multi-Agent Teams
Liang, P. P., Chen, J. W., Salakhutdinov, R., Morency, L., & Kottur, S. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/ghof1719 · Cited by 8
Self-citation
EASUM: Enhancing Affective State Understanding through Joint Sentiment and Emotion Modeling for Multimodal Tasks
Hwang, Y., & Kim, J.-H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00557 · Cited by 5
WeaveNet: End-to-End Audiovisual Sentiment Analysis
Yu, Y., Jia, Z., Shi, F., Zhu, M., Wang, W., & Li, X. (2022) · Communications in computer and information science · doi:10.1007/978-981-16-9247-5_1 · Cited by 5
ER-MRL: Emotion Recognition based on Multimodal Representation Learning
Guo, X., Wang, Y., Miao, Z., Yang, X., Guo, J., Hou, X., & Zao, F. (2022) · International Conference on Information Science and Technology · doi:10.1109/icist55546.2022.9926848 · Cited by 5
Learning Modality-Complementary and Eliminating-Redundancy Representations with Multi-Task Learning for Multimodal Sentiment Analysis
Zhao, X., Miao, X., Xu, X., Liu, Y., & Cao, Y. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650083 · Cited by 4
MultiViz: Towards User-Centric Visualizations and Interpretations of Multimodal Models
Liang, P. P., Lyu, Y., Chhablani, G., Jain, N., Deng, Z., Wang, X., Morency, L.-P., & Salakhutdinov, R. (2023) · Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544549.3585604 · Cited by 4
Self-citation
Model for Determining the Psycho-Emotional State of a Person Based on Multimodal Data Analysis
Shakhovska, N., Zherebetskyi, O., & Lupenko, S. A. (2024) · Applied Sciences · doi:10.3390/app14051920 · Cited by 4
Conditional selection with CNN augmented transformer for multimodal affective analysis
Wang, J., Wang, S., Xiao, S., Lin, R., Dong, M., & Guo, W. (2024) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.12320 · Cited by 5
Sentiment-oriented Sarcasm Integration for Video Sentiment Analysis Enhancement with Sarcasm Assistance
Fang, J.-F., Wang, W., Lin, G., & Lv, F. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680703 · Cited by 4
A gated leaky integrate-and-fire spiking neural network based on attention mechanism for multi-modal emotion recognition
Chen, G., Qian, Z., Qiu, S., Zhang, D., & Zhou, R. (2025) · Digital Signal Processing · doi:10.1016/j.dsp.2025.105322 · Cited by 5
Lightweight Models for Multimodal Sequential Data.
Sourav, S., & Ouyang, J. (2021) · Cited by 4
A New View of Multi-modal Language Analysis: Audio and Video Features as Text “Styles”
Sun, Z., Sarma, P. K., Liang, Y., & Sethares, W. A. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.167 · Cited by 4
Multimodal Privacy-preserving Mood Prediction from Mobile Data: A Preliminary StudyPreprint
Liu, T., Liang, P. P., Muszyński, M., Ishii, R., Brent, D. A., Auerbach, R. P., Allen, N. B., & Morency, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.02359 · Cited by 4
Self-citation
Modality-Invariant and -Specific Representations with Crossmodal Transformer for Multimodal Sentiment Analysis
Shan, Q., Wei, X., & Cai, Z. (2022) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2224/1/012024 · Cited by 4
Bi-attention Modal Separation Network for Multimodal Video Fusion
Du, P., Gao, Y., & Li, X. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-98358-1_46 · Cited by 3
IMCN: Identifying Modal Contribution Network for Multimodal Sentiment Analysis
Zhang, Q., Shi, L., Liu, P., Zhu, Z., & Xu, L. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956114 · Cited by 3
HumourHindiNet: Humour detection in Hindi web series using word embedding and convolutional neural network
Kumar, A., Mallik, A., & Kumar, S. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3661306 · Cited by 3
A Comprehensive Survey on Multimodal Sentiment Analysis
Dhal, D., Mohanty, J. P., & Rout, S. S. (2024) · International Conference on Intelligent Computing and Sustainable Innovations in Technology (IC-SIT) · doi:10.1109/ic-sit63503.2024.10862066 · Cited by 3
Crossmodal Language Comprehension—Psycholinguistic Insights and Computational Approaches
Alaçam, Ö., Li, X., Menzel, W., & Staron, T. (2020) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2020.00002 · Cited by 3
Multimodal Embeddings from Language ModelsPreprint
Tseng, S.-Y., Georgiou, P., & Narayanan, S. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.04302 · Cited by 3
Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion RecognitionPreprint
Dai, W., Liu, Z., Yu, T., & Fung, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.09629 · Cited by 5
Self-citation
MTAG: Modal-Temporal Attention Graph for Unaligned Human Multimodal Language SequencesPreprint
Yang, J., Wang, Y., Yi, R., Zhu, Y., Rehman, A., Zadeh, A., Poria, S., & Morency, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.11985 · Cited by 3
Self-citation
NFCMF: Noise Filtering and CrossModal Fusion for Multimodal Sentiment Analysis
Su, G., He, J., Li, X., Lu, M., & Yang, H. (2021) · International Conference on Asian Language Processing · doi:10.1109/ialp54817.2021.9675197 · Cited by 2
Information Bottleneck based Representation Learning for Multimodal Sentiment Analysis
Zhang, T., Zhang, H., Xiang, S., & Wu, T. (2022) · International Conference on Control Engineering and Artificial Intelligence (CCEAI) · doi:10.1145/3522749.3523069 · Cited by 2
Leveraging facial expressions as emotional context in image captioning
Das, R., Wu, N., & Dev, S. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17904-3 · Cited by 2
Multimodal sentiment analysis based on multi‐stage graph fusion networks under random missing modality conditions
Zhang, T., Song, B., Zhang, Z., & Zhang, Y. (2024) · IET Image Processing · doi:10.1049/ipr2.13310 · Cited by 2
CU-SEMLP: All-MLP-based multimodal interaction model for multimodal sentiment analysis
Liu, S., Zhao, H., Chen, Y., Kong, F., & Li, K. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07364-x · Cited by 2
HEROCA: Multimodal Sentiment Analysis Based on HEterogeneous Representation Optimization and Cross-Modal Attention
Zhao, T. G., Peng, J., Wang, L., & Zheng, C. (2023) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53992.2023.10393942 · Cited by 2
Multimodal Local-Global Ranking Fusion for Emotion RecognitionPreprint
Liang, P. P., Zadeh, A., & Morency, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.04931 · Cited by 6
Self-citation
Cross-Modality Disentangled Information Bottleneck Strategy for Multimodal Sentiment Analysis
Deng, Z., Huang, G., Zhong, G., Yuan, X., Huang, L., & Pun, C. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.10831481 · Cited by 1
Exploring Feature Interactions for Multimodal Sentiment Analysis
Xu, D., & Zhang, C. (2025) · International Journal of Cognitive Informatics and Natural Intelligence · doi:10.4018/ijcini.383754 · Cited by 1
A Lightweight Two-Stage Attention-Gating Network for Efficient Multimodal Sentiment Analysis
Wang, Y., Xie, W., Yang, Z., Xu, C., Lin, Y., & Chen, Y. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3626046 · Cited by 1
Multimodal Sentiment Analysis with Missing Modality: A Knowledge-Transfer Approach
Liu, W., & Zhan, H. (2025) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc65261.2025.11249068 · Cited by 2
Tree-Based Mix-Order Polynomial Fusion Network for Multimodal Sentiment Analysis
Tang, J., Hou, M., Jin, X., Zhang, J., Zhao, Q., & Kong, W. (2023) · Systems · doi:10.3390/systems11010044 · Cited by 1
Self-supervised Multimodal Representation Learning for Product Identification and Retrieval
Jiang, Y., Liao, K.-T., Lin, S.-D., Qiao, H., Yu, K., Yang, C., & Chen, Y. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-8145-8_44 · Cited by 1
Attention Information Fusion Emotion Analysis Model Based on BiLSTM-BERT
Cheng, Z., & Li, S. (2024) · International Conference on Consumer Electronics and Computer Engineering (ICCECE) · doi:10.1109/iccece61317.2024.10504175 · Cited by 1
Progressive Fusion Network with Mixture of Experts for Multimodal Sentiment Analysis
Liu, D., Yang, Z., & Guo, Z. (2024) · International Conference on Advanced Computational Intelligence (ICACI) · doi:10.1109/icaci60820.2024.10537022 · Cited by 1
Adaptive Language-Interacted Hyper-Modality Representation for Multimodal Sentiment Analysis
Pan, L., & Liu, W. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.0150746 · Cited by 2
Blending of Predictions Boosts Understanding for Multimodal Advertisements
Alekseev, A., Savchenko, A., Tutubalina, E. V., Myasnikov, E. V., & Nikolenko, S. (2024) · Journal of Mathematical Sciences · doi:10.1007/s10958-024-07438-w · Cited by 2

Authors: Liang, P. P., Liu, Z., Bagher Zadeh, A., & Morency, L.-P.
Affiliations: Carnegie Mellon University (US)
ORCID: Liang, Paul Pu
Morency, Louis-Philippe
Venue: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
Publisher: Association for Computational Linguistics
Published: 2018
Pages: 150-161
DOI: 10.18653/v1/d18-1014
OpenAlex: W2886193235
Cited by: 230
References: 69
Citations by year: 2026: 12 · 2025: 19 · 2024: 43 · 2023: 37 · 2022: 39 · 2021: 31 · 2020: 32 · 2019: 15 · 2018: 2
FWCI: 6.03
Type: conference paper
Language: en
Open access: hybrid
doi.org
Licence: cc-by
Locations: Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
Funders: National Science Foundation
Samsung
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
Keywords: cross-modal interaction, recurrent neural networks, multimodal sentiment analysis, emotion recognition

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Multimodal Language Analysis with Recurrent Multistage Fusion

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks