Multimodal Sentiment Intensity Analysis in Videos: Facial Gestures and Verbal Messages

Zadeh, Amir, Zellers, Rowan, Pincus, Eli, Morency, Louis-Philippe·2016·IEEE Intelligent Systems
DOI 10.1109/mis.2016.94

Cited by (852)

Items 401–500 of 852

  1. Multimodal Tweet Sentiment Classification Algorithm Based on Attention Mechanism
    Peiyu Zou, Shuangtao Yang · 2019 · Communications in computer and information science
  2. Speaker-Independent Multimodal Sentiment Analysis for Big Data
    Erik Cambria, Soujanya Poria, Amir Hussain · 2019 · Springer eBooks
  3. Cyclic Data Distillation Semi-Supervised Learning for Multi-Modal Emotion Recognition
    Shuzhen Li, Tong Zhang, C L Philip Chen · 2025 · IEEE Transactions on Knowledge and Data Engineering
  4. Self-attention mechanism prior to modality fusion for multimodal sentiment analysis
    Zhongyuan Chen, Chong Lu, Yihan Wang · 2025 · Multimedia Systems
  5. Boosting Modal-Specific Representations for Sentiment Analysis With Incomplete Modalities
    Xin Jiang, Lihuo He, Fei Gao, Kaifan Zhang, Jie Li, Xinbo Gao · 2025 · IEEE Transactions on Multimedia
  6. Multimodal sentiment analysis based on temporal perception and cross-modal interaction
    Changpeng Ji, TianYu Tan, Wei Dai · 2025 · Multimedia Systems
  7. Gradient and Structure Consistency in Multimodal Emotion Recognition
    Qinghongya Shi, Mang Ye, Wenke Huang, Bo Du, Xiaofen Zong · 2025 · IEEE Transactions on Image Processing
  8. Multimodal progressive contrastive learning for sentiment analysis
    Lianmin Zhou, Yuanyuan Pu, Zhengpeng Zhao, Jue Feng, Dan Xu, Jinjing Gu · 2025 · Neurocomputing
  9. A Prompt-Driven framework for compensation and fusion in multimodal sentiment analysis with missing modalities
    Jing Meng, Zhenfang Zhu, Qiang Lu, Hongli Pei, Kefeng Li, Yuzhi Ren, Meng (Leah) Li, Xiaowen Sun, Dawei Zhao · 2026 · Knowledge-Based Systems
  10. TPFN: A Text-Guided Progressive Fusion Network for Multimodal Sentiment Analysis
    Lisong Ou, Zhixin Li · 2026 · IEEE Transactions on Affective Computing
  11. Multimodal Sentiment Analysis Based on Dynamic Language Enhancement and Synergistic Cross-Modal Transformer
    Linqin Cai, Daohong Liu, Lanrui Liu, Renyun Mou · 2026 · IEEE Transactions on Neural Networks and Learning Systems
  12. Trusted Multimodal Socio-Cyber Sentiment Analysis Based on Disentangled Hierarchical Representation Learning
    Guoxia Xu, Lizhen Deng, Yansheng Li, Yantao Wei, Xiaokang Zhou, Hu Zhu · 2023 · IEEE Transactions on Computational Social Systems
  13. Multimodal sentiment analysis using Multi-Layer Fusion Convolution Neural Network
    D. Khalandar Basha, G Sunil, A. H. A. Hussein, Mukesh S, Myasar Mundher Adnan · 2023 · International Conference on Ambient Intelligence, Knowledge Informatics and Industrial Electronics (AIKIIE)
  14. Multimodal Sentiment Analysis Based on 3D Stereoscopic Attention
    Jian Huang, Yuanyuan Pu, Dongming Zhou, Hang Shi, Zhengpeng Zhao, Dan Xu, Jinde Cao · 2024 · IEEE International Conference on Acoustics Speech and Signal Processing
  15. Evaluation of Data Inconsistency for Multi-modal Sentiment Analysis
    Yufei Wang, Mengyue Wu · 2024 · Communications in computer and information science
  16. A dynamic weighted fusion model for multimodal sentiment analysis
    Liang Yang · 2025 · Signal Image and Video Processing
  17. ER-MRL: Emotion Recognition based on Multimodal Representation Learning
    Xiaoding Guo, Yadi Wang, Zhijun Miao, Xiaojin Yang, Jinkai Guo, Xianhong Hou, Feifei Zao · 2022 · International Conference on Information Science and Technology
  18. AtM-DNN: A Multimodal Attention Fusion Network with Auxiliary Function for Sentiment Classification
    Huang Ji, Xiaolong Xu · 2022 · International Conference on Computer Supported Cooperative Work in Design (CSCWD)
  19. A Personalized Affective Memory Neural Model for Improving Emotion Recognition
    Pablo V. A. Barros, German Ignacio Parisi, Stefan Wermter · 2019 · arXiv (Cornell University)
  20. ROSA: A Robust Self-Adaptive Model for Multimodal Emotion Recognition With Uncertain Missing Modalities
    Ziming Li, Yaxin Liu, Chuanpeng Yang, Yan Zhou, Songlin Hu · 2025 · IEEE Transactions on Multimedia
  21. DDSE: A Decoupled Dual-Stream Enhanced Framework for Multimodal Sentiment Analysis with Text-Centric SSM
    Shenjie Jiang, Zhuoyu Wang, Xuecheng Wu, Hongru Ji, M. X. Li, Xianghua Li, Chao Gao · 2025 · ACM International Conference on Multimedia (ACM MM)
  22. Language-dominated fusion and self-distillation for multimodal sentiment analysis with incomplete modalities
    Peng Shi, Jin Wang, Xuejie Zhang · 2025 · Pattern Recognition
  23. Unlocking explainable and effective multimodal affective reasoning via large language models
    Junjie Liao, Jiandian Zeng, Binbin Song, Mengting Zhou, Xiaopeng Fan B, Tian Wang · 2026 · Pattern Recognition
  24. MISSRAG: Addressing the Missing Modality Challenge in Multimodal Large Language Models
    Vittorio Pipoli, Alessia Saporita, Federico Bolelli, Marcella Cornia, Lorenzo Baraldi, Costantino Grana, Rita Cucchiara, Elisa Ficarra · 2025 · IEEE/CVF International Conference on Computer Vision (ICCV)
  25. Learning Modality-Complementary and Eliminating-Redundancy Representations with Multi-Task Learning for Multimodal Sentiment Analysis
    Xiaowei Zhao, Xinyu Miao, Xiujuan Xu, Yu Liu, Yifei Cao · 2024 · International Joint Conference on Neural Networks (IJCNN)
  26. Multi-HM: A Chinese Multimodal Dataset and Fusion Framework for Emotion Recognition in Human–Machine Dialogue Systems
    Yao Fu, Qiong Liu, Qing Song, Pengzhou Zhang, Gongdong Liao · 2025 · Applied Sciences
  27. Learning by Comparing: Boosting Multimodal Affective Computing through Ordinal Learning
    Sijie Mai, Ying Zeng, Haifeng Hu · 2025 · International World Wide Web Conference (WWW)
  28. Bimodal Sentiment Analysis Based on a Pre-Trained Model and Masked Attention Fusion
    Li Cai, Hongbin Li, Nan Zhang, Jing He · 2025 · IEEE Transactions on Audio Speech and Language Processing
  29. A Multimodal Sentiment Analysis Approach Based on Multiview Cross-Modal Fusion
    Yuxing Zhi, Junhuai Li, Huaijun Wang, Jing Chen · 2025 · IEEE Transactions on Computational Social Systems
  30. $$A^{2} H^{2}$$ for multimodal emotional data analysis
    Jun Wu, Jinyu Liu, Tianfeng Zhang, Shuai Guo, Yu Chen, Jiahui Huang, Fang An Deng · 2025 · Journal of Intelligent Information Systems
  31. SG-DTAM: Joint staged generation and dynamic time alignment for missing and unaligned modalities in sentiment analysis
    Deling Huang, Ran Gao, Geng Zhang, Jian Yu · 2025 · Expert Systems with Applications
  32. Sentiment Guided Aspect Conditioned Dialogue Generation in a Multimodal System
    Mauajama Firdaus, Nidhi Thakur, Asif Ekbal · 2022 · Lecture notes in computer science
  33. Hierarchical Supervised Contrastive Learning for Multimodal Sentiment Analysis
    Kezhou Chen, Shuo Wang, Yanbin Hao · 2024 · Lecture notes in computer science
  34. Conditional selection with CNN augmented transformer for multimodal affective analysis
    Jianwen Wang, Shiping Wang, Shunxin Xiao, Renjie Lin, Mianxiong Dong, Wenzhong Guo · 2024 · CAAI Transactions on Intelligence Technology
  35. SmartRAN: Smart Routing Attention Network for multimodal sentiment analysis
    Xueyu Guo, Shengwei Tian, Long Yu, Xiaoyu He · 2024 · Applied Intelligence
  36. DRSS: a multimodal sentiment analysis approach based on dual representation and self-supervised learning strategy
    Jing Meng, Zhenfang Zhu, Jiangtao Qi, Huaxiang Zhang · 2024 · The Journal of Supercomputing
  37. Sentiment-oriented Sarcasm Integration for Video Sentiment Analysis Enhancement with Sarcasm Assistance
    J.-F. Fang, Wenya Wang, Guosheng Lin, Fengmao Lv · 2024 · ACM International Conference on Multimedia (ACM MM)
  38. MST-ARGCN: modality-squeeze transformer with attentional recurrent graph capsule network for multimodal sentiment analysis
    Chengyu Hu, Jin Liu, Xingye Li, Meijing Li, Huihua He · 2024 · The Journal of Supercomputing
  39. Gated Fusion Network With Progressive Modality Enhancement for Sentiment Analysis
    Xin Feng, Wenjuan Gong, Tingbo Shi, Zhong Jing, Kechen Li, Jordi Gonzàlez · 2024 · IEEE Access
  40. Semantic-Driven Crossmodal Fusion for Multimodal Sentiment Analysis
    Pingshan Liu, Zhaoyang Wang, Fu Jie Huang · 2024 · International Journal on Semantic Web and Information Systems
  41. EmoShiftNet: a shift-aware multi-task learning framework with fusion strategies for emotion recognition in multi-party conversations
    Hinduja Nirujan, Y. H. P. P. Priyadarshana · 2025 · Frontiers in Artificial Intelligence
  42. Lightweight Models for Multimodal Sequential Data.
    Soumya Sourav, Jessica Ouyang · 2021
  43. Conversational Emotion Detection and Elicitation: A Preliminary Study
    Misbah Farooq, De Silva, Haileleol Tibebu, Xiyu Shi · 2023 · IEEE IAS Global Conference on Emerging Technologies (GlobConET)
  44. Mutual information maximization and feature space separation and bi-bimodal mo-dality fusion for multimodal sentiment analysis
    Kun Li, Shengwei Tian, Long Yu, Tiejun Zhou, Bo Wang, Fun Wang · 2023 · Journal of Intelligent & Fuzzy Systems
  45. Multimodal Learning with Incompleteness towards Multimodal Sentiment Analysis and Emotion Recognition Task
    Cam-Van Thi Nguyen, Huy-Sang Nguyen, Duc-Trong Le, Quang-Thuy Ha · 2023 · International Conference on Knowledge and Systems Engineering/Proceedings
  46. An Improved Model of Multi-attention LSTM for Multimodal Sentiment Analysis
    Anna Wang, Baoshan Sun, Rize Jin · 2019
  47. Hybrid grass bee optimization ‐ multikernal extreme learning classifier: Multimodular fusion strategy and optimal feature selection for multimodal sentiment analysis in social media videos
    Abdullah Saleh Alqahtani, Saravanan Pandiaraj, Maheswari Murali, Sami Alshmrany, Haytham Saleem Al-SARRAYRIH · 2021 · Concurrency and Computation Practice and Experience
  48. Exploring Multimodal Sentiment Analysis through Cartesian Product approach using BERT Embeddings and ResNet-50 encodings and comparing performance with pre-existing models
    Aruna Bhat, Rahul Mahar, Rahul Punia, Rahul Srivastava · 2022 · International Conference for Emerging Technology (INCET)
  49. Multilevel Representation Disentanglement Framework for Multimodal Sentiment Analysis
    Nan Jia, Zicong Bai, Tiancheng Xiong, Mingyang Guo · 2025 · IEEE Signal Processing Letters
  50. Efficient Hierarchical Multimodal Graph Neural Networks for AI-Driven Decision-Making in Consumer Data
    Jothi Prakash V, S. Arul Antran Vijay, S. Gopikrishnan, Maha Driss, Wadii Boulila · 2025 · IEEE Transactions on Consumer Electronics
  51. Hierarchical Text-Guided Refinement Network for Multimodal Sentiment Analysis
    Yue Su, X. Y. Zhao · 2025 · Entropy
  52. Personalized multimodal sentiment analysis under uncertain modalities missing via pretraining and online learning
    Hongxiang Sun, Zhizhong Liu, Dianhui Chu, Quan Z. Sheng, Zhaowei Liu, Jian Yu · 2025 · Knowledge-Based Systems
  53. Multi-modal sentiment analysis based on multi-level modal information interaction
    Hao Wang, Gulanbaier Tuerhong, Mairidan Wushouer, Xu Guo · 2025 · Procedia Computer Science
  54. KD-MSA: A Multimodal Implicit Sentiment Analysis Approach Based on KAN and Asymmetric Contribution-Aware Dynamic Fusion
    Zhiyuan Hou, Qiang Zhang, Ziwei Lei, Zheng Zeng, Ruijun Jia · 2025 · Symmetry
  55. MMKT: Multimodal Sentiment Analysis Model Based on Knowledge-Enhanced and Text-Guided Learning
    Chengkai Shi, Yunhua Zhang · 2025 · Applied Sciences
  56. DRLCCI: A hybrid fusion network leveraging disentangled representation learning and cross-modal collaborative interaction for multi-modal sentiment analysis
    Xue Lei, Miao Ma, Yutong Li · 2025 · Neurocomputing
  57. Text-Guided Enhanced Transformer Fusion for Multimodal Sentiment Analysis
    Xiangyu Liao, Xianxin Ke, Qinghua Liu, Guoliang Li · 2025 · Journal of Shanghai Jiaotong University (Science)
  58. TA-OEM : Multimodal sentiment analysis using two-step hierarchical attention and self-supervised label generation
    Jiaming Zhang, Wenqiang Li, Zhijia Zhang, Jian Fang, Huaici Zhao · 2025 · Knowledge-Based Systems
  59. Hybrid prompt learning and multilevel knowledge distillation for multimodal sentiment analysis with missing modalities
    Yiqiao Zhai, Qiuxia Yang, Chengchao Wang, Lianmin Zhou, Jue Feng, Fanghong Hu, Zhengpeng Zhao, Yuanyuan Pu · 2026 · Expert Systems with Applications
  60. TQ-CHAFN: Triple-query cross-modal hierarchical adaptive fusion network for multimodal sentiment analysis
    Gao Chien, Jianhua Liu, Jiacan Wang, Linying Chen · 2026 · Knowledge-Based Systems
  61. CMSBERT-CLR: Context-driven Modality Shifting BERT with Contrastive Learning for linguistic, visual, acoustic Representations
    Junghun Kim, Jihie Kim · 2022 · International Joint Conference on Neural Networks (IJCNN)
  62. Multi-task disagreement-reducing multimodal sentiment fusion network
    Wang Zijun, Jiang Naicheng, Chao Xinyue, Bin Sun · 2024 · Image and Vision Computing
  63. MATF: main-auxiliary transformer fusion for multi-modal sentiment analysis
    Qing Zhong, Xin-Hui Shao · 2024 · Soft Computing
  64. Global information regulation network for multimodal sentiment analysis
    Shufan Xie, Qiaohong Chen, Xianwen Fang, Qi Sun · 2024 · Image and Vision Computing
  65. Meta-Learning for Incomplete Multimodal Sentiment Analysis
    Geng Tu, Tianhao Wu, Xuan Luo, Xi Zeng, Wenjie Li, Ruifeng Xu · 2025 · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)
  66. Frequency Restoration and Modality Enforcement towards Resisting-corruption Multimodal Sentiment Analysis
    Weicheng Xie, Haijian Liang, Zenghao Niu, Xianxu Hou, Siyang Song, Zitong Yu, Linlin Shen · 2025 · ACM Transactions on Multimedia Computing Communications and Applications
  67. Multimodal prototypical network for interpretable sentiment classification
    Chenguang Song, Ke Chao, Bingjing Jia, Yiqing Shen · 2025 · Scientific Reports
  68. Multimodal Mixture of Low-Rank Experts for Sentiment Analysis and Emotion Recognition
    Shuo Zhang, Jinsong Zhang, Zhejun Zhang, Lei Li · 2025 · IEEE International Conference on Multimedia and Expo (ICME)
  69. ``The Squawk Bot'': Joint Learning of Time Series and Text Data Modalities for Automated Financial Information Filtering
    Xuan Hong Dang, Syed Yousaf Shah, Petros Zerfos · 2020 · International Joint Conference on Artificial Intelligence
  70. Impact Evaluation of Multimodal Information on Sentiment Analysis
    Luis N. Zúñiga-Morales, Jorge Ángel González Ordiano, J. Emilio Quiróz-Ibarra, Steve Simske · 2022 · Lecture notes in computer science
  71. Lightweight and Efficient Privacy-Preserving Multimodal Representation Inference via Fully Homomorphic Encryption
    Zhaojue Li, Yingpeng Sang, Xinru Deng, Hui Tian · 2023 · Lecture notes in computer science
  72. Unimodal Sentiment Analysis
    Hua Xu · 2023 · Springer eBooks
  73. Usefulness of Emotional Prosody in Neural Machine Translation
    Charles Brazier, Jean-Luc Rouas · 2024 · Speech prosody
  74. Multimodal reaching-position prediction for ADL support using neural networks
    Yutaka Takase, Kimitoshi Yamazaki · 2024 · ROBOMECH Journal
  75. A Residual Multi-Scale Convolutional Neural Network With Transformers for Speech Emotion Recognition
    Tianhao Yan, Hao Meng, Emilia Parada‐Cabaleiro, Jianhua Tao, Taihao Li, Björn Wolfgang Schuller · 2024 · IEEE Transactions on Affective Computing
  76. Web Semantic-Enhanced Multimodal Sentiment Analysis Using Multilayer Cross-Attention Fusion
    Yong Liu, YU Shi-qiu · 2024 · International Journal on Semantic Web and Information Systems
  77. PSACF: Parallel-Serial Attention-Based Cross-Fusion for Multimodal Emotion Recognition
    Yong Zhang, Yongqing Liu, Cheng Cheng · 2024 · IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
  78. Modality-Guided Refinement Learning for Multimodal Emotion Recognition
    Sunyoung Cho · 2025 · IEEE Access
  79. CNAMD Corpus: A Chinese Natural Audiovisual Multimodal Database of Conversations for Social Interactive Agents
    Jingyu Wu, Shi Chen, Wei Xiang, Lingyun Sun, Hongzeng Zhang, Zhengyu Zhang, Yanxu Li · 2023 · International Journal of Human-Computer Interaction
  80. Crossmodal Language Comprehension—Psycholinguistic Insights and Computational Approaches
    Özge Alaçam, Xingshan Li, Wolfgang Menzel, Tobias Staron · 2020 · Frontiers in Neurorobotics
  81. Variational Fusion for Multimodal Sentiment Analysis
    Navonil Majumder, Soujanya Poria, Gangeshwar Krishnamurthy, Niyati Chhaya, Rada F. Mihalcea, Alexander F. Gelbukh · 2019 · arXiv (Cornell University)
  82. MTAG: Modal-Temporal Attention Graph for Unaligned Human Multimodal Language Sequences
    Jianing Yang, Yongxin Wang, Ruitao Yi, Yuying Zhu, Azaan Rehman, Amir Zadeh, Soujanya Poria, Louis‐Philippe Morency · 2020 · arXiv (Cornell University)
  83. Cross-Modal Knowledge Transfer via Inter-Modal Translation and Alignment for Affect Recognition
    Vandana Rajan, Alessio Brutti, Andrea Cavallaro · 2021 · arXiv (Cornell University)
  84. Multimodal Sentiment Analysis Network Based on Distributional Transformation and Gated Cross-Modal Fusion
    Yuchen Zhang, Hong Zhong, Guilin Chen, Naji Alhusaini, Shenghui Zhao, Cheng Wu · 2024 · International Conference on Networking and Network Applications (NaNA)
  85. Emotion and Sentiment Analysis in Dialogue: A Multimodal Strategy Employing the BERT Model
    Mahesh Parmar, Akhilesh Tiwari · 2024 · Parul International Conference on Engineering and Technology (PICET)
  86. $\mathcal {P}$owMix: A Versatile Regularizer for Multimodal Sentiment Analysis
    Efthymios Georgiou, Yannis Avrithis, Alexandros Potamianos · 2024 · IEEE/ACM Transactions on Audio Speech and Language Processing
  87. Multi-level Multi-task representation learning with adaptive fusion for multimodal sentiment analysis
    Chuanbo Zhu, Min Chen, Haomin Li, Sheng Zhang, Han Liang, Chao Sun, Yifan Liu, Jincai Chen · 2024 · Neural Computing and Applications
  88. TCE-DBF: textual context enhanced dynamic bimodal fusion for hate video detection
    Haitao Xiong, Wei Jiao, Yuanyuan Cai · 2024 · Data Technologies and Applications
  89. Emotional boundaries and intensity aware model for incomplete multimodal sentiment analysis
    Yuqing Zhang, Dongliang Xie, Dawei Luo, Baosheng Sun · 2025 · Digital Signal Processing
  90. Enhanced Multimodal Depression Detection With Emotion Prompts
    Shiyu Teng, Jiaqing Liu, Hao Sun, Shurong Chai, Tomoko Tateyama, Lanfen Lin, Yen‐Wei Chen · 2025 · IEEE International Conference on Acoustics Speech and Signal Processing
  91. SIR-HCL: Semantic-Inconsistency Reasoning and Hybrid Contrastive Learning for Efficient Cross-Emotion Anomaly Detection
    Xin Liu, Qiyan Chen, Yiu‐ming Cheung, Shu‐Juan Peng · 2025 · IEEE Transactions on Cognitive and Developmental Systems
  92. Dynamic Tuning and Multi-Task Learning-Based Model for Multimodal Sentiment Analysis
    Yi Liang, Turdi Tohti, Wenpeng Hu, Bo Kong, Dongfang Han, Tianwei Yan, Askar Hamdulla · 2025 · Applied Sciences
  93. Dual-stream collaborative network (DSCN): Multimodal sentiment analysis via modality-invariant and modality-specific representation learning
    Shengbing Chen, Dongmei Zhou, Chao Wei, Xingyang Li, Hong-Yu Yao, Hao Luo · 2025 · Neurocomputing
  94. Seeking personality yet commonality via adversarial learning to enhance heterogeneous multimodal collaboration
    Ze Xiang Wu, Qi Zhang, Duoqian Miao, Xuerong Zhao, Guangyin Bao, Liang Hu, Kun Yi, Yu Zhou · 2025 · Information Fusion
  95. Hierarchical Signal Calibration and Refinement for Multimodal Sentiment Analysis
    Baojian Ren, Tao Cao, Zhengyang Zhang, Shuchen Bai, Na Liu · 2025 · IEEE Signal Processing Letters
  96. Tri-Modal grouping fusion network with optimal transport learning for unaligned multimodal sentiment analysis
    Aolin Xiong, Sijie Mai, Haifeng Hu · 2025 · Expert Systems with Applications
  97. D$$^2$$M: Decomposition-diffusion model for enhanced multimodal sentiment analysis
    Liansong Zong, Wenzhuo Xu, Dongfeng Hu, Jie Wang · 2025 · Applied Intelligence
  98. DAST-GAN: An Adversarial Learning Multimodal Sentiment Analysis Model Based on Dynamic Attention and Spatio-Temporal Fusion
    Wenlong Tan, Wenjuan Zhang · 2025 · Sensors
  99. Progressive intra- and inter-modality relation learning for multimodal sentiment analysis
    Jing Li, Zhibin Wu, Qiangchang Wang · 2025 · Knowledge-Based Systems
  100. Frequency-aware experts with multi-stage fusion for multimodal sentiment analysis
    Xiaofei Zhu, Yaochen Li · 2025 · Journal of Intelligent Information Systems