Multimodal sentiment analysis with word-level fusion and reinforcement learning

Cited by (330)

Items 101–200 of 330

1. See, hear, read: Leveraging multimodality with guided attention for abstractive text summarization
   Yash Kumar Atri, Shraman Pramanick, Vikram Goyal, Tanmoy Chakraborty · 2021 · Knowledge-Based Systems
2. A feature-based restoration dynamic interaction network for multimodal sentiment analysis
   Yufei Zeng, Zhixin Li, Zhenbin Chen, Huifang Ma · 2023 · Engineering Applications of Artificial Intelligence
3. Mimicking the Brain’s Cognition of Sarcasm From Multidisciplines for Twitter Sarcasm Detection
   Fanglong Yao, Xian Sun, Hongfeng Yu, Wenkai Zhang, Wei Liang, Kun Fu · 2021 · IEEE Transactions on Neural Networks and Learning Systems
4. A comprehensive review of visual–textual sentiment analysis from social media networks
   Israa Khalaf Salman Al-Tameemi, Mohammad‐Reza Feizi‐Derakhshi, Saeid Pashazadeh, Mohammad Asadpour · 2024 · Journal of Computational Social Science
5. Dialogue systems with audio context
   Tom Young, Vlad Pandelea, Soujanya Poria, Erik Cambria · 2020 · Neurocomputing
6. Factorized Multimodal Transformer for Multimodal Sequential Learning
   Amir Zadeh, Chengfeng Mao, Kelly Shi, Yiwei Zhang, Paul Pu Liang, Soujanya Poria, Louis‐Philippe Morency · 2019 · arXiv (Cornell University)
7. Human Conversation Analysis Using Attentive Multimodal Networks with Hierarchical Encoder-Decoder
   Yue Gu, Xinyu Li, Kaixiang Huang, Shiyu Fu, Kangning Yang, Shuhong Chen, Moliang Zhou, Ivan Marsic · 2018 · ACM International Conference on Multimedia (ACM MM)
8. Multi-Attention Multimodal Sentiment Analysis
   Taeyong Kim, Bowon Lee · 2020 · International Conference on Multimedia Retrieval
9. Sentiment analysis from Customer-generated online videos on product review using topic modeling and Multi-attention BLSTM
   Zheng Wang, Peng Gao, Xuening Chu · 2022 · Advanced Engineering Informatics
10. A multi-stage multimodal framework for sentiment analysis of Assamese in low resource setting
    Ringki Das, Thoudam Doren Singh · 2022 · Expert Systems with Applications
11. DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations
    Yiwei Lyu, Paul Pu Liang, Zihao Deng, Ruslan Salakhutdinov, Louis‐Philippe Morency · 2022 · Proceedings of the AAAI/ACM Conference on AI Ethics and Society
12. Fact-sentiment incongruity combination network for multimodal sarcasm detection
    Qiang Lu, Yunfei Long, Xia Sun, Jun Hong Feng, Hao Zhang · 2023 · Information Fusion
13. A context-sensitive multi-tier deep learning framework for multimodal sentiment analysis
    P. Ganeshkumar, S. Arul Antran Vijay, Jothi Prakash V, Anand Paul, Anand Nayyar · 2023 · Multimedia Tools and Applications
14. Modality-based Factorization for Multimodal Fusion
    Elham J. Barezi, Pascale Fung · 2019 · Workshop on Representation Learning for NLP (RepL4NLP)
15. Multilevel Sensor Fusion With Deep Learning
    Valentin Vielzeuf, Alexis Lechervy, Stéphane Pateux, Frédéric Jurie · 2018 · IEEE Sensors Letters
16. An Ensemble Method for Radicalization and Hate Speech Detection Online Empowered by Sentic Computing
    Óscar Araque, Carlos Á. Iglesias · 2021 · Cognitive Computation
17. Multimodal Deep Learning Framework for Sentiment Analysis from Text-Image Web Data
    Selvarajah Thuseethan, Sivasubramaniam Janarthan, Sutharshan Rajasegarar, Priya Kumari, John L. Yearwood · 2020 · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT)
18. Multimodal Graph for Unaligned Multimodal Sequence Analysis via Graph Convolution and Graph Pooling
    Sijie Mai, Songlong Xing, Jiaxuan He, Ying Zeng, Haifeng Hu · 2022 · ACM Transactions on Multimedia Computing Communications and Applications
19. Hybrid Attention based Multimodal Network for Spoken Language Classification.
    Yue Gu, Kangning Yang, Shiyu Fu, Shuhong Chen, Xinyu Li, Ivan Marsic · 2018 · PubMed
20. Integration of fuzzy logic and a convolutional neural network in three-way decision-making
    Leela Subhashini, Yuefeng Li, Jinglan Zhang, Ajantha Sanjeewa Atukorale · 2022 · Expert Systems with Applications
21. Diversity and Balance: Multimodal Sentiment Analysis Using Multimodal-Prefixed and Cross-Modal Attention
    Meng Li, Zhenfang Zhu, Kefeng Li, Hongli Pei · 2024 · IEEE Transactions on Affective Computing
22. Text-Oriented Modality Reinforcement Network for Multimodal Sentiment Analysis from Unaligned Multimodal Sequences
    Yuxuan Lei, Dingkang Yang, Mingcheng Li, Shunli Wang, Jiawei Chen, Lihua Zhang · 2024 · Lecture notes in computer science
23. Deep Hierarchical Fusion with Application in Sentiment Analysis
    Efthymios Georgiou, Charilaos Papaioannou, Alexandros Potamianos · 2019 · Interspeech (USB)
24. RBA-GCN: Relational Bilevel Aggregation Graph Convolutional Network for Emotion Recognition
    Lin Yuan, Guoheng Huang, Fenghuan Li, Xiaochen Yuan, Chi‐Man Pun, Guo Zhong · 2023 · IEEE/ACM Transactions on Audio Speech and Language Processing
25. A novel multi-modal fusion method based on uncertainty-guided meta-learning
    Duoyi Zhang, Md Abul Bashar, Richi Nayak · 2024 · Pattern Recognition
26. Sentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors
    Yang Wu, Yanyan Zhao, Hao Yang, Song Chen, Bing Qin, Xiaohuan Cao, Wenting Zhao · 2022 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
27. AcFormer: An Aligned and Compact Transformer for Multimodal Sentiment Analysis
    Daoming Zong, Chaoyue Ding, Baoxiang Li, Jiakui Li, Keru Zheng, Qunyan Zhou · 2023 · ACM International Conference on Multimedia (ACM MM)
28. Multimodal Knowledge-enhanced Interactive Network with Mixed Contrastive Learning for Emotion Recognition in Conversation
    Xudong Shen, Xianying Huang, Shihao Zou, Xinyi Gan · 2024 · Neurocomputing
29. Multi-Modal Sentiment Analysis Based on Image and Text Fusion Based on Cross-Attention Mechanism
    Hongchan Li, Yantong Lu, Haodong Zhu · 2024 · Electronics
30. Effective Sentiment-relevant Word Selection for Multi-modal Sentiment Analysis in Spoken Language
    Dong Zhang, Shoushan Li, Qiaoming Zhu, Guodong Zhou · 2019 · ACM International Conference on Multimedia (ACM MM)
31. Mutual Correlation Attentive Factors in Dyadic Fusion Networks for Speech Emotion Recognition
    Yue Gu, Xinyu Lyu, Weijia Sun, Weitian Li, Shuhong Chen, Xinyu Li, Ivan Marsic · 2019 · ACM International Conference on Multimedia (ACM MM)
32. Improving Multimodal fusion via Mutual Dependency Maximisation
    Pierre Colombo, Émile Chapuis, Matthieu Labeau, Chloé Clavel · 2021 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
33. Transformer-Based Interactive Multi-Modal Attention Network for Video Sentiment Detection
    Xuqiang Zhuang, Fangai Liu, Jian Hou, Jianhua Hao, Xiaohong Cai · 2022 · Neural Processing Letters
34. Multimodal Boosting: Addressing Noisy Modalities and Identifying Modality Contribution
    Sijie Mai, Sun Ya, Aolin Xiong, Ying Zeng, Haifeng Hu · 2023 · IEEE Transactions on Multimedia
35. A Unimodal Representation Learning and Recurrent Decomposition Fusion Structure for Utterance-Level Multimodal Embedding Learning
    Mai, Sijie, Hu, Haifeng, Xing, Songlong · 2022 · IEEE Transactions on Multimedia
36. GLoMo: Global-Local Modal Fusion for Multimodal Sentiment Analysis
    Zhuang Yan, Yanru Zhang, Zheng Hu, Xiaoyue Zhang, Jiawen Deng, Fuji Ren · 2024 · ACM International Conference on Multimedia (ACM MM)
37. Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysis
    Ying Zeng, Sijie Mai, Haifeng Hu · 2021 · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
38. A Optimized BERT for Multimodal Sentiment Analysis
    Jun Wu, Tianliang Zhu, Jiahui Zhu, Tianyi Li, Chunzhi Wang · 2022 · ACM Transactions on Multimedia Computing Communications and Applications
39. Analyzing Unaligned Multimodal Sequence via Graph Convolution and Graph Pooling Fusion
    Sijie Mai, Songlong Xing, Jiaxuan He, Ying Zeng, Haifeng Hu · 2020 · arXiv (Cornell University)
40. Emotion Recognition in Conversations: A Survey Focusing on Context, Speaker Dependencies, and Fusion Methods
    Yao Fu, Shaoyang Yuan, Chi Zhang, Juan Cao · 2023 · Electronics
41. Multi-modal graph context extraction and consensus-aware learning for emotion recognition in conversation
    Yijing Dai, Jinxing Li, Yingjian Li, Guangming Lu · 2024 · Knowledge-Based Systems
42. Dynamic Invariant-Specific Representation Fusion Network for Multimodal Sentiment Analysis
    Jing He, Haonan Yanga, Changfan Zhang, Hong-Run Chen, Yi-Shi Xua · 2022 · Computational Intelligence and Neuroscience
43. Deep learning on multi-view sequential data: a survey
    Zhuyang Xie, Yan Zhu Yang, Yiling Zhang, Jie Wang, Shengdong Du · 2022 · Artificial Intelligence Review
44. Multimodal Reaction: Information Modulation for Cross-Modal Representation Learning
    Ying Zeng, Sijie Mai, Wenjun Yan, Haifeng Hu · 2023 · IEEE Transactions on Multimedia
45. Incorporating End-to-End Speech Recognition Models for Sentiment Analysis
    Egor Lakomkin, Mohammad Ali Zamani, Cornelius Weber, Sven Magg, Stefan Wermter · 2019 · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings
46. Proceedings of Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML)
    2018
47. Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
    Isabelle Augenstein, Spandana Gella, Sebastian Ruder, Katharina Kann, Burcu Can, Alexis Conneau, Johannes Welbl, Xian Ren, Marek Rei · 2019
48. Multimodal Learning for Identifying Opportunities for Empathetic Responses
    Leili Tavabi, Kalin Stefanov, Setareh Nasihati Gilani, David R. Traum, Mohammad Soleymani · 2019 · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
49. Multimodal Gated Information Fusion for Emotion Recognition from EEG Signals and Facial Behaviors
    Soheil Rayatdoost, David Rudrauf, Mohammad Soleymani · 2020 · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
50. A Contextual Attention Network for Multimodal Emotion Recognition in Conversation
    Tana Wang, Yaqing Hou, Dongsheng Zhou, Qiang Zhang · 2021 · International Joint Conference on Neural Networks (IJCNN)
51. MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
    Liang, Paul Pu, Lyu, Yiwei, Fan, Xiang, Wu, Zetian, Cheng, Yun, Wu, Jason, Chen, Leslie, Wu, Peter, Lee, Michelle A., Zhu, Yuke et al. · 2021 · arXiv
52. Multimodal Sentiment Analysis with Multi-perspective Fusion Network Focusing on Sense Attentive Language
    Xia Li, Minping Chen · 2020 · Lecture notes in computer science
53. Enhancing emotion inference in conversations with commonsense knowledge
    Dayu Li, Xiaodan Zhu, Yang Li, Suge Wang, Deyu Li, Jian Liao, Jianxing Zheng · 2021 · Knowledge-Based Systems
54. AMSA: Adaptive Multimodal Learning for Sentiment Analysis
    Jingyao Wang, Luntian Mou, Лей Ма, Tiejun Huang, Wen Gao · 2022 · ACM Transactions on Multimedia Computing Communications and Applications
55. A deep learning approach for image and text classification using neutrosophy
    Mohd Anas Wajid, Aasim Zafar, Mohammad Saif Wajid · 2023 · International Journal of Information Technology
56. Exploring Semantic Relations for Social Media Sentiment Analysis
    Jiandian Zeng, Jiantao Zhou, Caishi Huang · 2023 · IEEE/ACM Transactions on Audio Speech and Language Processing
57. Sentiment Analysis on User-generated Video, Audio and Text
    Ashwini Rao, Akriti Ahuja, Shyam Kansara, Vrunda Patel · 2021 · International Conference on Computing, Communication, and Intelligent Systems (ICCCIS)
58. Dominant SIngle-Modal SUpplementary Fusion (SIMSUF) for Multimodal Sentiment Analysis
    Jian Huang, Yanli Ji, Zhen Qin, Yang Yang, Heng Tao Shen · 2023 · IEEE Transactions on Multimedia
59. Injecting Multimodal Information Into Pre-Trained Language Model for Multimodal Sentiment Analysis
    Sijie Mai, Ying Zeng, Aolin Xiong, Haifeng Hu · 2025 · IEEE Transactions on Affective Computing
60. Hierarchical multiples self-attention mechanism for multi-modal analysis
    Jun Wu, Tianliang Zhu, Jiahui Zhu, Tianyi Li, Wang Chunzhi · 2023 · Multimedia Systems
61. Moving From Narrative to Interactive Multi-Modal Sentiment Analysis: A Survey
    Junxia Ma, Lu Rong, Yazhou Zhang, Prayag Tiwari · 2023 · ACM Transactions on Asian and Low-Resource Language Information Processing
62. Joint training strategy of unimodal and multimodal for multimodal sentiment analysis
    Li Meng, Zhenfang Zhu, Kefeng Li, Lihua Zhou, Zhen Zhao, Hongli Pei · 2024 · Image and Vision Computing
63. SDRS: Sentiment-Aware Disentangled Representation Shifting for Multimodal Sentiment Analysis
    Sicheng Zhao, Zhenhua Yang, Henglin Shi, Xiaocheng Feng, Lingpengkun Meng, Bing Qin, Chenggang Clarence Yan, Jianhua Tao, Guiguang Ding · 2025 · IEEE Transactions on Affective Computing
64. Modality-collaborative Transformer with Hybrid Feature Reconstruction for Robust Emotion Recognition
    Chengxin Chen, Pengyuan Zhang · 2024 · ACM Transactions on Multimedia Computing Communications and Applications
65. Cross-Modal Dynamic Transfer Learning for Multimodal Emotion Recognition
    Soyeon Hong, Hyeoungguk Kang, Hyunsouk Cho · 2024 · IEEE Access
66. Improvement of Multimodal Emotion Recognition Based on Temporal-Aware Bi-Direction Multi-Scale Network and Multi-Head Attention Mechanisms
    Yuezhou Wu, Siling Zhang, Pengfei Li · 2024 · Applied Sciences
67. Sentiment Analysis of Persian Instagram Post: a Multimodal Deep Learning Approach
    Aria Naseri Karimvand, Reza Salehi Chegeni, Mohammad Ehsan Basiri, Shahla Nemati · 2021 · International Conference on Web Research (ICWR)
68. Human-in-the-Loop Low-Shot Learning
    Sen Wan, Yimin Hou, Feng Bao, Zhiquan Ren, Yunfeng Dong, Qionghai Dai, Yue Deng · 2020 · IEEE Transactions on Neural Networks and Learning Systems
69. Multimodal fusion via cortical network inspired losses
    Shiv Shankar · 2022 · Proceedings of the conference - Association for Computational Linguistics. Meeting
70. Multi-Scale Temporal Fusion Network for Real-Time Multimodal Emotion Recognition in IoT Environments
    Sungwook Yoon, Byungmun Kim · 2025 · Sensors
71. TPFN: Applying Outer Product Along Time to Multimodal Sentiment Analysis Fusion on Incomplete Data
    Binghua Li, Chao Li, Feng Duan, Ning Zheng, Qibin Zhao · 2020 · Lecture notes in computer science
72. Deep Modular Co-Attention Shifting Network for Multimodal Sentiment Analysis
    Piao Shi, Min Qiang Hu, Xuefeng Shi, Fuji Ren · 2023 · ACM Transactions on Multimedia Computing Communications and Applications
73. Learning-Based Cancer Treatment Outcome Prognosis Using Multimodal Biomarkers
    Maliazurina Binti Saad, Shenghua He, Wade L. Thorstad, Hiram Alberto Gay, Daniel Barnett, Yujie Zhao, Su Ruan, Xiaowei Wang, Hua Li · 2021 · IEEE Transactions on Radiation and Plasma Medical Sciences
74. CLAP: Contrastive Language-Audio Pre-training Model for Multi-modal Sentiment Analysis
    Tingzhen Zhao, Ming Chai Kong, Tian Liang, Qiang Zhu, Kun Kuang, Fei Qing Wu · 2023 · International Conference on Multimedia Retrieval
75. AFR-BERT: Attention-based mechanism feature relevance fusion multimodal sentiment analysis model
    Ji Mingyu, Jiawei Zhou, Wei Ning · 2022 · PLoS ONE
76. A Multi-Modal Fusion Method Based on Higher-Order Orthogonal Iteration Decomposition
    Liu, Fen, Chen, Jianfeng, Tan, Weijie, Cai, Chang · 2021 · Entropy
77. Multimodal Sentiment Analysis of Social Media Data: A Review
    Priyavrat, Nonita Sharma, Geeta Sikka · 2021 · Lecture notes in electrical engineering
78. Visual Sentiment Analysis Using Machine Learning for Entertainment Applications
    Anu Sayal, N Chaithra, Janhvi Jha, Balu Trilochan, G. Kalyan, Madhavan Priya, Veethika Gupta, Minakshi Memoria, Ashulekha Gupta · 2023 · International Conference on IoT, Communication and Automation Technology (ICICAT)
79. Mixture of Attention Variants for Modal Fusion in Multi-Modal Sentiment Analysis
    Chao He, Xinghua Zhang, Dongqing Song, Yingshan Shen, Chengjie Mao, Huosheng Wen, Dingju Zhu, Lihua Cai · 2024 · Big Data and Cognitive Computing
80. Text-Centric Multimodal Contrastive Learning for Sentiment Analysis
    Heng Peng, Xue Fan Gu, Jian Li, Zhaodan Wang, Hao Xu · 2024 · Electronics
81. A Two-Stage Stacked Transformer Framework for Multimodal Sentiment Analysis
    Guofeng Yi, Cunhang Fan, Jianhua Tao, Zhao Lv, Zhengqi Wen, Guanxiong Pei, Taihao Li · 2024 · Intelligent Computing
82. Multimodal Sentiment Analysis using Deep Learning Fusion Techniques and Transformers
    Muhaimin Bin Habib, Md. Ferdous Bin Hafiz, Niaz Ashraf Khan, Sohrab Hossain · 2024 · International Journal of Advanced Computer Science and Applications
83. Survey of Multimodal Sentiment Analysis Based on Deep Learning
    Jing Zhang, Siyuan Xue, Xiaoyi Wang, Jie Liu · 2023 · IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS)
84. Looking Beyond a Clever Narrative
    Abhinav Shukla, Harish Katti, Mohan Kankanhalli, Ramanathan Subramanian · 2018 · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
85. Sentiment Analysis using Imperfect Views from Spoken Language and Acoustic Modalities
    Imran Sheikh, Sri Harsha Dumpala, Rupayan Chakraborty, Sunil Kumar Kopparapu · 2018
86. AB-GRU: An attention-based bidirectional GRU model for multimodal sentiment fusion and analysis
    Jun Wu, Xinli Zheng, Jiangpeng Wang, Junwei Wu, Ji Wang · 2023 · Mathematical Biosciences & Engineering
87. Text data augmentation and pre-trained Language Model for enhancing text classification of low-resource languages
    Atabay Ziyaden, Amir Yelenov, Fuad Hajiyev, Samir Rustamov, Alexandr Pak · 2024 · PeerJ Computer Science
88. Multi-Modal Sentiment Analysis Based on Interactive Attention Mechanism
    Jun Wu, Tianliang Zhu, Xinli Zheng, Chunzhi Wang · 2022 · Applied Sciences
89. Multi-Modality Empowered Network for Facial Action Unit Detection
    Peng Liu, Zheng Zhang, Huiyuan Yang, Lijun Yin · 2019 · IEEE Winter Conference on Applications of Computer Vision
90. Multi-Modal Language Analysis with Hierarchical Interaction-Level and Selection-Level Attentions
    Dong Zhang, Liangqing Wu, Shoushan Li, Qiaoming Zhu, Guodong Zhou · 2019 · IEEE International Conference on Multimedia and Expo (ICME)
91. Second Grand-Challenge and Workshop on Multimodal Language (Challenge-HML)
    2020
92. An Ensemble-Based Hotel Reviews System Using Naive Bayes Classifier
    Joseph Bamidele Awotunde, Sanjay Misra, Vikash Katta, Oluwafemi Charles Adebayo · 2023 · Computer Modeling in Engineering & Sciences
93. Multimodal Sentiment Analysis Using BiGRU and Attention-Based Hybrid Fusion Strategy
    Zhizhong Liu, Bin Zhou, Lingqiang Meng, Guangyu Huang · 2023 · Intelligent Automation & Soft Computing
94. Multifeature Interactive Fusion Model for Aspect‐Based Sentiment Analysis
    Biqing Zeng, Xuli Han, Feng Zeng, Ruyang Xu, Heng Yang · 2019 · Mathematical Problems in Engineering
95. An error analysis for image-based multi-modal neural machine translation
    Iacer Calixto, Qun Liu · 2019 · Machine Translation
96. Global-Shared Text Representation Based Multi-Stage Fusion Transformer Network for Multi-Modal Dense Video Captioning
    Yulai Xie, Jingjing Niu, Yang Zhang, Fang Ren · 2023 · IEEE Transactions on Multimedia
97. Multilevel information compression and textual information enhancement for multimodal sentiment analysis
    Yuchen Zhang, Hong Zhong, Naji Alhusaini, Guilin Chen, Cheng Wu · 2025 · Knowledge-Based Systems
98. Incorporating Multimodal Sentiments into Conversational Bots for Service Requirement Elicitation
    Demin Yu, Junrui Tian, Tonghua Su, Zhiying Tu, Xiaofei Xu, Zhongjie Wang · 2021 · IEEE International Symposium on Service-Oriented System Engineering (SOSE)
99. Towards Asynchronous Multimodal Signal Interaction and Fusion via Tailored Transformers
    Dingkang Yang, Haopeng Kuang, Kun Yang, Mingcheng Li, Lihua Zhang · 2024 · IEEE Signal Processing Letters
100. EmoP3D: A Brain Like Pyramidal Deep Neural Network for Emotion Recognition
     Emanuel Di Nardo, Alfredo Petrosino, Ihsan Ullah · 2019 · Lecture notes in computer science