Multimodal Emotion Recognition using Deep Learning
Open accessThe DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
Abstract
New research into human-computer interaction seeks to consider the consumer's emotional status to provide a seamless human-computer interface. This would make it possible for people to survive and be used in widespread fields, including education and medicine. Multiple techniques can be defined through human feelings, including expressions, facial images, physiological signs, and neuroimaging strategies. This paper presents a review of emotional recognition of multimodal signals using deep learning and comparing their applications based on current studies. Multimodal affective computing systems are studied alongside unimodal solutions as they offer higher accuracy of classification. Accuracy varies according to the number of emotions observed, features extracted, classification system and database consistency. Numerous theories on the methodology of emotional detection and recent emotional science address the following topics. This would encourage studies to understand better physiological signals of the current state of the science and its emotional awareness problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Various experimental studies have attested to the superior performance of MER models over unimodal ones, placing MER at the vanguard of research within affective computing [77,78,79,80].”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [80]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Zebari, D. A., Haron, H., Zeebaree, S. R. M., & Zeebaree, D. Q. (2019) · International Conference on Advanced Science and Engineering (ICOASE) · doi:10.1109/icoase.2019.8723779 · Cited by 46 · cited 1×
“To ensure meaningful communication, accurate interpretation of these emotions is important [5, 6].”
BackgroundSelf-citationRetractedEkman, P., & Friesen, W. V. (1975) · Cited by 4,758
Valence focus and arousal focus: Individual differences in the structure of affective experience.
Feldman, L. A. (1995) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.69.1.153 · Cited by 465
Automatic, Dimensional and Continuous Emotion Recognition
Gunes, H., & Pantic, M. (2010) · International Journal of Synthetic Emotions · doi:10.4018/jse.2010101605 · Cited by 426 · cited 1×
“However, emotional variations can be observed in physiological signals for a very short period of around 315seconds [73].”
BackgroundHigh-level feature representation using recurrent neural network for speech emotion recognition
Lee, J., & Tashev, I. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-336 · Cited by 388 · cited 1×
“[54] shows that using RNN bidirectional- (Bi-LSTM) is better for extracting essential speech characteristics for better speech”
MethodFacial emotion recognition using deep learning: review and insights
Mellouk, W., & Handouzi, W. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.07.101 · Cited by 339 · cited 1×
“Mehrabian observed that 7% of knowledge moves between people through writing, 38% through voice, and 55% through facial expression [36].”
BackgroundMulti-Sequence LSTM-RNN Deep Learning and Metaheuristics for Electric Load Forecasting
Bouktif, S., Fiaz, A., Ouni, A., & Serhani, M. A. (2020) · Energies · doi:10.3390/en13020391 · Cited by 256 · cited 2×
“In [60], to boost the efficiency of the multimodal fusion, a new algorithm and new architecture using the kernel matrix and a deep neural network are proposed.”
BackgroundMethodZhang, X., Liu, J., Shen, J., Li, S., Hou, K., Hu, B., Gao, J., Zhang, T., & Hu, B. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2987575 · Cited by 206 · cited 1×
“] [69] proposed a revolutionary multimodal fusion system with regularization focused on a new kernel matrix perspective and a deep network architecture.”
BackgroundMohan, K., Seal, A., Krejcar, O., & Yazidi, A. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3031835 · Cited by 193 · cited 9×
“To efficiently map the subject's emotions with the extracted features from both ECG and PPG signals, a convolution neural network (CNN) was developed.”
BackgroundInfluentialLEARNet: Dynamic Imaging Network for Micro Expression Recognition
Verma, M., Vipparthi, S. K., Singh, G., & Murala, S. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2912358 · Cited by 174 · cited 9×
“Results show that the SVM algorithm has modified a lot to speech emotion recognition with the accuracy of 72.52 % using Berlin Emotional DB.”
BackgroundMethodInfluential
Research works printed but not linked: 5
- [4]4. O. F. Mohammad, M. S. M. Rahim, S. R. M. Zeebaree, and F. Y. Ahmed, "A survey and analysis of the image encryption methods," International Journal of Applied Engineering Research, vol. 12, pp. 13265-13280, 2017.
- [14]14. H. Dino, M. B. Abdulrazzaq, S. R. Zeebaree, A. B. Sallow, R. R. Zebari, H. M. Shukur, et al., "Facial Expression Recognition based on Hybrid Feature Extraction Techniques with Different Classifiers," TEST Engineering & Management, vol. 83, pp. 22319-22329, 2020.
- [15]15. S. S. R. Zeebaree, S. Ameen, and M. Sadeeq, "Social Media Networks Security Threats, Risks and Recommendation: A Case Study in the Kurdistan Region," International Journal of Innovation, Creativity and Change, vol. 13, pp. 349-365, 2020.
- [28]28. S. Y. Ameen, F. M. Almusailkh, and M. H. Al-Jammas, "FPGA Implementation of Neural Networks Based Symmetric Cryptosystem," in 6th International Conference: Sciences of Electronic, Technologies of Information and Telecommunications May, 2011, pp. 12-15.
- [59]59. F. Al-Naima, S. Y. Ameen, and A. F. Al-Saad, "Destroying steganography content in image files," in IEEE Proceedings of Fifth International Symposium on Communication Systems, Networks and Digital Signal Processing, University of Patras, Patras, Greece, 2006.
Cited by
Items 201–300 of 348
Yang, J., Wang, L., Qi, Y., Chen, H., & Li, J. (2024) · Applied Sciences · doi:10.3390/app14199121 · Cited by 2
Self-Supervised Learning-Based Multimodal Prediction on Prosocial Behavior Intentions
Naini, A. R., Zheng, Z., Misu, T., & Akash, K. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890650 · Cited by 2
“Such innovations have applications in intelligent vehicles [1], digital health [2], and emotion recognition [3].”
Using CNN and Transformer Model for Unimodal Speech Emotion Recognition on MELD and IEMOCAP
Aurelio, C., & Chowanda, A. (2025) · International Conference on Advancement in Data Science, E-learning and Information System (ICADEIS) · doi:10.1109/icadeis65852.2025.10933408 · Cited by 2
“The difficulty in researching emotion recognition is usually correlated to the dataset, as the results will mostly be overfit thus that will be one of the difficulties upon doing this research [7], [18].”
Alzubaidi, U. (2025) · Scientific Reports · doi:10.1038/s41598-025-11775-4 · Cited by 2
Self-Monitoring Deep Network Based on Multimodal Attention Fusion for Mental Emotion Recognition
Xu, G., Jiang, H., Hou, H., & Liu, G. (2025) · Journal of Cases on Information Technology · doi:10.4018/jcit.393784 · Cited by 4
A Comprehensive Study of Caching Effects on Fog Computing Performance
Abdulqadir, M. M., Salih, A. A., Ahmed, O. M., Hasan, D. A., Haji, L. M., Ahmed, S. H., Ibrahim, R. K., Sharif, K. H., & Dino, H. I. (2021) · Asian Journal of Research in Computer Science · doi:10.9734/ajrcos/2021/v10i430246 · Cited by 2
State of Art Survey for Fault Tolerance Feasibility in Distributed Systems
Hussein, A. A., Al-Zebari, A., Omar, N., Merceedi, K. J., Ahmed, A. J., Salim, N. O. M., Hasan, S. S., Kak, S. F., Ibrahim, I. M., Yasin, H. M., et al. (2021) · Asian Journal of Research in Computer Science · doi:10.9734/ajrcos/2021/v11i430268 · Cited by 2
Kumar, A., Lourens, M., Tiwari, N., Dass, P., Suresh Kumar, M., V, & Abdullah, K. H. (2022) · International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE) · doi:10.1109/icacite53722.2022.9823706 · Cited by 2
Liu, Y., & Ijaz, M. F. (2022) · Mobile Networks and Applications · doi:10.1007/s11036-022-02076-5 · Cited by 2
“In this regard, the traditional personalized auxiliary information presentation service system based on scene sensitivity uses location-based technology to provide network services for mobile information, and assists users to find appropriate information service content through friends or community recommendations [5,…”
BackgroundCorrelation Modeling of Moral Emotion based on Facial Image Emotion Recognition Algorithm
Li, X., & Lian, R. (2022) · International Conference on Augmented Intelligence and Sustainable Systems (ICAISS) · doi:10.1109/icaiss55157.2022.10011045 · Cited by 2
“For example, in the field of computer vision, it is proposed in recent years to extract the characteristics of color and light in the picture and predict the aesthetic emotion classification of painting works of art through linear classifier or regresso [4, 5, 6, 7].”
BackgroundApplying a Deep Learning Technique for Speech Recognition in Robotics
Jellali, A., Fredj, I. B., Zouhir, Y., & Ouni, K. (2022) · IEEE Information Technologies & Smart Industrial Systems (ITSIS) · doi:10.1109/itsis56166.2022.10118430 · Cited by 2
“Deep Learning has many uses, such as image recognition and classification [6] [7] [8]. language modeling [9] [10], natural language processing (NLP) [11] [12] and automatic speech recognition [13].”
BackgroundEmoLens: Pupil Diameter-based Emotion Classification using CNN and RF Algorithms
Rishu, Kukreja, V., & Sharma, V. (2023) · International Conference on Augmented Intelligence and Sustainable Systems (ICAISS) · doi:10.1109/icaiss58487.2023.10250592 · Cited by 2
“Numerous studies on emotions have been undertaken to utilize a mix of several physiological markers [4].”
BackgroundValidation of FEA-based breast deformation simulation using an artificial neural network
Wang, K., & Kesavadas, T. (2022) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2022.101044 · Cited by 1
Emotional Detection Based on Bio Information and Data Analytics
Alanya-Beltrán, J., Gehlot, A., Gurav, U., Valderrama-Zapata, C., Anandaram, H., & Jain, A. (2022) · International Conference on Innovative Computing, Intelligent Communication and Smart Electrical Systems (ICSES) · doi:10.1109/icses55317.2022.9914197 · Cited by 2
“According to Abdullah and colleagues' research, several computations, including as CNN, DCCN, and LSTM, may be used to analyse the articulated shift and significant modifications [7].”
MethodDhakal, I., Sah, N. K., Ling, K. W., Rauniyar, S. K., Kumar, G., & Mahesh, T. R. (2023) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt56998.2023.10308063 · Cited by 1
“The class with the greatest likelihood is chosen as the predicted class [14].”
BackgroundWang, Q., Fan, C., Jia, T., Han, Y., & Wu, X. (2023) · International Conference on Neuromorphic Computing (ICNC) · doi:10.1109/icnc59488.2023.10462792 · Cited by 1
“It also has made significant advancements in several applications, including multisensory emotion recognition [3]– [5], medical diagnosis [6]–[8], and intelligent robotics [9], [28], [30].”
BackgroundEmotion Recognition by Facial Expressions and Speech Using Deep Learning
Vardhan, J. V., Yelavarti, K. C., & Chand, A. J. (2024) · International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT) · doi:10.1109/icaect60202.2024.10469626 · Cited by 3
“LSTM (Long Short-Term Memory) [2] stands out as a highly successful deep learning architecture for tackling sequence modeling tasks.”
BackgroundMultimodal Approach to Emotion Recognition using Deep Learning
Ajay, K., Naseeba, B., Challa, N. P., & Karthick, A. A. (2023) · International Conference on Innovative Mechanisms for Industry Applications · doi:10.1109/icimia60377.2023.10426592 · Cited by 1
MEDA-CBLSTM: Data Acquisition, Processing and Emotion Recognition Based on Multi-Layer EDA
He, Y., Chen, L., Zhao, Q., Hong, Z., & Chen, Y. (2023) · International Conference on Artificial Intelligence, Human-Computer Interaction and Robotics (AIHCIR) · doi:10.1109/aihcir61661.2023.00066 · Cited by 1
Real-Time Facial Emotion Analysis for Adaptive Teaching Strategies Using Deep Learning
Suganthi, V., & Yogeshwari, M. (2024) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-1355-8.ch009 · Cited by 1
Multimodal optimal matching and augmentation method for small sample gesture recognition
Zhang, W., Liu, B., Zhao, T., & Qie, S. (2025) · BioScience Trends · doi:10.5582/bst.2024.01370 · Cited by 1
Hybrid Data Fusion for Elegant Emotion Recognition in Cloud-Based Big Data Systems
Jin, H., Sun, X., Jin, Z., & Kim, Y.-G. (2025) · International Conference on Multi-Agent Systems for Collaborative Intelligence (ICMSCI) · doi:10.1109/icmsci62561.2025.10894446 · Cited by 1
“Abdullah et al. [3] comprehensively introduced multi-modal emotion recognition technology in the form of a review, and focused on the emotion recognition method based on deep learning.”
Facial Emotion Detection and Music Recommendation using Deep Learning
Bagadi, G. (2025) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2025.60727 · Cited by 1
Shah, A., & Jain, R. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-00447-5_12 · Cited by 1
Pradhan, A., & Srivastava, S. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108541 · Cited by 1
Multimodal data analysis in emotion recognition: a review
Berdyshev, D. A., & Shishkin, A. G. (2025) · Modeling and Analysis of Information Systems · doi:10.18255/1818-1015-2025-3-252-281 · Cited by 1
Edge-Driven Real-Time UAV Thermal Imaging for Survivor Identification
Fayaz, F. A., Nasti, S. M., Iqbal, U., & Dar, A. H. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-6715-4_36 · Cited by 1
Feature fusion based transformer for sentiment analysis in social networks
Li, S., Li, H., Du, J., Yan, S., & Dong, C. (2025) · PLoS ONE · doi:10.1371/journal.pone.0333416 · Cited by 3
DPMMN: A dual performer-multi-modal network for emotion recognition
Gornale, S. S., Shivakumara, P., Unki, A., & Vadera, S. (2025) · Signal Processing Image Communication · doi:10.1016/j.image.2025.117464 · Cited by 1
A Virtual Music-Therapist Dialogue System Based on Natural Language Processing
Zhao, Y., Meng, N., Yuan, S., Mu, P., Zhao, L., & Yu, Z. (2025) · International Conference on Artificial Intelligence, Virtual Reality and Interaction Design · doi:10.1145/3777730.3777763 · Cited by 1
“…existing research: first, multimodal emotion recognition still has bottlenecks in feature alignment, weight distribution and real-time performance [5] [6]; Second, dialogue management lacks long-term emotional tracking and dynamic strategy adjustment [7] [8]; Third, the emotion driven…”
InfluentialSpatially Enhanced Pyramid Split attention for improved ECG-Based emotion recognition
Pan, C., Chen, H., Zhang, X., Su, T., & Ma, P. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.109729 · Cited by 1
RERS: A real‑time emotion‑aware recommender system for sustainable and eco‑friendly driving
Yıldız, E. R., & Bitirim, Y. (2026) · Journal of Intelligent Information Systems · doi:10.1007/s10844-026-01033-y · Cited by 1
“Moreover, in some studies aiming to increase emotion recognition accuracy, a fusion of physical and physiological factors, such as facial expression and EEG were utilized (Abdullah et al., 2021; Tan et al., 2021).”
CMSA: Addressing semantic discrepancy and context dependency in multimodal sentiment analysis
Zhao, J., Deng, O., & Jin, Q. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.133622 · Cited by 1
From Answer Engines to Learning Partners: A Dual-ZPD Design Framework for AI-Supported Learning
Klein, R., Benden, D. K., Schier, A., Stotko, D., & Lauermann, F. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791476 · Cited by 5
Guidelines for Emotion Recognition in Robot-Supported Interventions in Autism
Wróbel, M. R., Bartl-Pokorny, K. D., Zorčec, T., Barkana, D. E., Köse, H., Milling, M., Robins, B., Schuller, B. W., & Landowska, A. (2026) · International Journal of Social Robotics · doi:10.1007/s12369-026-01401-2 · Cited by 1
“More-over, multimodal emotion recognition systems offer higher classification accuracy than single-modality-based solutions [63].”
Huang, G., Matera, M., & Spitale, M. (2026) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg67764.2026.11556952 · Cited by 1
“Studies in emotion recognition [1] have demonstrated that combining multiple modalities leads to more accurate and resilient models, as each modality offers complementary information about the speaker’s emotional state [13].”
Retracted: Facial Recognition based on Deep Learning
Aanchal, Nijhawan, R., & Goel, S. (2021) · International Conference on Electronics, Communication and Aerospace Technology (ICECA) · doi:10.1109/iceca52323.2021.9676010 · Cited by 1
RetractedEmotion-Based Music Movie and TV Series Recommendation System Using Deep Learning Algorithm
Deepa, D., Vijayalakshmi, D. M., Shanmathi, B., Evelin, A. S., & Elakkiya, K. T. (2023) · Applied and Computational Engineering · doi:10.54254/2755-2721/2/20220656 · Cited by 1
“Researchers proposed work on multimodal emotion identification from voice and expression was published [8].”
BackgroundMultimodal Audio Emotion Recognition with Graph-based Consensus Pseudolabeling
Coutinho, G. N., Lima, A. D. V., Yugoshi, J., de Moraes, M. I., Gôlo, M. P. S., & Marcacini, R. M. (2023) · doi:10.5753/eniac.2023.234481 · Cited by 2
“However, recent advancements in the field have emphasized integrating multimodal information importance, particularly the textual modality derived from transcribed audio [Abdullah et al. 2021, Ezzameli and Mahersia 2023].”
BackgroundZou, Y., Fu, D., Mo, H., Chen, H., & Wang, D. (2024) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-237868 · Cited by 1
“The data sets used in the experiment are commonly used in the field of object recognitionanddetection,includingObjects365,Pas-calVOC,ImageNetAllandCOCOdatasets[27].”
RetractedFacial Expression Embrace: A CNN-SVM Approach for Multi-Class Emotional Classification
Kaur, A., Kukreja, V., Thapliyal, N., Manwal, M., & Sharma, R. (2024) · International Conference for Innovation in Technology (INOCON) · doi:10.1109/inocon60754.2024.10511549 · Cited by 1
Unlocking Human Emotions: The Power of Deep Learning in Sentiment Analysis
Waykar, Y. A., & Yambal, S. S. (2024) · Advances in psychology, mental health, and behavioral studies (APMHBS) book series · doi:10.4018/979-8-3693-4143-8.ch008 · Cited by 1
Boyko, N. I. (2024) · Journal of Ecohumanism · doi:10.62754/joe.v3i5.3877
Deep Learning-Based Emotion Recognition by Fusion of Facial Expressions and Speech Features
Vardhan, J. V., Yelavarti, K. C., & Chand, A. J. (2024) · World Conference on Communication & Computing (WCONF) · doi:10.1109/wconf61366.2024.10692119 · Cited by 1
“S. M. A. Abdullah et al. [7] This paper provides an overview of emotional recognition using deep learning across multimodal signals, comparing their applications through current research.”
BackgroundPolish Speech and Text Emotion Recognition in a Multimodal Emotion Analysis System
Skowroński, K., Gałuszka, A., & Probierz, E. (2024) · Applied Sciences · doi:10.3390/app142210284 · Cited by 5
FP-KDNet: Facial Perception and Knowledge Distillation Network for Emotion Recogniton in Coversation
Cai, C., Lin, X., Zhang, J., Bhattarai, A., Cai, C., Xia, X., & Pan, Z. (2024) · ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry (VRCAI) · doi:10.1145/3703619.3706033 · Cited by 1
“To reduce the impact of environmental noise on emotion recognition, deep learning methods have demonstrated strong capabilities in extracting complex features from videos, accurately identifying subtle emotional changes [Abdullah et al. 2021; Hazmoune and Bougamouza 2024].”
Machine Recognition and Subconscious Perception Characteristics of Musical Emotions
Xu, D., Fan, A., Liao, X., Yu, G., & Dai, W. (2024) · China Automation Congress (CAC) · doi:10.1109/cac63892.2024.10865611 · Cited by 1
UMEDNet: a multimodal approach for emotion detection in the Urdu language
Majeed, A. A., & Mujtaba, H. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.2861 · Cited by 2
“Emotions are an intrinsic part of human nature and part of our daily life ( Baltru š aitis, Ahuja & Morency, 2018 ; Abdullah et al., 2021 ).”
Tiwari, A., Shukla, P., Snehlata, S., Sharma, K., & Dubey, A. K. (2024) · International Conference on Control, Computing, Communication and Materials (ICCCCM) · doi:10.1109/iccccm61016.2024.11039925 · Cited by 1
“Authors [20] discussed the application of Convolutional Neural Networks (CNNs) for detecting complex emotions in social media texts, demonstrating significant improvements in classification accuracy.”
Intelligent Gaming Input Device for Tilt Recognition
Shazhaev, I., Mikhaylov, D., Shafeeg, A., & Mulyarchik, E. (2022) · Journal of Intelligent Learning Systems and Applications · doi:10.4236/jilsa.2022.144008 · Cited by 1
Can One Hear the Position of Nodes?
Puzis, R. (2023) · Studies in computational intelligence · doi:10.1007/978-3-031-21131-7_50 · Cited by 1
A Survey on Multi-modal Emotion Detection TechniquesPreprint
Chatterjee, C., Shah, N., Bhatt, S., & Chandi, S. (2024) · Research Square · doi:10.21203/rs.3.rs-3942784/v1 · Cited by 1
Facial and speech Emotional Recognition based on Improved Deep ModelPreprint
Shahrestani, M. R., Motamed, S., & Yamaghani, M. (2024) · Research Square · doi:10.21203/rs.3.rs-3996203/v1 · Cited by 1
Stress Prediction and Detection Using IoT and Deep Learning: A Comprehensive Review
Malik, K. S. (2021) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2021.38291
“Large numbers of them utilize huge information methods, semantic standards and profound learning [29].”
BackgroundTorcate, A. S., de Santana, M. A., Gomes, J. C., Nunes, I. B., Fonseca, F. S., Moreno, G. M. M., & dos Santos, W. P. (2022) · Research Square · doi:10.21203/rs.3.rs-1363614/v1
Enhanced neural network scheme for recognising criminals using face detection
Ramesh, T., & Shalini, R. (2022) · International Journal of Health Sciences · doi:10.53730/ijhs.v6ns4.9056
Deep Learning Systems Integrated into the Digital Strategy of a Company Involved in e-commerce
Rusu, R., & Avram, C. (2022) · Annals of Dunarea de Jos University of Galati Fascicle I Economics and Applied Informatics · doi:10.35219/eai15840409238
Bangla Song Suggestion Using Face Detection
Hamim, M. A., Tahseen, J., Hossain, K. M. I., & Islam, M. S. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-19958-5_109
Yanyuan, 苏. S., Guangyu, 范. F., Haimei, 龚. G., Xue, 李. L., & Yongping, 陈. C. (2022) · Infrared and Laser Engineering · doi:10.3788/irla20220078
Content recommendation based on recognised Emotion
Challa, S. S. R. K., Gowri, M. V., Roobini, M. S., Nandhini, P., & Clemency, C. A. D. D. (2023) · International Conference on Recent Advances in Electrical, Electronics, Ubiquitous Communication, and Computational Intelligence (RAEEUCCI) · doi:10.1109/raeeucci57140.2023.10134505
“CF approaches [10-12], thanks to the enormous success of deep learning in Computer Vision (CV) and (NLP).”
BackgroundImproving information fusion in deep learning
Don, D. P. M. K. (2023) · Queensland University of Technology · doi:10.5204/thesis.eprints.241291
Music Recommendation System Based on Facial Emotion
Sreedhar, M. B., Shahida, S., Tanmai, P., Nithin, M., & Neelima, G. S. (2023) · Journal of Humanities Music and Dance · doi:10.55529/jhmd.33.11.22
Evidence-Based Research on Multimodal Fusion Emotion Recognition
Huang, Z., & Liao, M. (2023) · Atlantis Highlights in Intelligent Systems/Atlantis highlights in intelligent systems · doi:10.2991/978-94-6463-200-2_61 · Cited by 1
Centrality Learning: Auralization and Route Fitting
Li, X., Bachar, L., & Puzis, R. (2023) · Entropy · doi:10.3390/e25081115
“Many deep convolutional neural networks have been developed in recent years to recognize speech [31], emotions [32], background sounds [33], etc.”
BackgroundDeep Learning Emotion Recognition Method
Xiao, W., Tan, W., Xiong, N. N., Yang, C., Chen, L., & Xie, R. (2023) · IEEE International Conference on Cyber Security and Cloud Computing / IEEE International Conference on Edge Computing and Scalable Cloud (CSCloud/EdgeCom) · doi:10.1109/cscloud-edgecom58631.2023.00067 · Cited by 1
Psychophysiology in the digital age
van der Mee, D. J. (2023) · doi:10.5463/thesis.258
Rahman, S. M. A., Armi, N., Abdelhag, M. E., Amin, S. T., & Eishah, H. A. (2023) · International Conference on Radar, Antenna, Microwave, Electronics, and Telecommunications (ICRAMET) · doi:10.1109/icramet60171.2023.10366566 · Cited by 3
“There is no requirement to combine the feature maps [31].”
BackgroundFace Emotion Recognition (FER) Using Convolutional Neural Network (CNN) in Machine Learning
Jain, L., & Kavita, K. (2024) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2024.58077
Multimodal Emotion Recognition System Using Machine Learning Classifier
Prabha, R. (2023) · Journal Of Advanced Zoology · doi:10.53555/jaz.v44is8.3499
Su, Y. (2023) · International Conference on Integrated Intelligence and Communication Systems (ICIICS) · doi:10.1109/iciics59993.2023.10421121
“This automatic feature extraction method has high accuracy and can improve recognition accuracy, completing end-to-end learning, that is, the entire process from the original image data to the output results does not require manual design of features and classifiers [17-18].”
MethodExploration on multimodal data recognition method for Internet of Things based on deep learning
Zheng, X., & Sun, Z. (2024) · Intelligent Decision Technologies · doi:10.3233/idt-230267
“Sharmeen M Saleem Abdullah used DL for emotion recognition of multimodal signals and compared their current research based on DL applications [3].”
Multimodality in Misinformation Detection
Ekbal, A., & Kumari, R. (2024) · The information retrieval series · doi:10.1007/978-3-031-54834-5_5
Zhang, T., Dahlan, H. A., Xie, Z., Jin-feng, W., Chen, Y., Pan, Q., & Huang, Y. (2024) · Applied Mathematics and Nonlinear Sciences · doi:10.2478/amns-2024-2156 · Cited by 3
Decoding Emotions: Advancements in CNN-SVM Hybrid Models for Affective Computing
Garg, O., Kukreja, V., Chattopadhyay, S., Verma, A., & Sharma, R. (2024) · International Conference for Emerging Technology (INCET) · doi:10.1109/incet61516.2024.10593102
“Ekman and his colleagues [2] were the first to identify six essential emotions: pleasure, sadness, surprise, fear, and fury.”
BackgroundEmotion Unveiled: Enhancing Facial Expression Classification Through CNN-SVM Fusion
Dhawan, N., Kukreja, V., Chattopadhyay, S., Verma, A., & Sharma, R. (2024) · International Conference for Emerging Technology (INCET) · doi:10.1109/incet61516.2024.10593392
“Emotions represent a scale from the highest levels of happiness to the lowest levels of desolation and serve as a prism with the help of which all the diversity of the world becomes clearer to us [1].”
BackgroundChinese Speaking Emotion Recognition and Emotion Synthesis based on BERT Model
Chen, Q. (2024) · International Conference on Intelligent Algorithms for Computational Intelligence Systems (IACIS) · doi:10.1109/iacis61494.2024.10721813
“By fusing text, audio and video data, deep neural network was used for emotion recognition [14].”
MethodAdvancements, Methods, and Obstacles in Emotion Recognition with Multimodal Approaches
Arunkumar, M., Athiban, R., Selva Jothi, M., & Pavithra, S. (2025) · International Conference on Intelligent Data Communication Technologies and Internet of Things (IDCIoT) · doi:10.1109/idciot64235.2025.10914926
“Vision Transformers (ViTs), a specialized adaptation, analyze images through patch-based processing [5].”
InfluentialGender-Based Bimodal Analysis of Speech and Text in Non-Contextual Emotion Recognition
Angkasa, A. R., Zahra, A., & Fung, W.-K. (2024) · Beyond Technology Summit on Informatics International Conference (BTS-I2C) · doi:10.1109/bts-i2c63534.2024.10942075
“Due to the distinct computational methods applied to each modality, from feature extraction through classification, the processing of each modality yields different output predictions[4], [5].”
Analysis of intelligent movie recommender system using facial expression
Suganya, A., Shanmugaiah, M., Chandrasekar, S., Xavier, S. W. R., & Chidambaram, U. (2025) · AIP conference proceedings · doi:10.1063/5.0263226
Advancing Pain Recognition Through Statistical Correlation-Driven Multimodal Fusion
Gu, X., Wang, Z., Jin, I., & Wu, Z. (2024) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw63320.2024.00056 · Cited by 8
“Its application in understanding human pain behavior is particularly promising, as it acknowledges pain as a complex emotional state rather than merely a physical condition [2] [3].”
MODALITY-SPECIFIC DOMAIN ADAPTATION FOR MULTIMODAL EMOTION DETECTION
Harutyunyan, E. A. (2024) · P R O C E E D I N G S OF NATIONAL POLYTECHNIC UNIVERSITY OF ARMENIA INFORMATION TECHNOLOGIES ELECTRONICS RADIO ENGINEERING · doi:10.53297/18293336-2024.2-57
Parmar, A. (2025) · INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · doi:10.55041/ijsrem47033 · Cited by 1
In-Vehicle Emotion Recognition and Regulation System Based on Manual Feature ExtractionPreprint
Zhong, Z., Yang, Y., Chen, Y., Chen, M., & Zhang, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5257710
Mental Health Companion for Trauma Recovery Using Multi-Modal Emotion Recognition
Balasubramanian, S., Sridhar, K., & Nandagopalan, K. (2025) · International Conference on Data Science, Agents & Artificial Intelligence (ICDSAAI) · doi:10.1109/icdsaai65575.2025.11011760
Multimodal Emotion Recognition Using Visual and Thermal Image Fusion: A Deep Learning Approach
Mahmud, I., Das, P. R., Rifa, N., Hossain, I., Rahman, R., & Farid, D. M. (2024) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit64611.2024.11022356
“Recent research has shown that using multimodal approaches, such as combining visual and thermal data, can significantly improve the accuracy of emotion recognition compared to traditional single-modal systems [1].”
Intelligent Multimodal Music Style Classification Method for Vocal Stage
Guan, P. (2024) · Asian Conference on Artificial Intelligence Technology (ACAIT) · doi:10.1109/acait63902.2024.11022110
Fusion-based Model for Social Gathering Emotion Classification
Aonty, S. S., Akter, A., & Arefin, M. S. (2024) · International Conference on Electrical and Computer Engineering (ICECE) · doi:10.1109/icece64886.2024.11025091
“Over the years, emotion detection tasks have evolved from traditional machine learning models such as SVM, k-NN, Naive Bayes, Logistic Regression, Random Forest, and Multilayer Perceptron (MLP) [2] to more advanced deep learning-based multimodal systems [3].”
Game Analysis Using Multimodal Emotion Recognition
Kulkarni, R. G., Rajeevalochana, S., Vishwanatha, A., Sharanya, S., & Narayan, S. (2025) · International Conference on Computer and Communication Systems (ICCCS) · doi:10.1109/icccs65393.2025.11069702
Real Time Emotion Recognition using ML
Joshi, M., Mathur, B., & Nehra, V. (2025) · International Conference on Self Sustainable Artificial Intelligence Systems (ICSSAS) · doi:10.1109/icssas66150.2025.11080884
“…SFS optimization, multi-dataset evaluation [11] Real-time CNN+BiLSTM, data augmentation [12] End-to-end deep learning SER review [13] Multimodal model-level fusion, high accuracy [14] Multimodal SER review using deep learning [15] Text and speech emotion recognition, transformer-based models”
Image emotion classification technology based on polarity-aware attention mechanism
Hao, J., & Su, J. (2025) · Journal of Computational Methods in Sciences and Engineering · doi:10.1177/14727978251361546
Emotion Recognition using Voice with Machine Learning Model - Android App
Saranya, G., Sanjay, A., Vimal Kumar, S., & Daniel Jebin, J. (2025) · International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV) · doi:10.1109/icicv64824.2025.11085573
Human Emotion Detection using Speech Recognition in Real Time
Singh, P., Khan, A. A., & Tiwari, A. K. (2025) · Asset analytics · doi:10.1007/978-981-96-7556-2_4
Audio Emotion Detection Application Utilizing AWS Cloud
Bello, R.-W., Owolawi, P. A., Wyk, E. V., & Tu, C. (2025) · Recent Advances in Computer Science and Communications · doi:10.2174/0126662558384328250622220909 · Cited by 1
Shen, Z., Li, X., Bai, J., Wang, K., & Xu, Y. (2025) · IET Biometrics · doi:10.1049/bme2/7626919 · Cited by 1
“This situation has resulted in the emergence of the method of multimodal fusion [18, 19], which aims to improve the overall ef fi ciency and adaptability of recognition systems by integrating complementary information from different modalities.”
Xu, C., Zhang, Y., Li, H., & Mao, Z. (2025) · Cyberpsychology Behavior and Social Networking · doi:10.1177/21522715251371092 · Cited by 4
Deep Learning-Based Multiclass Image Classifier for Emotion Identification
Aworinde, H. O., Olufemi, L. O., & Adeniyi, A. E. (2025) · Conference on Information Communications Technology and Society (ICTAS) · doi:10.1109/ictas64866.2025.11155848
“Abdullah et al. [7] proposed multimodal deep learning using audio and video inputs to improve accuracy for emotion recognition.”
Liu, S., & Hao, W. (2025) · International Conference on Education, Knowledge and Information Management (ICEKIM) · doi:10.1145/3756580.3756672
“On the teacher side, assessment of teaching outcomes and optimisation of strategies [8] are key teachers’ requirements.”
Ասրիյան, Է., Gevorgyan, R., Sargsyan, H., Makaryan, A., & Aghasyan, S. (2025) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2025.102014
Tong, J., Li, S., Zhuang, Z., Hu, J., & Zou, Y. (2025) · International Workshop on Multimodal and Responsible Affective Computing (MRAC) · doi:10.1145/3746270.3760231
Shi, Y., Kong, F., & Zhang, L. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755272 · Cited by 3
“Currently, numerous studies have separately investigated multi-modal sentiment analysis [1] and intent recognition [11].”
Metadata
- Authors
- Abdullah, S. M. S. A., Ameen, S. Y. A., M. Sadeeq, M. A., & Zeebaree, S.
- Affiliations
- Duhok Polytechnic University (IQ)
- Venue
- Journal of Applied Science and Technology Trends
- Published
- 2021-05-30
- Volume
- 2
- Issue
- 01
- Pages
- 73-79
- ISSN
- 2708-0757 (ISSN-L)
- OpenAlex
- W3156576211
- Cited by
- 347
- References
- 72
- Citations by year
- 2026: 40 · 2025: 91 · 2024: 70 · 2023: 66 · 2022: 46 · 2021: 34
- FWCI
- 52.20
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Journal of Applied Science and Technology Trends (publishedVersion, cc-by-nc-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Keywords
- multimodal emotion recognition, affective computing, human-computer interaction, facial expression recognition, physiological signals, neuroimaging, deep learning, emotion classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.