Multimodal Emotion Recognition using Deep Learning
Open accessThe DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index.
Abstract
New research into human-computer interaction seeks to consider the consumer's emotional status to provide a seamless human-computer interface. This would make it possible for people to survive and be used in widespread fields, including education and medicine. Multiple techniques can be defined through human feelings, including expressions, facial images, physiological signs, and neuroimaging strategies. This paper presents a review of emotional recognition of multimodal signals using deep learning and comparing their applications based on current studies. Multimodal affective computing systems are studied alongside unimodal solutions as they offer higher accuracy of classification. Accuracy varies according to the number of emotions observed, features extracted, classification system and database consistency. Numerous theories on the methodology of emotional detection and recent emotional science address the following topics. This would encourage studies to understand better physiological signals of the current state of the science and its emotional awareness problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Various experimental studies have attested to the superior performance of MER models over unimodal ones, placing MER at the vanguard of research within affective computing [77,78,79,80].”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [80]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Zebari, D. A., Haron, H., Zeebaree, S. R. M., & Zeebaree, D. Q. (2019) · International Conference on Advanced Science and Engineering (ICOASE) · doi:10.1109/icoase.2019.8723779 · Cited by 46 · cited 1×
“To ensure meaningful communication, accurate interpretation of these emotions is important [5, 6].”
BackgroundSelf-citationRetractedEkman, P., & Friesen, W. V. (1975) · Cited by 4,758
Valence focus and arousal focus: Individual differences in the structure of affective experience.
Feldman, L. A. (1995) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.69.1.153 · Cited by 465
Automatic, Dimensional and Continuous Emotion Recognition
Gunes, H., & Pantic, M. (2010) · International Journal of Synthetic Emotions · doi:10.4018/jse.2010101605 · Cited by 426 · cited 1×
“However, emotional variations can be observed in physiological signals for a very short period of around 315seconds [73].”
BackgroundHigh-level feature representation using recurrent neural network for speech emotion recognition
Lee, J., & Tashev, I. (2015) · Interspeech 2015 · doi:10.21437/interspeech.2015-336 · Cited by 388 · cited 1×
“[54] shows that using RNN bidirectional- (Bi-LSTM) is better for extracting essential speech characteristics for better speech”
MethodFacial emotion recognition using deep learning: review and insights
Mellouk, W., & Handouzi, W. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.07.101 · Cited by 339 · cited 1×
“Mehrabian observed that 7% of knowledge moves between people through writing, 38% through voice, and 55% through facial expression [36].”
BackgroundMulti-Sequence LSTM-RNN Deep Learning and Metaheuristics for Electric Load Forecasting
Bouktif, S., Fiaz, A., Ouni, A., & Serhani, M. A. (2020) · Energies · doi:10.3390/en13020391 · Cited by 256 · cited 2×
“In [60], to boost the efficiency of the multimodal fusion, a new algorithm and new architecture using the kernel matrix and a deep neural network are proposed.”
BackgroundMethodZhang, X., Liu, J., Shen, J., Li, S., Hou, K., Hu, B., Gao, J., Zhang, T., & Hu, B. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2987575 · Cited by 206 · cited 1×
“] [69] proposed a revolutionary multimodal fusion system with regularization focused on a new kernel matrix perspective and a deep network architecture.”
BackgroundMohan, K., Seal, A., Krejcar, O., & Yazidi, A. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3031835 · Cited by 193 · cited 9×
“To efficiently map the subject's emotions with the extracted features from both ECG and PPG signals, a convolution neural network (CNN) was developed.”
BackgroundInfluentialLEARNet: Dynamic Imaging Network for Micro Expression Recognition
Verma, M., Vipparthi, S. K., Singh, G., & Murala, S. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2912358 · Cited by 174 · cited 9×
“Results show that the SVM algorithm has modified a lot to speech emotion recognition with the accuracy of 72.52 % using Berlin Emotional DB.”
BackgroundMethodInfluential
Research works printed but not linked: 5
- [4]4. O. F. Mohammad, M. S. M. Rahim, S. R. M. Zeebaree, and F. Y. Ahmed, "A survey and analysis of the image encryption methods," International Journal of Applied Engineering Research, vol. 12, pp. 13265-13280, 2017.
- [14]14. H. Dino, M. B. Abdulrazzaq, S. R. Zeebaree, A. B. Sallow, R. R. Zebari, H. M. Shukur, et al., "Facial Expression Recognition based on Hybrid Feature Extraction Techniques with Different Classifiers," TEST Engineering & Management, vol. 83, pp. 22319-22329, 2020.
- [15]15. S. S. R. Zeebaree, S. Ameen, and M. Sadeeq, "Social Media Networks Security Threats, Risks and Recommendation: A Case Study in the Kurdistan Region," International Journal of Innovation, Creativity and Change, vol. 13, pp. 349-365, 2020.
- [28]28. S. Y. Ameen, F. M. Almusailkh, and M. H. Al-Jammas, "FPGA Implementation of Neural Networks Based Symmetric Cryptosystem," in 6th International Conference: Sciences of Electronic, Technologies of Information and Telecommunications May, 2011, pp. 12-15.
- [59]59. F. Al-Naima, S. Y. Ameen, and A. F. Al-Saad, "Destroying steganography content in image files," in IEEE Proceedings of Fifth International Symposium on Communication Systems, Networks and Digital Signal Processing, University of Patras, Patras, Greece, 2006.
Cited by
Items 101–200 of 348
Xu, X., & Zhang, T. (2025) · Scientific Reports · doi:10.1038/s41598-025-04681-2 · Cited by 13
Cháves-Villota, A., Jiménez-Martín, A., Jojoa, M., Bahillo, A., & García-Domínguez, J. J. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101873 · Cited by 15
Affective computing methods for multimodal embodied AI human–computer interaction
Song, X., Liu, C., Xu, L., Gao, B., Lu, Z., & Zhang, Y. (2025) · Aslib Journal of Information Management · doi:10.1108/ajim-08-2024-0653 · Cited by 18
Scheduling Algorithms Implementation for Real Time Operating Systems: A Review
Ismael, G. A., Salih, A. A., Al-Zebari, A., Omar, N., Merceedi, K. J., Ahmed, A. J., Salim, N. O. M., Hasan, S. S., Kak, S. F., Ibrahim, I. M., et al. (2021) · Asian Journal of Research in Computer Science · doi:10.9734/ajrcos/2021/v11i430269 · Cited by 8
Analysis of Facial Expression for Emotion Recognition using CNN-SVM
Rishu, Kukreja, V., & Chauhan, S. (2023) · International Conference on Inventive Research in Computing Applications (ICIRCA) · doi:10.1109/icirca57980.2023.10220858 · Cited by 8
“William James presents the crucial physiological theory of emotion in 1884 [2], which states that a person's emotions are anchored in the bodily experience.”
BackgroundBeyond Empirical Windowing: An Attention-Based Approach for Trust Prediction In Autonomous Vehicles
Niu, M., Zheng, Z., Akash, K., & Misu, T. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446116 · Cited by 8
“Unlike short-term states such as emotions [2, 3], modeling protracted human states like satisfaction and trust presents unique challenges due to long signals and sparse labeling.”
BackgroundMultimodal Emotion Captioning Using Large Language Model with Prompt Engineering
Xu, Y., Zhou, Y., Cai, Y., Xie, J., Ye, R., & Wu, Z. (2024) · International Workshop on Multimodal and Responsible Affective Computing (MRAC) · doi:10.1145/3689092.3689403 · Cited by 9
Advancing music emotion recognition: large-scale dataset construction and evaluator impact analysis
Hu, Q., Murad, M. A. A., & Li, Q. L. (2025) · Multimedia Systems · doi:10.1007/s00530-025-01701-z · Cited by 13
Multimodal emotion recognition method in complex dynamic scenes
Liu, L., Luo, Q., Zhang, W., Zhang, M., & Zhai, B. (2025) · Journal of Information and Intelligence · doi:10.1016/j.jiixd.2025.02.004 · Cited by 13
Lightweight Spatio-Temporal Convolutional Neural Network for Audio-Visual Emotion Recognition
Ding, S. Y., Tang, T. B., & Lu, C. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3566773 · Cited by 8
Ai, Y., Zhang, Y., Wang, J., & Xu, N. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00280-2 · Cited by 10
Technical Solutions to Emotion AI's Privacy Harms: A Systematic Literature Review
Chowdhary, S., Ingber, A. S., & Andalibi, N. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732074 · Cited by 12
Multimodal Knowledge Distillation for Emotion Recognition
Zhang, Z., & Lu, G. (2025) · Brain Sciences · doi:10.3390/brainsci15070707 · Cited by 13
“As a result, researchers have increasingly investigated the use of multimodal data for emotion recognition [1,2].”
Emo-BERT: A Multi-Modal Teacher Speech Emotion Recognition Method
Zhao, G. L., Zhang, Y., & Chu, J. (2023) · International Workshop on Artificial Intelligence and Education (WAIE) · doi:10.1109/waie60568.2023.00022 · Cited by 9
“1 [3][4] .”
Emotion Recognition from Facial Expression using Explainable Deep Learning
Cesarelli, M., Martinelli, F., Mercaldo, F., & Santone, A. (2022) · IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) · doi:10.1109/dasc/picom/cbdcom/cy55231.2022.9927951 · Cited by 8
Emotion Recognition of Humans using modern technology of AI: A Survey
Alheeti, A. A. M., Salih, M. M. M., Mohammed, A. H., Hamood, M. A., Khudhair, N. R., & Shakir, A. T. (2023) · International Symposium on Innovative Approaches in Smart Technologies (ISAS) · doi:10.1109/isas60782.2023.10391385 · Cited by 8
A State of the Art Survey of Machine Learning Algorithms for IoT Security
Jahwar, A. F., & Zeebaree, S. R. M. (2021) · Asian Journal of Research in Computer Science · doi:10.9734/ajrcos/2021/v9i430226 · Cited by 13
“This type of network has been successfully used to recognize images [53] and video, classification, and text processing.”
MethodSelf-citationTinyM2Net: A Flexible System Algorithm Co-designed Multimodal Learning Framework for Tiny DevicesPreprint
Rashid, H.-A., Ovi, P. R., Busart, C., Gangopadhyay, A., & Mohsenin, T. (2022) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/e8px7 · Cited by 10
Rao, Y., Cao, M., & Ye, M. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3568519 · Cited by 9
“Vanilla emotion recognition focuses solely on the multiple modalities of a single sentence without contextual support, whereas emotion recognition in conversations requires interpreting emotions within a broader context to avoid ambiguity [5], [6], [7], [8].”
Wang, Y., Lei, J., Zhu, X., & Zhang, T. (2025) · Speech Communication · doi:10.1016/j.specom.2025.103278 · Cited by 7
Beyond Emotion Recognition: A Multi-Turn Multimodal Emotion Understanding and Reasoning Benchmark
Hu, J., Shi, H., Dai, C., Li, Z., Song, P., & Wang, M. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755726 · Cited by 31
Smart Healthcare for ECG Telemonitoring System
Saeed, J. N., & Ameen, S. Y. (2021) · Journal of Soft Computing and Data Mining · doi:10.30880/jscdm.2021.02.02.007 · Cited by 8
“Currently, deep learning is the most common detection [64], [65], [67], [69], [70].”
BackgroundSelf-citationCustom emoji based emotion recognition system for dynamic business webpages
Isiaka, F., & Adamu, Z. (2022) · International Journal of Intelligent Computing and Cybernetics · doi:10.1108/ijicc-11-2021-0254 · Cited by 3
Multimodal Art Pose Recognition and Interaction With Human Intelligence Enhancement
Ma, C., Liu, Q., & Dang, Y. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.769509 · Cited by 7
“…as innovative and novel, which serves as a special language for communication between humans and computers, such as body movements, facial expressions, etc., to express specific interaction intentions and achieve the exchange of information between humans and computers (Abdullah et al., 2021).”
BackgroundTangibleMoments: Embedding XR Memories onto Physical Objects
Khan, O., Ahmed, Z., Nam, H., & Kim, K. (2025) · IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · doi:10.1109/vrw66409.2025.00227 · Cited by 7
“Furthermore, emotional analysis models [1] can analyze a user’s emotional state and adapt the content replay accordingly, personalizing the experience based on the emotional context.”
Accuracy and robustness evaluation of deep learning algorithms in facial recognition systems
Zhang, J., & Hu, N. (2025) · Systems and Soft Computing · doi:10.1016/j.sasc.2025.200252 · Cited by 10
Yu, J., Ru, Y., Lei, B., & Chen, H. (2025) · Sensors · doi:10.3390/s25206397 · Cited by 11
“While unimodal recognition using facial expressions or physiological signals is well-established, emotion as a complex psychophysiological phenomenon often lacks robustness when analyzed through single modalities or even fused physiological signals alone [5].”
Hossain, M. A., Osman, M. H., Hamdan, A. A., Abdelhag, M. E., & Kechadi, T. (2023) · International Conference on Computing, Communication and Networking Technologies (ICCCNT) · doi:10.1109/icccnt56998.2023.10308392 · Cited by 6
Compound facial expressions recognition approach using DCGAN and CNN
Ullah, S., Ou, J., Xie, Y., & Tian, W. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20138-6 · Cited by 8
Bhavana, N., Guthur, A. S., Reddy, K. L. S., Ahmed, S. T., & Ahmed, A. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.04.088 · Cited by 7
A Multi-Model Framework for Grading of Human Emotion Using CNN and Computer Vision
Kulkarni, P., & Rajesh, T. M. (2021) · International Journal of Computer Vision and Image Processing · doi:10.4018/ijcvip.2022010102 · Cited by 6
Cao, Y., & Park, J. (2023) · IEEE Access · doi:10.1109/access.2023.3341926 · Cited by 6
“They proposed a DL-based approach that combined LSTM with other models to achieve more accurate emotion recognition [42].”
MethodGovindaraju, V., & Thangam, D. (2024) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-6806-0.ch018 · Cited by 8
Bi, X., & Zhang, T. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.2890 · Cited by 10
Li, F., Luo, J., & Xia, W. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113915 · Cited by 6
A Feature Selection Algorithm Based on Differential Evolution for English Speech Emotion Recognition
Yue, L., Hu, P., Chu, S., & Pan, J. (2023) · Applied Sciences · doi:10.3390/app132212410 · Cited by 5
Cultural-Aware AI Model for Emotion Recognition
Baradaran, M., Zohari, P., Mahyar, A., Motamednia, H., Rahmati, D., & Gorgin, S. (2024) · Iranian Conference on Machine Vision and Image Processing · doi:10.1109/mvip62238.2024.10491176 · Cited by 5
Binary Emotion Classification of Music Using Deep Neural Networks
Revathy, V. R., & Pillai, A. S. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-030-96302-6_45 · Cited by 5
Multiview nonlinear discriminant structure learning for emotion recognition
Guo, S., Song, L., Xie, R., Li, L., & Liu, S. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110042 · Cited by 5
Zhu, C. (2023) · International Journal on Recent and Innovation Trends in Computing and Communication · doi:10.17762/ijritcc.v11i8.7737 · Cited by 5
Srinivasarao, V. (2023) · International Journal of Speech Technology · doi:10.1007/s10772-023-10044-x · Cited by 5
“In work (Abdullah et al., 2021), the augmented time-domain signal is rebuilt utilizing noisy phase during training, with the goal being to maximise a loss function described in time domain.”
Torcate, A. S., de Santana, M. A., & dos Santos, W. P. (2023) · IEEE Latin American Conference on Computational Intelligence (LA-CCI) · doi:10.1109/la-cci58595.2023.10409408 · Cited by 5
“There are several ways for humans to manifest/communicate their emotions [3], whether through Galvanic Skin Response, Facial Expressions, Electroencephalographic signals, voice, Electrocardiograms, Eye Tracking and others.”
BackgroundApplication of emotion recognition technology in psychological counseling for college students
Li, X. (2024) · Journal of Intelligent Systems · doi:10.1515/jisys-2023-0290 · Cited by 6
Luo, Y., Wang, M., & Ye, Q. (2024) · Electronic Commerce Research · doi:10.1007/s10660-024-09937-z · Cited by 8
“In the task of emotion identification, multimodal feature fusion has demonstrated superior performance compared to single-modal approaches [70].”
Xiu, T., Sun, Y., Zhang, X., Gao, Y., Wu, J., Zhang, A. Y., & Li, H. (2025) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.368008 · Cited by 6
“This allowed us to assess the impact of each module on the overall model performance (Abdullah et al., 2021; Razzaq et al., 2023).”
A State of Art Survey of Nano Technology: Implementation, Challenges, and Future Trends
Ageed, Z. S., Ahmed, A. M., Omar, N., Kak, S. F., Ibrahim, I. M., Yasin, H. M., Rashid, Z. N., Salih, A. A., & Salim, N. O. M. (2021) · Asian Journal of Research in Computer Science · doi:10.9734/ajrcos/2021/v10i330245 · Cited by 5
“Nanotechnology was not well-known when it was first coined, even though it referred to manipulating atoms and molecules to make them into anything we wished [11,12].”
BackgroundImplementasi Deep Learning untuk Entity Matching pada Dataset Obat (Studi Kasus K24 dan Farmaku)
Pratama, R. P., Hidayat, R. S. A., Fano, N. F., Akbar, A., & Rakhmawati, N. A. (2021) · JISKA (Jurnal Informatika Sunan Kalijaga) · doi:10.14421/jiska.2021.6.3.130-138 · Cited by 5
“Selain itu, deep learning telah berhasil diterapkan diberbagai bidang seperti signal processing, artificial intelligence , dan emotion detection (Abdullah et al., 2021).”
MethodMultimodal Fusion Strategies for Physiological-emotion Analysis
Zhang, T., Huang, Z., Li, R., Zhao, J., & Jin, Q. (2021) · doi:10.1145/3475957.3484452 · Cited by 5
Attention-based CNN-BiGRU for Bengali Music Emotion Classification
Ghosh, S., & Md. Omar Faruk, R. I. A. D. (2022) · Indonesian Journal of Computer Science · doi:10.33022/ijcs.v11i3.3111 · Cited by 5
Design a Clustering Document based Semantic Similarity System using TFIDF and K-Mean
Ibrahim, R. K., Zeebaree, S. R. M., Jacksi, K., Sadeeq, M. A. M., Shukur, H. M., & Alkhayyat, A. H. (2021) · International Conference on Engineering Technology and its Applications (IICETA) · doi:10.1109/iiceta51758.2021.9717942 · Cited by 5
Self-citationMultimodal Strategy To Defend Mobile Devices Against Vishing Attacks
Mishra, M., Jain, S. K., Suraj Patel, M., Mani, S., Boragule, A., Sahni, N., & Nair, R. C. (2024) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3636534.3690683 · Cited by 7
“[11, 12] , emotion recognition [13–15], medicine [16–18], education [19–21], including Vishing [22].”
Multimodal fusion for athlete state prediction leveraging XLNet and deep generative models
Feng, Y., Sun, Y., & Hang, C. (2025) · Alexandria Engineering Journal · doi:10.1016/j.aej.2025.07.021 · Cited by 5
Text and Sound-Based Feature Extraction and Speech Emotion Classification for Korean
Jo, J., Kim, S. K., & Yoon, Y. C. (2024) · International Journal on Advanced Science Engineering and Information Technology · doi:10.18517/ijaseit.14.3.18544 · Cited by 5
“Emotion recognition research has historically concentrated on visual and textual cues, such as facial expressions and linguistic patterns [12]–[21].”
BackgroundAlhussen, A., Ansari, A. S., & Mohammadi, M. S. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.059382 · Cited by 4
“It’s critical to accurately perceive these emotions to facilitate effective conversation [7].”
Tomaz, M. S. D. L. E. S., Fernandes, B., & Sciutti, A. (2025) · IEEE Access · doi:10.1109/access.2025.3540034 · Cited by 6
“Accurate interpretation of emotional signals originating from multiple modalities, such as visual expressions, physiological signals and vocal tones is essential to ensure meaningful communication, both in human-to-human and in human-to-machine interactions [2], [3].”
Taware, S., & Thakare, A. (2025) · Circuits Systems and Signal Processing · doi:10.1007/s00034-025-03080-2 · Cited by 4
Venkataraman, J., & Mohandoss, L. (2023) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.7683 · Cited by 7
Lu, Y.-Y., Chen, Z., Zheng, Q., Zhu, Y., & Wang, M. (2023) · Soft Computing · doi:10.1007/s00500-023-08760-2 · Cited by 5
RetractedYang, H. (2023) · IEEE Access · doi:10.1109/access.2023.3301127 · Cited by 4
“However, the emotion expressed by many words in different contexts is different from the emotion marked in the emotion dictionary, and the accuracy of emotion classification of texts is low [3], [4].”
BackgroundAre You Sure? - Multi-Modal Human Decision Uncertainty Detection in Human-Robot Interaction
Scherf, L., Gasche, L. A., Chemangui, E., & Koert, D. (2024) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3610977.3634926 · Cited by 11
“While there is a large body of literature for methods to enable robots to recognize basic human emotions such as anger, happiness, sadness, or fear [1, 2, 27, 31], there are fewer works that explore human uncertainty recognition [11, 13, 15, 30].”
BackgroundDai, Z., Fei, H., & Lian, C. (2024) · Internet Technology Letters · doi:10.1002/itl2.520 · Cited by 5
“11,12 Specific datasets (such as eNTERFACE’05, RML, and AFEW6.”
2D CNN vs 3D CNN: An Empirical Study on Deep Learning-based Facial Emotion Recognition
Dhoot, A., Hadj-Alouane, N. B., & Alouane, M. T.-H. (2023) · International Conference on Modeling, Simulation & Intelligent Computing (MoSICom) · doi:10.1109/mosicom59118.2023.10458797 · Cited by 4
“The use of deep learning architectures, specifically, neural networks, is very common for the detection of emotion [13].”
BackgroundLeveraging Emotional AI for Improved Human-Computer Interactions: An Interdisciplinary Perspective
Govindaraju, V., & Thangam, D. (2024) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-2794-4.ch005 · Cited by 4
Real-Time Speech Emotion Recognition Using Hybrid Transformer Networks for Mental Health Monitoring
Durga, P. J., Basha, C. H., Babu, C. N. K., Senthilkumar, S., Kumar, N., & Kavitha, M. (2025) · International Conference on Multi-Agent Systems for Collaborative Intelligence (ICMSCI) · doi:10.1109/icmsci62561.2025.10894225 · Cited by 5
“1D dilated CNN architecture [5] 85.56% Convolutional Neural Network [6] 86.32% CNN+LSTM [7] 87.15% Deep neural network [8] 91.6% Proposed 92.17% High recall is important in healthcare-related applications, as missing signs of emotional distress (false negatives) could result in missed intervention…”
InfluentialVarying the Context to Advance Affect Modelling: A Study on Game Engagement Prediction
Pinitas, K., Rašajski, N., Barthet, M., Kaselimi, M., Makantasis, K., Liapis, A., & Yannakakis, G. N. (2024) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii63134.2024.00027 · Cited by 10
“Beyond visual cues, the capacity of other input modalities as affect predictors, such as audio and physiology, has also been studied extensively [15]–[18].”
The Establishment and Optimization of Public Emotion Network Communication Model using Deep Learning
Zhang, Y., Dai, B., & Zhong, Y. (2022) · International Journal of Humanoid Robotics · doi:10.1142/s0219843622400102 · Cited by 4
Zhang, W. (2022) · Wireless Communications and Mobile Computing · doi:10.1155/2022/7691586 · Cited by 4
“As shown in Figure 2, a general multimodal emotion recognition system consists of three steps: feature extraction, multimodal information fusion, and emotion classifier design [44].”
BackgroundPleural Effusion Detection Using Machine Learning and Deep Learning Based on Computer Vision
Ibrahim, R. F., Yhiea, N. M., Mohammed, A. M., & Magdy, A. (2022) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-20601-6_19 · Cited by 4
Emotion-Recognition Algorithm Based on Weight-Adaptive Thought of Audio and Video
Cheng, Y., Zhou, D., Wang, S., & Wen, L. (2023) · Electronics · doi:10.3390/electronics12112548 · Cited by 5
Yang, Z., Li, Z., Zhu, D., & Zhou, Y. (2024) · Information Technology And Control · doi:10.5755/j01.itc.53.1.35060 · Cited by 5
“In addition, multimodal sentiment recognition is essential in application scenarios such as intelligent customer service, social robotics, mental health assistance, and online education [1].”
BackgroundLai, X., Xue, Y., Tursun, T., Yadikarl, N., & Ubul, K. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-8499-8_31 · Cited by 3
Advancements in Real-Time Emotion Detection System Using Deep Neural Networks
Sundhararajan, V. S., & Nagappan, G. (2025) · IGI Global eBooks · doi:10.4018/979-8-3693-6910-4.ch002 · Cited by 3
Pan, Y. (2025) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2025.0160613 · Cited by 6
A hybrid approach to Twitter sentiment analysis using integration of ESN ISPBO and BERT
Chai, Z., Sun, N., Zhang, Q., & Razmjooy, N. (2025) · Scientific Reports · doi:10.1038/s41598-025-32399-8 · Cited by 4
DDLS: Distributed Deep Learning Systems: A Review
Kako, E. A. N. A. (2021) · Türk bilgisayar ve matematik eğitimi dergisi · doi:10.17762/turcomat.v12i10.5632 · Cited by 3
“…efficiency compared to a single processor by installing high-speed calculators (for instance, Nvidia V100 GPUs) with low-speed interconnections (for example, a 1/10 Gbps Ethernet) to train a deep model (for example, the calculated to communicate ratio is low) (S. M. S. A. Abdullah et al., 2021).”
BackgroundEmotion Detection using Deep Learning
Chaithra, G., Shrinivas, P. A., Naidu, B. G., Vinay, G. S., Supreeth, S., & Biradar, A. (2024) · International Conference on Networks, Multimedia and Information Technology (NMITCON) · doi:10.1109/nmitcon62075.2024.10699079 · Cited by 3
“In [8], focused on multimodal emotion recognition methods utilizing neural network architectures and deep learning techniques.”
BackgroundSamonte, M. J. C., Nitta, K., Datu, C., & Erpelo, R. (2024) · International Conference on Information Management and Management Science (IMMS) · doi:10.1145/3695652.3695658 · Cited by 3
Meta-learning Improves Emotion Recognition
Wang, W., Zhang, J., Lin, Z., Cui, L., & Zhang, X. (2023) · Smart innovation, systems and technologies · doi:10.1007/978-981-19-7184-6_2 · Cited by 4
Hyper-Enhanced Feature Learning System for Emotion Recognition
Fordson, H. P., Xing, X., Guo, K., Xu, X., Anderson, A. K., & Rosa, E. D. (2023) · Springer eBooks · doi:10.1007/978-3-031-21236-9_1 · Cited by 3
Zawadzka, T., Wierciński, T., Waloszek, W., & Wróbel, M. R. (2023) · IEEE Access · doi:10.1109/access.2023.3257573 · Cited by 5
“In combination with multimodal approaches, this makes it possible to achieve the most accurate results in emotion recognition [2], [4].”
BackgroundGuiding Interactive Film With Emotion-Profiling Chatbots
Celina, L., Wang, H., & Wang, M. (2023) · 2012 International Conference on Computing, Networking and Communications (ICNC) · doi:10.1109/icnc57223.2023.10074513 · Cited by 6
“The optional audio and video inputs can provide more information to improve prediction accuracy [11].”
BackgroundDiseases Diagnosis Using Machine Learning of Medical Images
Abas, S. M., Hassan, O. M. S., Ali, I. M., Nori, S. S., & Hassan, H. S. (2022) · International Conference on Advanced Science and Engineering (ICOASE) · doi:10.1109/icoase56293.2022.10075581 · Cited by 3
“This approach prepare the dataset as first step to be suitable for learning algorithms and apply many processes on dataset such as filtration and segmentation [17], [18].”
MethodInfluential(2023) · International journal of intelligent engineering and systems · doi:10.22266/ijies2023.1031.46 · Cited by 3
“Abdullah, et al [35] utilized a multi-modal emotion recognition system with the use of deep learning technique for facial emotion recognition.”
BackgroundAbdalraheem, M. H. O., Hossain, M. A., Hamdan, A. A., Kechadi, T., & Limkar, S. V. (2023) · International Conference on Computing Communication Control and automation · doi:10.1109/iccubea58933.2023.10392279 · Cited by 3
Multimodal Emotion Recognition in Video, Audio, and Text using Deep and Transfer Learning
Panchal, V., Deshpande, T., Kandoi, T., Deulkar, K., & Narvekar, M. (2023) · International Conference on Advanced Computing Technologies and Applications (ICACTA) · doi:10.1109/icacta58201.2023.10392410 · Cited by 3
“But they may frequently be quickly changed in response to diverse social situations [1].”
BackgroundAseniero, B. A., Lee, M. A., Wang, Y., Zhou, Q., Shahmansouri, N., & Goldstein, R. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650815 · Cited by 3
“Human-computer interaction has a long history of modeling human emotional responses [3] as part of computing systems such as affective computing research [7].”
Audio-visual scene recognition using attention-based graph convolutional model
Wang, Z., Wu, Y., Wang, Y., Gong, W., & Gonzàlez, J. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19654-2 · Cited by 3
Enhancing Facial Expression Recognition through Light Field Cameras
Oucherif, S. D., Nawaf, M. M., Boï, J.-M., Nicod, L., Mallor, É., Dubuisson, S., & Merad, D. (2024) · Sensors · doi:10.3390/s24175724 · Cited by 4
“In the field of affective computing, multimodal fusion involves integrating various data modalities, including images, videos, audio, and bio-electrical signals such as EEG or ECG [22–25].”
BackgroundMultimodal Chinese Agricultural News Classification Method Based on Interactive Attention
Duan, X., Li, Z., Liu, L., & Liu, Y. (2024) · IEEE Access · doi:10.1109/access.2024.3482868 · Cited by 5
“Recently, multimodal learning has become extensively prevalent and applied in diverse fields such as visual question answering [2], [3], sentiment analysis [4], [5], neural machine translation [6], cross-modal retrieval [7], speech recognition [8], and image captioning.”
BackgroundAudio-Guided Fusion Techniques for Multimodal Emotion Analysis
Gao, F., & Shi, P. (2024) · International Workshop on Multimodal and Responsible Affective Computing (MRAC) · doi:10.1145/3689092.3689414 · Cited by 6
Systematic mapping study of tools to identify emotions and personality traits
Islam, A., Noor, N. F. B. M., & Rahman, S. S. A. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00271-3 · Cited by 6
Optimal Spatial Attention Network based Convolutional Neural Network for Facial Emotion Recognition
Mahendar, M., Malik, A., & Batra, I. (2022) · International Conference on Smart Generation Computing, Communication and Networking (SMART GENCON) · doi:10.1109/smartgencon56628.2022.10084147 · Cited by 3
“Emotion recognition utilizes traditional methods with the disadvantage of mutual optimization of classification and feature extraction [3, 4].”
MethodDesign of smart elderly care emotion recognition and elderly care system based on transfer learning
Pu, Y., Zhang, J., Li, M., & Mou, Y. (2025) · Journal of Computational Methods in Sciences and Engineering · doi:10.1177/14727978251337893 · Cited by 2
“23,24 Research shows that facial expressions provide overt emotional information, while voice and text reveal more subtle emotional changes.”
A multi-modal speech emotion recognition method based on graph neural networks
Li, Y., Wang, Y., Yang, X., Hoi, L. M., & Im, S. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06953-w · Cited by 2
Chittepu, S., Martha, S., & Banik, D. (2026) · Cognitive Processing · doi:10.1007/s10339-026-01330-y · Cited by 4
Khanra, M., & Roy, S. S. (2026) · Frontiers in Robotics and AI · doi:10.3389/frobt.2026.1810478 · Cited by 4
Sun, J. (2024) · Highlights in Science Engineering and Technology · doi:10.54097/agvjvq19 · Cited by 5
Music Recommendation based on Emotions recognized through Speech and Body Gestures
Nambiar, K. R., & Palaniswamy, S. (2024) · International Conference for Innovation in Technology (INOCON) · doi:10.1109/inocon60754.2024.10511354 · Cited by 2
LeMiNo – Let Me Know how to Negotiate: Virtual Simulator for Negotiation Training
Votintseva, A., & Johnson, R. (2024) · International Conference on Smart Computing, IoT and Machine Learning (SIML) · doi:10.1109/siml61815.2024.10578255 · Cited by 5
“In our exploration of human emotion recognition, we rely on the insightful work of Abdullah et al. [13], who have made significant progress in multimodal emotion identification using deep learning techniques.”
MethodAbdellaoui, B., Remaida, A., Sabri, Z., Abdellaoui, M., Idrissi, Y. E. B. E., & Moumen, A. (2024) · International Conference on Intelligent Systems and Computer Vision (ISCV) · doi:10.1109/iscv60512.2024.10620124 · Cited by 4
Metadata
- Authors
- Abdullah, S. M. S. A., Ameen, S. Y. A., M. Sadeeq, M. A., & Zeebaree, S.
- Affiliations
- Duhok Polytechnic University (IQ)
- Venue
- Journal of Applied Science and Technology Trends
- Published
- 2021-05-30
- Volume
- 2
- Issue
- 01
- Pages
- 73-79
- ISSN
- 2708-0757 (ISSN-L)
- OpenAlex
- W3156576211
- Cited by
- 347
- References
- 72
- Citations by year
- 2026: 40 · 2025: 91 · 2024: 70 · 2023: 66 · 2022: 46 · 2021: 34
- FWCI
- 52.20
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Journal of Applied Science and Technology Trends (publishedVersion, cc-by-nc-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Keywords
- multimodal emotion recognition, affective computing, human-computer interaction, facial expression recognition, physiological signals, neuroimaging, deep learning, emotion classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.