The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index; 5 of the 12 sources its claims cite were checked for support.
Multimodal Emotion Recognition From EEG Signals and Facial Expressions
Open accessAbstract
Emotion recognition has attracted a lot of attention in recent years. It is widely used in health care, teaching, human-computer interaction, and other fields. Various human emotional features are often used to recognize different emotions. Currently, there is more and more research on multimodal emotion recognition based on the fusion of multiple features. This paper proposes a deep learning model of multimodal emotion recognition based on the fusion of electroencephalogram (EEG) signals and facial expressions to achieve an excellent classification effect. First, the pre-trained convolution neural network (CNN) is used to extract the facial features from the facial expressions. Next, the attention mechanism is introduced to extract more critical facial frame features. Then, we apply CNNs to extract spatial features from original EEG signals, which use a local convolution kernel and a global convolution kernel to learn the features of left and right hemispheres channels and all EEG channels. After feature-level fusion, the fusion features of the facial expression features and EEG features are fed into the classifier for emotion recognition. This paper conducted experiments on the DEAP and MAHNOB-HCI datasets to evaluate the performance of the proposed model. The accuracy of valence dimension classification is 96.63%, and arousal dimension classification is 97.15% on the DEAP dataset, while 96.69% and 96.26% on the MAHNOB-HCI dataset. The experimental results show that the proposed model can effectively carry out emotion recognition.
Overview
CoChat Summary
TL;DR
This paper proposes a deep learning model that fuses EEG signals and facial expressions for multimodal emotion recognition, using a pre-trained CNN with attention for facial features and CNNs with local/global kernels for EEG spatial features. It reports high binary valence/arousal classification accuracy on DEAP and MAHNOB-HCI, with fusion outperforming either modality alone.
Key findings
- On DEAP, the proposed model achieved 96.63% accuracy for valence and 97.15% for arousal in binary classification.
- On MAHNOB-HCI, it achieved 96.69% accuracy for valence and 96.26% for arousal.
- Ablation results show multimodal fusion outperformed single modalities: DEAP valence was 93.59% for facial expressions, 85.72% for EEG, and 96.63% for fusion; DEAP arousal was 94.03%, 87.97%, and 97.15%, respectively. MAHNOB-HCI valence was 93.34%, 85.98%, and 96.69%; arousal was 93.06%, 85.23%, and 96.26%.
- The paper reports that its model outperformed several existing approaches on DEAP and MAHNOB-HCI, such as Huang et al. (DEAP valence 80.30%, arousal 74.23%), Zhu et al. (70.01%, 77.08%), Li et al. (71.00%, 58.75%), and Koelstra et al. (74.00%, 70.00%).
Methods
- The model extracts facial features using a pre-trained CNN (DeepVANet, pre-trained on AFEW-VA) as an encoder, then applies an attention mechanism to weight important facial frames; input samples are T × 3 × 64 × 64 with T = 5 frames per second.
- EEG spatial features are extracted with CNNs using a local convolution kernel for left/right brain channels and a global convolution kernel for all EEG channels; in DEAP and MAHNOB-HCI, c = 32 channels, h = 64, and K = 20 kernels.
- Facial and EEG features are fused at the feature level and fed into a fully connected classifier. Experiments use DEAP (32 subjects, 40 trials each, 2400 samples per subject) and MAHNOB-HCI (25 participants used, 20 trials each, 1611 samples per subject), with 5-fold cross-validation; training used PyTorch, batch size 128, learning rate 0.00001, Adam optimizer, and binary cross entropy.
Limitations
- The paper does not include an explicit limitations section; its conclusion states future work to explore a more reliable pre-training model to reduce resources and time, and to introduce more modalities such as non-physiological signals.
- Evident from the text: MAHNOB-HCI experiments used only 25 of 30 participants because three recording files were corrupted and two participants’ data were incomplete.
- Evident from the text: the tasks are binary high/low valence and arousal classifications, not continuous or multi-class emotion recognition; also, DEAP facial expressions were recorded for the first 22 subjects, while the experiments state both datasets use data of all subjects, suggesting a possible sample mismatch.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 0 Not valid · 4 Not checked · 16 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Still, more and more researchers use multimodal models that integrate multiple data and features to carry out emotion recognition, which can extract more abundant features and improve the effect of emotion recognition.”
What the sources say
- Not checked — no full text of the sourcecites [10] source
- Supportedcites [11] source
The Introduction says recent research uses multimodal data to improve emotion-classification performance and that multiple sources provide complementary emotion-related information. The Multimodal Data Fusion section states that multimodal inputs improve recognition accuracy over single-modality inputs by providing additional information. (LLM, conf=0.93)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Through the above optimization methods, the model proposed in this paper outperforms other similar studies mentioned above in the classification tasks of DEAP and MAHNOB-HCI datasets.”
What the sources say
- Could not verifycites [29] sourceExperimental — accuracy not yet measured
The cited paper reports 2D face-alignment comparisons on 300-VW, 300-W and Menpo (Section 5, p. 5), not classification results on DEAP or MAHNOB-HCI. Searches for those datasets and classification tasks found no passage addressing the citing paper’s claim about its own model. (LLM, conf=0.78)
- Could not verifycites [30] sourceExperimental — accuracy not yet measured
The Discussion (p. 11) describes EEGLAB as a toolbox for EEG processing, while the comparison on p. 12 concerns ICA algorithms, not classification models. Searches for DEAP, MAHNOB-HCI, and classification performance found no passage assessing the citing paper’s proposed model or its claimed outperformance. (LLM, conf=0.78)
- Supportedcites [31] source
Section 3.2, “Comparison with Other Literature” (page not provided), compares the method with earlier studies on DEAP and MAHNOB-HCI and states that its performance surpassed previous results for both valence and arousal classification. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Could not verify
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Still, more and more researchers use multimodal models that integrate multiple data and features to carry out emotion recognition, which can extract more abundant features and improve the effect of emotion recognition.”
What the sources say
- Not checked — no full text of the sourcecites [10] source
- Supportedcites [11] source
The Introduction says data from multiple sources provide complementary emotion-related information. Section B, “Multimodal Data Fusion for Emotion Classification,” explains that features from each modality can be combined and that the additional information improves emotion-recognition accuracy. This supports the attributed claim; page numbers are not provided in our copy. (LLM, conf=0.93)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The accuracy of valence dimension classification is 96.63%, and arousal dimension classification is 97.15% on the DEAP dataset, while 96.69% and 96.26% on the MAHNOB-HCI dataset.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The accuracy of valence dimension classification is 96.63%, and arousal dimension classification is 97.15% on the DEAP dataset, while 96.69% and 96.26% on the MAHNOB-HCI dataset.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
9 more claims
Create a free account to see all 23 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 2
- Probably not a substantial claim. 8
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The proposed study demonstrated a marginally higher performance, achieving an accuracy of 97.80%, compared to Wang et al.'s [23] 96.89% using the DEAP dataset by combining EEG and facial expressions.”
From Multimodal Emotion Recognition: Emotion Classification Through the Integration of EEG and Facial Expressions (2025) · cites this paper as [23]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“the number of samples per subject in the DEAP dataset is 2400”
Reported 2400 · recomputed 1200
Citations
Cites
Delorme, A., & Makeig, S. (2004) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2003.10.009 · Cited by 24,371 · cited 1×
“For EEG data, the sampling rate is reduced to 128Hz, and the band-pass filter in the toolbox of EEGLAB [30] is used to filter and remove artifacts.”
MethodAn argument for basic emotions
Ekman, P. (1992) · Cognition and Emotion · doi:10.1080/02699939208411068 · Cited by 9,646 · cited 1×
“In the dimensional model, the emotions are mapped into the valence, arousal, and The associate editor coordinating the review of this manuscript and approving it for publication was Seifedine Kadry . dominance dimensions [2].”
BackgroundCore affect and the psychological construction of emotion.
Russell, J. A. (2003) · Psychological Review · doi:10.1037/0033-295x.110.1.145 · Cited by 6,113 · cited 2×
“Currently, the most commonly used dimensional model is the valence-arousal model proposed by Russell [3], which only includes valence and arousal.”
BackgroundDEAP: A Database for Emotion Analysis ;Using Physiological Signals
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., & Patras, I. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.15 · Cited by 5,093 · cited 1×
“This paper uses two standard datasets: DEAP [26] and MAHNOB-HCI [27] datasets to verify the model performance.”
MethodUniversals and cultural differences in the judgments of facial expressions of emotion.
Ekman, P., Friesen, W. V., O'Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., Krause, R., LeCompte, W. A., Pitcairn, T., Ricci-Bitti, P. E., et al. (1987) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.53.4.712 · Cited by 1,763
A Multimodal Database for Affect Recognition and Implicit Tagging
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.25 · Cited by 1,664 · cited 1×
“This paper uses two standard datasets: DEAP [26] and MAHNOB-HCI [27] datasets to verify the model performance.”
MethodBulat, A., & Tzimiropoulos, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.116 · Cited by 1,464
Bulat, A., & Tzimiropoulos, G. (2017) · Nottingham ePrints (University of Nottingham) · Cited by 1,277
Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers
Atkinson, J., & Campos, D. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.10.049 · Cited by 506 · cited 1×
“Traditional emotion recognition often uses machine learning methods, such as support vector machine (SVM), decision tree, and k -nearest neighbor (KNN) to classify [4], [5].”
MethodAFEW-VA database for valence and arousal estimation in-the-wild
Kossaifi, J., Tzimiropoulos, G., Todorovic, S., & Pantic, M. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.02.001 · Cited by 236 · cited 1×
“The AFEW-VA database [25] contains 600 videos selected from movies showing various facial expressions.”
Background
Cited by
Multimodal Emotion Recognition with Deep Learning: Advancements, challenges, and future directions
A.V., G., T., M., D., P., & E., U. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102218 · Cited by 225
A review on EEG-based multimodal learning for emotion recognition
Pillalamarri, R., & Shanmugam, U. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11126-9 · Cited by 183
Using transformers for multimodal emotion recognition: Taxonomies and state of the art review
Hazmoune, S., & Bougamouza, F. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108339 · Cited by 166
Multimodal emotion recognition: A comprehensive review, trends, and challenges
Ramaswamy, M. P. A., & Palaniswamy, S. (2024) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1563 · Cited by 70
A survey on state-of-the-art deep learning applications and challenges
Noor, M. H. M., & Ige, A. O. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111225 · Cited by 120
CVGG-19: Customized Visual Geometry Group Deep Learning Architecture for Facial Emotion Recognition
Kim, J. H., Poulose, A., & Han, D. S. (2024) · IEEE Access · doi:10.1109/access.2024.3377235 · Cited by 45
EEG-based affective brain–computer interfaces: recent advancements and future challenges
Chen, Y., Peng, Y. K., Tang, J., Camilleri, T. A., Camilleri, K. P., Kong, W., & Cichocki, A. S. (2025) · Journal of Neural Engineering · doi:10.1088/1741-2552/ade290 · Cited by 49
Li, J., & Peng, J. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3430310 · Cited by 40
“Thus, multimodal fusion has been used to recognizeemotionsinsomestudies,suchasfusingphysiological signalswithfacialexpressions[15],[16],[17].”
MethodInfluentialTian, F., Zhang, L., Zhu, L., Zhao, M., Liu, J., Dong, Q., & Zhao, Q. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3420445 · Cited by 33
Emotion recognition using cross-modal attention from EEG and facial expression
Cui, R., Chen, W., & Li, M. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112587 · Cited by 32
Govea, J., Navarro, A. M., Sánchez-Viteri, S., & Villegas-Ch, W. E. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1458230 · Cited by 38
Erdem Güler, S., & Patlar Akbulut, F. (2025) · IEEE Access · doi:10.1109/access.2025.3538642 · Cited by 21
Khan, M. R., Tania, A. A., & Ahmad, M. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.107826 · Cited by 16
Emotion recognition with multiple physiological parameters based on ensemble learning
Liao, Y., Gao, Y., Wang, F., Zhang, L., Xu, Z., & Wu, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-96616-0 · Cited by 23
“Notably, Wang 20 and Wang 21 developed advanced emotion recognition strategies tailored for noisy environments, combining EEG with other modalities, such as speech and facial expressions.”
Emotion recognition based on EEG signals and face images
Lian, Y., Zhu, M., Sun, Z., Liu, J., & Hou, Y. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.107462 · Cited by 16
Khubani, J., & Kulkarni, S. S. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106217 · Cited by 13
GJFusion: A Channel-Level Correlation Construction Method for Multimodal Physiological Signal Fusion
Huang, W., Chen, Y., Jiang, X., Zhang, T., & Chen, Q. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3617503 · Cited by 14
“[40]: This work proposes a multimodal method based on the fusion of EEG signals and facial expressions (FE).”
MethodHosseini, I., Hossain, M. Z., Zhang, Y., & Rahman, S. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104121 · Cited by 16
Self-citationState-of-the-art Multimodal Emotion Recognition: A comprehensive survey and taxonomy
Yazici, A., Kucukyilmaz, T., Dokeroglu, T., Sharipbay, A., Lee, M.-H., & Tyler, B. (2026) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2026.200642 · Cited by 11
EEG emotion recognition approach using multi-scale convolution and feature fusion
Zhang, Y., Shan, Q., Chen, W., & Liu, W. (2024) · The Visual Computer · doi:10.1007/s00371-024-03652-4 · Cited by 13
Self-citationKapase·, A. B., & Uke, N. (2025) · International Journal of Speech Technology · doi:10.1007/s10772-025-10202-3 · Cited by 11
Sun, W., Yan, X., Su, Y., Wang, G., & Zhang, Y. (2025) · Sensors · doi:10.3390/s25072029 · Cited by 15
“Wang et al. fused the physiological features of EEG signals extracted by CNN and the visual features of facial expressions extracted by a pre-training network using the attention mechanism, which significantly improved the performance of emotion classification [30].”
Self-citationSafavi, F., Patel, K., & Vinjamuri, R. K. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3567966 · Cited by 14
“This significant achievement is highlighted in Table V, which +ConvLSTM[59],DCNN[60]andDeep ResidualNetwork[61]andthefusionmodelscombiningNeuro-physiologicalandFacialfeatureslike3D-CNN[62],CNN[63], CNN+Bi-LSTM[64].”
Hassan, C. A. U., Ehatisham-ul-Haq, M., Murtaza, F., Yasin, A. U., & Ullah, S. S. (2025) · Scientific Reports · doi:10.1038/s41598-025-98404-2 · Cited by 10
A Unified Biosensor–Vision Multi-Modal Transformer network for emotion recognition
Ali, K., & Hughes, C. E. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.107232 · Cited by 17
Facial Expression Recognition Based on Deep Learning: A Review
Jajan, K. I. K., & Abdulazeez, A. M. (2024) · Indonesian Journal of Computer Science · doi:10.33022/ijcs.v13i1.3705 · Cited by 12
Sarikaya, M. A., & İnce, G. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.2649 · Cited by 11
Electroencephalogram Emotion Recognition via AUC Maximization
Xiao, M., & Shi, B. (2024) · Algorithms · doi:10.3390/a17110489 · Cited by 14
A Unified Transformer-based Network for Multimodal Emotion RecognitionPreprint
Ali, K., & Hughes, C. E. (2023) · doi:10.36227/techrxiv.23916123 · Cited by 10
Lixun, X. I. E., Sun, W., Zhang, J., & Zhao, X. (2025) · Digital Signal Processing · doi:10.1016/j.dsp.2025.105261 · Cited by 9
Semisupervised Deep Features of Time‐Frequency Maps for Multimodal Emotion Recognition
Zali-Vargahan, B., Charmin, A., Kalbkhani, H., & Barghandan, S. (2023) · International Journal of Intelligent Systems · doi:10.1155/2023/3608115 · Cited by 9
“Te EEG and facial expressions were used in [17] for multimodal emotion recognition.”
MethodEEG-based Multimodal Emotion Recognition: Recent Progress, Challenges, and Future Directions
Muhammad, G., Almuntasheri, S. A., Alenezi, F., Alhadi, N., & Leung, V. C. M. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3774428 · Cited by 14
Revolutionizing neurostimulator care: enhancing remote health monitoring through SDN-cloud networks
(2025) · doi:10.1007/s11235-024-01255-x · Cited by 7
“The additional datasets discussed include the DEAP Dataset, providing EEG and peripheral physiological signals recorded during music video viewing for emotion analysis, captured via a 32-channel cap with a 128 Hz sampling rate [26].”
Music Recommendation System Using Facial Detection Based Emotion Analysis
Guthula, M. S., & Bordoloi, M. (2024) · International Conference on Emerging Techniques in Computational Intelligence (ICETCI) · doi:10.1109/icetci62771.2024.10704201 · Cited by 9
Wu, Q., Zhu, M., Xu, W., Wang, J., Mao, Z., Gao, Q., & Song, Y. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3400341 · Cited by 6
“…extracted facial expression and EEG features based on fully connected (FC) neural networks, and fused these features using layered LSTM. Wang et al. [33] used the pretrained CNN to extract facial expression features and then applied the attention mechanism to extract critical facial frame features.”
MethodRDA-MTE: an innovative model for emotion recognition in sports behavior decision-making
Zhang, S. (2024) · Frontiers in Neuroscience · doi:10.3389/fnins.2024.1466013 · Cited by 8
“Physiological signals include heart rate, galvanic skin response (GSR), and electroencephalography (EEG) (Wang S. et al., 2023).”
EA-FUSION: A multimodal emotion recognition model using EEG and facial expression data
Xu, C., Li, W., Li, S., Zhao, P., Xing, F., & Song, A. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108629 · Cited by 6
Xiu, T., Sun, Y., Zhang, X., Gao, Y., Wu, J., Zhang, A. Y., & Li, H. (2025) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.368008 · Cited by 6
“A multimodal fusion model is then used to combine the emotional features from various modalities into a unified emotional state representation (S. Wang et al., 2023).”
Research on cross-modal emotion recognition based on multi-layer semantic fusion
Xu, Z., & Gao, Y. (2024) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2024110 · Cited by 9
A Unified Transformer-based Network for Multimodal Emotion RecognitionPreprint
Ali, K., & Hughes, C. E. (2023) · doi:10.36227/techrxiv.23916123.v1 · Cited by 9
Alotaibi, Y. A., & Vuyyuru, V. A. (2023) · AIMS Mathematics · doi:10.3934/math.20231169 · Cited by 7
KUŞ, İ., Koçak, C., & Keleş, A. (2025) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2025.200615 · Cited by 17
Wicaksana, I. P. R. E., Davinsi, G. R., Afriyanto, M. A., Wibowo, A., & Suri, P. A. (2024) · Research Square · doi:10.21203/rs.3.rs-3856084/v1 · Cited by 5
A Dual-Modal Silent Speech Interface via Surface Electromyography (sEMG) and Vibration Sensing
Gou, G., Guo, Y.-S., Song, Z., Huo, P., Lu, Y., Li, N., Huo, X., Fang, Z., Chen, D., Liu, C., et al. (2025) · ACS Applied Materials & Interfaces · doi:10.1021/acsami.5c15136 · Cited by 5
Lu, C.-K., Lu, C.-W., & Bo Lin, G. (2026) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3621086 · Cited by 4
“Sharafi et al. [24] and Wang et al. [25] used CNNs to further extract features.”
Vision Transformer Algorithm for Plant Disease Detection: A Systematic Literature Review
(2024) · doi:10.1109/icic64337.2024.10956258 · Cited by 3
“Thirdly, feature fusion methods also play a role in reducing the effects of irrelevant features or noise, thus helping the model to focus on features that are more meaningful and important for the given task [9], [15], [18], [29], [30], [31].”
Abdulwahhab, A. H., Myderrizi, I., & Asaad, A. Y. (2026) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2026.111175 · Cited by 3
Step-Wise Prompting Meets Uncertainty-Aware Dynamic Fusion for Robust EEG-Visual Emotion Recognition
Hao, D., Meng, M., Gao, Y., Lou, X., & Kong, W. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3632304 · Cited by 3
“Following prior work [29], each trial is segmented into non-overlapping 1-second windows, resulting in 2400 samples per subject.”
InfluentialA dual-branch deep learning framework for emotion recognition from EEG signals
Saha, D., Ali, A., Gulvanskii, V., Kaplun, D., & Sarkar, R. (2026) · Scientific Reports · doi:10.1038/s41598-026-42998-8 · Cited by 3
“More recently, research has concentrated on multimodal fusion 16,17 for emotion recognition and ensemble learning 18 methods that combine different signal sources and classifier outputs to improve stability and reliability, especially under noisy real-world conditions.”
Singh, P., Dahale, S. R., Yash, Gupta, S., & Raman, B. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114735 · Cited by 5
Sun, J., Xu, T., & Yao, Y. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1459446 · Cited by 4
Modeling EEG Data into Graphs for the Prognostic of Patients in Coma Using Graph Neural Networks
Nascimento, O. O., de Assis, D. L. C., Filho, J. B. D., Zhao, L., & Carneiro, M. G. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-79038-6_16 · Cited by 5
Wen, Y., & Chen, W. (2025) · Measurement · doi:10.1016/j.measurement.2025.119247 · Cited by 2
A Unified Physiological Signal Interaction Network for Cross-Dataset Emotion Recognition
Cai, Z., Gao, H., Wu, M., Li, J., & Liu, C. (2025) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2025.3566229 · Cited by 2
Label Feature Co-Learning for Facial and EEG Emotion Recognition
Cai, M., Chen, C. L. P., Li, S., & Zhang, T. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3600267 · Cited by 3
“…include traditional methods (SVM [55] and MLP [56]), CNN-based methods (Multi-task CNN [7], TCMA [57], DST-EIF [17]), GCN-based methods (UAGCFNet [43]), Transformer-based methods (UBVMT [58], Husformer [42]), and hybrid-architecture methods (PreVGG-LSTM [8], MindLink-Eumpy [9], MER-ESFE [59]).”
Bharaneedharan, M., Kavya, G., Annalakshmi, T., & Bapu, B. R. T. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-04736-x · Cited by 1
Khanra, M., & Roy, S. S. (2026) · Frontiers in Robotics and AI · doi:10.3389/frobt.2026.1810478 · Cited by 3
Modeling High-Order Relationships Between Human and Video for Emotion Recognition in Video Learning
Ai, H., Tao, X., Li, X., & Gan, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-2064-7_1 · Cited by 1
Interactive graph emotion recognition based on multi-modal data enhancement
(2026) · doi:10.1038/s41598-025-34853-z · Cited by 1
“This method is better than related unilateral multi-mode emotional identification models 23 .”
Shi, J., Xi, T., Fan, X., Tang, J., & Yu, H. (2024) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2024.3479314 · Cited by 1
“Wang et al. [39] presented a deep learning model for multimodal emotion recognition by fusing EEG signals and facial expressions, achieving high classification accuracy on public datasets, and demonstrating the effectiveness of this approach in recognizing emotions.”
Shilpa, T., Siddappa, M., & Nasik, S. (2024) · International Conference on Electrical, Electronics, Information and Communication Technologies (ICEEICT) · doi:10.1109/iceeict61591.2024.10718491 · Cited by 1
“Wang, S., et al. [14] developed a fully connected neural network (FCNN) for multimodal emotion recognition.”
BackgroundDPMMN: A dual performer-multi-modal network for emotion recognition
Gornale, S. S., Shivakumara, P., Unki, A., & Vadera, S. (2025) · Signal Processing Image Communication · doi:10.1016/j.image.2025.117464 · Cited by 1
Facial expression recognition for emotion perception: A comprehensive science mapping
Kan, H., Chen, L., Zhang, Y., et al. (2026) · Ibrain · doi:10.1002/ibra.70010 · Cited by 1
“The current technologies that can perform multimodal analysis 38 ‐ 41 with FER include EEG, galvanic skin response (GSR), heart rate variability (HRV), electromyography (EMG), eye movement, functional magnetic resonance imaging (fMRI), and functional near ‐ infrared spectroscopy (fNIRS) (Figure 7).”
Self-citationAffective Speech Processing Using MFCC
(2025) · doi:10.1109/ictcs65341.2025.10989400 · Cited by 1
Kim, T., Moon, E., Kang, H., & Kim, H. S. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11368-2 · Cited by 2
Sreenivasarao, D., & Saheb, S. K. (2026) · Iran Journal of Computer Science · doi:10.1007/s42044-025-00365-2 · Cited by 1
Enhancing Facial Emotion Recognition Using Advanced Convolutional Neural Networks
Kalas, M. S., & Gore, S. R. (2025) · International Conference on Smart & Sustainable Technology (INCSST) · doi:10.1109/incsst64791.2025.11210351 · Cited by 1
“Shuai Wang et al. [7] further expanded this concept by integrating EEG signals with facial expressions, achieving high classification accuracy on DEAP and MAHNOB-HCI datasets.”
Umair, M., Rashid, N., Khan, U. S., Hamza, A., Zeb, A., Nawaz, T. H., & Ansari, A. R. (2025) · Results in Engineering · doi:10.1016/j.rineng.2025.106601 · Cited by 1
Babu, K. R., Suneetha, A., & Kumar, K. K. (2024) · Journal of Computer Science · doi:10.3844/jcssp.2024.1339.1348 · Cited by 1
A Hierarchical Multi-Domain Attention Network for EEG-Based Emotion Recognition
Sun, M., & You, Q. (2026) · International Conference on Advanced Electronic Materials, Computers and Software Engineering (AEMCSE) · doi:10.1109/aetcse69203.2026.11504412
Tabita, S., Narasimharao, P., Pamulaparthy, L., Janbhasha, S., GOPAIAH, J., & Nimmala, S. (2026) · International Conference on Innovations in Cybersecurity and Data Science (ICICDS) · doi:10.1109/icicds70526.2026.11604656 · Cited by 1
Reddy, C. H. N., Kumar, K. S. A., Manjunathachari, K., & Abdul, A. M. (2026) · Iranian Journal of Science and Technology Transactions of Electrical Engineering · doi:10.1007/s40998-025-00979-z
(2025) · doi:10.1109/otcon65728.2025.11070912
“Wang et al. (2023) adopted a multimodal approach by combining EEG with facial expressions using CNNs, achieving 96.63% (Valence) and 97.15% (Arousal) on the DEAP and MAHNOB-HCI datasets [11].”
Influential“In [23] and [24], deep learning models combining CNNs with attention mechanisms are used to recognize emotions using EEG and facial expressions, demonstrating high accuracy on the DEAP and MAHNOB-HCI datasets and highlighting the success of multimodal approaches.”
QoS-aware deployment of emotion recognition system through Fog computing infrastructure
Islam, M. S. U., Kumar, A., & Bhushan, M. (2026) · Cluster Computing · doi:10.1007/s10586-026-06263-x
Hybrid CNN-LBP Model for Realtime Emotion Classification
Sanaboina, C. S. (2025) · IOSR Journal of Computer Engineering · doi:10.9790/0661-2703050619
Detecting Fake News Belief via Skin and Blood Flow Signals
Nguyen, G. T., Wang, L., Jiang, Y., & Gedeon, T. (2026) · IEEE Access · doi:10.1109/access.2026.3667798 · Cited by 3
Vimala, P., Selvakani, R., Kavitha, R., Raaghav., R. S. T., Logeshwaran., T., & Jyotheesh., S. (2026) · International Conference on Expert Clouds and Applications (ICOECA) · doi:10.1109/icoeca68095.2026.11485509
Emotion recognition and human-robot interaction: a systematic review of methods and applications
Barquin, I. S., Castano, J. A., Garate, V. R., & del‐Ama, A. J. (2026) · Frontiers in Robotics and AI · doi:10.3389/frobt.2026.1891746
Wei, Y., & Zheng, Y. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-04862-6 · Cited by 1
Mahaseni, B., & Khan, N. (2025) · Empathic Computing · doi:10.70401/ec.2025.0010 · Cited by 4
Multimodal Anti-Spoofing Emotion Recognition Using GRU and BERT
Ge, C., Peng, G., Guo, A., Zhu, W., Fan, Z., & Zhu, X. (2024) · doi:10.1109/ictech63197.2024.00096
Metadata
- Authors
- Wang, S., Qu, J., Zhang, Y., & Zhang, Y.
- Affiliations
- Liaoning Normal University (CN)
- Venue
- IEEE Access
- Published
- 2023
- Volume
- 11
- Pages
- 33061-33068
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4361987556
- Semantic Scholar
- 00cf5823372d502b5d81f35177757c06201d458d
- Cited by
- 124
- References
- 36
- Citations by year
- 2026: 29 · 2025: 56 · 2024: 32 · 2023: 7
- FWCI
- 29.96
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- IEEE Access (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Natural Science Foundation of China (61772252)
- Natural Science Foundation of Liaoning Province of China (2019-MS-216)
- Scientific Research Foundation of the Education Department of Liaoning Province (LJKZ0965)
- Huzhou Science and Technology Plan Project (2022GZ08)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- multimodal emotion recognition, EEG signals, facial expressions, feature-level fusion, attention mechanism, convolutional neural networks, valence and arousal classification, DEAP dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.