Multimodal Emotion Recognition from Eye Image, Eye Movement and EEG Using Deep Neural Networks
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
In consideration of the complexity of recording electroencephalography(EEG), some researchers are trying to find new features of emotion recognition. In order to investigate the potential of eye tracking glasses for multimodal emotion recognition, we collect and use eye images to classify five emotions along with eye movements and EEG. We compare four combinations of the three different types of data and two kinds of fusion methods, feature level fusion and Bimodal Deep AutoEncoder (BDAE). According to the three-modality fusion features generated by BDAE, the best mean accuracy of 79.63% is achieved. By analyzing the confusion matrices, we find that the three modalities can provide complementary information for recognizing five emotions. Meanwhile, the experimental results indicate that the classifiers with eye image and eye movement fusion features can achieve a comparable classification accuracy of 71.99%.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 5,839
Zheng, W.-L., & Lu, B.-L. (2015) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2015.2431497 · Cited by 2,595
Self-citationDifferential entropy feature for EEG-based emotion classification
Duan, R.-N., Zhu, J.-Y., & Lu, B.-L. (2013) · 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER) · doi:10.1109/ner.2013.6695876 · Cited by 1,100
Self-citationFacial Expression Recognition via a Boosted Deep Belief Network
Liu, P., Han, S., Meng, Z., & Tong, Y. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.233 · Cited by 650
Eye movement assessment of selective attentional capture by emotional pictures.
Nummenmaa, L., Hyönä, J., & Calvo, M. G. (2006) · Emotion · doi:10.1037/1528-3542.6.2.257 · Cited by 454 · cited 1×
“experiments and demonstrated the existence of correlations between eye movements and emotional states [5].”
BackgroundEEG-based emotion recognition during watching movies
Nie, D., Wang, X.-W., Shi, L.-C., & Lu, B.-L. (2011) · 2011 5th International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner.2011.5910636 · Cited by 360
Self-citationEmotion Recognition Using Multimodal Deep Learning
Liu, W., Zheng, W.-L., & Lu, B.-L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46672-9_58 · Cited by 301
Self-citationCombining eye movements and EEG to enhance emotion recognition
Lu, Y., Zheng, W., Li, B., & Lu, B. (2015) · Cited by 282
Self-citation
Cited by
Review of Eye Tracking Metrics Involved in Emotional and Cognitive Processes
Skaramagkas, V., Giannakakis, G., Ktistakis, E., Manousos, D., Karatzanis, I., Tachos, N., Tripoliti, E., Marias, K., Fotiadis, D. I., & Tsiknakis, M. (2023) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2021.3066072 · Cited by 419
Emotion recognition in EEG signals using deep learning methods: A review
Jafari, M., Shoeibi, A., Khodatars, M., Bagherzadeh, S., Shalbaf, A., García, D. L., Gorriz, J. M., & Acharya, U. R. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107450 · Cited by 240
EEG-Based Multimodal Emotion Recognition: A Machine Learning Perspective
Liu, H., Lou, T., Zhang, Y., Wu, Y., Xiao, Y., Jensen, C. S., & Zhang, D. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3369130 · Cited by 175
Lan, Y.-T., Liu, W., & Lu, B. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207625 · Cited by 106
Self-citationEye-Tracking Feature Extraction for Biometric Machine Learning
Lim, J. Z., Mountstephens, J., & Teo, J. (2022) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2021.796895 · Cited by 97
Liu, Y., Jia, Z., & Wang, H. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612277 · Cited by 88
Research Progress of EEG-Based Emotion Recognition: A Survey
Wang, Y., Zhang, B., & Di, L. (2024) · ACM Computing Surveys · doi:10.1145/3666002 · Cited by 87
Multimodal Emotion Recognition Using a Hierarchical Fusion Convolutional Neural Network
Zhang, Y., Cheng, C., & Zhang, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3049516 · Cited by 84
A new approach for product evaluation based on integration of EEG and eye-tracking
Zhu, S., Qi, J., Hu, J., & Hao, S. (2022) · Advanced Engineering Informatics · doi:10.1016/j.aei.2022.101601 · Cited by 83
Emerging Wearable Biosensor Technologies for Stress Monitoring and Their Real-World Applications
Wu, J., Ching, C. T., Wang, H., & Liao, L. (2022) · Biosensors · doi:10.3390/bios12121097 · Cited by 62
Zhu, F., Zhang, J., Dang, R., Hu, B., & Wang, Q. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106996 · Cited by 61
Generative technology for human emotion recognition: A scoping review
Ma, F., Yuan, Y., Xie, Y., Ren, H., Liu, I., He, Y., Ren, F., Yu, F. R., & Ni, S. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102753 · Cited by 61
Vortmann, L.-M., Ceh, S. M., & Putze, F. (2022) · Frontiers in Computer Science · doi:10.3389/fcomp.2022.780580 · Cited by 56
Bao, J., Tao, X., & Zhou, Y. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3221128 · Cited by 52
Chen, H., Li, Z., Jin, M., & Li, J. (2021) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc46164.2021.9630277 · Cited by 48
Wu, C., Gong, X., Luo, L., Zhao, Q., Hu, S., Mou, Y., & Jing, B. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.738959 · Cited by 45
Zheng, H., Zhou, T., Han, T., Li, S., & Yu, C. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102683 · Cited by 32
Mou, Y., Jing, B., Li, Y.-C., Fang, N., & Wu, C. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.1066164 · Cited by 29
Toward the Construction of Affective Brain-Computer Interface: A Systematic Review
Chen, H., Li, J., He, H., Zhu, J., Sun, S., Li, X., & Hu, B. (2025) · ACM Computing Surveys · doi:10.1145/3712259 · Cited by 26
Ma, Z., Li, A., Tang, J., Zhang, J., & Yin, Z. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110004 · Cited by 25
Hybrid multi-modal emotion recognition framework based on InceptionV3DenseNet
Alamgir, F. M., & Alam, M. S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15066-w · Cited by 24
Supervised Contrastive Learning for Affect Modelling
Pinitas, K., Makantasis, K., Liapis, A., & Yannakakis, G. N. (2022) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3536221.3556584 · Cited by 22
E-MFNN: an emotion-multimodal fusion neural network framework for emotion recognition
Guo, Z., Yang, M., Lin, L., Li, J., Zhang, S., He, Q., Gao, J., Meng, H., Chen, X., Tao, Y., et al. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1977 · Cited by 20
Multimodal Emotion Recognition Using a Modified Dense Co-Attention Symmetric Network
Zhao, Z.-W., Liu, W., & Lu, B.-L. (2021) · 2021 10th International IEEE/EMBS Conference on Neural Engineering (NER) · doi:10.1109/ner49283.2021.9441352 · Cited by 19
Self-citationCan Brain Signals Reveal Inner Alignment with Human Languages?
Qiu, J., Han, W., Zhu, J., Xu, M., Weber, D. J., Li, B., & Zhao, D. X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.120 · Cited by 19
Du, G., Zeng, Y., Su, K., Li, C., Wang, X., Teng, S., Li, D., & Liu, P. X. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3204940 · Cited by 19
Building a Behavioral Profile and Assessing the Skill of Video Game Players
Lange, A., Somov, A., Stepanov, A., & Burnaev, E. V. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3127083 · Cited by 17
eSports Players Professional Level and Tiredness Prediction using EEG and Machine Learning
Melentev, N., Somov, A., Burnaev, E. V., Strelnikova, I., Strelnikova, G. V., Melenteva, E., & Menshchikov, A. (2020) · Proceedings of IEEE Sensors/Proceedings of IEEE Sensors ... · doi:10.1109/sensors47125.2020.9278704 · Cited by 15
Wang, Y., Liu, J.-W., Lu, B., & Zheng, W. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3524418 · Cited by 13
Self-citationChen, X., Wang, X., Fang, C., Fang, L., Gong, W., Liu, C., & Wang, S. J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713571 · Cited by 13
A Review of AI in Human-Machine Cooperation: Machine Perspective
Sourav, M. S. G., Javaid, A. Y., & Cheng, L. (2025) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3774318 · Cited by 13
Machine Learning Human Behavior Detection Mechanism Based on Python Architecture
Jinnuo, Z., Goyal, S. B., Verma, C., Raboacă, M. S., & Mihălţan, T. C. (2022) · Mathematics · doi:10.3390/math10173159 · Cited by 13
PSPN: Pseudo-Siamese Pyramid Network for multimodal emotion analysis
Yin, Y., Kong, W., Tang, J., Li, J., & Babiloni, F. (2024) · Cognitive Neurodynamics · doi:10.1007/s11571-024-10123-y · Cited by 12
Orthogonal-Moment-Based Attraction Measurement With Ocular Hints in Video-Watching Task
Yang, M., Feng, X., Ma, R., Li, X., & Mao, C. (2023) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2023.3268505 · Cited by 12
Zhou, Y., Lu, H., Wan, X., Dong, W., & Xie, X. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108209 · Cited by 11
A Survey of EEG-Based Approaches to Classroom Attention Assessment in Education
Wei, L., Yu, Y., Qin, Y., & Zhang, S. (2025) · Information · doi:10.3390/info16100860 · Cited by 11
Collins, M. L., & Davies, T. C. (2023) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc40787.2023.10340178 · Cited by 11
Ramaswamy, M. P. A., & Palaniswamy, S. (2024) · IEEE Access · doi:10.1109/access.2024.3506157 · Cited by 9
Recognizing Human Emotions from Eyes and Surrounding Features: A Deep Learning Approach
Shuvo, M. N. R., Islam, M. A., Shamsojjaman, M., Akter, S., Hasan, S. M. N., & Khatun, T. (2021) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2021.0120346 · Cited by 9
Application of Combined Brain Computer Interface and Eye Tracking
Wang, P., Song, Z., Chen, H., Fang, T., Zhang, Y., Zhang, X., Wang, S., Li, H., Lin, Y., Jia, J., et al. (2021) · International Winter Conference on Brain-Computer Interface (BCI) · doi:10.1109/bci51272.2021.9385344 · Cited by 9
Multimodal Emotion Recognition System Using Machine Learning and Psychological Signals: A Review
Rishu, Singh, J., & Gill, R. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-981-16-1740-9_54 · Cited by 9
Eye-blinking analysis as a marker of emotional states
Goshvarpour, A., & Goshvarpour, A. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11304-1 · Cited by 9
Exploring Emotions in EEG: Deep Learning Approach with Feature Fusion
Mridula, D. T., Ferdaus, A. A., & Pias, T. S. (2023) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit60459.2023.10441204 · Cited by 9
Interactive EEG Emotion Recognition with Incremental Gaussian Processes
Ping, X., & Huang, W. (2025) · International Journal of Neural Systems · doi:10.1142/s0129065725500418 · Cited by 8
Li, J., Zhou, Y., Liu, T., Jung, T., Wan, X., Duan, D., Li, D., Yu, H., Song, H., Dong, X., et al. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125089 · Cited by 7
Eye-Tracking as a Component of Multimodal Emotion Recognition Systems
Celniak, W., & Augustyniak, P. (2022) · Advances in intelligent systems and computing · doi:10.1007/978-3-031-09135-3_6 · Cited by 7
EEG Signal-Based Eye Blink Classifier Using Convolutional Neural Network For BCI Systems
Ngo, B.-V., Nguyen, T.-H., & Nguyen, T.-N. (2021) · International Conference on Advanced Computing and Applications (ACOMP) · doi:10.1109/acomp53746.2021.00031 · Cited by 7
Brandstetter, J., Knoch, E.-M., & Gauterin, F. (2025) · Sensors · doi:10.3390/s25227028 · Cited by 6
Can Brain Signals Reveal Inner Alignment with Human Languages?Preprint
Han, W., Qiu, J., Zhu, J., Xu, M., Weber, D. J., Li, B., & Zhao, D. X. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.06348 · Cited by 6
Differentially Private Multimodal Laplacian Dropout (DP-MLD) for EEG representative learning
Fu, X., Wang, B., Guo, X., Liu, G., & Xiang, Y. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107657 · Cited by 5
Transformer-Based Bimodal Emotion Recognition: Signal Measurement and Transformer Fusion Approach
Soleimani, S., & Osman, H. A. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3615268 · Cited by 5
MSNet: Multimodal Self-attention Network for Depression Detection via Fusion of Eye Tracking and EEG
Zhu, F., Wu, B., Huo, Y., Dang, R., Hu, B., & Wang, Q. (2025) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/3715669.3723106 · Cited by 4
Leveraging Affective Friction to Improve Online Creative Collaboration: An Experimental Design
Saigot, M. (2022) · Lecture notes in information systems and organisation · doi:10.1007/978-3-031-13064-9_25 · Cited by 4
Korotin, A. A., Stepanov, A., Lange, A., Nikolaev, D., Abramov, S., Klyuchnikov, N., Burnaev, E. V., & Somov, A. (2021) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-76063-2_28 · Cited by 3
Xin, T., Wang, J., Jin, X., Ning, X., Feng, Z., & Lin, Y. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755354 · Cited by 3
Sarkar, A., Chatterjee, I., Dhar, A., Das, J., Roy, P., Ghosh, P., & Das, S. (2024) · International Conference on Computer, Electrical & Communication Engineering (ICCECE) · doi:10.1109/iccece58645.2024.10497244 · Cited by 3
Deeply Coupling EEG Signals and Eye Movements for Multi-Modal and Region-Aware Emotion Recognition
Chen, Y., Gao, Y., Fu, X., He, H., Ouyang, T., Chen, S., & Fu, X. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888584 · Cited by 3
Human Emotion Recognition using Polar-Based Lagged Poincare Plot Indices of Eye-Blinking Data
Goshvarpour, A., & Goshvarpour, A. (2021) · International Journal of Computational Intelligence and Applications · doi:10.1142/s1469026821500231 · Cited by 3
Surprise Emotion Recognition of Pilots inside the Simulator Based on Multi-view Learning
Zhang, J., Wu, D., Chen, D., Shang, Y., Zhang, W., & Wei, F. (2022) · IEEE International Conference on Civil Aviation Safety and Information Technology (ICCASIT) · doi:10.1109/iccasit55263.2022.9986697 · Cited by 3
Deep Learning Architectures for Enhanced Emotion Recognition from EEG and Facial Expressions
Soleimani, S. (2024) · uO Research (University of Ottawa) · doi:10.20381/ruor-30202 · Cited by 3
DV Emotion Net: An Integrated Multimodal Approach for Emotion Recognition
Dommeti, D., Nallapati, S. R., & Alfaris, R. (2024) · Open access journal of criminology investigation & justice. · doi:10.23880/oajcij-16000112 · Cited by 2
Exploring Emotions in EEG: Deep Learning Approach with Feature FusionPreprint
Mridula, D. T., Ferdaus, A. A., & Pias, T. S. (2023) · medRxiv · doi:10.1101/2023.11.17.23298680 · Cited by 2
The role of teachers’ emotions in students’ outcomes: from the perspective of interpersonal emotions
(2023) · Frontiers research topics · doi:10.3389/978-2-83251-074-2 · Cited by 1
Preference detection of the humanoid robot face based on EEG and eye movement
Wang, P., Mu, W., Zhan, G., Wang, A., Song, Z., Fang, T., Zhang, X., Wang, J., Niu, L., Bin, J., et al. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09765-0 · Cited by 1
Feature-Based Emotion Classification Using Eye-Tracking Data
(2024) · Slovenian Conference on Artificial Intelligence · doi:10.70314/is.2024.scai.9988 · Cited by 1
A Review of Machine Learning-Based Assessment of Depression
Wang, Z., Cai, Z., Dong, S., & Hei, W. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-3755-3_15 · Cited by 1
Markerless emotion recognition from full-body movements for Social XR
Neri, M., Baldoni, S., Carli, M., & Battisti, F. (2026) · Signal Processing Image Communication · doi:10.1016/j.image.2026.117489 · Cited by 1
Pillalamarri, R., & Shanmugam, U. (2026) · Journal on Information Security · doi:10.1186/s13635-026-00230-0 · Cited by 1
Recognizing Polychronic-Monochronic Tendency of Individuals Using Eye Tracking and Machine Learning
Barth, S., Langner, M., Toreini, P., & Maedche, A. (2022) · Lecture notes in information systems and organisation · doi:10.1007/978-3-031-13064-9_9 · Cited by 1
Quan, X., & Wu, D. (2023) · Wiley eBooks · doi:10.1002/9781119863663.ch33 · Cited by 1
EmoGO: Emotion Estimation on Gazed Object by Using Mobile Eye-Tracker
Karasawa, T., Hamanaka, S., Kume, T., Sasaki, W., Nakazawa, J., & Okoshi, T. (2023) · International Conference on the Internet of Things (IoT) · doi:10.1145/3627050.3630729 · Cited by 1
Sowmiya, S. R., & Fatima, N. S. (2024) · International Conference on Inventive Computation Technologies (ICICT) · doi:10.1109/icict60155.2024.10544473 · Cited by 1
Fine-grained Dual-space Context Analysis Network for Multimodal Emotion Recognition
Li, J., Tang, J., Kong, W., & Ying, Y. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651517 · Cited by 1
Fındık, H., Kaim, M., Uzun, F., Kanat, A., Keleş, O. N., & Aydın, M. D. (2025) · Life · doi:10.3390/life15040536 · Cited by 1
Differentially Private Multimodal Laplacian Dropout (Dp-Mld) for Eeg Representative LearningPreprint
Fu, X., Wang, B., Guo, X., Liu, G., & Xiang, Y. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4990230 · Cited by 1
Jin, D., Yu, W., Yang, X., Wang, H., & Peng, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17615-9_17
(2023) · International Research Journal of Modernization in Engineering Technology and Science · doi:10.56726/irjmets-ncascte202216
Evaluating The Usability of Massive Open Online Course User Interface Based on Eye-Tracker Data
Herlina, H., & Nugraha, K. A. (2023) · International Conference on Intelligent Cybernetics Technology & Applications (ICICyTA) · doi:10.1109/icicyta60173.2023.10429001
Exploring Smartphone Shells Based on Mental Health Monitoring Needs
Wan, R., & Li, S. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia240077
Jiao, Y., & He, X. (2025) · Computer Methods in Biomechanics & Biomedical Engineering · doi:10.1080/10255842.2025.2456996
Souza, A. P. S. D. (2023) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas)
Learning Rule-based Decision-Making Systems from Heterogeneous Longitudinal Data
Shakur, A. H. (2023) · ResearchWorks at the University of Washington (University of Washington)
Касов, А. М., & Kasov, A. M. (2024) · Electronic scientific archive of UrFU (Ural Federal University)
Decoding Emotions: Single-Signal and Fusion-based Models using Temporal Neural NetworksPreprint
Faezi, S., Mahmoudi, D., Salehpour, P., Aghdasi, H. S., Shirmarz, A., Khooyooz, S., Wei, Y., TaheriNejad, N., & Haghi, M. (2025) · doi:10.36227/techrxiv.176583431.12811489/v1
Wen, S., Liu, Q., Wang, X., Chen, K., Ju, J., Ding, J., Min, B., & Zhao, X.-M. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6876239
A Hybrid Multimodal Approach to Emotion Recognition with Spiking Neural Networks and Transformers
Babydasan, A., Khanolkar, A., & Indolia, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-26352-0_37
Zhu, F., Huo, Y., Rong, Y., Jiang, Y., Dang, R., Hu, B., & Huang, Y. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111340
Emotion detection using eye-gaze in both typically developed and clinical populations.
Collins, M. L. (2026) · QSpace (Queen's University Library)
Liu, Y., Liu, B., & Guo, X. (2026) · Frontiers of Urban and Rural Planning · doi:10.1007/s44243-026-00099-0
CP-Net: A Cognitive-Guided Multimodal Approach for Cross-subject Emotion Recognition
Liu, C., Yuan, M., Huang, F., Hu, B., & Tao, X. (2026) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3767308.3836444
Metadata
- Authors
- Guo, J.-J., Zhou, R., Zhao, L.-M., & Lu, B.-L.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Venue
- 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
- Publisher
- IEEE
- Published
- 2019-07
- Volume
- 2019
- Pages
- 3071-3074
- ISSN
- 2375-7477 (ISSN-L)
- OpenAlex
- W2979656641
- Cited by
- 73
- References
- 17
- Citations by year
- 2026: 4 · 2025: 13 · 2024: 16 · 2023: 12 · 2022: 13 · 2021: 13 · 2020: 2
- FWCI
- 8.83
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Keywords
- multimodal emotion recognition, eye-tracking glasses, eye images, eye movements, EEG, feature-level fusion, emotion classification, data fusion
- MeSH terms
- Electroencephalography, Emotions, Eye Movements, Humans, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.