Core-Brain-Network-Based Multilayer Convolutional Neural Network for Emotion Recognition
The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
Abstract
In this article, we propose a method for emotion classification based on multilayer convolutional neural network (MCNN) and combining differential entropy (DE) and brain network. First, we use continuous wavelet transform (CWT) to get the time-frequency representation (TFR) of electroencephalogram (EEG) signals on each channel and extract rich information from different frequency bands for subsequent analysis. Brain networks are then constructed in multiple bands to characterize the spatial connections hidden in the multichannel EEG signals. Based on brain networks, we further develop core brain networks through a set of key nodes determined by DE. These core brain networks are associated with brain activities and differ markedly between different emotional states. The final designed MCNN model takes DE features and core brain networks as inputs for emotion recognition. We evaluate our method on the SJTU emotion EEG dataset, and the average accuracy of 15 subjects achieves 91.45%. Utilizing the complementary features of DE and brain network, the proposed method provides an efficient framework for accurate emotion recognition from EEG signals.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
1 retracted
Bhattacharyya, A., Sharma, M., Pachori, R. B., Sircar, P., & Acharya, U. R. (2018) · Neural Computing and Applications · doi:10.1007/s00521-016-2646-4 · Cited by 204
RetractedBackpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
Zheng, W.-L., & Lu, B.-L. (2015) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2015.2431497 · Cited by 2,595
EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks
Song, T., Zheng, W., Song, P., & Cui, Z. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2817622 · Cited by 1,597
Emotions Recognition Using EEG Signals: A Survey
Alarcao, S. M., & Fonseca, M. J. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2714671 · Cited by 1,201
Differential entropy feature for EEG-based emotion classification
Duan, R.-N., Zhu, J.-Y., & Lu, B.-L. (2013) · 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER) · doi:10.1109/ner.2013.6695876 · Cited by 1,100
Hinton, G. (2009) · Scholarpedia · doi:10.4249/scholarpedia.5947 · Cited by 885
Emotional state classification from EEG data using machine learning approach
Wang, X.-W., Nie, D., & Lu, B.-L. (2014) · Neurocomputing · doi:10.1016/j.neucom.2013.06.046 · Cited by 755
Brain–computer interfaces: communication and restoration of movement in paralysis
Birbaumer, N., & Cohen, L. G. (2007) · The Journal of Physiology · doi:10.1113/jphysiol.2006.125633 · Cited by 754
Emotion Recognition From EEG Using Higher Order Crossings
Petrantonakis, P. C., & Hadjileontiadis, L. J. (2010) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2009.2034649 · Cited by 687
Truong, N. D., Nguyen, A. D., Kuhlmann, L., Bonyadi, M. R., Yang, J., Ippolito, S., & Kavehei, O. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.04.018 · Cited by 635
Spatial–Temporal Recurrent Neural Network for Emotion Recognition
Zhang, T., Zheng, W., Cui, Z., Zong, Y., & Li, Y. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2788081 · Cited by 595
EEG-Based Spatio–Temporal Convolutional Neural Network for Driver Fatigue Evaluation
Gao, Z., Wang, X., Yang, Y., Mu, C., Cai, Q., Dang, W., & Zuo, S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2886414 · Cited by 482
Self-citationEEG-based emotion classification using deep belief networks
Zheng, W.-L., Zhu, J.-Y., Peng, Y., & Lu, B.-L. (2014) · 2014 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2014.6890166 · Cited by 367
A real-time automated system for the recognition of human facial expressions
Anderson, K., & McOwan, P. W. (2006) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2005.854502 · Cited by 355
Emotion Recognition Using Multimodal Deep Learning
Liu, W., Zheng, W.-L., & Lu, B.-L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46672-9_58 · Cited by 301
Jadhav, P., Rajguru, G., Datta, D., & Mukhopadhyay, S. (2020) · Biocybernetics and Biomedical Engineering · doi:10.1016/j.bbe.2020.01.010 · Cited by 181
APPROACHING AUTOMATIC RECOGNITION OF EMOTION FROM VOICE: A ROUGH BENCHMARK
McGilloway, S., Cowie, R., Douglas‐Cowie, E., Gielen, S. C. A. M., Westerdijk, M., & Stroeve, S. (2000) · Cited by 167
Damping identification using a continuous wavelet transform: application to real data
Slavič, J., Simonovski, I., & Boltežar, M. (2003) · Journal of Sound and Vibration · doi:10.1016/s0022-460x(02)01032-5 · Cited by 165
Recognizing Emotions Evoked by Music Using CNN-LSTM Networks on EEG Signals
Sheykhivand, S., Mousavi, Z., Rezaii, T. Y., & Farzamnia, A. (2020) · IEEE Access · doi:10.1109/access.2020.3011882 · Cited by 157
Structure out of chaos: Functional brain network analysis with EEG, MEG, and functional MRI
van Straaten, E. C. W., & Stam, C. J. (2013) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2012.10.010 · Cited by 154
Wang, X., Zhang, T., Xu, X., Chen, L., Xing, X., & Chen, C. L. P. (2018) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2018.8621147 · Cited by 132
Classification of Human Emotions from EEG Signals using Statistical Features and Neural Network
Chai, T.-Y., San, W. S., Seong, T. C., & Rizon, M. (2009) · International Journal of Integrated Engineering · Cited by 127
Asghar, M. A., Khan, M. J., Fawad, Amin, Y., Rizwan, M., Rahman, M., Badnava, S., & Mirjavadi, S. S. (2019) · Sensors · doi:10.3390/s19235218 · Cited by 104
Bonita, J. D., Ambolode, L. C. C., Rosenberg, B. M., Cellucci, C. J., Watanabe, T. A. A., Rapp, P. E., & Albano, A. M. (2014) · Cognitive Neurodynamics · doi:10.1007/s11571-013-9267-8 · Cited by 95
Classification of EEG Signals on VEP-Based BCI Systems With Broad Learning
Gao, Z., Dang, W., Liu, M., Guo, W., Ma, K., & Chen, G. (2020) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2020.2964684 · Cited by 75
Self-citationWavelet Multiresolution Complex Network for Analyzing Multivariate Nonlinear Time Series
Gao, Z., Li, S., Dang, W., Yang, Y., Do, Y., & Grebogi, C. (2017) · International Journal of Bifurcation and Chaos · doi:10.1142/s0218127417501231 · Cited by 64
Self-citationHan, C., Sun, X., Yang, Y., Che, Y., & Qin, Y. (2019) · Entropy · doi:10.3390/e21040353 · Cited by 64
Self-citationA Coincidence-Filtering-Based Approach for CNNs in EEG-Based Recognition
Gao, Z., Li, Y., Yang, Y., Dong, N., Yang, X., & Grebogi, C. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2955447 · Cited by 64
Self-citationIntegrating Facial Expression and Body Gesture in Videos for Emotion Recognition
Yan, J., Zheng, W., Minhai, X. I. N., & Yan, J. (2014) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.e97.d.610 · Cited by 42
Zhang, J., Liu, W., & Zhang, Z. (2018) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2018.2851422 · Cited by 40
Makkuva, A. V., & Wu, Y. (2018) · IEEE Transactions on Information Theory · doi:10.1109/tit.2018.2815687 · Cited by 39
Rhythm-Dependent Multilayer Brain Network for the Detection of Driving Fatigue
Dang, W., Gao, Z., Lv, D., Sun, X., & Cheng, C. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3008229 · Cited by 38
Self-citationEEG-based emotion recognition using discriminative graph regularized extreme learning machine
Zhu, J.-Y., Zheng, W., Peng, Y. K., Duan, R.-N., & Lu, B. (2014) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2014.6889618 · Cited by 33
Children Emotion Regulation: Development of Neural Marker by Investigating Human Brain Signals
Mehmood, R. M., Yang, H.-J., & Kim, S. (2020) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3011817 · Cited by 32
Eye movement based emotion recognition using electrooculography
Soundariya, R. S., & Renuga, R. (2017) · Innovations in Power and Advanced Computing Technologies (i-PACT) · doi:10.1109/ipact.2017.8245212 · Cited by 29
Yang, Y., & Gao, Z. (2020) · International Journal of Bifurcation and Chaos · doi:10.1142/s0218127420501187 · Cited by 18
Self-citationSimple Convolutional Neural Network for Left-Right Hands Motor Imagery EEG Signals Classification
Tian, G., & Liu, Y. (2019) · International Journal of Cognitive Informatics and Natural Intelligence · doi:10.4018/ijcini.2019070103 · Cited by 16
Juárez-Castillo, E., Acosta‐Mesa, H., Fernández-Ruíz, J., & Cruz-Ramírez, N. (2017) · Intelligent Data Analysis · doi:10.3233/ida-170881 · Cited by 6
Cited by
Bearing fault diagnosis via generalized logarithm sparse regularization
Zhang, Z., Huang, W., Liao, Y., Song, Z., Shi, J., Jiang, X., Shen, C., & Zhu, Z. (2022) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2021.108576 · Cited by 243
Emotion recognition in EEG signals using deep learning methods: A review
Jafari, M., Shoeibi, A., Khodatars, M., Bagherzadeh, S., Shalbaf, A., García, D. L., Gorriz, J. M., & Acharya, U. R. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107450 · Cited by 240
Multimodal Emotion Recognition From EEG Signals and Facial Expressions
Wang, S., Qu, J., Zhang, Y., & Zhang, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3263670 · Cited by 124
“Recently, the research of emotion recognition based on EEG has also begun to use deep learning models to extract features [7], [19].”
BackgroundMethodInfluentialTC-Net: A Transformer Capsule Network for EEG-based emotion recognition
Wei, Y., Liu, Y., Li, C., Cheng, J., Song, R., & Chen, X. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106463 · Cited by 138
Vempati, R., & Sharma, L. D. (2023) · Results in Engineering · doi:10.1016/j.rineng.2023.101027 · Cited by 99
Metadata
- Authors
- Gao, Z., Li, R., Ma, C., Rui, L., & Sun, X.
- Affiliations
- Tianjin University (CN)
- Venue
- IEEE Transactions on Instrumentation and Measurement
- Published
- 2021
- Volume
- 70
- Pages
- 1-9
- ISSN
- 0018-9456 (ISSN-L)
- 1557-9662
- OpenAlex
- W3176984140
- Cited by
- 61
- References
- 40
- Citations by year
- 2026: 3 · 2025: 8 · 2024: 11 · 2023: 18 · 2022: 7 · 2021: 14
- FWCI
- 12.08
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Instrumentation and Measurement (publishedVersion)
- Funders
- National Natural Science Foundation of China (61922062, 61903270, 61873181)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Physical Sciences › Engineering › Biomedical Engineering › Muscle activation and electromyography studies
- Keywords
- emotion recognition, EEG, brain networks, differential entropy, continuous wavelet transform, time-frequency representation, frequency bands, key nodes, emotion classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.