A Multi-Domain Adaptive Graph Convolutional Network for EEG-based Emotion Recognition
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
Among all solutions of emotion recognition tasks, electroencephalogram (EEG) is a very effective tool and has received broad attention from researchers. In addition, information across multimedia in EEG often provides a more complete picture of emotions. However, few of the existing studies concurrently incorporate EEG information from temporal domain, frequency domain and functional brain connectivity. In this paper, we propose a Multi-Domain Adaptive Graph Convolutional Network (MD-AGCN), fusing the knowledge of both the frequency domain and the temporal domain to fully utilize the complementary information of EEG signals. MD-AGCN also considers the topology of EEG channels by combining the inter-channel correlations with the intra-channel information, from which the functional brain connectivity can be learned in an adaptive manner. Extensive experimental results demonstrate that our model exceeds state-of-the-art methods in most experimental settings. At the same time, the results show that MD-AGCN could extract complementary domain information and exploit channel relationships for EEG-based emotion recognition effectively.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“φ (1 × 1) in Figure 3 corresponds to the weight vectorW in Equation 3, and the residual branch [6] achieved by the 1 × 1 convolution operation is introduced to ensure the stability of the network.”
BackgroundHämäläinen, M., Hari, R., Ilmoniemi, R. J., Knuutila, J., & Lounasmaa, O. V. (1993) · Reviews of Modern Physics · doi:10.1103/revmodphys.65.413 · Cited by 4,684 · cited 1×
“It has been proven that EEG signals have the ability to reveal delicate changes in emotions with high time resolution [5], which is more objective and precise when analyzing emotion states [37].”
BackgroundZheng, W.-L., & Lu, B.-L. (2015) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2015.2431497 · Cited by 2,595 · cited 7×
“• DBN[36]: The deep belief network is a probabilistic generative model with a deep architecture.”
BackgroundMethodInfluentialSelf-citationMauss, I. B., & Robinson, M. D. (2009) · Cognition and Emotion · doi:10.1080/02699930802204677 · Cited by 2,078 · cited 1×
“Mauss [19] proposed that emotional procedures should be recognized as involving distributed circuits instead of specific brain regions in isolation.”
BackgroundTwo-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition
Shi, L., Zhang, Y., Cheng, J., & Lu, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01230 · Cited by 2,071 · cited 2×
“[24] proposed an adaptive graph convolutional network for skeleton-based action recognition that achieves significant improvement in the field by adding adaptive graph convolutional layers and second-order information.”
BackgroundMethodEEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks
Song, T., Zheng, W., Song, P., & Cui, Z. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2817622 · Cited by 1,597 · cited 7×
“For SparseDGCNN [31], they perform leave-one-clip-out validation which take only one clip as test set.”
BackgroundMethodInfluentialEmotionMeter: A Multimodal Framework for Recognizing Human Emotions
Zheng, W.-L., Liu, W., Lu, Y., Lu, B.-L., & Cichocki, A. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2797176 · Cited by 1,276 · cited 4×
“Model SEED[36] SEED-IV[35] SEED-V[12] SVM[12, 36, 38] 83.”
MethodInfluentialSelf-citationDifferential entropy feature for EEG-based emotion classification
Duan, R.-N., Zhu, J.-Y., & Lu, B.-L. (2013) · 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER) · doi:10.1109/ner.2013.6695876 · Cited by 1,100 · cited 1×
“Due to the non-stationary characteristics of EEG signals as well as the interference of noise and artifacts of raw EEG data, frequency domain features such as Power Spectral Density (PSD) [4] and Differential Entropy (DE) [3] , and time-frequency domain features such as Hilbert-Huang Spectrum (HHS) [8] have shown good…”
BackgroundSelf-citationFeature Extraction and Selection for Emotion Recognition from EEG
Jenke, R., Peer, A., & Buss, M. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2339834 · Cited by 1,081 · cited 2×
“In the time domain, the most widely used EEG features are Hjorth Features, Fractal Dimension (FD) and Higher Order Crossings (HOC) [8].”
BackgroundMethodIdentifying Stable Patterns over Time for Emotion Recognition from EEG
Zheng, W.-L., Zhu, J.-Y., & Lu, B.-L. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2712143 · Cited by 990 · cited 2×
“It has been proven that EEG signals have the ability to reveal delicate changes in emotions with high time resolution [5], which is more objective and precise when analyzing emotion states [37].”
BackgroundSelf-citationEEG-Based Emotion Recognition Using Regularized Graph Neural Networks
Zhong, P., Wang, D., & Miao, C. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2994159 · Cited by 910 · cited 10×
“[38] proposed a regularized graph neural network to investigate inter-channel relations for emotion recognition with pre-computed frequency features from EEG signals, which utilized the frequency domain information from EEG data.”
BackgroundMethodResultInfluentialEmotional state classification from EEG data using machine learning approach
Wang, X.-W., Nie, D., & Lu, B.-L. (2014) · Neurocomputing · doi:10.1016/j.neucom.2013.06.046 · Cited by 755 · cited 1×
“Wang et al. [26] implemented a support vector machine (SVM) in the frequency domain to identify different classes of emotions.”
BackgroundSelf-citationFrontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions
Schmidt, L. A., & Trainor, L. J. (2001) · Cognition and Emotion · doi:10.1080/0269993004200187 · Cited by 617 · cited 1×
“Among them, EEG signals are collected by placing electrodes on the scalp to measure voltage fluctuations from the cortex in the brain [22].”
BackgroundSpatial–Temporal Recurrent Neural Network for Emotion Recognition
Zhang, T., Zheng, W., Cui, Z., Zong, Y., & Li, Y. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2788081 · Cited by 595 · cited 3×
“[32] proposed a recurrent neural network (RNN) to learn spatial-temporal information from EEG signals.”
BackgroundMethodResting State Cortical Connectivity Reflected in EEG Coherence in Individuals With Autism
Murias, M., Webb, S. J., Greenson, J., & Dawson, G. (2007) · Biological Psychiatry · doi:10.1016/j.biopsych.2006.11.012 · Cited by 531 · cited 2×
“In addition, growing evidences [7, 20, 27, 30] have shown that functional brain connectivity is associated with multiple psychophysiological disorders of cognitive deficit disease.”
BackgroundEEG emotion recognition using fusion model of graph convolutional neural networks and LSTM
Yin, Y., Zheng, X., Hu, B., Zhang, Y., & Cui, X. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106954 · Cited by 466 · cited 1×
“[29] fused Poster Session 6 MM ’21, October 20–24, 2021, Virtual Event, China”
BackgroundLiu, W., Qiu, J.-L., Zheng, W.-L., & Lu, B.-L. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2021.3071170 · Cited by 452 · cited 11×
“Extensive experiments on the SEED, SEED-IV and SEED-V datasets demonstrate the extraordinary performance of our model in comparison with various competitive baseline models, especially on problems with more than three emotion classes.”
MethodInfluentialSelf-citationA Novel Bi-Hemispheric Discrepancy Model for EEG Emotion Recognition
Li, Y., Wang, L., Zheng, W., Zong, Y., Qi, L., Cui, Z., Zhang, T., & Song, T. (2021) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.2999337 · Cited by 369 · cited 3×
“• BiHDM[13]: The bi-hemispheres discrepancy model focuses on the asymmetric differences of the right and left hemispheres for EEG emotion recognition.”
BackgroundMethodAutocorrelation Function and Power Spectral Density of Laser-Produced Speckle Patterns
Goldfischer, L. I. (1965) · Journal of the Optical Society of America · doi:10.1364/josa.55.000247 · Cited by 313 · cited 1×
“Due to the non-stationary characteristics of EEG signals as well as the interference of noise and artifacts of raw EEG data, frequency domain features such as Power Spectral Density (PSD) [4] and Differential Entropy (DE) [3] , and time-frequency domain features such as Hilbert-Huang Spectrum (HHS) [8] have shown good…”
BackgroundClassifying Different Emotional States by Means of EEG-Based Functional Connectivity Patterns
Lee, Y.-Y., & Hsieh, S. (2014) · PLoS ONE · doi:10.1371/journal.pone.0095415 · Cited by 301 · cited 1×
“Pearson’s correlation coefficient [11] is used as the connectivity index.”
MethodBrain–machine interfaces from motor to mood
Shanechi, M. M. (2019) · Nature Neuroscience · doi:10.1038/s41593-019-0488-y · Cited by 263 · cited 1×
“Emotion recognition plays an important role in affective brain computer interface (aBCI) [23] and mental health assessment [2].”
BackgroundMultimodal emotion recognition using EEG and eye tracking data
Zheng, W.-L., Dong, B.-N., & Lu, B.-L. (2014) · 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society · doi:10.1109/embc.2014.6944757 · Cited by 230
Self-citationEmotion-dependent functional connectivity of the default mode network in adolescent depression
Ho, T. C., Connolly, C. G., Henje Blom, E., LeWinn, K. Z., Strigo, I. A., Paulus, M. P., Frank, G., Max, J. E., Wu, J., Chan, M., et al. (2015) · Biological Psychiatry · doi:10.1016/j.biopsych.2014.09.002 · Cited by 227 · cited 2×
“In addition, growing evidences [7, 20, 27, 30] have shown that functional brain connectivity is associated with multiple psychophysiological disorders of cognitive deficit disease.”
BackgroundSparseDGCNN: Recognizing Emotion From Multichannel EEG Signals
Zhang, G., Yu, M., Liu, Y.-J., Zhao, G., Zhang, D., & Zheng, W. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3051332 · Cited by 185 · cited 4×
“For SparseDGCNN [31], they perform leave-one-clip-out validation which take only one clip as test set.”
BackgroundMethodResultInfluentialJia, Z., Lin, Y., Cai, X., Chen, H., Gou, H., & Wang, J. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413724 · Cited by 167 · cited 3×
“[9] proposed an attention 3D dense network with fusing EEG features in the time domain and the frequency domain.”
BackgroundMethodResultA Novel Neural Network Model based on Cerebral Hemispheric Asymmetry for EEG Emotion Recognition
Li, Y., Zheng, W., Cui, Z., Zhang, T., & Zong, Y. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/216 · Cited by 147 · cited 1×
“• BiDANN[14]: The bi-hemispheres domain adversarial neural network investigates both the right and left sides of the hemisphere EEG features and maps them to discriminative feature spaces separately.”
BackgroundWhitton, A. E., Deccy, S., Ironside, M. L., Kumar, P., Beltzer, M., & Pizzagalli, D. A. (2018) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2017.07.001 · Cited by 107 · cited 2×
“[27] considered that the increase in the high frequency band based on the functional connection of EEG signals could be a neural model of the recurrent course of major depression.”
BackgroundEEG-based emotion recognition using Recurrence Plot analysis and K nearest neighbor classifier
Bahari, F., & Janghorbani, A. (2013) · National and International Iranian Conference on Biomedical Engineering (ICBME) · doi:10.1109/icbme.2013.6782224 · Cited by 92 · cited 1×
“[1] practiced a non-linear k nearest neighbor classifier (kNN) based on a recurrence plot.”
MethodLi, T.-H., Liu, W., Zheng, W.-L., & Lu, B.-L. (2019) · 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER) · doi:10.1109/ner.2019.8716943 · Cited by 75 · cited 6×
“Our proposed method is evaluated on three emotional EEG datasets (SEED [36], SEED-IV[35], and SEED-V[12, 16]).”
BackgroundMethodInfluentialSelf-citationIdentifying Functional Brain Connectivity Patterns for EEG-Based Emotion Recognition
Wu, X., Zheng, W., & Lu, B. (2019) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner.2019.8717035 · Cited by 75 · cited 3×
“Inspired by the effective method of extracting temporal EEG features [28], we construct the functional connectivity presented by the weighed adjacency matrix A in the temporal domain, as shown in Figure 2.”
BackgroundMethodInfluentialSelf-citationYin, Z., Li, J., Zhang, Y., Ren, A., Meneen, K. M. V., & Huang, L. (2016) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2016.08.013 · Cited by 71 · cited 2×
“[30] concluded that functional brain connections based on EEG signals are usually slower and inefficient in patients with schizophrenia.”
BackgroundDepression and implicit emotion processing: An EEG study
Bocharov, A. V., Knyazev, G. G., & Savostyanov, A. N. (2017) · Neurophysiologie Clinique · doi:10.1016/j.neucli.2017.01.009 · Cited by 57 · cited 2×
“More-over, many mental disorders are related to emotions, and thus accurately evaluating patients’ emotion states will enhance the treatment of neurological diseases [2].”
BackgroundZhao, L.-M., Li, R., Zheng, W.-L., & Lu, B.-L. (2019) · 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER) · doi:10.1109/ner.2019.8717055 · Cited by 54 · cited 11×
“Extensive experiments on the SEED, SEED-IV and SEED-V datasets demonstrate the extraordinary performance of our model in comparison with various competitive baseline models, especially on problems with more than three emotion classes.”
MethodInfluentialSelf-citation
Research works printed but not linked: 5
- Yann LeCun , Yoshua Bengio , and Geoffrey Hinton . 2015 . Deep learning . Nature , Vol. 521 , 7553 (2015), 436--444. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature, Vol. 521, 7553 (2015), 436--444.
- Yang Li , Wenming Zheng , Lei Wang , Yuan Zong , and Zhen Cui . 2019 b. From regional to global brain: A novel hierarchical spatial-temporal neural network model for EEG emotion recognition . IEEE Transactions on Affective Computing ( 2019 ). Yang Li, Wenming Zheng, Lei Wang, Yuan Zong, and Zhen Cui. 2019 b. From regional to global brain: A novel hierarchical spatial-temporal neural network model for EEG emotion recognition. IEEE Transactions on Affective Computing (2019).
- Yifei Lu , Wei-Long Zheng , Binbin Li , and Bao-Liang Lu . 2015 . Combining eye movements and EEG to enhance emotion recognition . In Twenty-Fourth International Joint Conference on Artificial Intelligence. Yifei Lu, Wei-Long Zheng, Binbin Li, and Bao-Liang Lu. 2015. Combining eye movements and EEG to enhance emotion recognition. In Twenty-Fourth International Joint Conference on Artificial Intelligence.
- P MatLab . 2018. 9.7. 0.1190202 (R2019b). MathWorks Inc Natick MA USA ( 2018 ). P MatLab. 2018. 9.7. 0.1190202 (R2019b). MathWorks Inc Natick MA USA (2018).
- 10.1080/02699930126048 · doi:10.1080/02699930126048 · Paper page
Cited by
Feng, L., Cheng, C., Zhao, M., Deng, H., & Zhang, Y. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3198688 · Cited by 164
“For the hand-crafted features, they can separate into time-domain features [16], [17], frequency-domain features [18], [19] and time-frequency domain features [20], [21].”
BackgroundPGCN: Pyramidal Graph Convolutional Network for EEG Emotion Recognition
Jin, M., Du, C., He, H., Cai, T., & Li, J. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3385676 · Cited by 190
“In the subject-dependent experiments, following the previous experimental settings [52], [53], we performed three-fold cross-validation.”
BackgroundMethodGraph Neural Network-Based EEG Classification: A Survey
Klepl, D., Wu, M., & He, F. (2024) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2024.3355750 · Cited by 185
Multimodal Emotion Recognition From EEG Signals and Facial Expressions
Wang, S., Qu, J., Zhang, Y., & Zhang, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3263670 · Cited by 124
“For example, Li et al. [8] used various deep learning models to extract features from EEG signals for emotion recognition.”
BackgroundMethodYe, M., Chen, C. L. P., & Zhang, T. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3225855 · Cited by 134
“Li et al. [49] developed MD-AGCN to learn the topology of EEG channels in an adaptive manner.”
Metadata
- Authors
- Li, R., Wang, Y., & Lu, B.-L.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Venue
- Proceedings of the 29th ACM International Conference on Multimedia
- Publisher
- ACM
- Published
- 2021-10-17
- Pages
- 5565-5573
- OpenAlex
- W3205895886
- Cited by
- 73
- References
- 32
- Citations by year
- 2026: 5 · 2025: 22 · 2024: 21 · 2023: 16 · 2022: 9
- FWCI
- 12.79
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Multimedia (ACM MM) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Physical Sciences › Engineering › Biomedical Engineering › Muscle activation and electromyography studies
- Keywords
- EEG-based emotion recognition, graph convolutional networks, frequency domain, time domain, brain functional connectivity, adaptive graph learning, inter-channel correlation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.