The DOI matches its Crossref record; 50 of the 51 references it cites were found in a registry or an index.
Data augmentation for enhancing EEG-based emotion recognition with deep generative models.
Open accessAbstract
Objective. The data scarcity problem in emotion recognition from electroencephalography (EEG) leads to difficulty in building an affective model with high accuracy using machine learning algorithms or deep neural networks. Inspired by emerging deep generative models, we propose three methods for augmenting EEG training data to enhance the performance of emotion recognition models. Approach. Our proposed methods are based on two deep generative models, variational autoencoder (VAE) and generative adversarial network (GAN), and two data augmentation ways, full and partial usage strategies. For the full usage strategy, all of the generated data are augmented to the training dataset without judging the quality of the generated data, while for the partial usage, only high-quality data are selected and appended to the training dataset. These three methods are called conditional Wasserstein GAN (cWGAN), selective VAE (sVAE), and selective WGAN (sWGAN). Main results. To evaluate the effectiveness of these proposed methods, we perform a systematic experimental study on two public EEG datasets for emotion recognition, namely, SEED and DEAP. We first generate realistic-like EEG training data in two forms: power spectral density and differential entropy. Then, we augment the original training datasets with a different number of generated realistic-like EEG data. Finally, we train support vector machines and deep neural networks with shortcut layers to build affective models using the original and augmented training datasets. The experimental results demonstrate that our proposed data augmentation methods based on generative models outperform the existing data augmentation approaches such as conditional VAE, Gaussian noise, and rotational data augmentation. We also observe that the number of generated data should be less than 10 times of the original training dataset to achieve the best performance. Significance. The augmented training datasets produced by our proposed sWGAN method significantly enhance the performance of EEG-based emotion recognition models.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Experiments show that two representations of EEG signals were suitable for the requirement of the artificial datasets that enhances the performance of the classifier.”
From Data Augmentation for Deep Neural Networks Model in EEG Classification Task: A Review (2021) · cites this paper as Luo et al., 2020
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
(2008) · Journal of Machine Learning Research · Cited by 33,051
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,119
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
Improved training of Wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017) · arXiv · doi:10.48550/arxiv.1704.00028 · Cited by 6,800
DEAP: A Database for Emotion Analysis ;Using Physiological Signals
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., & Patras, I. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.15 · Cited by 5,093
Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition
Dahl, G. E., Yu, D., Deng, L., & Acero, A. (2012) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2134090 · Cited by 3,011
Best practices for convolutional neural networks applied to visual document analysis
Simard, P. Y., Steinkraus, D., & Platt, J. C. (2005) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2003.1227801 · Cited by 2,862
Zheng, W.-L., & Lu, B.-L. (2015) · IEEE Transactions on Autonomous Mental Development · doi:10.1109/tamd.2015.2431497 · Cited by 2,477
Self-citationUnlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro
Zheng, Z., Zheng, L., & Yang, Y. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.405 · Cited by 2,142
The Unreasonable Effectiveness of Data
Halevy, A. Y., Norvig, P., & Pereira, F. M. B. (2009) · IEEE Intelligent Systems · doi:10.1109/mis.2009.36 · Cited by 1,776
Deep learning for electroencephalogram (EEG) classification tasks: a review
Craik, A., He, Y., & Contreras-Vidal, J. L. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab0ab5 · Cited by 1,758
A Multimodal Database for Affect Recognition and Implicit Tagging
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.25 · Cited by 1,664
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
Wu, J., Zhang, C., Xue, T., Freeman, W. T., & Tenenbaum, J. B. (2016) · arXiv · doi:10.48550/arxiv.1610.07584 · Cited by 1,393
EmotionMeter: A Multimodal Framework for Recognizing Human Emotions
Zheng, W.-L., Liu, W., Lu, Y., Lu, B.-L., & Cichocki, A. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2797176 · Cited by 1,276
Self-citationA survey on deep learning for big data
Zhang, Q., Yang, L. T., Chen, Z., & Li, P. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.10.006 · Cited by 1,226
Differential entropy feature for EEG-based emotion classification
Duan, R.-N., Zhu, J., & Lu, B. (2013) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner.2013.6695876 · Cited by 1,225
Self-citationEmotions Recognition Using EEG Signals: A Survey
Alarcao, S. M., & Fonseca, M. J. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2714671 · Cited by 1,201
Katsigiannis, S., & Ramzan, N. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2688239 · Cited by 1,135
Feature Extraction and Selection for Emotion Recognition from EEG
Jenke, R., Peer, A., & BUSS, M. J. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2339834 · Cited by 1,087
van Dyk, D. A., & Meng, X.-L. (2001) · Journal of Computational and Graphical Statistics · doi:10.1198/10618600152418584 · Cited by 1,052
Identifying Stable Patterns over Time for Emotion Recognition from EEG
Zheng, W., Zhu, J., & Lu, B. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2712143 · Cited by 1,023
Self-citationEmotional state classification from EEG data using machine learning approach
Wang, X.-W., Nie, D., & Lu, B.-L. (2014) · Neurocomputing · doi:10.1016/j.neucom.2013.06.046 · Cited by 755
Self-citationDeep Convolutional Inverse Graphics NetworkPreprint
Kulkarni, T. D., Whitney, W., Kohli, P., & Tenenbaum, J. B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.03167 · Cited by 746
Emotion Recognition From EEG Using Higher Order Crossings
Petrantonakis, P. C., & Hadjileontiadis, L. J. (2009) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2009.2034649 · Cited by 691
Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
Salimans, T., Kingma, D. P., & Welling, M. (2014) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1410.6460 · Cited by 400
Differential entropy feature for EEG-based vigilance estimation
Shi, L.-C., Jiao, Y.-Y., & Lu, B.-L. (2013) · 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2013.6611075 · Cited by 396
Self-citationBänziger, T., Grandjean, D., & Scherer, K. R. (2009) · Emotion · doi:10.1037/a0017088 · Cited by 380
MPED: A Multi-Modal Physiological Emotion Database for Discrete Emotion Recognition
Song, T., Zheng, W., Cheng, L., Zong, Y., Zhang, X., & Cui, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2891579 · Cited by 339
Lotte, F. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2404941 · Cited by 327
A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges
Mühl, C., Allison, B. Z., Nijholt, A., & Chanel, G. (2014) · Brain-Computer Interfaces · doi:10.1080/2326263x.2014.912881 · Cited by 296
Data augmentation for EEG-based emotion recognition with deep convolutional neural networks
Wang, F., Zhong, S.-H., Peng, J., Jiang, J., & Liu, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-73600-6_8 · Cited by 256
Advances in Neural Networks - ISNN 2006
Wang, J., Żurada, J. M., Yi, Z., Lu, B., & Yin, H. (2006) · Lecture notes in computer science · doi:10.1007/11759966 · Cited by 253
Self-citationEEG Data Augmentation for Emotion Recognition Using a Conditional Wasserstein GAN
Luo, Y., & Lu, B. (2018) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2018.8512865 · Cited by 228
Self-citationEmotion Classification with Data Augmentation Using Generative Adversarial Networks
Zhu, X., Liu, Y., Li, J., Wan, T., & Qin, Z. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93040-4_28 · Cited by 203
A facial expression emotion recognition based human-robot interaction system
Liu, Z., Wu, M. H., Cao, W., Chen, L., Xu, J., Zhang, R., Zhou, M., & Mao, J.-W. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510622 · Cited by 192
EEG-Based Emotion Recognition Using Hierarchical Network With Subnetwork Nodes
Yang, Y., Wu, Q. M. J., Zheng, W.-L., & Lu, B.-L. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2017.2685338 · Cited by 141
Self-citationRotational data augmentation for electroencephalographic data
Krell, M. M., & Kim, S. K. (2017) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2017.8036864 · Cited by 73
Unsupervised Learning in Reservoir Computing for EEG-Based Emotion Recognition
Fourati, R., Ammar, B., Medina, J., & Alimi, A. M. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2982143 · Cited by 68
Zhao, L.-M., Li, R., Zheng, W., & Lu, B. (2019) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner.2019.8717055 · Cited by 56
Self-citationDepression and implicit emotion processing: An EEG study
Bocharov, A. V., Knyazev, G. G., & Savostyanov, A. N. (2017) · Neurophysiologie Clinique · doi:10.1016/j.neucli.2017.01.009 · Cited by 55
Prediction of subjective ratings of emotional pictures by EEG features
McFarland, D. J., Parvaz, M. A., Sarnacki, W. A., Goldstein, R. Z., & Wolpaw, J. R. (2016) · Journal of Neural Engineering · doi:10.1088/1741-2552/14/1/016009 · Cited by 41
Feature Extraction for Emotion Recognition and Modelling Using Neurophysiological Data
Samara, A., Menezes, M. L. R., & Galway, L. (2016) · International Conference on Ubiquitous Computing and Communications (IUCC) · doi:10.1109/iucc-css.2016.027 · Cited by 37
Using body movement and posture for emotion detection in non-acted scenarios
Garber-Barron, M., & Si, M. (2012) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2012.6250780 · Cited by 26
Research works printed but not linked: 1
- Meneses Alarcro, 2017, Emotions recognition using EEG signals: A survey, IEEE Trans. Affective Computing, 10, 374, 10.1109/TAFFC.2017.2714671 · DOI · Paper page
Cited by
Sorted by citations
EEG Based Emotion Recognition: A Tutorial and Review
Li, X., Zhang, Y., Tiwari, P., Song, D., Hu, B., Yang, M., Zhao, Z., Kumar, N., & Marttinen, P. (2023) · ACM Computing Surveys · doi:10.1145/3524499 · Cited by 508
Feng, L., Cheng, C., Zhao, M., Deng, H., & Zhang, Y. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3198688 · Cited by 165
GANSER: A Self-Supervised Data Augmentation Framework for EEG-Based Emotion Recognition
Zhang, Z., Liu, Y., & Zhong, S.-H. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3170369 · Cited by 154
Samal, P., & Hashmi, M. F. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10690-2 · Cited by 170
Generative adversarial networks in EEG analysis: an overview
Habashi, A. G., Azab, A. M., Eldawlatly, S., & Aly, G. M. (2023) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-023-01169-w · Cited by 145
Metadata
- Authors
- Luo, Y., Zhu, L.-Z., Wan, Z.-Y., & Lu, B.-L.
- Affiliations
- Shanghai Jiao Tong University (CN)
- Corresponding author
- Lu, Bao-Liang
- Venue
- Journal of Neural Engineering
- Publisher
- IOP Publishing
- Published
- 2020-10-01
- Volume
- 17
- Issue
- 5
- Pages
- 056021
- ISSN
- 1741-2552 (ISSN-L)
- 1741-2560
- OpenAlex
- W3083255269
- Cited by
- 162
- References
- 51
- Citations by year
- 2026: 15 · 2025: 28 · 2024: 39 · 2023: 37 · 2022: 33 · 2021: 8 · 2020: 2
- FWCI
- 19.32
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Journal of Neural Engineering (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (61673266)
- 111 Project
- National Key Research and Development Program of China (2017YFB1002501)
- SJTU Trans-med Awards Research (WF540162605)
- Fundamental Research Funds for the Central Universities
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Physical Sciences › Computer Science › Signal Processing › Biometric Identification and Security
- Keywords
- EEG-based emotion recognition, data augmentation, deep generative models, variational autoencoder, generative adversarial networks, conditional Wasserstein GAN, data scarcity, power spectral density, differential entropy, affective model, SEED dataset, DEAP dataset
- MeSH terms
- Machine Learning, Algorithms, Electroencephalography, Emotions, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
