Affective states classification using EEG and semi-supervised deep learning approaches
Abstract
Affective states of a user provide important information for many applications such as, personalized information (e.g., multimedia content) retrieval/delivery or intelligent human-computer interface design. In recently years, physiological signals, Electroencephalogram (EEG) in particular, have been shown to be very effective in estimating a user's affective states during social interaction or under video or audio stimuli. However, due to the large number of parameters associated with the neural expression of emotion, there is still a lot of unknowns on the specific spatial and spectral correlation of the EEG signal and the affective states expression. To investigate on such correlation, two types of semi-supervised deep learning approaches, stacked denoising autoencoder (SDAE) and deep belief networks (DBN), were applied as application specific feature extractors for the affective states classification problem using EEG signals. To evaluate the efficacy of the proposed semi-supervised approaches, a subject-specific affective states classification experiment were carried out on the DEAP database to classify 2-dimensional affect states. The DBN based model achieved averaged F1 scores of 86.67%, 86.60% and 86.69% for arousal, valence and liking states classification respectively, which has significantly improved the state-of-art classification performance. By examining the weight vectors at each layer, we were also able to gain insights on the spatial or spectral locations of the most discriminating features. Another main advantage of applying the semi-supervised learning methods is that only a small fraction of labeled data, e.g., 1/6 of the training samples, were used in this study.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Xu and Plataniotis [168] showed that DBNs can build on PSD features to outperform state-of-the-art methods on the DEAP dataset.
“Similarly, Xu and Plataniotis [168] showed that DBNs can build on PSD features to outperform state-of-the-art methods on the DEAP dataset.”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [168]
Cites
LIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918 · see citing works
Wu, Y., & Vapnik, V. N. (1999) · Technometrics · doi:10.2307/1271368 · Cited by 27,236 · see citing works
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · see citing works
Russell, J. A. (1980) · Journal of Personality and Social Psychology · doi:10.1037/h0077714 · Cited by 16,334 · cited 1× · see citing works
“D model of emotion [9], which included scores for arousal, valence, liking, dominance and familiarity.”
BackgroundHe, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359 · see citing works
Libsvm : A library for support vector machines
Ferrari, V. (2008) · Medical Entomology and Zoology · Cited by 10,124 · see citing works
DEAP: A Database for Emotion Analysis ;Using Physiological Signals
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., & Patras, I. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.15 · Cited by 5,093 · see citing works
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051 · see citing works
Vapnik, V. N. 1. (1998) · Cited by 2,163 · see citing works
EEG-Based Emotion Recognition in Music Listening
Lin, Y., Wang, C., Jung, T., Wu, T.-L., Jeng, S., Duann, J., & Chen, J. (2010) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2010.2048568 · Cited by 1,022 · see citing works
Emotional state classification from EEG data using machine learning approach
Wang, X.-W., Nie, D., & Lu, B.-L. (2014) · Neurocomputing · doi:10.1016/j.neucom.2013.06.046 · Cited by 755 · see citing works
Learning Deep Generative Models
Salakhutdinov, R. (2015) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-010814-020120 · Cited by 433 · see citing works
Brain oscillations in perception and memory
Başar, E., Başar‐Eroğlu, C., Karakaş, S., & Schürmann, M. (2000) · International Journal of Psychophysiology · doi:10.1016/s0167-8760(99)00047-1 · Cited by 394 · see citing works
EEG-based emotion classification using deep belief networks
Zheng, W.-L., Zhu, J.-Y., Peng, Y., & Lu, B.-L. (2014) · 2014 IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2014.6890166 · Cited by 367 · see citing works
Fusion of facial expressions and EEG for implicit affective tagging
Koelstra, S., & Patras, I. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2012.10.002 · Cited by 228 · see citing works
Robust EEG emotion classification using segment level decision fusion
Rozgić, V., Vitaladevuni, S. N. P., & Prasad, R. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6637858 · Cited by 116 · see citing works
Modeling Physiological Data with Deep Belief Networks
Wang, D., & Shang, Y. (2013) · International Journal of Information and Education Technology · doi:10.7763/ijiet.2013.v3.326 · Cited by 87 · see citing works
Affective state recognition from EEG with deep belief networks
Li, K., Li, X., Zhang, Y., & Zhang, A. (2013) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2013.6732507 · Cited by 65 · see citing works
Affect classification using genetic-optimized ensembles of fuzzy ARTMAPs
Liew, W. S., Seera, M., Loo, C. K., & Lim, E. (2014) · Applied Soft Computing · doi:10.1016/j.asoc.2014.10.032 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Deep learning for electroencephalogram (EEG) classification tasks: a review
Craik, A., He, Y., & Contreras-Vidal, J. L. (2019) · Journal of Neural Engineering · doi:10.1088/1741-2552/ab0ab5 · Cited by 1,758 · see citing works
A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers
Zhang, X., Yao, L., Wang, X., Monaghan, J., McAlpine, D., & Zhang, Y. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/abc902 · Cited by 296 · see citing works
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243 · see citing works
- Authors
- Xu, H., & Plataniotis, K. N.
- Affiliations
- University of Toronto (CA)
- Venue
- 2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP)
- Publisher
- IEEE
- Published
- 2016-09
- Volume
- 2016
- Pages
- 1-6
- OpenAlex
- W2574833558
- Cited by
- 86
- References
- 19
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 4 · 2023: 15 · 2022: 11 · 2021: 11 · 2020: 14 · 2019: 11 · 2018: 10 · 2017: 3
- FWCI
- 9.25
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Workshop on Multimedia Signal Processing (MMSP) (publishedVersion)
- IEEE Conference Proceedings
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Keywords
- EEG, semi-supervised deep learning, stacked denoising autoencoder, deep belief network, arousal, valence, liking, DEAP database, feature extraction, emotion recognition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
