Using Deep Convolutional Neural Network for Emotion Detection on a Physiological Signals Dataset (AMIGOS)
Open accessAbstract
Recommender systems have been based on context and content, and now the technological challenge of making personalized recommendations based on the user emotional state arises through physiological signals that are obtained from devices or sensors. This paper applies the deep learning approach using a deep convolutional neural network on a dataset of physiological signals (electrocardiogram and galvanic skin response), in this case, the AMIGOS dataset. The detection of emotions is done by correlating these physiological signals with the data of arousal and valence of this dataset, to classify the affective state of a person. In addition, an application for emotion recognition based on classic machine learning algorithms is proposed to extract the features of physiological signals in the domain of time, frequency, and non-linear. This application uses a convolutional neural network for the automatic feature extraction of the physiological signals, and through fully connected network layers, the emotion prediction is made. The experimental results on the AMIGOS dataset show that the method proposed in this paper achieves a better precision of the classification of the emotional states, in comparison with the originally obtained by the authors of this dataset.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382 · see citing works
An overview of heart rate variability metrics and norms
Shaffer, F., & Ginsberg, J. P. (2017) · Frontiers in Public Health · doi:10.3389/fpubh.2017.00258 · Cited by 7,185 · see citing works
The Cognitive Structure of Emotions
Ortony, A., Clore, G. L., & Collins, A. (1988) · doi:10.1017/cbo9780511571299 · Cited by 6,208 · see citing works
DEAP: A Database for Emotion Analysis ;Using Physiological Signals
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., & Patras, I. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.15 · Cited by 5,093 · see citing works
Emotion elicitation using films
Gross, J. J., & Levenson, R. W. (1995) · Cognition & Emotion · doi:10.1080/02699939508408966 · Cited by 2,179 · see citing works
Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Calvo, R. A., & D'Mello, S. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.1 · Cited by 1,893 · cited 1× · see citing works
“algorithms, to ensure that the emotion recognition system has a high precision and robustness in the processing of physiological data [2].”
BackgroundA Multimodal Database for Affect Recognition and Implicit Tagging
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.25 · Cited by 1,664 · see citing works
The biopsychology of mood and arousal
Thayer, R. E. (1990) · Choice Reviews Online · doi:10.5860/choice.28-1830 · Cited by 1,242 · see citing works
Katsigiannis, S., & Ramzan, N. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2688239 · Cited by 1,135 · see citing works
The Handbook of emotion elicitation and assessment
(2008) · Choice Reviews Online · doi:10.5860/choice.46-1769 · Cited by 1,127 · see citing works
Smart Tourism Destinations Enhancing Tourism Experience Through Personalisation of Services
Buhalis, D., & Amaranggana, A. (2015) · Information and Communication Technologies in Tourism 2015 · doi:10.1007/978-3-319-14343-9_28 · Cited by 969 · see citing works
The International Affective Picture System (IAPS) in the Study of Emotion and Attention
Bradley, M. M., & Lang, P. J. (2007) · Handbook of Emotion Elicitation and Assessment · doi:10.1093/oso/9780195169157.003.0003 · Cited by 915 · see citing works
Chollet, F. (2017) · Cited by 742 · see citing works
AMIGOS: A Dataset for Affect, Personality and Mood Research on Individuals and Groups
Miranda-Correa, J. A., Abadi, M. K., Sebe, N., & Patras, I. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2884461 · Cited by 740 · see citing works
Raschka, S. (2015) · Cited by 656 · see citing works
Discrete Emotions or Dimensions? The Role of Valence Focus and Arousal Focus
Barrett, L. F. (1998) · Cognition & Emotion · doi:10.1080/026999398379574 · Cited by 645 · cited 1× · see citing works
“into their conscious affective experience [7].”
ASCERTAIN: Emotion and Personality Recognition Using Commercial Sensors
Subramanian, R., Wache, J., Abadi, M. K., Vieriu, R. L., WINKLER, S. S., & Sebe, N. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2625250 · Cited by 536 · see citing works
A 12-point circumplex structure of core affect.
Yik, M., Russell, J. A., & Steiger, J. H. (2011) · Emotion · doi:10.1037/a0023980 · Cited by 528 · see citing works
Recognition of emotions using multimodal physiological signals and an ensemble deep learning model
Yin, Z., Zhao, M., Wang, Y., Yang, J., & Zhang, J. (2017) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2016.12.005 · Cited by 416 · see citing works
DECAF: MEG-Based Multimodal Database for Decoding Affective Physiological Responses
Abadi, M. K., Subramanian, R., Kia, S. M., Avesani, P., Patras, I., & Sebe, N. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2392932 · Cited by 367 · see citing works
Emotion recognition from multi-channel EEG data through Convolutional Recurrent Neural Network
Li, X., Song, D., Zhang, P., Yu, G., Hou, Y., & Hu, B. (2016) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2016.7822545 · Cited by 307 · see citing works
A Practical Guide to Tourism Destination Management
World Tourism Organization (2007) · World Tourism Organization (UNWTO) eBooks · doi:10.18111/9789284412433 · Cited by 305 · see citing works
Emotion Recognition Using Multimodal Deep Learning
Liu, W., Zheng, W.-L., & Lu, B.-L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46672-9_58 · Cited by 301 · see citing works
Learning deep physiological models of affect
Martinez, H. P., Bengio, Y., & Yannakakis, G. N. (2013) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2013.2247823 · Cited by 287 · see citing works
Automatic identification of artifacts in electrodermal activity data
Taylor, S., Jaques, N., Chen, W., Fedor, S., Sano, A., & Picard, R. (2015) · 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2015.7318762 · Cited by 279 · see citing works
Classification of ECG signals based on 1D convolution neural network
Li, D. X., Zhang, J., Zhang, Q., & Wei, X. (2017) · IEEE International Conference on e-Health Networking, Applications and Services (Healthcom) · doi:10.1109/healthcom.2017.8210784 · Cited by 236 · see citing works
Russell, J. A. (1989) · Elsevier eBooks · doi:10.1016/b978-0-12-558704-4.50010-4 · Cited by 204 · see citing works
Time-series analysis for rapid event-related skin conductance responses
Bach, D. R., Flandin, G., Friston, K. J., & Dolan, R. J. (2009) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2009.08.005 · Cited by 203 · see citing works
Emotion Elicitation: A Comparison of Pictures and Films
Uhrig, M. K., Trautmann, N., Baumgärtner, U., Treede, R.-D., Henrich, F., Hiller, W., & Marschall, S. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00180 · Cited by 184 · see citing works
Analysis of physiological signals for recognition of boredom, pain, and surprise emotions
Jang, E.-H., Park, B.-J., Park, M.-S., Kim, S.-H., & Sohn, J.-H. (2015) · Journal of Physiological Anthropology · doi:10.1186/s40101-015-0063-5 · Cited by 159 · see citing works
Deep Learning for ECG Classification
Pyakillya, B., Казаченко, Н. Я., & Mikhailovsky, N. (2017) · Journal of Physics Conference Series · doi:10.1088/1742-6596/913/1/012004 · Cited by 155 · see citing works
Advances in Electrodermal Activity Processing with Applications for Mental Health
Greco, A., Valenza, G., & Scilingo, E. (2016) · Cambridge International Law Journal · doi:10.1007/978-3-319-46705-4 · Cited by 98 · see citing works
Emotion Classification in Arousal Valence Model using MAHNOB-HCI Database
Ben, M., & Lachiri, Z. (2017) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2017.080344 · Cited by 79 · see citing works
Lin, W., Li, C., & Sun, S. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-71589-6_33 · Cited by 72 · see citing works
Computational Approaches to Modeling Artificial Emotion – An Overview of the Proposed Solutions
Kowalczuk, Z., & Czubenko, M. (2016) · Frontiers in Robotics and AI · doi:10.3389/frobt.2016.00021 · Cited by 62 · see citing works
Comparing features from ECG pattern and HRV analysis for emotion recognition system
Ferdinando, H., Seppänen, T., & Alasaarela, E. L. I. (2016) · IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) · doi:10.1109/cibcb.2016.7758108 · Cited by 51 · see citing works
Emotional arousal estimation while reading comics based on physiological signal analysis
Matsubara, M., Augereau, O., Sanches, C. L., et al. (2016) · doi:10.1145/3011549.3011556 · Cited by 38 · see citing works
Selection of the Most Relevant Physiological Features for Classifying Emotion
Godin, C., Prost-Boucle, F., Campagne, A., Charbonnier, S., Bonnet, S., & Vidal, A. (2015) · International Conference on Physiological Computing Systems (PhyCS) · doi:10.5220/0005238600170025 · Cited by 32 · see citing works
Measuring Human Senses and the Touristic Experience: Methods and Applications
Kim, J., & Fesenmaier, D. R. (2016) · Tourism on the verge · doi:10.1007/978-3-319-44263-1_4 · Cited by 22 · see citing works
Decoding emotional experiences through physiological signal processing
Perez-Rosero, M. S., Rezaei, B., Akçakaya, M., & Ostadabbas, S. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952282 · Cited by 21 · see citing works
Python Online and Offline ECG QRS Detector based on the Pan-Tomkins algorithm
Sznajder, M., & Łukowska, M. (2017) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.583770 · Cited by 19 · see citing works
Makikawa, M., Shiozawa, N., & Okada, S. (2014) · Elsevier eBooks · doi:10.1016/b978-0-12-418662-0.00007-6 · Cited by 13 · see citing works
Evaluation of consistency of HRV indices change among different emotions
Zhao, L., Yang, L., Shi, H., Xia, Y., Li, F., & Liu, C. (2017) · Chinese Automation Congress · doi:10.1109/cac.2017.8243625 · Cited by 13 · see citing works
Emotion Recognition Based on Physiological Sensor Data Using Codebook Approach
Shirahama, K., & Grzegorzek, M. (2016) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-39904-1_3 · Cited by 11 · see citing works
Ferreira, P., & Simons, C. L. (2017) · UWE Research Repository (UWE Bristol) · Cited by 10 · see citing works
Deep affect recognition from R-R intervals
Gjoreski, M., Gjoreski, H., Luštrek, M., & Gams, M. (2017) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3123024.3125608 · Cited by 10 · see citing works
Multimodal affect recognition in V-A-D space using deep learning
Kawde, P., & Verma, G. K. (2017) · International Conference On Smart Technologies For Smart Nation (SmartTechCon) · doi:10.1109/smarttechcon.2017.8358500 · Cited by 8 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016.
Cited by
Sorted by citations
Self-Supervised Learning for ECG-Based Emotion Recognition
Sarkar, P., & Etemad, A. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9053985 · Cited by 146 · see citing works
Multimodal Emotion Recognition Using a Hierarchical Fusion Convolutional Neural Network
Zhang, Y., Cheng, C., & Zhang, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3049516 · Cited by 84 · see citing works
Multimodal emotion recognition based on manifold learning and convolution neural network
Zhang, Y., Cheng, C., & Zhang, Y. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13149-8 · Cited by 67 · see citing works
Pre-training in Medical Data: A Survey
Qiu, Y., Lin, F., Chen, W., & Xu, M. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1382-8 · Cited by 30 · see citing works
- Authors
- Santamaria-Granados, L., Munoz-Organero, M., Ramirez-Gonzalez, G., Abdulhay, E., & Arunkumar, N.
- Affiliations
- Jordan University of Science and Technology (JO)
- SASTRA University (IN)
- Venue
- IEEE Access
- Published
- 2019
- Volume
- 7
- Pages
- 57-67
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2901027086
- Cited by
- 289
- References
- 47
- Citations by year
- 2026: 15 · 2025: 26 · 2024: 46 · 2023: 53 · 2022: 58 · 2021: 41 · 2020: 32 · 2019: 17 · 2018: 1
- FWCI
- 29.79
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion)
- LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- e-Archivo (Carlos III University of Madrid) (submittedVersion)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, other-oa)
- Funders
- Government of Colombia
- Colciencias
- Governorate of Boyacá
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Heart Rate Variability and Autonomic Control
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Keywords
- emotion recognition, physiological signals, AMIGOS dataset, electrocardiogram, skin conductance, valence and arousal, affective state classification, personalized recommendation systems, automatic feature extraction, time-frequency features, nonlinear characteristics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
