On the classification of emotional biosignals evoked while viewing affective pictures: An integrated data-mining-based approach for healthcare applications
Abstract
Recent neuroscience findings demonstrate the fundamental role of emotion in the maintenance of physical and mental health. In the present study, a novel architecture is proposed for the robust discrimination of emotional physiological signals evoked upon viewing pictures selected from the International Affective Picture System (IAPS). Biosignals are multichannel recordings from both the central and the autonomic nervous systems. Following the bidirectional emotion theory model, IAPS pictures are rated along two dimensions, namely, their valence and arousal. Following this model, biosignals in this paper are initially differentiated according to their valence dimension by means of a data mining approach, which is the C4.5 decision tree algorithm. Then, the valence and the gender information serve as an input to a Mahalanobis distance classifier, which dissects the data into high and low arousing. Results are described in Extensible Markup Language (XML) format, thereby accounting for platform independency, easy interconnectivity, and information exchange. The average recognition (success) rate was 77.68% for the discrimination of four emotional states, differing both in their arousal and valence dimension. It is, therefore, envisaged that the proposed approach holds promise for the efficient discrimination of negative and positive emotions, and it is hereby discussed how future developments may be steered to serve for affective healthcare applications, such as the monitoring of the elderly or chronically ill people.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
To remove frequency noise, methods in frequency domain such as PSD, FFT, and low-pass/high-pass filtering are common to remove fluctuations in sensor signals.
“To remove frequency noise, the other methods in frequency domain such as such as power spectral density (PSD) fast Fourier transforms (FFT), and low-pass/high-pass filtering tools are common to remove the fluctuations in sensor signals [51,62].”
From Data Mining for Wearable Sensors in Health Monitoring Systems: A Review of Recent Trends and Challenges (2013) · cites this paper as [51]
Other tools for feature selection used in literature include threshold-based rules, ANOVA, and Fourier transforms.
“Other tools for feature selection used in the literature includes threshold-based rules [61], analysis of variance (ANOVA) [51], and Fourier transforms [52].”
From Data Mining for Wearable Sensors in Health Monitoring Systems: A Review of Recent Trends and Challenges (2013) · cites this paper as [51]
The decision tree method provides an efficient representation of rule classification.
“To make an accurate discrimination for selected features, the decision tree method is one of the significant learning a technique which provides an efficient representation of rule classification [51].”
From Data Mining for Wearable Sensors in Health Monitoring Systems: A Review of Recent Trends and Challenges (2013) · cites this paper as [51]
The C4.5 algorithm is used when dealing with complex and noisy data; it estimates the error rate of initial nodes and prunes the tree to make a more efficient sub-tree.
“Nowadays, upon dealing with complex and noisy data, the C4.5 algorithm is used which is estimating the error rate of initial nodes and pruning the tree to make a more efficient sub-tree [41,51].”
From Data Mining for Wearable Sensors in Health Monitoring Systems: A Review of Recent Trends and Challenges (2013) · cites this paper as [51]
Cites
(1992) · Cited by 23,431 · see citing works
Russell, J. A. (1980) · Journal of Personality and Social Psychology · doi:10.1037/h0077714 · Cited by 16,334 · see citing works
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382 · see citing works
Event-related EEG/MEG synchronization and desynchronization: basic principles
Pfurtscheller, G., & Lopes da Silva, F. H. (1999) · Clinical Neurophysiology · doi:10.1016/s1388-2457(99)00141-8 · Cited by 7,328 · see citing works
International Affective Picture System (IAPS) : Technical Manual and Affective Ratings
Lang, P. J. (1995) · Medical Entomology and Zoology · Cited by 2,870 · see citing works
Report of the committee on methods of clinical examination in electroencephalography
(1958) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(58)90053-1 · Cited by 2,717 · see citing works
Toward machine emotional intelligence: analysis of affective physiological state
Picard, R. W., Vyzas, E., & Healey, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.954607 · Cited by 2,334 · see citing works
Cacioppo, J. T., & Berntson, G. G. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.115.3.401 · Cited by 1,782 · see citing works
Emotion, motivation, and anxiety: brain mechanisms and psychophysiology
Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (1998) · Biological Psychiatry · doi:10.1016/s0006-3223(98)00275-3 · Cited by 1,043 · see citing works
Review: Electrodermal Responses: What Happens in the Brain
Critchley, H. D. (2002) · The Neuroscientist · doi:10.1177/107385840200800209 · Cited by 997 · see citing works
Affective computing: challenges
Picard, R. W. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00052-1 · Cited by 978 · see citing works
Hughes, C. W. (1982) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-198205000-00017 · Cited by 975 · see citing works
The Structure of Current Affect: Controversies and Emerging Consensus
Barrett, L. F., & Russell, J. A. (1999) · Current Directions in Psychological Science · doi:10.1111/1467-8721.00003 · Cited by 974 · see citing works
Discrete Emotions or Dimensions? The Role of Valence Focus and Arousal Focus
Barrett, L. F. (1998) · Cognition & Emotion · doi:10.1080/026999398379574 · Cited by 645 · cited 1× · see citing works
“An emerging research issue concerning such applications is their enrichment with machine emotional intelligence through the robust emotion recognition in order to mimic what is required for human– human communication and the support of decisions about daily tasks [1].”
BackgroundPleasure-arousal theory and the intensity of emotions.
Reisenzein, R. (1994) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.67.3.525 · Cited by 471 · see citing works
Emotion Recognition Using Bio-sensors: First Steps towards an Automatic System
Haag, A., Goronzy, S., Schaich, P., & Williams, J. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24842-2_4 · Cited by 451 · see citing works
Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals
Lisetti, C., & Nasoz, F. (2004) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/s1110865704406192 · Cited by 432 · see citing works
Picard, R. W., & Healey, J. (1997) · Personal and Ubiquitous Computing · doi:10.1007/bf01682026 · Cited by 292 · see citing works
Russell, J. A. (1989) · Elsevier eBooks · doi:10.1016/b978-0-12-558704-4.50010-4 · Cited by 204 · see citing works
Oscillatory brain theory: a new trend in neuroscience
Başar, E., Basar-Eroglu, C., Karakaş, S., & Schürmann, M. (1999) · IEEE Engineering in Medicine and Biology Magazine · doi:10.1109/51.765190 · Cited by 163 · see citing works
A Bayesian approach to joint feature selection and classifier design
Krishnapuram, B., Harternink, A. J., Carin, L., & Figueiredo, M. A. T. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.55 · Cited by 158 · see citing works
Personal and Ubiquitous Computing
Comput, P. U., Yu, S.-H., Wang, L.-S., Chu, H., Chen, S.-H., Chai, C., Chen, H., You, C.-W., Yu, S.-H., Wang, L.-S., et al. (2013) · Personal and Ubiquitous Computing · doi:10.1007/779.1617-4917 · Cited by 146 · see citing works
Klados, M. A., Frantzidιs, C. A., Vivas, A. B., Papadelis, C. L., Lithari, C. D., Pappas, C., & Bamidis, P. D. (2009) · Computational Intelligence and Neuroscience · doi:10.1155/2009/549419 · Cited by 79 · see citing works
Self-citationA social informatics approach to human-robot interaction with a service social robot
Lisetti, C. L., Brown, S. M., Alvarez, K., & Marpaung, A. H. (2004) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2004.826278 · Cited by 66 · see citing works
Zeidner, M. (2017) · The SAGE Encyclopedia of Abnormal and Clinical Psychology · doi:10.4135/9781483365817.n530 · Cited by 56 · see citing works
Affective computing in the era of contemporary neurophysiology and health informatics
Bamidis, P. D., Papadelis, C. L., Kourtidou‐Papadeli, C., et al. (2004) · Interacting with Computers · doi:10.1016/j.intcom.2004.06.009 · Cited by 45 · see citing works
Self-citationFebo Cincotti, A., Donatella Mattia, A., Babiloni, C., Carducci, F., Bianchi, L., Millán, J. D. R., Mouriño, J., Salinari, S., Marciani, M. G., & Babiloni, F. (2002) · Methods of Information in Medicine · doi:10.1055/s-0038-1634391 · Cited by 34 · see citing works
Bratsas, C., Koutkias, V. G., Kaimakamis, E., et al. (2007) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2007.06.005 · Cited by 27 · see citing works
Self-citationFrantzidιs, C. A., Lithari, C. D., Vivas, A. B., et al. (2008) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe.2008.4696823 · Cited by 25 · see citing works
Self-citationA Framework to Support Automated Classification and Labeling of Brain Electromagnetic Patterns
Frishkoff, G. A., Frank, R. M., Rong, J., Dou, D., Dien, J., & Halderman, L. K. (2007) · Computational Intelligence and Neuroscience · doi:10.1155/2007/14567 · Cited by 24 · see citing works
Affective computing and medical informatics: state of the art in emotion-aware medical applications.
Luneski, A., Bamidis, P. D., & Hitoglou-Antoniadou, M. (2008) · PubMed · Cited by 23 · see citing works
Self-citationThe Removal Of Ocular Artifacts From EEG Signals: A Comparison of Performances For Different Methods
Klados, M. A., Papadelis, C. L., Lithari, C. D., et al. (2009) · World Congress on Medical Physics and Biomedical Engineering, September 7 - 12, 2009, Munich, Germany · doi:10.1007/978-3-540-89208-3_300 · Cited by 23 · see citing works
Self-citationKaimakamis, E., Bratsas, C., Sichletidis, L. T., Karvounis, C., & Maglaveras, N. (2009) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2009.5334605 · Cited by 21 · see citing works
Self-citationAn Integrated Approach to Emotion Recognition for Advanced Emotional Intelligence
Bamidis, P. D., Frantzidιs, C. A., Konstantinidis, E. I., Luneski, A., Lithari, C. D., Klados, M. A., Bratsas, C., Papadelis, C. L., & Pappas, C. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02580-8_62 · Cited by 20 · see citing works
Self-citationAffective computer-aided learning for autistic children.
Luneski, A., Konstantinidis, E. I., Hitoglou-Antoniadou, M., et al. (2008) · Cited by 19 · see citing works
Self-citationBamidis, P. D., Luneski, A., Vivas, A. B., et al. (2007) · PubMed · Cited by 11 · see citing works
Self-citationTowards an Emotion Specification Method: Representing Emotional Physiological Signals
Luneski, A., & Bamidis, P. D. (2007) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms.2007.113 · Cited by 6 · see citing works
Self-citationLovis, C., Spahni, S., Appel, R. D., et al. (2002) · Methods of Information in Medicine · doi:10.1055/s-0038-1634442 · Cited by 5 · see citing works
Bratsas, C., Frantzidιs, C. A., Klados, M. A., et al. (2009) · Studies in health technology and informatics · doi:10.3233/978-1-60750-044-5-322 · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Banaee, H., Ahmed, M., & Loutfi, A. (2013) · Sensors · doi:10.3390/s131217472 · Cited by 452 · see citing works
Gunes, H., & Schuller, B. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.06.016 · Cited by 360 · see citing works
Telemental Health Care, an Effective Alternative to Conventional Mental Care: a Systematic Review
Langarizadeh, M., Tabatabaei, M., Tavakol, K., Naghipour, M., & Moghbeli, F. (2017) · Acta Informatica Medica · doi:10.5455/aim.2017.25.240-246 · Cited by 314 · see citing works
SMIL: Multimodal Learning with Severely Missing Modality
Ma, M., Ren, J., Zhao, L., Tulyakov, S., Wu, C., & Peng, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i3.16330 · Cited by 332 · see citing works
“Multimodal learning attracts intensive research interest because of broad applications such as intelligent tutoring (Petrovica, Anohina-Naumeca, and Ekenel 2017), robotics (Noda et al. 2014), and healthcare (Frantzidis et al. 2010).”
BackgroundStress and Perception of Emotional Stimuli: Long-Term Stress Rewiring the Brain
Khosrowabadi, R., & Institute for Cognitive and Brain Sciences, S. B. U. (2018) · Basic and Clinical Neuroscience Journal · doi:10.29252/nirp.bcn.9.2.107 · Cited by 39 · see citing works
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
Liang, P. P., Lyu, Y., Fan, X., Wu, Z., Cheng, Y., Wu, J., Chen, L., Wu, P., Lee, M. A., Zhu, Y., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.07502 · Cited by 85 · see citing works
“…recognition [48] to the recent explosion of interest in language, vision, and video understanding [48] for applications such as multimedia [102, 116], affective computing [101, 127], robotics [84, 91], finance [70], dialogue [126], human-computer interaction [47, 117], and healthcare [51, 172].”
Background
- Authors
- Frantzidis, C. A., Bratsas, C., Klados, M. A., Konstantinidis, E., Lithari, C. D., Vivas, A. B., Papadelis, C. L., Kaldoudi, E., Pappas, C., & Bamidis, P. D.
- Affiliations
- Aristotle University of Thessaloniki (GR)
- University of Sheffield (GB)
- University of Trento (IT)
- Democritus University of Thrace (GR)
- Venue
- IEEE Transactions on Information Technology in Biomedicine
- Published
- 2010-03
- Volume
- 14
- Issue
- 2
- Pages
- 309-318
- ISSN
- 1089-7771 (ISSN-L)
- OpenAlex
- W2120911927
- Cited by
- 166
- References
- 39
- Citations by year
- 2025: 4 · 2024: 8 · 2023: 3 · 2022: 15 · 2021: 9 · 2020: 10 · 2019: 11 · 2018: 13 · 2017: 10 · 2016: 16 · 2015: 15 · 2014: 13 · 2013: 10 · 2012: 13
- FWCI
- 15.41
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Information Technology in Biomedicine (publishedVersion)
- PubMed (publishedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Heart Rate Variability and Autonomic Control
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- emotion recognition, emotional pictures, International Affective Picture System, valence and arousal, C4.5 decision tree, Mahalanobis distance classifier, autonomic nervous system, elderly monitoring, chronic disease patients, XML data exchange
- MeSH terms
- Adult, Algorithms, Autonomic Nervous System, Central Nervous System, Electroencephalography, Emotions, Evoked Potentials, Female, Galvanic Skin Response, Humans, Male, Monitoring, Physiologic, Pattern Recognition, Automated, Signal Processing, Computer-Assisted, Reproducibility of Results, Recognition, Psychology, Data Mining
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
