Emotion recognition from facial images with simultaneous occlusion, pose and illumination variations using meta-learning
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Automatic facial emotion recognition in real-world situations like partial occlusions, varying head poses and illumination conditions are challenging to the machine learning community. The main reason is the lack of sufficient samples with the aforementioned conditions in the baseline datasets which throws difficulty in training a well performing machine learning or deep learning model. To overcome this challenge, we have adopted the concept of meta-learning. Meta-learning using prototypical networks (metric-based meta-learning) has been proven to be well-fit for few-shot problems without severe overfitting. We leverage the quick adaptation power of prototypical networks for emotion recognition in the scarcity of such diverse samples. We have used CMU Multi-PIE dataset which contains images with partial occlusions, varying head-poses and illumination levels for training and evaluating the model. For testing the adaptability of the system to intra-class and inter-dataset variations, AffectNet face database images have been used. The proposed method is named as ERMOPI (Emotion Recognition using Meta-learning across Occlusion, Pose and Illumination) which performs emotion recognition from facial expressions using meta-learning approach for still images and it is robust to partial occlusions, varying head poses and illumination levels which is the novelty of this work. The key benefit is the usage of less number of training samples compared to the existing work in emotion recognition and achieved comparable results with the state-of-the-art approaches. The proposed method achieved 90% accuracy for CMU Multi-PIE database images and 68% accuracy for AffectNet database images.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
An argument for basic emotions
Ekman, P. (1992) · Cognition and Emotion · doi:10.1080/02699939208411068 · Cited by 9,646 · cited 1×
“In order to carry out this study, we have considered Ekman model of basic emotions (Ekman, 1992) augmented with neutral as a no-emotion state.”
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks
Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2603342 · Cited by 6,214
Learning to Compare: Relation Network for Few-Shot Learning
Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P. H. S., & Hospedales, T. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00131 · Cited by 5,816
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338 · cited 1×
“As reported in (Mollahosseini et al., 2019), the average agreement between two annotators on the 5 basic emotions over a part of AffectNet dataset is 65.36%.”
InfluentialOcclusion Aware Facial Expression Recognition Using CNN With Attention Mechanism
Li, Y., Zeng, J., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2886767 · Cited by 916 · cited 2×
“Also, models trained on one dataset do not generalize well for a new dataset because of inter-dataset variations in head poses, colour levels, contrasts, occlusions and illuminations, etc. (Li et al., 2019; Mollahosseini et al., 2016).”
Compound facial expressions of emotion
Du, S., Tao, Y., & Martinez, A. M. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1322355111 · Cited by 806
Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2008) · 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition · doi:10.1109/afgr.2008.4813399 · Cited by 373
Cited by
A comprehensive review on ensemble deep learning: Opportunities and challenges
Mohammed, A., & Kora, R. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.01.014 · Cited by 1,127
Khare, S. K., Blanes-Vidal, V., Nadimi, E. S., & Acharya, U. R. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102019 · Cited by 469
Automated emotion recognition: Current trends and future perspectives
Maithri, M., Raghavendra, U., Gudigar, A., Samanth, J., Barua, P. D., Murugappan, M., Chakole, Y., & Acharya, U. R. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106646 · Cited by 172
Multimodal emotion recognition: A comprehensive review, trends, and challenges
Ramaswamy, M. P. A., & Palaniswamy, S. (2024) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1563 · Cited by 70
Self-citationMoussaoui, K., & Farah, M. (2024) · 2024 IEEE 7th International Conference on Advanced Technologies, Signal and Image Processing (ATSIP) · doi:10.1109/atsip62566.2024.10638977 · Cited by 2
Metadata
- Authors
- Kuruvayil, S., & Palaniswamy, S.
- Venue
- Journal of King Saud University - Computer and Information Sciences
- Published
- 2022-10
- Volume
- 34
- Issue
- 9
- Pages
- 7271-7282
- ISSN
- 1319-1578 (ISSN-L)
- OpenAlex
- W3175450530
- Cited by
- 106
- References
- 10
- FWCI
- 17.18
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Emotion recognition from facial images with simultaneous occlusion, pose and illumination variations using meta-learning. https://app.cochat.ai/papers/10.1016/j.jksuci.2021.06.012
- The paper
- Kuruvayil, S., & Palaniswamy, S. (2022). Emotion recognition from facial images with simultaneous occlusion, pose and illumination variations using meta-learning. Journal of King Saud University - Computer and Information Sciences. https://doi.org/10.1016/j.jksuci.2021.06.012
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.