The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index.
Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition
Open accessAbstract
Occlusion and pose variations, which can change facial appearance significantly, are two major obstacles for automatic Facial Expression Recognition (FER). Though automatic FER has made substantial progresses in the past few decades, occlusion-robust and pose-invariant issues of FER have received relatively less attention, especially in real-world scenarios. This paper addresses the real-world pose and occlusion robust FER problem in the following aspects. First, to stimulate the research of FER under real-world occlusions and variant poses, we annotate several in-the-wild FER datasets with pose and occlusion attributes for the community. Second, we propose a novel Region Attention Network (RAN), to adaptively capture the importance of facial regions for occlusion and pose variant FER. The RAN aggregates and embeds varied number of region features produced by a backbone convolutional neural network into a compact fixed-length representation. Last, inspired by the fact that facial expressions are mainly defined by facial action units, we propose a region biased loss to encourage high attention weights for the most important regions. We validate our RAN and region biased loss on both our built test datasets and four popular datasets: FERPlus, AffectNet, RAF-DB, and SFEW. Extensive experiments show that our RAN and region biased loss largely improve the performance of FER with occlusion and variant pose. Our method also achieves state-of-the-art results on FERPlus, AffectNet, RAF-DB, and SFEW. Code and the collected test data will be publicly available.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382
SIFT: predicting amino acid changes that affect protein function
Ng, P. C. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg509 · Cited by 6,577
Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks
Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2603342 · Cited by 6,214
Self-citationParkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479
A Discriminative Feature Learning Approach for Deep Face Recognition
Wen, Y., Zhang, K., Li, Z., & Qiao, Y. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46478-7_31 · Cited by 4,077
Self-citationVGGFace2: A Dataset for Recognising Faces across Pose and Age
Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00020 · Cited by 3,032
One millisecond face alignment with an ensemble of regression trees
Kazemi, V., & Sullivan, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.241 · Cited by 2,831
Comprehensive database for facial expression analysis
Kanade, T., Cohn, J. F., & Tian, Y. (2002) · Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) · doi:10.1109/afgr.2000.840611 · Cited by 2,591
Recurrent Models of Visual Attention
Mnih, V., Heess, N., Graves, A., & Kavukcuoglu, K. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.6247 · Cited by 2,568
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338
Facial expression recognition based on Local Binary Patterns: A comprehensive study
Shan, C., Gong, S., & McOwan, P. W. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.08.005 · Cited by 2,300
Deep Learning Face Representation from Predicting 10,000 Classes
Sun, Y., Wang, X., & Tang, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.244 · Cited by 1,997
Liu, C., & Wechsler, H. (2002) · IEEE Transactions on Image Processing · doi:10.1109/tip.2002.999679 · Cited by 1,886
Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 1,753
Challenges in Representation Learning: A Report on Three Machine Learning Contests
Goodfellow, I. J., Erhan, D., Carrier, P. L., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D.-H., et al. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-42051-1_16 · Cited by 1,719
Learning Face Representation from Scratch
Yi, D., Lei, Z., Liao, S., & Li, S. Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.7923 · Cited by 1,521
Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild
Li, S., Deng, W., & Du, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.277 · Cited by 1,470
Long Short-Term Memory-Networks for Machine Reading
Cheng, J., Dong, L., & Lapata, M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1053 · Cited by 1,172
Training deep networks for facial expression recognition with crowd-sourced label distribution
Barsoum, E., Zhang, C., Ferrer, C. C., & Zhang, Z. (2016) · Proceedings of the 18th ACM International Conference on Multimodal Interaction · doi:10.1145/2993148.2993165 · Cited by 926
Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism
Li, Y., Zeng, J., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2886767 · Cited by 916
(2016) · Cited by 844
Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition
Jung, H., Lee, S., Yim, J., Park, S., & Kim, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.341 · Cited by 781 · cited 1×
“Due to its potential applications in various fields, such as intelligent tutoring systems, service robots, driver fatigue monitoring, Facial Expression Recognition (FER) has attracted increasing attention in the computer vision community recently [28], [5],”
BackgroundDeep Learning using Linear Support Vector Machines
Tang, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.0239 · Cited by 708
Challenges in representation learning: A report on three machine learning contests
Goodfellow, I. J., Erhan, D., Luc Carrier, P., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D.-H., et al. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.005 · Cited by 697
Collecting Large, Richly Annotated Facial-Expression Databases from Movies
Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2012) · IEEE MultiMedia · doi:10.1109/mmul.2012.26 · Cited by 663
Li, S., & Deng, W. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2868382 · Cited by 660
Benitez-Quiroz, C. F., Srinivasan, R., & Martinez, A. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.600 · Cited by 607
Automatic facial expression recognition using features of salient facial patches
Happy, S. L., & Routray, A. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2386334 · Cited by 599
Image based Static Facial Expression Recognition with Multiple Deep Network Learning
Yu, Z., & Zhang, C. (2015) · Proceedings of the 2015 ACM on International Conference on Multimodal Interaction · doi:10.1145/2818346.2830595 · Cited by 598
Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark
Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130508 · Cited by 541
FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition
Ding, H., Zhou, S. K., & Chellappa, R. (2017) · 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) · doi:10.1109/fg.2017.23 · Cited by 416
Neural Aggregation Network for Video Face Recognition
Yang, J., Ren, P., Zhang, D., Chen, D., Wen, F., Li, H., & Hua, G. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.554 · Cited by 409
Self-citationCombining modality specific deep neural networks for emotion recognition in video
Kahou, S. E., Pal, C., Bouthillier, X., Froumenty, P., Gülçehre, Ç., Memisevic, R., Vincent, P., Courville, A., Bengio, Y., Ferrari, R. C., et al. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2531745 · Cited by 391
Facial Expression Recognition with Inconsistently Annotated Datasets
Zeng, J., Shan, S., & Chen, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_14 · Cited by 341
Emotion Recognition in the Wild via Convolutional Neural Networks and Mapped Binary Patterns
Levi, G., & Hassner, T. (2015) · Proceedings of the 2015 ACM on International Conference on Multimodal Interaction · doi:10.1145/2818346.2830587 · Cited by 336
A Comprehensive Survey on Pose-Invariant Face Recognition
Ding, C., & Tao, D. (2016) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2845089 · Cited by 329
Island Loss for Learning Discriminative Features in Facial Expression Recognition
Cai, J., Meng, Z., Khan, A. S., Li, Z., OReilly, J., & Tong, Y. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00051 · Cited by 321
Identity-Aware Convolutional Neural Network for Facial Expression Recognition
Meng, Z., Liu, P., Cai, J., Han, S., & Tong, Y. (2017) · 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) · doi:10.1109/fg.2017.140 · Cited by 319
Partial Face Recognition: Alignment-Free Approach
Liao, S., Jain, A. K., & Li, S. Z. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.191 · Cited by 313
Video and Image based Emotion Recognition Challenges in the Wild: EmotiW 2015
Dhall, A., Ramana Murthy, O. V., Goecke, R., Joshi, J., & Gedeon, T. (2015) · Proceedings of the 2015 ACM on International Conference on Multimodal Interaction · doi:10.1145/2818346.2829994 · Cited by 305
Peak-Piloted Deep Network for Facial Expression Recognition
Zhao, X., Liang, X., Liu, L., Li, T., Han, Y., Vasconcelos, N., & Yan, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_27 · Cited by 304
Deep facial expression recognition: a survey
Shan, L., 北京邮电大学人工智能学院, 北. 1. S. O. A. I., & Weihong, D. (2020) · Journal of Image and Graphics · doi:10.11834/jig.200233 · Cited by 291
Emotion Recognition in Speech using Cross-Modal Transfer in the Wild
Albanie, S., Nagrani, A., Vedaldi, A., & Zisserman, A. (2018) · Proceedings of the 26th ACM international conference on Multimedia · doi:10.1145/3240508.3240578 · Cited by 290
Attention Clusters: Purely Attention Based Local Feature Integration for Video Classification
Long, X., Gan, C., de Melo, G., Wu, J., Liu, X., & Wen, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00817 · Cited by 233
Japanese Female Facial Expression (JAFFE) Database
Lyons, M. J., Kamachi, M., & Gyoba, J. (2017) · Figshare · doi:10.6084/m9.figshare.5245003 · Cited by 226
AU-inspired Deep Networks for Facial Expression Feature Learning
Liu, M., Li, S., Shan, S., & Chen, X. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.02.011 · Cited by 224
An analysis of facial expression recognition under partial facial image occlusion
Kotsia, I., Buciu, I., & Pitas, I. (2008) · Image and Vision Computing · doi:10.1016/j.imavis.2007.11.004 · Cited by 221
Frame Attention Networks for Facial Expression Recognition in Videos
Meng, D., Peng, X., Wang, K., & Qiao, Y. (2019) · 2019 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2019.8803603 · Cited by 196
Self-citationYovel, G., & Duchaine, B. (2006) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2006.18.4.580 · Cited by 168
Face Attention Network: An Effective Face Detector for the Occluded FacesPreprint
Wang, J., Yuan, Y., & Yu, G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.07246 · Cited by 162
Coupled Gaussian processes for pose-invariant facial expression recognition
Rudovic, O., Pantic, M., & Patras, I. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.233 · Cited by 159
EmotiW 2018: Audio-Video, Student Engagement and Group-Level Affect Prediction
Dhall, A., Kaur, A., Goecke, R., & Gedeon, T. (2018) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3242969.3264993 · Cited by 140
Robust face analysis using convolutional neural networks
Fasel, B. (2003) · doi:10.1109/icpr.2002.1048231 · Cited by 108
Random Gabor based templates for facial expression recognition in images with facial occlusion
Zhang, L., Tjondronegoro, D., & Chandran, V. (2014) · Neurocomputing · doi:10.1016/j.neucom.2014.05.008 · Cited by 91
Robust Facial Expression Recognition via Compressive Sensing
Zhang, S., Zhao, X., & Lei, B. (2012) · Sensors · doi:10.3390/s120303747 · Cited by 75
Recognition of Facial Expressions in the Presence of Occlusion
Bourel, F., Chibelushi, C. C., & Low, A. A. (2001) · British Machine Vision Conference (BMVC) · doi:10.5244/c.15.23 · Cited by 74
Group emotion recognition with individual facial emotion CNNs and global image based CNNs
Tan, L., Zhang, K., Wang, K., Zeng, X., Peng, X., & Qiao, Y. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3143008 · Cited by 73
Self-citationSparse Representation for accurate classification of corrupted and occluded facial expressions
Cotter, S. F. (2010) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2010.5494903 · Cited by 73
Combining convolutional neural networks for emotion recognition
Huang, C. (2017) · IEEE MIT Undergraduate Research Technology Conference (URTC) · doi:10.1109/urtc.2017.8284175 · Cited by 65
Gender and Smile Classification Using Deep Convolutional Neural Networks
Zhang, K., Tan, L., Li, Z., & Qiao, Y. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.97 · Cited by 63
Self-citationBatista, J. C., Albiero, V., Bellon, O. R. P., & da Silva, L. T. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.111 · Cited by 49
Zhang, L., Tjondronegoro, D., & Chandran, V. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2013.12.008 · Cited by 33
Liu, S.-S., Zhang, Y., Liu, K., & Li, Y. (2013) · International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP) · doi:10.1109/iih-msp.2013.63 · Cited by 17
Weighted Voting Of Sparse Representation Classifiers For Facial Expression Recognition
Cotter, S. F. (2010) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.41873 · Cited by 15
Cited by
Suppressing Uncertainties for Large-Scale Facial Expression Recognition
Wang, K., Peng, X., Yang, J., Lu, S., & Qiao, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00693 · Cited by 772
Self-citationAd-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild
Fard, A. P., & Mahoor, M. H. (2022) · IEEE Access · doi:10.1109/access.2022.3156598 · Cited by 157
Face2Exp: Combating Data Biases for Facial Expression Recognition
Zeng, D., Lin, Z., Yan, X., Liu, Y., Wang, F., & Tang, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01965 · Cited by 141
Adaptive Multilayer Perceptual Attention Network for Facial Expression Recognition
Liu, H., Cai, H., Lin, Q., Li, X., & Xiao, H. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3165321 · Cited by 129
An Overview of Emotion in Artificial Intelligence
Assunção, G., Patrão, B., Castelo-Branco, M., & Menezes, P. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3159614 · Cited by 103
Metadata
- Authors
- Wang, K., Peng, X., Yang, J., Meng, D., & Qiao, Y.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2020
- Volume
- 29
- Pages
- 4057-4069
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W3003720578
- Cited by
- 940
- References
- 73
- FWCI
- 120.06
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.