The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Learning Deep Global Multi-Scale and Local Attention Features for Facial Expression Recognition in the Wild
Abstract
Facial expression recognition (FER) in the wild received broad concerns in which occlusion and pose variation are two key issues. This paper proposed a global multi-scale and local attention network (MA-Net) for FER in the wild. Specifically, the proposed network consists of three main components: a feature pre-extractor, a multi-scale module, and a local attention module. The feature pre-extractor is utilized to pre-extract middle-level features, the multi-scale module to fuse features with different receptive fields, which reduces the susceptibility of deeper convolution towards occlusion and variant pose, while the local attention module can guide the network to focus on local salient features, which releases the interference of occlusion and non-frontal pose problems on FER in the wild. Extensive experiments demonstrate that the proposed MA-Net achieves the state-of-the-art results on several in-the-wild FER benchmarks: CAER-S, AffectNet-7, AffectNet-8, RAFDB, and SFEW with accuracies of 88.42%, 64.53%, 60.29%, 88.40%, and 59.40% respectively. The codes and training logs are publicly available at https://github.com/zengqunzhao/MA-Net.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
(2016) · Cited by 31,590
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
CBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428
SIFT: predicting amino acid changes that affect protein function
Ng, P. C. (2003) · Nucleic Acids Research · doi:10.1093/nar/gkg509 · Cited by 6,577
Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks
Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2603342 · Cited by 6,214
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
Res2Net: A New Multi-Scale Backbone Architecture
Gao, S.-H., Cheng, M.-M., Zhao, K., Zhang, X.-Y., Yang, M.-H., & Torr, P. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2938758 · Cited by 3,385
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338
Coding facial expressions with Gabor wavelets
Lyons, M., Akamatsu, S., Kamachi, M., & Gyoba, J. (2002) · Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670949 · Cited by 2,210
MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition
Guo, Y., Zhang, L., Hu, Y., He, X., & Gao, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_6 · Cited by 2,122
Liu, C., & Wechsler, H. (2002) · IEEE Transactions on Image Processing · doi:10.1109/tip.2002.999679 · Cited by 1,886
Challenges in Representation Learning: A Report on Three Machine Learning Contests
Goodfellow, I. J., Erhan, D., Carrier, P. L., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D.-H., et al. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-42051-1_16 · Cited by 1,719
RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild
Deng, J., Guo, J., Ververas, E., Kotsia, I., & Zafeiriou, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00525 · Cited by 1,681
Bulat, A., & Tzimiropoulos, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.116 · Cited by 1,464
Bulat, A., & Tzimiropoulos, G. (2017) · Nottingham ePrints (University of Nottingham) · Cited by 1,277
Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition
Wang, K., Peng, X., Yang, J., Meng, D., & Qiao, Y. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2956143 · Cited by 940
Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism
Li, Y., Zeng, J., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2886767 · Cited by 916
Suppressing Uncertainties for Large-Scale Facial Expression Recognition
Wang, K., Peng, X., Yang, J., Lu, S., & Qiao, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00693 · Cited by 772
Challenges in representation learning: A report on three machine learning contests
Goodfellow, I. J., Erhan, D., Luc Carrier, P., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D.-H., et al. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.005 · Cited by 697
Collecting Large, Richly Annotated Facial-Expression Databases from Movies
Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2012) · IEEE MultiMedia · doi:10.1109/mmul.2012.26 · Cited by 663
Li, S., & Deng, W. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2868382 · Cited by 660
Automatic facial expression recognition using features of salient facial patches
Happy, S. L., & Routray, A. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2386334 · Cited by 599
SSH: Single Stage Headless Face Detector
Najibi, M., Samangouei, P., Chellappa, R., & Davis, L. S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.522 · Cited by 487
Facial Expression Recognition by De-expression Residue Learning
Yang, H., Ciftci, U., & Yin, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00231 · Cited by 439
FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition
Ding, H., Zhou, S. K., & Chellappa, R. (2017) · 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) · doi:10.1109/fg.2017.23 · Cited by 416
Learning active facial patches for expression analysis
Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., & Metaxas, D. N. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247974 · Cited by 369
Self-citationFacial Expression Recognition with Inconsistently Annotated Datasets
Zeng, J., Shan, S., & Chen, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_14 · Cited by 341
Context-Aware Emotion Recognition Networks
Lee, J., Kim, S., Kim, S., Park, J., & Sohn, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01024 · Cited by 323
Island Loss for Learning Discriminative Features in Facial Expression Recognition
Cai, J., Meng, Z., Khan, A. S., Li, Z., OReilly, J., & Tong, Y. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00051 · Cited by 321
Identity-Aware Convolutional Neural Network for Facial Expression Recognition
Meng, Z., Liu, P., Cai, J., Han, S., & Tong, Y. (2017) · 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) · doi:10.1109/fg.2017.140 · Cited by 319
Joint Pose and Expression Modeling for Facial Expression Recognition
Zhang, F., Zhang, T., Mao, Q., & Xu, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00354 · Cited by 256
Label Distribution Learning on Auxiliary Label Space Graphs for Facial Expression Recognition
Chen, S., Wang, J., Chen, Y., Shi, Z., Geng, X., & Rui, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01400 · Cited by 229
Learning Multiscale Active Facial Patches for Expression Analysis
Zhong, L., Liu, Q., Yang, P., Huang, J., & Metaxas, D. N. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2354351 · Cited by 216
Self-citationXie, S., & Hu, H. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2844085 · Cited by 214
Xie, S., Hu, H., & Wu, Y. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.03.019 · Cited by 209
Adaptive Deep Metric Learning for Identity-Aware Facial Expression Recognition
Liu, X., Kumar, B. V. K. V., You, J., & Jia, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.79 · Cited by 207
Driver’s Facial Expression Recognition in Real-Time for Safe Driving
Jeong, M., & Ko, B. C. (2018) · Sensors · doi:10.3390/s18124270 · Cited by 195
Duric, Z., Gray, W. D., Heishman, R., Li, F., Rosenfeld, A., Schoelles, M. J., Schunn, C., & Wechsler, H. (2002) · Proceedings of the IEEE · doi:10.1109/jproc.2002.801449 · Cited by 185
Covariance Pooling for Facial Expression Recognition
Acharya, D., Huang, Z., Paudel, D. P., & Van Gool, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00077 · Cited by 181
Efficient Facial Feature Learning with Wide Ensemble-Based Convolutional Neural Networks
Siqueira, H. L., Magg, S., & Wermter, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6037 · Cited by 175
Yovel, G., & Duchaine, B. (2006) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.2006.18.4.580 · Cited by 168
Spatio-temporal convolutional features with nested LSTM for facial expression recognition
Yu, Z., Liu, G., Liu, Q., & Deng, J. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.07.028 · Cited by 162
Self-citationPatch-Gated CNN for Occlusion-aware Facial Expression Recognition
Li, Y., Zeng, J., Shan, S., & Chen, X. (2018) · 2018 24th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr.2018.8545853 · Cited by 158
Deep Neural Network Augmentation: Generating Faces for Affect Analysis
Kollias, D., Cheng, S., Ververas, E., Kotsia, I., & Zafeiriou, S. P. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01304-3 · Cited by 149
Geometry Guided Pose-Invariant Facial Expression Recognition
Zhang, F., Zhang, T., Mao, Q., & Xu, C. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2972114 · Cited by 143
Lai, Y.-H., & Lai, S. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00046 · Cited by 113
Facial Motion Prior Networks for Facial Expression Recognition
Chen, Y., Wang, J., Chen, S., Shi, Z., & Cai, J. (2019) · IEEE International Conference on Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip47243.2019.8965826 · Cited by 109
Conditional convolution neural network enhanced random forest for facial expression recognition
Liu, Y., Yuan, X., Gong, X., Xie, Z., Fang, F., & Luo, Z. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.07.016 · Cited by 97
Identity- and Pose-Robust Facial Expression Recognition through Adversarial Feature Learning
Wang, C., Wang, S., & Liang, G. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350872 · Cited by 90
Self-citationVideo-based Emotion Recognition Using Deeply-Supervised Neural Networks
Fan, Y., Lam, J. C. K., & Li, V. O. K. (2018) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3242969.3264978 · Cited by 85
Recognition of Facial Expressions in the Presence of Occlusion
Bourel, F., Chibelushi, C. C., & Low, A. A. (2001) · British Machine Vision Conference (BMVC) · doi:10.5244/c.15.23 · Cited by 79
A Facial Affect Analysis System for Autism Spectrum Disorder
Li, B., Mehta, S., Aneja, D., Foster, C. E., Ventola, P. E., Shic, F., & Shapiro, L. G. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803604 · Cited by 79
Occluded Facial Expression Recognition Enhanced through Privileged Information
Pan, B., Wang, S., & Xia, B. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351049 · Cited by 70
Self-citationRankBoost with l1 regularization for facial expression recognition and intensity estimation
Yang, P., Liu, Q., & Metaxas, D. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459371 · Cited by 69
Self-citationHand-Crafted Feature Guided Deep Learning for Facial Expression Recognition
Zeng, G., Zhou, J., Jia, X., Xie, W., & Shen, L. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00068 · Cited by 66
Separate Loss for Basic and Compound Facial Expression Recognition in the Wild
Li, Y., Lu, Y., Li, J., & Lu, G. (2019) · Asian Conference on Machine Learning · Cited by 51
CNN-based Facial Affect Analysis on Mobile DevicesPreprint
Hewitt, C., & Güneş, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.08775 · Cited by 47
Wang, S., Shuai, H., & Liu, Q. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3007531 · Cited by 45
Self-citationSpatiotemporal analysis of RGB-D-T facial images for multimodal pain level recognition
Irani, R., Nasrollahi, K., Simon, M. O., Corneanu, C. A., Escalera, S., Bahnsen, C., Lundtoft, D. H., Moeslund, T. B., Pedersen, T. L., Klitgaard, M.-L., et al. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301341 · Cited by 42
Feature-Level and Model-Level Audiovisual Fusion for Emotion Recognition in the Wild
Cai, J., Meng, Z., Khan, A. S., Li, Z., O’Reilly, J. D., Han, S., Liu, P., Chen, M., & Tong, Y. (2019) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr.2019.00089 · Cited by 42
Semantic Neighborhood-Aware Deep Facial Expression Recognition
Fu, Y., Wu, X., Li, X., Pan, Z., & Luo, D. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2991510 · Cited by 42
Deep Covariance Descriptors for Facial Expression RecognitionPreprint
Otberdout, N., Kacem, A., Daoudi, M., Ballihi, L., & Berretti, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.03869 · Cited by 40
Pose-specific non-linear mappings in feature space towards multiview facial expression recognition
Jampour, M., Lepetit, V., Mauthner, T., & Bischof, H. (2016) · Image and Vision Computing · doi:10.1016/j.imavis.2016.05.002 · Cited by 38
Facial Expression Recognition Using Deep Siamese Neural Networks with a Supervised Loss function
Hayale, W. S. A., Negi, P., & Mahoor, M. H. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756571 · Cited by 32
Robust facial expression recognition based on RPCA and AdaBoost
Mao, X., Xue, Y., Li, Z., Huang, K., & Shan-Wei, L. (2009) · International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS) · doi:10.1109/wiamis.2009.5031445 · Cited by 29
A Graph-Structured Representation with BRNN for Static-based Facial Expression Recognition
Zhong, L., Bai, C., Li, J., Chen, T., Li, S., & Liu, Y. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756615 · Cited by 29
Hammal, Z., Arguin, M., & Gosselin, F. (2009) · Journal of Vision · doi:10.1167/9.2.22 · Cited by 21
Deep Multi-Facial Patches Aggregation Network For Facial Expression RecognitionPreprint
Hazourli, A. R., Djeghri, A., Salam, H., & Othmani, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09298 · Cited by 9
Cited by
Adaptive Multilayer Perceptual Attention Network for Facial Expression Recognition
Liu, H., Cai, H., Lin, Q., Li, X., & Xiao, H. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3165321 · Cited by 129
Metadata
- Authors
- Zhao, Z., Liu, Q., & Wang, S.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2021
- Volume
- 30
- Pages
- 6544-6556
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W3179103990
- Cited by
- 375
- References
- 77
- FWCI
- 48.99
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.