The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Robust Lightweight Facial Expression Recognition Network with Label Distribution Training
Open accessAbstract
This paper presents an efficiently robust facial expression recognition (FER) network, named EfficientFace, which holds much fewer parameters but more robust to the FER in the wild. Firstly, to improve the robustness of the lightweight network, a local-feature extractor and a channel-spatial modulator are designed, in which the depthwise convolution is employed. As a result, the network is aware of local and global-salient facial features. Then, considering the fact that most emotions occur as combinations, mixtures, or compounds of the basic emotions, we introduce a simple but efficient label distribution learning (LDL) method as a novel training strategy. Experiments conducted on realistic occlusion and pose variation datasets demonstrate that the proposed EfficientFace is robust under occlusion and pose variation conditions. Moreover, the proposed method achieves state-of-the-art results on RAF-DB, CAER-S, and AffectNet-7 datasets with accuracies of 88.36%, 85.87%, and 63.70%, respectively, and a comparable result on the AffectNet-8 dataset with an accuracy of 59.89%. The code is public available at https://github.com/zengqunzhao/EfficientFace.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356 · cited 1×
“To achieve this purpose, it is intuitive to utilize the state-of-the-art lightweight models which are proposed by researchers on model compression, such as the MobileNet series (Howard et al. 2017; San-dler et al. 2018), and ShuffleNet series (Zhang et al. 2018; Ma et al. 2018).”
MethodRes2Net: A New Multi-Scale Backbone Architecture
Gao, S.-H., Cheng, M.-M., Zhao, K., Zhang, X.-Y., Yang, M.-H., & Torr, P. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2938758 · Cited by 3,385
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338
MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition
Guo, Y., Zhang, L., Hu, Y., He, X., & Gao, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_6 · Cited by 2,122
Dapretto, M., Davies, M. S., Pfeifer, J. H., Scott, A. A., Sigman, M., Bookheimer, S. Y., & Iacoboni, M. (2006) · Nature Neuroscience · doi:10.1038/nn1611 · Cited by 1,694
RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild
Deng, J., Guo, J., Ververas, E., Kotsia, I., & Zafeiriou, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00525 · Cited by 1,681
A GENERAL PSYCHOEVOLUTIONARY THEORY OF EMOTION
PLUTCHIK, R. (1980) · Theories of Emotion · doi:10.1016/b978-0-12-558701-3.50007-7 · Cited by 1,370
Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition
Wang, K., Peng, X., Yang, J., Meng, D., & Qiao, Y. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2956143 · Cited by 940
Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism
Li, Y., Zeng, J., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2886767 · Cited by 916 · cited 7×
“To verify the effectiveness of the proposed method, we conduct the experiments on three popular in-the-wild facial expression datasets: RAF-DB (Li and Deng 2018), CAER-S (Lee et al. 2019), and AffectNet (Mollahosseini, Hasani, and Mahoor 2017), and five realistic occlusion and pose variation datasets: FED-RO (Li et al.…”
BackgroundMethodInfluentialSuppressing Uncertainties for Large-Scale Facial Expression Recognition
Wang, K., Peng, X., Yang, J., Lu, S., & Qiao, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00693 · Cited by 772
Li, S., & Deng, W. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2868382 · Cited by 660
Automatic prediction of frustration
Kapoor, A., Burleson, W., & Picard, R. W. (2007) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2007.02.003 · Cited by 592
(2014) · doi:10.5555/2615731.2617415 · Cited by 553
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527
Deep Label Distribution Learning With Label Ambiguity
Gao, B.-B., Xing, C., Xie, C.-W., Wu, J., & Geng, X. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2689998 · Cited by 508
Learning active facial patches for expression analysis
Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., & Metaxas, D. N. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247974 · Cited by 369
Self-citationWang, A. T., Dapretto, M., Hariri, A. R., Sigman, M., & Bookheimer, S. Y. (2004) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-200404000-00015 · Cited by 363
Facial Expression Recognition with Inconsistently Annotated Datasets
Zeng, J., Shan, S., & Chen, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_14 · Cited by 341
Context-Aware Emotion Recognition Networks
Lee, J., Kim, S., Kim, S., Park, J., & Sohn, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01024 · Cited by 323
Peak-Piloted Deep Network for Facial Expression Recognition
Zhao, X., Liang, X., Liu, L., Li, T., Han, Y., Vasconcelos, N., & Yan, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_27 · Cited by 304
Automatically Detecting Pain in Video Through Facial Action Units
Lucey, P., Cohn, J. F., Matthews, I., Lucey, S., Sridharan, S., Howlett, J., & Prkachin, K. M. (2011) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2010.2082525 · Cited by 267 · cited 1×
“In the field of medicine, the pain detection is used for monitoring patient progress in the clinical setting (Lucey et al. 2010; Kaltwang, Rudovic, and Pantic 2012; Irani et al. 2015), the facial attributes such as expressions, action units, arousal, and valence are used for classifying autism…”
MethodContinuous Pain Intensity Estimation from Facial Expressions
Kaltwang, S., Rudovic, O., & Pantic, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33191-6_36 · Cited by 245 · cited 1×
“In the field of medicine, the pain detection is used for monitoring patient progress in the clinical setting (Lucey et al. 2010; Kaltwang, Rudovic, and Pantic 2012; Irani et al. 2015), the facial attributes such as expressions, action units, arousal, and valence are used for classifying autism…”
MethodLabel Distribution Learning on Auxiliary Label Space Graphs for Facial Expression Recognition
Chen, S., Wang, J., Chen, Y., Shi, Z., Geng, X., & Rui, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01400 · Cited by 229
Learning Multiscale Active Facial Patches for Expression Analysis
Zhong, L., Liu, Q., Yang, P., Huang, J., & Metaxas, D. N. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2354351 · Cited by 216
Self-citationDriver’s Facial Expression Recognition in Real-Time for Safe Driving
Jeong, M., & Ko, B. C. (2018) · Sensors · doi:10.3390/s18124270 · Cited by 195
Efficient Facial Feature Learning with Wide Ensemble-Based Convolutional Neural Networks
Siqueira, H. L., Magg, S., & Wermter, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.6037 · Cited by 175
Emotion Distribution Recognition from Facial Expressions
Zhou, Y., Xue, H., & Geng, X. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806328 · Cited by 172
Spatio-temporal convolutional features with nested LSTM for facial expression recognition
Yu, Z., Liu, G., Liu, Q., & Deng, J. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.07.028 · Cited by 162
Self-citationDeep Neural Network Augmentation: Generating Faces for Affect Analysis
Kollias, D., Cheng, S., Ververas, E., Kotsia, I., & Zafeiriou, S. P. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01304-3 · Cited by 149
Loth, E., Garrido, L., Ahmad, J., Watson, E., Duff, A., & Duchaine, B. (2018) · Molecular Autism · doi:10.1186/s13229-018-0187-7 · Cited by 139
Facial Motion Prior Networks for Facial Expression Recognition
Chen, Y., Wang, J., Chen, S., Shi, Z., & Cai, J. (2019) · IEEE International Conference on Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip47243.2019.8965826 · Cited by 109
Facial Emotion Distribution Learning by Exploiting Low-Rank Label Correlations Locally
Jia, X., Zheng, X., Li, W., Zhang, C., & Li, Z. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01007 · Cited by 97
Identity- and Pose-Robust Facial Expression Recognition through Adversarial Feature Learning
Wang, C., Wang, S., & Liang, G. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350872 · Cited by 90
Discriminant Distribution-Agnostic Loss for Facial Expression Recognition in the Wild
Farzaneh, A. H., & Qi, X. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00211 · Cited by 90
Data-Dependent Label Distribution Learning for Age Estimation
He, Z., Li, X., Zhang, Z. M., Wu, F. Q., Geng, X., Zhang, Y., Yang, M., & Zhuang, Y. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2655445 · Cited by 88
Separate Loss for Basic and Compound Facial Expression Recognition in the Wild
Li, Y., Lu, Y., Li, J., & Lu, G. (2019) · Asian Conference on Machine Learning · Cited by 86
A Facial Affect Analysis System for Autism Spectrum Disorder
Li, B., Mehta, S., Aneja, D., Foster, C. E., Ventola, P. E., Shic, F., & Shapiro, L. G. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803604 · Cited by 79
Khan, S. A., Hussain, S., Sun, X., & Yang, S. (2018) · IEEE Access · doi:10.1109/access.2018.2878601 · Cited by 67
Indoor Crowd Counting by Mixture of Gaussians Label Distribution Learning
Ling, M., & Geng, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2922818 · Cited by 59
A Deep Learning System for Recognizing Facial Expression in Real-Time
Yu, M., Dong, H., Alja’am, J. M., & Saddik, A. E. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3311747 · Cited by 56
STCAM: Spatial-Temporal and Channel Attention Module for Dynamic Facial Expression Recognition
Chen, W., Zhang, D., Li, M., & Lee, D.-J. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3027340 · Cited by 56
Real-time Facial Expression Recognition “In The Wild” by Disentangling 3D Expression from Identity
Koujan, M. R., Alharbawee, L., Giannakakis, G. A., Pugeault, N., & Roussos, A. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00084 · Cited by 48
Wang, S., Shuai, H., & Liu, Q. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3007531 · Cited by 45
Self-citationGaze-X: Adaptive, Affective, Multimodal Interface for Single-User Office Scenarios
Maat, L., & Pantic, M. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-72348-6_13 · Cited by 44
Spatiotemporal analysis of RGB-D-T facial images for multimodal pain level recognition
Irani, R., Nasrollahi, K., Simon, M. O., Corneanu, C. A., Escalera, S., Bahnsen, C., Lundtoft, D. H., Moeslund, T. B., Pedersen, T. L., Klitgaard, M.-L., et al. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301341 · Cited by 42
Lee, J. R. H., & Wong, A. K. C. (2020) · Canadian Conference on Computer and Robot Vision · doi:10.1109/crv50864.2020.00010 · Cited by 30
Real Time Facial Expression Recognition
Srivastava, S. (2012) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-642-27317-9_13 · Cited by 12
Ferro-Pérez, R., & Mitre-Hernández, H. A. (2020) · arXiv (Cornell University) · Cited by 8
Cited by
Distract Your Attention: Multi-Head Cross Attention Network for Facial Expression Recognition
Wen, Z., Lin, W., Wang, T., & Xu, G. (2023) · Biomimetics · doi:10.3390/biomimetics8020199 · Cited by 313
Metadata
- Authors
- Zhao, Z., Liu, Q., & Zhou, F.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2021-05-18
- Volume
- 35
- Issue
- 4
- Pages
- 3510-3519
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W3175546442
- Cited by
- 283
- References
- 57
- FWCI
- 53.16
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.