The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Deep Region and Multi-label Learning for Facial Action Unit Detection
Abstract
Region learning (RL) and multi-label learning (ML) have recently attracted increasing attentions in the field of facial Action Unit (AU) detection. Knowing that AUs are active on sparse facial regions, RL aims to identify these regions for a better specificity. On the other hand, a strong statistical evidence of AU correlations suggests that ML is a natural way to model the detection task. In this paper, we propose Deep Region and Multi-label Learning (DRML), a unified deep network that simultaneously addresses these two problems. One crucial aspect in DRML is a novel region layer that uses feed-forward functions to induce important facial regions, forcing the learned weights to capture structural information of the face. Our region layer serves as an alternative design between locally connected layers (i.e., confined kernels to individual pixels) and conventional convolution layers (i.e., shared kernels across an entire image). Unlike previous studies that solve RL and ML alternately, DRML by construction addresses both problems, allowing the two seemingly irrelevant problems to interact more directly. The complete network is end-to-end trainable, and automatically learns representations robust to variations inherent within a local region. Experiments on BP4D and DISFA benchmarks show that DRML performs the highest average F1-score and AUC within and across datasets in comparison with alternative methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
Understanding Neural Networks Through Deep Visualization
Yosinski, J., Clune, J., Nguyen, A. M., Fuchs, T. J., & Lipson, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06579 · Cited by 1,358
Learning a Deep Compact Image Representation for Visual Tracking
Wang, N., & Yeung, D. (2013) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · Cited by 836
Compound facial expressions of emotion
Du, S., Tao, Y., & Martinez, A. M. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1322355111 · Cited by 806
DISFA: A Spontaneous Facial Action Intensity Database
Mavadati, S. M., Mahoor, M. H., Bartlett, K., Trinh, P., & Cohn, J. F. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2013.4 · Cited by 793
Facial Expression Recognition via a Boosted Deep Belief Network
Liu, P., Han, S., Meng, Z., & Tong, Y. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.233 · Cited by 650
Automatic Recognition of Facial Actions in Spontaneous Expressions
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C., Fasel, I. R., & Movellan, J. R. (2006) · Journal of Multimedia · doi:10.4304/jmm.1.6.22-35 · Cited by 468
Learning hierarchical representations for face verification with convolutional deep belief networks
Huang, G. B., Lee, H., & Learned-Miller, E. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247968 · Cited by 407
Dynamics of facial expression extracted automatically from video
Littlewort, G., Bartlett, M. S., Fasel, I., Susskind, J., & Movellan, J. (2006) · Image and Vision Computing · doi:10.1016/j.imavis.2005.09.011 · Cited by 358
LIBLINEAR: A Library for Large Linear Classification
FanRong-En, ChangKai-Wei, HsiehCho-Jui, WangXiang-Rui, & LinChih-Jen (2008) · Journal of Machine Learning Research · doi:10.5555/1390681.1442794 · Cited by 328
Selective Transfer Machine for Personalized Facial Action Unit Detection
Chu, W.-S., De la Torre, F., & Cohn, J. F. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.451 · Cited by 307
Self-citationDeep Convolutional Ranking for Multilabel Image AnnotationPreprint
Gong, Y., Jia, Y., Leung, T., Toshev, A., & Ioffe, S. (2014) · Cited by 272
A high-resolution spontaneous 3D dynamic facial expression database
Zhang, X., Yin, L., Cohn, J. F., Canavan, S., Reale, M., Horowitz, A., & Liu, P. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553788 · Cited by 236
Action unit detection using sparse appearance descriptors in space-time video volumes
Jiang, B., Valstar, M. F., & Pantic, M. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771416 · Cited by 233
Web-scale training for face identification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298891 · Cited by 227
AU-aware Deep Networks for facial expression recognition
Liu, M., Li, S., Shan, S., & Chen, X. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553734 · Cited by 218
Learning Multiscale Active Facial Patches for Expression Analysis
Zhong, L., Liu, Q., Yang, P., Huang, J., & Metaxas, D. N. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2354351 · Cited by 216
Disentangling Factors of Variation for Facial Expression Recognition
Rifai, S., Bengio, Y., Courville, A., Vincent, P., & Mirza, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_58 · Cited by 215
Joint patch and multi-label learning for facial action unit detection
Zhao, K., Chu, W.-S., De la Torre, F., Cohn, J. F., & Zhang, H. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298833 · Cited by 196
Self-citationSiddiqi, M. H., Ali, R., Khan, A. M., Park, Y.-T., & Lee, S. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2405346 · Cited by 183
Towards Class-Imbalance Aware Multi-Label Learning
Zhang, M.-L., Li, Y.-K., Yang, H., & Liu, X.-Y. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3027509 · Cited by 170
Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face
Lucey, S., Bilal, A., & F., J. (2007) · Face Recognition · doi:10.5772/4841 · Cited by 161
Robust Representation and Recognition of Facial Emotions Using Extreme Sparse Learning
Shojaeilangari, S., Yau, W., Nandakumar, K., Li, J., & Teoh, E. K. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2416634 · Cited by 154
Capturing Global Semantic Relationships for Facial Action Unit Recognition
Wang, Z., Li, Y., Wang, S., & Ji, Q. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.410 · Cited by 126
Multi-conditional Latent Variable Model for Joint Facial Action Unit Detection
Eleftheriadis, S., Rudovic, O. O., & Pantić, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.432 · Cited by 98
Structure-Preserving Sparse Decomposition for Facial Expression Analysis
Taheri, S., Qiu, Q., & Chellappa, R. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2331141 · Cited by 75
Facial Action Unit Event Detection by Cascade of Tasks
Ding, X., Chu, W.-S., de la Torre, F. N., Cohn, J. F., & Wang, Q. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.298 · Cited by 73
Self-citationConfidence Preserving Machine for Facial Action Unit Detection
Zeng, J., Chu, W.-S., la Torre, F. D., Cohn, J. F., & Xiong, Z. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.413 · Cited by 67
Self-citationLiu, P., Zhou, J. T., Tsang, I. W.-H., Meng, Z., Han, S., & Tong, Y. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10593-2_11 · Cited by 55
Zhu, Y., la Torre, F. D., Cohn, J. F., & Zhang, Y. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.10 · Cited by 54
Task-dependent multi-task multiple kernel learning for facial action unit detection
Zhang, X., & Mahoor, M. H. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2015.08.026 · Cited by 37
Learning to combine local models for facial Action Unit detection
Jaiswal, S., Martínez, B., & Valstar, M. (2015) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2015.7284872 · Cited by 18
Cited by
Openface 2.0: facial behavior analysis toolkit
Baltrusaitis, T., Zadeh, A., Lim, Y. C., & Morency, L.-P. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00019 · Cited by 1,861
Metadata
- Authors
- Zhao, K., Chu, W.-S., & Zhang, H.
- Venue
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2016-06
- Pages
- 3391-3399
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2421475762
- Cited by
- 329
- References
- 38
- FWCI
- 11.04
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.