The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Facial Expression Recognition Using Enhanced Deep 3D Convolutional Neural Networks
Open accessAbstract
Deep Neural Networks (DNNs) have shown to outperform traditional methods in various visual recognition tasks including Facial Expression Recognition (FER). In spite of efforts made to improve the accuracy of FER systems using DNN, existing methods still are not generalizable enough in practical applications. This paper proposes a 3D Convolutional Neural Network method for FER in videos. This new network architecture consists of 3D Inception-ResNet layers followed by an LSTM unit that together extracts the spatial relations within facial images as well as the temporal relations between different frames in the video. Facial landmark points are also used as inputs to our network which emphasize on the importance of facial components rather than the facial regions that may not contribute significantly to generating facial expressions. Our proposed method is evaluated using four publicly available databases in subject-independent and cross-database tasks and outperforms state-of-the-art methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Active Shape Models-Their Training and Application
Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995) · Computer Vision and Image Understanding · doi:10.1006/cviu.1995.1004 · Cited by 7,190
Constants across cultures in the face and emotion.
Ekman, P., & Friesen, W. V. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0030377 · Cited by 5,509
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927467 · Cited by 5,409
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
Facial expression recognition based on Local Binary Patterns: A comprehensive study
Shan, C., Gong, S., & McOwan, P. W. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.08.005 · Cited by 2,300
Challenges in Representation Learning: A Report on Three Machine Learning Contests
Goodfellow, I. J., Erhan, D., Carrier, P. L., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D.-H., et al. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-42051-1_16 · Cited by 1,719
Human Detection Using Oriented Histograms of Flow and Appearance
Dalal, N., Triggs, B., & Schmid, C. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744047_33 · Cited by 1,652
Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2010) · Image and Vision Computing · doi:10.1016/j.imavis.2009.08.002 · Cited by 1,621
Going deeper in facial expression recognition using deep neural networks
Mollahosseini, A., Chan, D., & Mahoor, M. H. (2016) · 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2016.7477450 · Cited by 1,086
Self-citationWeb-Based Database for Facial Expression Analysis
Pantic, M., Valstar, M., Rademaker, R., & Maat, L. (2005) · 2005 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2005.1521424 · Cited by 1,039
Long-term Recurrent Convolutional Networks for Visual Recognition and Description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., & Darrell, T. (2014) · doi:10.21236/ada623249 · Cited by 968
Facial expression recognition from video sequences: temporal and static modeling
Cohen, I., Sebe, N., Garg, A., Chen, L. S., & Huang, T. S. (2003) · Computer Vision and Image Understanding · doi:10.1016/s1077-3142(03)00081-x · Cited by 874
Face Alignment at 3000 FPS via Regressing Local Binary Features
Ren, S., Cao, X., Wei, Y., & Sun, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.218 · Cited by 853
DISFA: A Spontaneous Facial Action Intensity Database
Mavadati, S. M., Mahoor, M. H., Bartlett, K., Trinh, P., & Cohn, J. F. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2013.4 · Cited by 793
Self-citationMolchanov, P., Yang, X., Gupta, S., Kim, K., Tyree, S., & Kautz, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.456 · Cited by 695
Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition
Sariyanidi, E., Gunes, H., & Cavallaro, A. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2366127 · Cited by 685
Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images
Song, S., & Xiao, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.94 · Cited by 681
300 Faces In-The-Wild Challenge: Database and results
Sagonas, C., Antonakos, E., Tzimiropoulos, G., Zafeiriou, S., & Pantic, M. (2016) · Image and Vision Computing · doi:10.1016/j.imavis.2016.01.002 · Cited by 638
Benitez-Quiroz, C. F., Srinivasan, R., & Martinez, A. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.600 · Cited by 607
Video-based emotion recognition using CNN-RNN and C3D hybrid networks
Fan, Y., Lu, X., Li, D., & Liu, Y. (2016) · Proceedings of the 18th ACM International Conference on Multimodal Interaction · doi:10.1145/2993148.2997632 · Cited by 556
Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark
Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130508 · Cited by 541 · cited 2×
“Since our method is designed mainly for classifying sequences of inputs, databases that contain only independent unrelated still images of facial expressions such as MultiPie [18] , SFEW [11] , FER2013 [17] cannot be examined by our method.”
MethodActive and dynamic information fusion for facial expression understanding from image sequences
Zhang, Y., & Ji, Q. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.93 · Cited by 461
Hidden Conditional Random Fields for Gesture Recognition
Wang, S. B., Quattoni, A., Morency, L.-P., Demirdjian, D., & Darrell, T. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.132 · Cited by 451
Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition
Liu, M., Shan, S., Wang, R., & Chen, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.226 · Cited by 392
A Semi-automatic Methodology for Facial Landmark Annotation
Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., & Pantic, M. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2013.132 · Cited by 387
The first facial expression recognition and analysis challenge
Valstar, M. F., Jiang, B., Mehu, M., Pantic, M., & Scherer, K. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771374 · Cited by 385
Scene labeling with LSTM recurrent neural networks
Byeon, W., Breuel, T. M., Raue, F., & Liwicki, M. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298977 · Cited by 375
Deeply Learning Deformable Facial Action Parts Model for Dynamic Expression Analysis
Liu, M., Li, S., Shan, S., Wang, R., & Chen, X. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16817-3_10 · Cited by 329
Conditional models for contextual human motion recognition
Sminchisescu, C., Kanaujia, A., Li, Z., & Metaxas, D. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.59 · Cited by 282
Authentic facial expression analysis
Sebe, N., Lew, M. S., Sun, Y., Cohen, I., Gevers, T., & Huang, T. S. (2007) · Image and Vision Computing · doi:10.1016/j.imavis.2005.12.021 · Cited by 257
Recognition of facial expressions and measurement of levels of interest from video
Yeasin, M., Bullot, B., & Sharma, R. (2006) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2006.870737 · Cited by 235
AU-aware Deep Networks for facial expression recognition
Liu, M., Li, S., Shan, S., & Chen, X. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553734 · Cited by 218
Conditional models for contextual human motion recognition
Sminchisescu, C., Kanaujia, A., & Metaxas, D. (2006) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.07.014 · Cited by 178
Introducing the Geneva Multimodal Emotion Portrayal (GEMEP) corpus
Bänziger, & Scherer, T. A. (2010) · Cited by 170
Deep learning the dynamic appearance and shape of facial action units
Jaiswal, S., & Valstar, M. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477625 · Cited by 169
Facial expression recognition with temporal modeling of shapes
Jain, S. D., Hu, C., & Aggarwal, J. K. (2011) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2011.6130446 · Cited by 140
Lee, S. H., Plataniotis, K. N., & Ro, Y. M. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2346515 · Cited by 124
Hasani, B., & Mahoor, M. H. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.99 · Cited by 108
Self-citationPCA-based dictionary building for accurate facial expression recognition via sparse representation
Mohammadi, M. R., Fatemizadeh, E., & Mahoor, M. H. (2014) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2014.03.006 · Cited by 103
Self-citationRecognizing lower face action units for facial expression analysis
Tian, Y., Takeo Kanade, A., & Cohn, J. F. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840678 · Cited by 94
Using moment invariants and HMM in facial expression recognition
Zhu, Y., Silva, L. C. D., & Ko, C. C. (2002) · Pattern Recognition Letters · doi:10.1016/s0167-8655(01)00108-8 · Cited by 93
Facial Expression Recognition from World Wild Web
Mollahosseini, A., Hassani, B., Salvador, M. J., Abdollahi, H., Chan, D., & Mahoor, M. H. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.188 · Cited by 83
Self-citationTracking Vertex Flow and Model Adaptation for Three-Dimensional Spatiotemporal Face Analysis
Sun, Y., Chen, X., Rosato, M., & Yin, L. (2010) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/tsmca.2010.2041659 · Cited by 82
Structure-Preserving Sparse Decomposition for Facial Expression Analysis
Taheri, S., Qiu, Q., & Chellappa, R. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2331141 · Cited by 75
Facial Expression Recognition Based on Local Phase Quantization and Sparse Representation
Wang, Z., & Ying, Z. (2012) · International Conference on Natural Computation (ICNC) · doi:10.1109/icnc.2012.6234551 · Cited by 70
Facial expression recognition using $${l}_{p}$$ l p -norm MKL multiclass-SVM
Zhang, X., Mahoor, M. H., & Mavadati, S. M. (2015) · Machine Vision and Applications · doi:10.1007/s00138-015-0677-y · Cited by 70
Self-citationDynamic Facial Expression Recognition Using A Bayesian Temporal Manifold Model
Shan, C., Gong, S., & McOwan, P. W. (2006) · British Machine Vision Conference (BMVC) · doi:10.5244/c.20.31 · Cited by 60
Cross-database evaluation for facial expression recognition
Mayer, C., Eggers, M., & Radig, B. (2014) · Pattern Recognition and Image Analysis · doi:10.1134/s1054661814010106 · Cited by 55
Cross-Domain Facial Expression Recognition Using Supervised Kernel Mean Matching
Miao, Y.-Q., Araujo, R., & Kamel, M. S. (2012) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2012.178 · Cited by 35
Facial expression recognition using lp-norm MKL multiclass-SVM
Zhang, X., Mahoor, M. H., & Mavadati, S. M. (2015) · Cited by 27
Self-citationModeling Spatial and Temporal Cues for Multi-label Facial Action Unit DetectionPreprint
Chu, W.-S., de la Torre, F. N., & Cohn, J. F. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.00911 · Cited by 20
Facial expression recognition with discriminatory graphical models
Hasani, B., Arzani, M. M., Fathy, M., & Raahemifar, K. (2016) · International Conference on Signal Processing and Intelligent Systems (ICSPIS) · doi:10.1109/icspis.2016.7869885 · Cited by 6
Self-citation
Cited by
Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 1,753
A Brief Review of Facial Emotion Recognition Based on Visual Information
Ko, B. (2018) · Sensors · doi:10.3390/s18020401 · Cited by 724
Attention mechanism-based CNN for facial expression recognition
Li, J., Jin, K., Zhou, D., Kubota, N., & Ju, Z. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.06.014 · Cited by 347
Local Learning With Deep and Handcrafted Features for Facial Expression Recognition
Georgescu, M.-I., Ionescu, R. T., & Popescu, M. (2019) · IEEE Access · doi:10.1109/access.2019.2917266 · Cited by 331
Facial Expression Recognition: A Survey
Huang, Y., Chen, F., Lv, S., & Wang, X. (2019) · Symmetry · doi:10.3390/sym11101189 · Cited by 195
Metadata
- Authors
- Hasani, B., & Mahoor, M. H.
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 2278-2288
- OpenAlex
- W2617750261
- Cited by
- 287
- References
- 65
- FWCI
- 35.85
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.