The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition
Abstract
Temporal information has useful features for recognizing facial expressions. However, to manually design useful features requires a lot of effort. In this paper, to reduce this effort, a deep learning technique, which is regarded as a tool to automatically extract useful features from raw data, is adopted. Our deep network is based on two different models. The first deep network extracts temporal appearance features from image sequences, while the other deep network extracts temporal geometry features from temporal facial landmark points. These two models are combined using a new integration method in order to boost the performance of the facial expression recognition. Through several experiments, we show that the two models cooperate with each other. As a result, we achieve superior performance to other state-of-the-art methods in the CK+ and Oulu-CASIA databases. Furthermore, we show that our new integration method gives more accurate results than traditional methods, such as a weighted summation and a feature concatenation method.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In benchmarking experiments across various datasets, many studies found that features extracted with CNNs lead to higher recognition accuracies than handcrafted features (e.g., [87], [88], [89], [90], [91], [64], [92], [93], [94]).”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [88]
In the citing paper
“This way, researchers artificially increase the number of available examples or training epochs by a factor typically between 10 and 30 [88], [113], [114], and up to 300 [115].”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [88]
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846
Face detection, pose estimation, and landmark localization in the wild
Zhu, X., & Ramanan, D. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248014 · Cited by 2,171
Supervised Descent Method and Its Applications to Face Alignment
Xiong, X., & la Torre, F. D. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.75 · Cited by 1,926
A Spatio-Temporal Descriptor Based on 3D-Gradients
Klaeser, A., Marszałek, M., & Schmid, C. (2008) · British Machine Vision Conference (BMVC) · doi:10.5244/c.22.99 · Cited by 1,842
A 3-dimensional sift descriptor and its application to action recognition
Scovanner, P., Ali, S., & Shah, M. A. (2007) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1291233.1291311 · Cited by 1,605
Facial expression recognition from near-infrared videos
Zhao, G., Huang, X., Taini, M., Li, S. Z., & Pietikäinen, M. (2011) · Image and Vision Computing · doi:10.1016/j.imavis.2011.07.002 · Cited by 771
A data-driven approach to cleaning large face datasets
Ng, H., & WINKLER, S. S. (2014) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2014.7025068 · Cited by 673
Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition
Liu, M., Shan, S., Wang, R., & Chen, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.226 · Cited by 392
Learning active facial patches for expression analysis
Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., & Metaxas, D. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247974 · Cited by 380
Deeply Learning Deformable Facial Action Parts Model for Dynamic Expression Analysis
Liu, M., Li, S., Shan, S., Wang, R., & Chen, X. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16817-3_10 · Cited by 333
Ghimire, D., & Lee, J. (2013) · Sensors · doi:10.3390/s130607714 · Cited by 303
Dense 3D face alignment from 2D videos in real-time
Jeni, L. A., Cohn, J. F., & Takeo Kanade, A. (2015) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2015.7163142 · Cited by 179
Towards a practical lipreading system
Zhou, Z., Zhao, G., & Pietikäinen, M. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995345 · Cited by 162
Capturing Complex Spatio-temporal Relations among Facial Muscles for Facial Expression Recognition
Wang, Z., Wang, S., & Ji, Q. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.439 · Cited by 154
Spatio-temporal covariance descriptors for action and gesture recognition
Sanin, A., Sanderson, C., Harandi, M. T., & Lovell, B. C. (2013) · Proceedings · doi:10.1109/wacv.2013.6475006 · Cited by 154
Facial expression recognition with temporal modeling of shapes
Jain, S. D., Hu, C., & Aggarwal, J. K. (2011) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2011.6130446 · Cited by 140
Spatio-temporal covariance descriptors for action and gesture recognition
Sanin, A., Sanderson, C., Harandi, M. T., & Lovell, B. C. (2013) · Cited by 123
Dynamic Facial Expression Recognition Using Longitudinal Facial Expression Atlases
Guo, Y., Zhao, G., & Pietikäinen, M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33709-3_45 · Cited by 96
Exploring Bag of Words Architectures in the Facial Expression Domain
Sikka, K., Wu, T., Susskind, J., & Bartlett, M. S. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33868-7_25 · Cited by 94
Facial Expression Recognition in Perceptual Color Space
Lajevardi, S. M., & Wu, H. R. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2197628 · Cited by 91
Automatic Facial Expression Recognition System Based on Geometric and Appearance Features
Youssif, A. A. A., & Asker, W. A. A. (2011) · Computer and Information Science · doi:10.5539/cis.v4n2p115 · Cited by 55
Manifold based Sparse Representation for robust expression recognition without neutral subtraction
Ptucha, R., Tsagkatakis, G., & Savakis, A. E. (2011) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2011.6130512 · Cited by 48
Spatio-temporal Event Classification Using Time-Series Kernel Based Structured Sparsity
Jeni, L. A., Lörincz, A., Szabó, Z., Cohn, J. F., & Takeo Kanade, A. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10593-2_10 · Cited by 30
Flagged first, then by citations
Research works printed but not linked: 4
- sanin, 2013, Spatiotemporal covariance descriptors for action and gesture recognition, WACV 2013, 103
- valstar, 2010, Induced disgust, happiness and surprise: an addition to the mmi facial expression database, Proceedings of Int'l Conf Language Resources and Evaluation Workshop on EMOTION, 65
- glorot, 2011, Deep sparse rectifier networks, Proceedings of the 14th International Conference on Artificial Intelligence and Statistics JMLR W&CP Volume, 15, 315
- girard, 2013, Ground truth facs action unit coding on the group formation task, Tech Rep
Cited by
Sorted by citations
Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 2,175
“In static-based methods [12], [13], [14], the feature representation is encoded with only spatial information from the current single image, whereas dynamicbased methods [15], [16], [17] consider the temporal relation among contiguous frames in the input facial expression sequence.”
MethodInfluentialRegion Attention Networks for Pose and Occlusion Robust Facial Expression Recognition
Wang, K., Peng, X., Yang, J., Meng, D., & Qiao, Y. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2956143 · Cited by 970
“Due to its potential applications in various fields, such as intelligent tutoring systems, service robots, driver fatigue monitoring, Facial Expression Recognition (FER) has attracted increasing attention in the computer vision community recently [28], [5],”
BackgroundA Brief Review of Facial Emotion Recognition Based on Visual Information
Ko, B. C. (2018) · Sensors · doi:10.3390/s18020401 · Cited by 790
“• Joint fine-tuning in deep neural networks [48] Sequence 70.”
BackgroundMethodInfluentialLi, S., & Deng, W. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2868382 · Cited by 688
“In DTAGN [30], two different models were combined to extract temporal appearance and geometry features simultaneously.”
MethodA systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 713
Metadata
- Authors
- Jung, H., Lee, S., Yim, J., Park, S., & Kim, J.
- Affiliations
- Korea Advanced Institute of Science and Technology (KR)
- Venue
- 2015 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2015-12
- Pages
- 2983-2991
- OpenAlex
- W2217426128
- Cited by
- 781
- References
- 27
- Citations by year
- 2026: 9 · 2025: 29 · 2024: 45 · 2023: 82 · 2022: 87 · 2021: 127 · 2020: 115 · 2019: 125 · 2018: 92 · 2017: 55 · 2016: 15
- FWCI
- 72.14
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- KAIST Institutional Repository (KAIST) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- facial expression recognition, joint fine-tuning, facial landmarks, feature integration, image sequences, deep neural network, CK+ database
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
