The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition
Abstract
Temporal information has useful features for recognizing facial expressions. However, to manually design useful features requires a lot of effort. In this paper, to reduce this effort, a deep learning technique, which is regarded as a tool to automatically extract useful features from raw data, is adopted. Our deep network is based on two different models. The first deep network extracts temporal appearance features from image sequences, while the other deep network extracts temporal geometry features from temporal facial landmark points. These two models are combined using a new integration method in order to boost the performance of the facial expression recognition. Through several experiments, we show that the two models cooperate with each other. As a result, we achieve superior performance to other state-of-the-art methods in the CK+ and Oulu-CASIA databases. Furthermore, we show that our new integration method gives more accurate results than traditional methods, such as a weighted summation and a feature concatenation method.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In benchmarking experiments across various datasets, many studies found that features extracted with CNNs lead to higher recognition accuracies than handcrafted features (e.g., [87], [88], [89], [90], [91], [64], [92], [93], [94]).”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [88]
In the citing paper
“This way, researchers artificially increase the number of available examples or training epochs by a factor typically between 10 and 30 [88], [113], [114], and up to 300 [115].”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [88]
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846
Face detection, pose estimation, and landmark localization in the wild
Zhu, X., & Ramanan, D. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248014 · Cited by 2,171
Supervised Descent Method and Its Applications to Face Alignment
Xiong, X., & la Torre, F. D. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.75 · Cited by 1,926
A Spatio-Temporal Descriptor Based on 3D-Gradients
Klaeser, A., Marszałek, M., & Schmid, C. (2008) · British Machine Vision Conference (BMVC) · doi:10.5244/c.22.99 · Cited by 1,842
A 3-dimensional sift descriptor and its application to action recognition
Scovanner, P., Ali, S., & Shah, M. A. (2007) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1291233.1291311 · Cited by 1,605
Facial expression recognition from near-infrared videos
Zhao, G., Huang, X., Taini, M., Li, S. Z., & Pietikäinen, M. (2011) · Image and Vision Computing · doi:10.1016/j.imavis.2011.07.002 · Cited by 771
Research works printed but not linked: 4
- sanin, 2013, Spatiotemporal covariance descriptors for action and gesture recognition, WACV 2013, 103
- valstar, 2010, Induced disgust, happiness and surprise: an addition to the mmi facial expression database, Proceedings of Int'l Conf Language Resources and Evaluation Workshop on EMOTION, 65
- glorot, 2011, Deep sparse rectifier networks, Proceedings of the 14th International Conference on Artificial Intelligence and Statistics JMLR W&CP Volume, 15, 315
- girard, 2013, Ground truth facs action unit coding on the group formation task, Tech Rep
Cited by
Sorted by citations
Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 2,175
“In static-based methods [12], [13], [14], the feature representation is encoded with only spatial information from the current single image, whereas dynamicbased methods [15], [16], [17] consider the temporal relation among contiguous frames in the input facial expression sequence.”
MethodInfluentialRegion Attention Networks for Pose and Occlusion Robust Facial Expression Recognition
Wang, K., Peng, X., Yang, J., Meng, D., & Qiao, Y. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2956143 · Cited by 970
“Due to its potential applications in various fields, such as intelligent tutoring systems, service robots, driver fatigue monitoring, Facial Expression Recognition (FER) has attracted increasing attention in the computer vision community recently [28], [5],”
BackgroundA Brief Review of Facial Emotion Recognition Based on Visual Information
Ko, B. C. (2018) · Sensors · doi:10.3390/s18020401 · Cited by 790
“• Joint fine-tuning in deep neural networks [48] Sequence 70.”
BackgroundMethodInfluentialLi, S., & Deng, W. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2868382 · Cited by 688
“In DTAGN [30], two different models were combined to extract temporal appearance and geometry features simultaneously.”
MethodA systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 713
Spatial–Temporal Recurrent Neural Network for Emotion Recognition
Zhang, T., Zheng, W., Cui, Z., Zong, Y., & Li, Y. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2788081 · Cited by 595
Facial Expression Recognition Based on Deep Evolutional Spatial-Temporal Networks
Zhang, K., Huang, Y., Du, Y., & Wang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2689999 · Cited by 493
“of previous state-of-the-art methods [7], [6], [24] on this database is lower than the results on CK+.”
BackgroundMethodResultInfluentialFacial Expression Recognition by De-expression Residue Learning
Yang, H., Çiftçi, U. A., & Yin, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00231 · Cited by 466
“Our DeRL method achieves the highest accuracy compared to those of state-of-the-art methods, including CNN-based methods (DTAGN-Joint [12], FN2EN [6], and PPDN [35]) and handcrafted features based methods (LBP-TOP [34], HOG 3D [14] and STM-Explet [18]).”
BackgroundMethodFacial emotion recognition using convolutional neural networks (FERC)
Mehendale, N. D. (2020) · SN Applied Sciences · doi:10.1007/s42452-020-2234-1 · Cited by 440
“Jung et al. [38] tried to work with fixed frames which make the system not so efficient with video input.”
BackgroundDeep Affect Prediction in-the-Wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond
Kollias, D., Tzirakis, P., Nicolaou, M. A., Papaioannou, A., Zhao, G., Schuller, B. W., Kotsia, I., & Zafeiriou, S. P. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01158-4 · Cited by 406
“still used in many recent works [27], are the Cohn-Kanade database [34,54], MMI database [40,56], Multi-PIE database [22] and the BU-3D and BU-4D databases [61, 62].”
MethodFacial Expression Recognition with Inconsistently Annotated Datasets
Zeng, J., Shan, S., & Chen, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_14 · Cited by 362
“Based on these datasets, lots of FER approaches are proposed and achieve the state-of-the-art performance[27, 25, 30, 4, 14, 34, 18].”
BackgroundIsland Loss for Learning Discriminative Features in Facial Expression Recognition
Cai, J., Meng, Z., Khan, A. S., Li, Z., O’Reilly, J. D., & Tong, Y. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00051 · Cited by 339
“Benefiting from the advance in feature learning, features can be learned either unsupervised by sparse coding [25], [59], [35], [29] or supervised by deep learning [36], [44], [24], [5], [8], [30], [21], [15], [22], [58], [9], [51], [3], [49],”
BackgroundMethodInfluentialIdentity-Aware Convolutional Neural Network for Facial Expression Recognition
Meng, Z., Liu, P., Cai, J., Han, S., & Tong, Y. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.140 · Cited by 335
“deep learning [34], [42], [23], [29], [20], [16], [49], [30], [13], [21], [53].”
MethodPeak-Piloted Deep Network for Facial Expression Recognition
Zhao, X., Liang, X., Liu, L., Li, T., Han, Y., Vasconcelos, N. M., & Yan, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46475-6_27 · Cited by 317
“59% DTAGN(Joint) [7] sequence-based 81.”
BackgroundInfluentialA Deep Neural Network-Driven Feature Learning Method for Multi-view Facial Expression Recognition
Zhang, T., Zheng, W., Cui, Z., Zong, Y., Yan, J., & Yan, K. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2598092 · Cited by 314
“Table II shows the accuracies of the comparison between 2D JFDNN transformed from [18] and our neural network corresponding to the seven yaw head poses (0◦, 15◦, 30◦, 45◦, 60◦, 75◦ and 90◦).”
BackgroundMethodInfluentialKim, J., Kim, B., Roy, P. P., & Jeong, D.-M. (2019) · IEEE Access · doi:10.1109/access.2019.2907327 · Cited by 315
Self-citationAff-Wild: Valence and Arousal ‘In-the-Wild’ Challenge
Zafeiriou, S. P., Kollias, D., Nicolaou, M. A., Papaioannou, A., Zhao, G., & Kotsia, I. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.248 · Cited by 297
“Some representative datasets, which are still used in many recent works [18], include the Cohn-Kanade database [33, 24], the MMI database [29, 35], the Multi-PIE database [17] and the BU-3D and BU-4D databases [41, 40].”
MethodFrom Facial Expression Recognition to Interpersonal Relation Prediction
Zhang, Z., Luo, P., Loy, C. C., & Tang, X. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-017-1055-1 · Cited by 315
“Social signal processing mainly involves facial expression recognition (Zhao and Pietikainen 2007; Tian et al. 2011; Liu et al. 2014b; Ruiz et al. 2015; Wu and Ji 2016; Fabian Benitez-Quiroz et al. 2016; Liu et al. 2014a, c; Jung et al. 2015;Mollahosseini et al. 2016; Fabian Benitez-Quiroz et al. 2016; Zhao et al. 201…”
BackgroundJoint Pose and Expression Modeling for Facial Expression Recognition
Zhang, F., Zhang, T., Mao, Q., & Xu, C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00354 · Cited by 270
“, Support Vector Machines (SVMs) [16], softmax [18], and logistic regression [36], to train a facial expression recognizer for a target expression.”
BackgroundKim, D. H., Baddar, W. J., Jang, J., & Ro, Y. M. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2695999 · Cited by 257
“In [32], a temporal geometric feature was jointly learned in order to reduce the effect of the identity on the learned spatio-temporal appearance features.”
BackgroundMethodInfluentialZhang, S., Yang, Y., Chen, C., Zhang, X., Leng, Q., & Zhao, X. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121692 · Cited by 290
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243
Jiang, Y., Li, W., Hossain, M. S., Chen, M., Alelaiwi, A. A., & Al-Hammadi, M. H. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.06.019 · Cited by 224
Xie, S., & Hu, H. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2844085 · Cited by 216
“In this paper, we compare the recognition accuracy of our model with several competitive approaches, including conventional methods (LAP [44], MPSD [45] and MCSPL [30]) and deep-based methods (DTAN [37], DTGN [37] and 3DCNN-DAP [46]).”
BackgroundMethodInfluentialFrame Attention Networks for Facial Expression Recognition in Videos
Meng, D., Peng, X., Wang, K., & Qiao, Y. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803603 · Cited by 222
“ST network [23] S: the last frame T: all frames S: the last frame T: all frames 98.50 DTAGN [12] Fixed length Fixed length 97.25 CNN+Island loss[24] The last three frames and the first frame The last three frames and the first frame”
BackgroundMethodInfluentialAdaptive Deep Metric Learning for Identity-Aware Facial Expression Recognition
Liu, X., Kumar, B. V. K. V., You, J., & Jia, P. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.79 · Cited by 209
“12% DTAGN [16] 97.”
Xie, S., Hu, H., & Wu, Y. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.03.019 · Cited by 209
Huang, X., Wang, S., Liu, X., Zhao, G., Feng, X., & Pietikäinen, M. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2713359 · Cited by 212
“sion recognition [18], [19], [20], [21], [22].”
Karnati, M., Seal, A., Krejcar, O., & Yazidi, A. (2020) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3031835 · Cited by 198
“In recent years, it is observed from the literature that deep-learning-based methods are superior to handcrafted features-based methods for FER tasks [17], [18], [43], [44].”
BackgroundFacial Expression Recognition: A Survey
Huang, Y., Chen, F., Lv, S., & Wang, X. (2019) · Symmetry · doi:10.3390/sym11101189 · Cited by 195
“In order to take advantage of temporal features for recognising facial expression, a new integration method named Deep Temporal Appearance-Geometry Network (DTAGN) is proposed in [86].”
MethodAU-assisted Graph Attention Convolutional Network for Micro-Expression Recognition
Xie, H., Lo, L., Shuai, H.-H., & Cheng, W.-H. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3414012 · Cited by 199
“, cropping [45], flipping [40], rotating [15, 44].”
Three convolutional neural network models for facial expression recognition in the wild
Shao, J., & Qian, Y. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.05.005 · Cited by 192
Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network
Minaee, S., Minaei, M., & Abdolrashidi, A. (2021) · Sensors · doi:10.3390/s21093046 · Cited by 251
Automatic depression recognition using CNN with attention mechanism from videos
He, L., Chan, J. C.-W., & Wang, Z. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.10.015 · Cited by 179
Automated emotion recognition: Current trends and future perspectives
Maithri, M., Raghavendra, U., Gudigar, A., Samanth, J., Barua, P. D., Murugappan, M., Chakole, Y., & Acharya, U. R. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106646 · Cited by 172
A Compact Deep Learning Model for Robust Facial Expression Recognition
Kuo, C.-M., Lai, S., & Sarkis, M. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00286 · Cited by 170
“The joint-fine tuning method [16] adopted data augmentation strategy with 7 different rotation angles to obtain 14 times more data.”
MethodInfluentialMicro-expression recognition based on 3D flow convolutional neural network
Li, J., Wang, Y., See, J., & Liu, W. (2018) · Pattern Analysis and Applications · doi:10.1007/s10044-018-0757-5 · Cited by 177
“Subsequent works by [15] and [13] proposed models that jointly learn from two types of data, i.”
BackgroundAnalysing Affective Behavior in the second ABAW2 Competition
Kollias, D., & Zafeiriou, S. P. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00408 · Cited by 168
POSTER: A Pyramid Cross-Fusion Transformer Network for Facial Expression Recognition
Zheng, C., Mendieta, M., & Chen, C. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00339 · Cited by 170
“[16] proposed two networks, where one receives the image as input and another receives facial landmark as input.”
BackgroundMethodInfluentialKhan, A. R. (2022) · Information · doi:10.3390/info13060268 · Cited by 179
Spatio-temporal convolutional features with nested LSTM for facial expression recognition
Yu, Z., Liu, G., Liu, Q., & Deng, J. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.07.028 · Cited by 162
Web-based fully automated cephalometric analysis by deep learning
Kim, H. H., Shim, E., Park, J., Kim, Y., Lee, U., & Kim, Y. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105513 · Cited by 161
Learning Affective Video Features for Facial Expression Recognition via Hybrid Deep Learning
Zhang, S., Pan, X., Cui, Y., Zhao, X., & Liu, L. (2019) · IEEE Access · doi:10.1109/access.2019.2901521 · Cited by 154
“[10] propose a small temporal CNN to extract temporal geometric features from facial landmark points.”
BackgroundModeling Stylized Character Expressions via Deep Learning
Aneja, D., Colburn, A., Faigin, G., Shapiro, L. G., & Mones, B. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-54184-6_9 · Cited by 150
“Joint Fine-Tuning in Deep Neural Networks [33] have also been used to combine temporal appearance features from image sequences and temporal geometry features from temporal facial landmark points to enhance the performance of the facial expression recognition.”
MethodDetection of Cattle Using Drones and Convolutional Neural Networks
Rivas, A., Chamoso, P., González‐Briones, A., & Corchado, J. M. (2018) · Sensors · doi:10.3390/s18072048 · Cited by 144
“For this purpose, softmax [45] is the selected activation function, which is defined in Equation (1).”
BackgroundAutomatic Recognition of Student Engagement Using Deep Learning and Facial Expression
Nezami, O. M., Dras, M., Hamey, L., Richards, D., Wan, S., & Paris, C. L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-46133-1_17 · Cited by 138
“Deep learning models have been successful in automatically recognizing facial expressions in images [15,23,24,30,37,38,39].”
BackgroundResNet-Based Vehicle Classification and Localization in Traffic Surveillance Systems
Jung, H., Choi, M.-K., Jung, J., Lee, J.-H., Kwon, S., & Jung, W. Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.129 · Cited by 137
Self-citationAdvances in Facial Expression Recognition: A Survey of Methods, Benchmarks, Models, and Datasets
Kopalidis, T., Solachidis, V., Vretos, N., & Daras, P. (2024) · Information · doi:10.3390/info15030135 · Cited by 205
Can Micro-Expression be Recognized Based on Single Apex Frame?
Li, Y., Huang, X., & Zhao, G. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451376 · Cited by 134
“Deep learning has been shown efficient on facial expression recognition [14], we present to use deep learning to recognize ME.”
MethodLi, S., & Deng, W. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1131-1 · Cited by 134
Geometry Guided Pose-Invariant Facial Expression Recognition
Zhang, F., Zhang, T., Mao, Q., & Xu, C. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2972114 · Cited by 135
“[57], [58] and learning-based features [8], [9], [59], [60].”
BackgroundFacial Expression Recognition with Identity and Emotion Joint Learning
Li, M., Xu, H., Huang, X., Song, Z., Liu, X., & Li, X. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2880201 · Cited by 128
“[1], we use feature-level concatenation of deeplearned identity and emotion features to form a new network with”
BackgroundMethodInfluentialRecognition of Affect in the Wild Using Deep Neural Networks
Kollias, D., Nicolaou, M. A., Kotsia, I., Zhao, G., & Zafeiriou, S. P. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.247 · Cited by 123
“In general, the procedure of freezing a part of the network and fine-tuning [11] the rest can be deemed very useful, in particular when the given dataset is incremented with more videos.”
BackgroundDeep spatial-temporal feature fusion for facial expression recognition in static images
Sun, N., Li, Q., Huan, R., Liu, J., & Han, G. (2017) · Pattern Recognition Letters · doi:10.1016/j.patrec.2017.10.022 · Cited by 121
Facial Expression Recognition in Videos Using Dynamic Kernels
Perveen, N., Roy, D., & Mohan, C. K. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3011846 · Cited by 116
“networks [24], recurrent neural network [13], etc are explored in past few decades for efficient facial expression recognition”
BackgroundMukhiddinov, M., Djuraev, O., Akhmedov, F., Mukhamadiyev, A., & Cho, J. (2023) · Sensors · doi:10.3390/s23031080 · Cited by 125
“Other studies [40,41] have introduced a variety of landmark and image fusion processes.”
Yang, H., Zhang, Z., & Yin, L. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00050 · Cited by 112
“DTAGN-Joint [29] sequence-based 81.”
Supervised Committee of Convolutional Neural Networks in Automated Facial Expression Analysis
Pons, G., & Masip, D. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2753235 · Cited by 110
“In [31],”
Transformer-based Multimodal Information Fusion for Facial Expression Analysis
Zhang, W., Qiu, F., Wang, S., Zeng, H., Zhang, Z., An, R., Ma, B., & Ding, Y. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00271 · Cited by 114
“Some works [3, 15] employ CNN to extract the spatial and temporal features and improve expression recognition.”
MethodFacial Expression Recognition via Deep Learning
Fathallah, A., Abdi, L., & Douik, A. (2017) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa.2017.124 · Cited by 111
“Performance comparison on the CK+ database, A1[20] , A2 [19], A3 [21].”
MethodLai, Y.-H., & Lai, S. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00046 · Cited by 106
“However, the proposed model still can compete with another deep learning based approach [7] that utilized image feature and landmarks to jointly train a DNN discriminator.”
BackgroundMethodInfluentialFacial Motion Prior Networks for Facial Expression Recognition
Chen, Y., Wang, J., Chen, S., Shi, Z., & Cai, J. (2019) · IEEE International Conference on Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip47243.2019.8965826 · Cited by 107
“facial landmarks [3], or optical flow [4].”
CogEmoNet: A Cognitive-Feature-Augmented Driver Emotion Recognition Model for Smart Cockpit
Li, W., Zeng, G., Zhang, J., Xu, Y. H., Xing, Y., Zhou, R., Guo, G., Shen, Y., Cao, D., & Wang, F. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3127935 · Cited by 104
“[33] used a dual-stream network to extract temporal appearance features and temporal geometry features to improve facial expression recognition ability.”
Aff-Wild2: Extending the Aff-Wild Database for Affect RecognitionPreprint
Kollias, D., & Zafeiriou, S. P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.07770 · Cited by 142
“Some representative datasets developed in labs, which are still used in many recent works [12], are the Cohn-Kanade database [43], [29], the MMI database”
MethodDeep Joint Spatiotemporal Network (DJSTN) for Efficient Facial Expression Recognition
Jeong, D.-M., Kim, B., & Dong, S.-Y. (2020) · Sensors · doi:10.3390/s20071936 · Cited by 102
“[33], who used convolution and pooling layer, respectively.”
BackgroundMethodInfluentialOAENet: Oriented attention ensemble for accurate facial expression recognition
Wang, Z., Zeng, F., Liu, S., & Zeng, B. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107694 · Cited by 105
Deep convolutional BiLSTM fusion network for facial expression recognition
Liang, D., Liang, H., Yu, Z., & Zhang, Y. (2019) · The Visual Computer · doi:10.1007/s00371-019-01636-3 · Cited by 105
“The recent focus has been more on algorithms capturing spatial–temporal features [5, 8, 13, 14, 20, 21].”
BackgroundMethodInfluentialAdaptive Feature Mapping for Customizing Deep Learning Based Facial Expression Recognition Model
Wu, B., & Lin, C.-H. (2018) · IEEE Access · doi:10.1109/access.2018.2805861 · Cited by 100
“[17], and Byeon and Kwak [18] predict the expressions by passing a video, which seemsmore reasonable in practice; nevertheless, labeling video data is more labor intensive.”
BackgroundMulti-cue fusion for emotion recognition in the wild
Yan, J., Zheng, W., Cui, Z., Tang, C., Zhang, T., & Zong, Y. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.03.068 · Cited by 98
Facial expression recognition in videos using hybrid CNN & ConvLSTM
Singh, R., Saurav, S., Kumar, T., Saini, R., Vohra, A., & Singh, S. K. (2023) · International Journal of Information Technology · doi:10.1007/s41870-023-01183-0 · Cited by 105
Hard negative generation for identity-disentangled facial expression recognition
Liu, X., Kumar, B. V. K. V., Jia, P., & You, J. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.11.001 · Cited by 95
A comprehensive review of facial expression recognition techniques
Rashmi Adyapady, R., & Annappa, B. (2022) · Multimedia Systems · doi:10.1007/s00530-022-00984-w · Cited by 101
Incremental Boosting Convolutional Neural Network for Facial Action Unit RecognitionPreprint
Han, S., Meng, Z., Khan, A.-S., & Tong, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05395 · Cited by 97
“Specifically, a set of IB-3 CNNs, with the number of input neurons of 64, 128, 256, 512, 1042, and 2048, were trained and tested on the SEMAINE database.”
BackgroundInfluentialFacial expression recognition with FRR‐CNN
Xie, S., & Hu, H. (2017) · Electronics Letters · doi:10.1049/el.2016.4328 · Cited by 96
Exploring Deep Physiological Models for Nociceptive Pain Recognition
Thiam, P., Bellmann, P., Kestler, H. A., & Schwenker, F. (2019) · Sensors · doi:10.3390/s19204503 · Cited by 106
“…(in order to include the temporal aspect of the physiological signals in the inference model), could also potentially improve the performance of the current system, since such approaches have been successfully applied in other domains of application such as facial expression recognition [51–53].”
Conditional convolution neural network enhanced random forest for facial expression recognition
Liu, Y., Yuan, X., Gong, X., Xie, Z., Fang, F., & Luo, Z. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.07.016 · Cited by 94
k-Same-Net: k-Anonymity with Generative Deep Neural Networks for Face Deidentification
Meden, B., Emeršič, Ž., Štruc, V., & Peer, P. (2018) · Entropy · doi:10.3390/e20010060 · Cited by 94
Multimodal Emotion Recognition Fusion Analysis Adapting BERT With Heterogeneous Feature Unification
Lee, S., Han, D. K., & Ko, H. (2021) · IEEE Access · doi:10.1109/access.2021.3092735 · Cited by 99
“[28]–[30] built an emotion recognition model by fusing handcrafted features and deep learning based features from facial images.”
BackgroundSelf-citationEfficient facial expression recognition using histogram of oriented gradients in wavelet domain
Nigam, S., Singh, R. K., & Misra, A. K. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6040-3 · Cited by 92
“Recognition accuracy of the proposed method is shown 3DCNN-DAP [32] DTDN [26] DTGN [26] DTAGN (WS) [26] DTAGN (Joint) [26] DNN [25] CNN [25] TOP-1 [58] TOP-2 [58] CNN [43] Exp-Net [43] IACNN [43] AUDN [31] BDBN [34] FP+SAE [41] CNN [29] Proposed in the form of confusion matrices.”
MethodInfluentialZhang, H., Huang, B., & Tian, G. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2019.12.013 · Cited by 88
Facial Expression Recognition With Deeply-Supervised Attention Network
Fan, Y., Li, V. O. K., & Lam, J. C. K. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2988264 · Cited by 92
Facial Expression Recognition Using Frequency Neural Network
Tang, Y. D., Zhang, X., Hu, X., Wang, S., & Wang, H. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3037467 · Cited by 89
“For another example, some researches [6], [14] combined geometric feature and learned feature.”
BackgroundMethodInfluentialJi, Y., Hu, Y., Yang, Y., Shen, F., & Shen, H. T. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.12.037 · Cited by 86
Multiple Attention Network for Facial Expression Recognition
Gan, Y., Chen, J., Yang, Z., & Xu, L. (2020) · IEEE Access · doi:10.1109/access.2020.2963913 · Cited by 85
“[18] presented a joint fine-tuning strategy with”
Chang, W.-Y., Hsu, S.-H., & Chien, J.-H. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.246 · Cited by 81
“Moreover, CNNs also have been applied to face-related tasks [22, 29, 41] and facial expression recognition [17, 9].”
MethodFilntisis, P. P., Efthymiou, N., Koutras, P., Potamianos, G., & Maragos, P. A. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2930434 · Cited by 90
“However, to date, most research has focused on automatic visual recognition of emotion from facial expressions [15], [16], with only few including emotional body expressions into the recognition loop [8].”
BackgroundSAANet: Siamese action-units attention network for improving dynamic facial expression recognition
Liu, D., Ouyang, X., Xu, S., Zhou, P., He, K., & Wen, S. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.06.062 · Cited by 85
Clip-aware expressive feature learning for video-based facial expression recognition
Liu, Y., Feng, C., Yuan, X., Zhou, L., Wang, W., Qin, J., & Luo, Z. (2022) · Information Sciences · doi:10.1016/j.ins.2022.03.062 · Cited by 83
“In [21], a temporal geometric feature was proposed to improve the discriminative capacity of the learned spatio-temporal appearance features.”
BackgroundMethodFacial expression recognition algorithm based on parameter adaptive initialization of CNN and LSTM
An, F., & Liu, Z. (2019) · The Visual Computer · doi:10.1007/s00371-019-01635-4 · Cited by 81
Co-attentive multi-task convolutional neural network for facial expression recognition
Yu, W., & Xu, H. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108401 · Cited by 84
Chen, Y., Li, J., Shan, S., Wang, M., & Hong, R. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3453443 · Cited by 89
“Performance comparison of dynamic facial expression recognition on the DFEW [4] testing set.”
Deeper cascaded peak-piloted network for weak expression recognition
Yu, Z., Liu, Q., & Liu, G. (2017) · The Visual Computer · doi:10.1007/s00371-017-1443-0 · Cited by 79
“Deep learning algorithms have shown excellent performance of facial expression recognition in latest significant conferences [10,14,15,36] and competitions [2,7,8,30].”
BackgroundMethodInfluentialBrain-Machine Coupled Learning Method for Facial Emotion Recognition
Liu, D., Dai, W., Zhang, H., Jin, X., Cao, J., & Kong, W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3257846 · Cited by 83
“As shown in Table XI, compared with the state-of-the-art methods [94], [95], [96], [97], [98] on the CK+ dataset, the proposed method achieves competitive”
ResultLearning Bases of Activity for Facial Expression Recognition
Sarıyanidi, E., Güneş, H., & Cavallaro, A. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662237 · Cited by 79
Facial expression recognition based on a multi-task global-local network
Yu, M., Zheng, H., Peng, Z., Dong, J., & Du, H. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2020.01.016 · Cited by 78
Deep Disturbance-Disentangled Learning for Facial Expression Recognition
Ruan, D., Yan, Y., Chen, S., Xue, J., & Wang, H. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413907 · Cited by 77
Facial Micro-Expression Recognition Using Two-Dimensional Landmark Feature Maps
Choi, D. Y., & Song, B. C. (2020) · IEEE Access · doi:10.1109/access.2020.3006958 · Cited by 80
“Also, CLFM shows about 15% higher performance than DTAGN even for mixed dataset.”
BackgroundMethodInfluentialFacial Expression Recognition With Two-Branch Disentangled Generative Adversarial Network
Xie, S., Hu, H., & Chen, Y. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3024201 · Cited by 76
“Note that for CK+, DTAGN [3] and DeRF [4] both extract temporal expressional information from”
BackgroundMethodInfluentialFacial Expression Recognition of Instructor Using Deep Features and Extreme Learning Machine
Bhatti, Y. K., Jamil, A., Nida, N., Yousaf, M. H., Viriri, S., & Velastín, S. A. (2021) · Computational Intelligence and Neuroscience · doi:10.1155/2021/5570870 · Cited by 80
Facial Landmark-Based Emotion Recognition via Directed Graph Neural Network
Ngoc, Q. T., Lee, S., & Song, B. C. (2020) · Electronics · doi:10.3390/electronics9050764 · Cited by 73
“For example, a study [5] demonstrated performance improvement by associating the landmark modality with other modalities.”
BackgroundMethodResultInfluentialSelf-citation
Metadata
- Authors
- Jung, H., Lee, S., Yim, J., Park, S., & Kim, J.
- Affiliations
- Korea Advanced Institute of Science and Technology (KR)
- Venue
- 2015 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2015-12
- Pages
- 2983-2991
- OpenAlex
- W2217426128
- Cited by
- 781
- References
- 27
- Citations by year
- 2026: 9 · 2025: 29 · 2024: 45 · 2023: 82 · 2022: 87 · 2021: 127 · 2020: 115 · 2019: 125 · 2018: 92 · 2017: 55 · 2016: 15
- FWCI
- 72.14
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- KAIST Institutional Repository (KAIST) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- facial expression recognition, joint fine-tuning, facial landmarks, feature integration, image sequences, deep neural network, CK+ database
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
