The DOI matches its Crossref record; 31 of the 32 references it cites were found in a registry or an index.
Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning
Abstract
This paper presents the techniques employed in our team's submissions to the 2015 Emotion Recognition in the Wild contest, for the sub-challenge of Static Facial Expression Recognition in the Wild. The objective of this sub-challenge is to classify the emotions expressed by the primary human subject in static images extracted from movies. We follow a transfer learning approach for deep Convolutional Neural Network (CNN) architectures. Starting from a network pre-trained on the generic ImageNet dataset, we perform supervised fine-tuning on the network in a two-stage process, first on datasets relevant to facial expressions, followed by the contest's dataset. Experimental results show that this cascading fine-tuning approach achieves better results, compared to a single stage fine-tuning with the combined datasets. Our best submission exhibited an overall accuracy of 48.5% in the validation set and 55.6% in the test set, which compares favorably to the respective 35.96% and 39.13% of the challenge baseline.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Since the number of labeled examples for FER remains relatively limited, large 11 models are likely to overfit the data [106], [107], [89], [108].”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [107]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019
Return of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
Recognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745
Flagged first, then by citations
1 of the 32 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- J. R. S. Widen and A. Brooks . Anger and disgust: Discrete or overlapping categories ? In Proceedings of the 2004 APS Annual Convention , 2004 . J. R. S. Widen and A. Brooks. Anger and disgust: Discrete or overlapping categories? In Proceedings of the 2004 APS Annual Convention, 2004.
Cited by
Sorted by citations
Items 301–400 of 655
Mišeikis, J., Brijačak, I., Yahyanejad, S., Glette, K., Elle, O. J., & Tørresen, J. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794077 · Cited by 12
“This has been proven to work for mid-level image representations in object classification, using the pre-trained network on natural images to adapt for medical image recognition and even emotion recognition [10] [11] [12].”
Coarse-to-Fine Activity Annotation and Recognition Algorithm for Solitary Older Adults
Hu, X. Y., Chu, D., Li, Z., Dai, R., Cui, Y., Zhou, Z., An, B., Han, Y., Jiang, C., & Ding, D. (2019) · IEEE Access · doi:10.1109/access.2019.2962843 · Cited by 11
“overfitting, whereas transfer learning requires much smaller dataset [23].”
BackgroundMendez, M., Calderon, S., & Tyrrell, P. N. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-41005-6_21 · Cited by 12
Transfer Learning Code Vectorizer based Machine Learning Models for Software Defect Prediction
Singh, R., Singh, J., Gill, M. S., Malhotra, R., & Garima, G. (2020) · International Conference on Computational Performance Evaluation (ComPE) · doi:10.1109/compe49325.2020.9200076 · Cited by 12
“Computer vision is one field where transfer learning is used extensively from image recognition to video analysis [21][22][23][24].”
BackgroundPredicting Actions in Videos and Action-Based Segmentation Using Deep Learning
Memon, F. A., Khan, U. A., Shaikh, A., Alghamdi, A., Kumar, P., & Alrizq, M. (2021) · IEEE Access · doi:10.1109/access.2021.3101175 · Cited by 12
“In order to cope with this issue, transfer learning [21] can be used to train a pre-trained network on an entirely different task by transferring its knowledge from a previous learning task.”
BackgroundLevel-K Classification from EEG Signals Using Transfer Learning
Mizrahi, D., Zuckerman, I., & Laufer, I. (2021) · Sensors · doi:10.3390/s21237908 · Cited by 12
“, [29,30]) that focuses on the use of knowledge generated in a particular problem to solve another problem that has similar characteristics.”
BackgroundDeepDet: YAMNet with BottleNeck Attention Module (BAM) for TTS synthesis detection
Mahum, R., Irtaza, A., Javed, A., Mahmoud, H. A., & Hassan, H. (2024) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-024-00335-9 · Cited by 17
“We employed features extraction utilizing a feature extraction layer through which mel-spectrograms have been generated and passed to an improved architecture of MobileNet [54] guided by the attention module as a base network in YamNet [55].”
MethodKaneko, T., Hiramatsu, K., & Kashino, K. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2967236 · Cited by 12
“We ignored the classes said to be nuanced and of which labeling is ambiguous [23] and selected three classes: anger, happiness, and neutral.”
BackgroundMulti-Network Fusion Based on CNN for Facial Expression Recognition
Li, C., Ma, N., & Deng, Y. (2018) · International Conference on Computer Science, Electronics and Communication Engineering (CSECE) · doi:10.2991/csece-18.2018.35 · Cited by 12
“The image of FER-2013 has low resolution, not aligned, that means in early training of the facial expression classification models, quantity is more important than quality to improve experiment results [1].”
BackgroundMethodInfluentialMišeikis, J., Brijačak, I., Yahyanejad, S., Glette, K., Elle, O. J., & Tørresen, J. (2018) · IEEE International Conference on Intelligence and Safety for Robotics (ISR) · doi:10.1109/iisr.2018.8535937 · Cited by 12
“This has been proven to work for midlevel image representations in object classification, using the pre-trained network on natural images to adapt for medical image recognition and even emotion recognition [3] [4] [5].”
BackgroundSecProCT: In Silico Prediction of Human Secretory Proteins Based on Capsule Network and Transformer
Du, W., Zhao, X., Sun, Y., Zheng, L., Li, Y., & Zhang, Y. (2021) · International Journal of Molecular Sciences · doi:10.3390/ijms22169054 · Cited by 11
“Transfer learning refers to the transfer of knowledge learned in related tasks to new tasks [46], thereby improving the learning effect, which has been successfully applied to a deep learning framework for small sample problems [47].”
BackgroundFacial Emotions Recognition using Convolutional Neural NetPreprint
Ghaffar, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01456 · Cited by 18
“Hidden Morkov Model Neural Network based models has also been proposed in several studies for facial emotion detections [26] In our project we are using Convolutional neural network.”
MethodA New Intra Fine-Tuning Method Between Histopathological Datasets in Deep Learning
Dif, N., & Elberrichi, Z. E. (2020) · International Journal of Service Science Management Engineering and Technology · doi:10.4018/ijssmet.2020040102 · Cited by 13
Bian, J., Abdelrahman, S., Shi, J., & Fiol, G. D. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.11.010 · Cited by 11
Effects of Emotion Grouping for Recognition in Human-Robot Interactions
Tozadore, D. C., Ranieri, C. M., Nardari, G. V., Romero, R. A. F., & Guizilini, V. (2018) · Brazilian Conference on Intelligent Systems · doi:10.1109/bracis.2018.00082 · Cited by 11
“Besides, negative emotions are usually underrepresented on almost all real-world datasets, which leads to poor precision scores [25], making it almost impossible to build refined interaction strategies based on these particular emotions.”
BackgroundPrivileged Information for Modeling Affect In The Wild
Makantasis, K., Melhárt, D., Liapis, A., & Yannakakis, G. N. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597417 · Cited by 11
Pinitas, K., Makantasis, K., Liapis, A., & Yannakakis, G. N. (2022) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3512290.3528744 · Cited by 11
“[28] used a deep Convolutional Neural Network (CNN) pretrained on the generic ImageNet dataset to perform emotion recognition on small datasets.”
MethodVideo Emotion Recognition using Hand-Crafted and Deep Learning Features
Xia, X., Liu, J., Yang, T., Jiang, D., Han, W., & Hichem Sahli, B. (2018) · Asian Conference on Affective Computing and Intelligent Interaction (ACII Asia) · doi:10.1109/aciiasia.2018.8470326 · Cited by 11
“To deal with the problem of training a complex model on small scale datasets, researchers propose using transfer learning [19], in which a certain pre-trained DCNN model is fine-tuned to the particular application settings with the target domain data [13], [15].”
BackgroundClassification of 3D Archaeological Objects Using Multi-View Curvature Structure Signatures
Canul‐Ku, M., Hasimoto-Beltrán, R., Jiménez-Badillo, D., Ruíz-Correa, S., & Román-Rangel, E. (2018) · IEEE Access · doi:10.1109/access.2018.2886791 · Cited by 10
“There exist different methods for the construction of 3D shape descriptors, the most popular approaches are based on Vertices [9], [35], [41], [53], [80], Multi-View Projections (MVP), and recently Transfer Learning [13], [20], [29], [59], [88].”
BackgroundMethodImproved Generalization of Arabic Text Classifiers
Khaddaj, A., Hajj, H. M., & El-Hajj, W. (2019) · Arabic Natural Language Processing Workshop (WANLP) · doi:10.18653/v1/w19-4618 · Cited by 11
“…DA techniques to more Arabic dialects, trying other domain adaptation algorithms in Arabic, developing new domain adaptation techniques, evaluating the DA tasks using better text representation ( e.g. sequence models...) and integrating transfer learning techniques in the models (Ng et al., 2015).”
BackgroundTowards Real-Time Multimodal Emotion Recognition among Couples
Boateng, G. (2020) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3382507.3421154 · Cited by 11
“Transfer learning has also been used in emotion recognition tasks ( [34, 57]).”
MethodSpeech Emotion Recognition among Couples using the Peak-End Rule and Transfer Learning
Boateng, G., Sels, L., Kuppens, P., Hilpert, P., & Kowatsch, T. (2020) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3395035.3425253 · Cited by 11
“It is used to address the limitations of using small labeled datasets and has shown success in various fields including emotion recognition tasks ([35, 42]).”
BackgroundMethodEmotion Recognition and Discrimination of Facial Expressions using Convolutional Neural Networks
Manewa, R. M. A. H., & Mayurathan, B. (2020) · IEEE Region Humanitarian Technology Conference · doi:10.1109/r10-htc49770.2020.9357008 · Cited by 11
“Performance Reported [14] 48.5 [11] 75.2 [2] 66.4 Our method 64.56 Table II compares the classification rate of the different state-of-the-art methods and our method.”
BackgroundMethodInfluentialTransfer Learning-Based Received Power Prediction Using RGB-D Camera in mmWave Networks
Tomoya, M., Nishio, T., Morikura, M., Yamamoto, K., Asai, Y., & Miyatake, R. (2019) · IEEE Vehicular Technology Conference · doi:10.1109/vtcspring.2019.8746643 · Cited by 11
“production environments, and this has been demonstrated in [10]–[12].”
BackgroundHuman Action Adverb Recognition: ADHA Dataset and a Three-Stream Hybrid Model
Pang, B., Zha, K., & Lu, C. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00308 · Cited by 11
Lightweight Facial Expression Recognition Based on Class-Rebalancing Fusion Cumulative Learning
Mou, X., Song, Y., Wang, R., Tang, Y., & Xin, Y. (2023) · Applied Sciences · doi:10.3390/app13159029 · Cited by 10
“Based on an existing pre-training model, Ng et al. [15] solved the problem of insufficient samples and poor characterization through multi-round fine-tuning.”
MethodGemstone Classification Using Deep Convolutional Neural Network
Chakraborty, B., Mukherjee, R., & Das, S. (2024) · Journal of The Institution of Engineers (India) Series B · doi:10.1007/s40031-024-01003-4 · Cited by 10
“Fine-tuning indicates training of newly added layers of the model with a very low learning rate which, can remarkably increase the accuracy of the model by progressively adapting the previous trained resources to the newly targeted data [40].”
Son, H., & Park, Y. E. (2024) · Current Issues in Tourism · doi:10.1080/13683500.2024.2368152 · Cited by 14
“Through this process, we can address the data scarcity issues for labelling and training dataset creation (Ng et al., 2015; Son & Park, 2023; Weiss et al., 2016).”
Multimodal Models for Contextual Affect Assessment in Real-Time
Vice, J., Khan, M. M., & Yanushkevich, S. N. (2019) · IEEE International Conference on Cognitive Machine Intelligence (CogMI) · doi:10.1109/cogmi48466.2019.00020 · Cited by 10
“These affective states are considered universal; independent of cultural background [2].”
BackgroundGanai, A. F., & Khursheed, F. (2022) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-022-00414-7 · Cited by 10
“While training a CNN from scratch requires a huge dataset, some studies use the concept of transfer learning [26] where pre-trained CNNs can be fine-tuned to smaller datasets [26,27] while as some studies use CNN to extract features [28] and then classification is performed by a distinct classi-fier.”
MethodClassification of wireless capsule endoscopy images for bleeding using deep features fusion
Rani, K., Devi, G., Kumar, S., Figueiredo, I. N., & Figueiredo, P. (2022) · International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME) · doi:10.1109/iceccme55909.2022.9987916 · Cited by 10
Plant Disease Detection Using Deep Learning and Machine Learning Algorithms
Chopra, N., Singh, A., Garg, R., Sarangi, P. K., Sahoo, A. K., & Dutta, M. (2023) · International Conference on Technological Advancements in Computational Sciences (ICTACS) · doi:10.1109/ictacs59847.2023.10390154 · Cited by 10
Deep Convolutional Neural Network for Facial Expression Recognition
Zhai, Y., Liu, J., Zeng, J., Piuri, V., Scotti, F., Ying, Z., Xu, Y. W., & Gan, J. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-71607-7_19 · Cited by 11
The Many Variations of Emotion
Vielzeuf, V., Kervadec, C., Pateux, S., & Jurie, F. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756560 · Cited by 10
HRI Physio Lib: A Software Framework to Support the Integration of Physiological Adaptation in HRI
Kothig, A., Muñoz, J. E., Mahdi, H., Aroyo, A. M., & Dautenhahn, K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-62056-1_4 · Cited by 10
Gerłowska, J., Dmitruk, K., & Rejdak, K. (2021) · AIMS neuroscience · doi:10.3934/neuroscience.2021012 · Cited by 11
A Survey on Factors Affecting Facial Expression Recognition based on Convolutional Neural Networks
Dufourq, E. (2020) · South African Institute of Computer Scientists and Information Technologists · doi:10.1145/3410886.3410891 · Cited by 10
“Good speed and accuracy are listed as reasons for using AlexNet in [76].”
MethodCho, S., Kim, Y., Jang, J., & Hwang, I. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3580812 · Cited by 10
“…sensors surrounding the living area, continuously monitoring the resident’s physiology [3, 56], home activities [66, 78, 79, 98], emotions [99, 100, 115], exercises [109, 111], verbal expressions [30, 38, 39, 138], social interactions [62, 75, 158], etc., producing contextual ingredients…”
BackgroundSurvey on Current Trend in Emotion Recognition Techniques Using Deep Learning
Adebiyi, M. O., Olaniyan, D., Adebiyi, A. A., & Okunola, A. A. (2023) · International Conference on Science, Engineering and Business for Sustainable Development Goals (SEB-SDG) · doi:10.1109/seb-sdg57117.2023.10124548 · Cited by 11
“Reference [22] outlines the methods used in our team's submissions to the Static Facial Expression Recognition in the Wild sub-challenge of the 2015 Emotion Recognition in the Wild competition.”
MethodMultimodal Transfer Learning for Oral Presentation Assessment
Tun, S. S. Y., Okada, S., Huang, H., & Leong, C. W. (2023) · IEEE Access · doi:10.1109/access.2023.3295832 · Cited by 13
“approach includes the extraction of facial features [39], [40] or acoustic features [37], [38] using pretrained convolutional neural network (CNN) architectures or using pretrained”
MethodLatest trends in emotion recognition methods: case study on emotiw challenge
Naz, H. U., & Ahuja, S. (2020) · International Journal of Advanced Computer Research · doi:10.19101/ijacr.2019.940117 · Cited by 9
“Ng et al. [19] proposed an idea of adding deep supervision in emotion recognition, It is recent and great idea to deal with the problem of emotion grouping, because the EmotiW challenge has been successfully organized by six times, emotion recognition in audio-video and „in the wild‟ has always…”
BackgroundI Only Have Eyes for You: The Impact of Masks On Convolutional-Based Facial Expression Recognition
Barros, P. V. A., & Sciutti, A. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00134 · Cited by 11
“Most of the current state-of-the-art solutions for automatic facial expression recognition, however, claim to have addressed the problem by approaching maximum generalization [20, 4, 17].”
BackgroundAn Acoustical and Lexical Machine-Learning Pipeline to Identify Connectional Silences
Matt, J. E., Rizzo, D. M., Javed, A., Eppstein, M. J., Manukyan, V., Gramling, C. J., Dewoolkar, A., & Gramling, R. E. (2023) · Journal of Palliative Medicine · doi:10.1089/jpm.2023.0087 · Cited by 10
Combining Deep Learning with Signal-image Encoding for Multi-Modal Mental Wellbeing Classification
Woodward, K., Kanjo, E., & Tsanas, A. (2023) · ACM Transactions on Computing for Healthcare · doi:10.1145/3631618 · Cited by 18
“When using models pre-trained on the ImageNet dataset and testing using images of faces expressing seven emotions an accuracy of 55.6% was achieved compared with a baseline performance of 39.13% [58].”
MethodInfluentialOptimized-CNN enabled Facial Emotion Recognition within Collaborative Edge Computing
Tao, M., Liao, L., Li, K., Xu, J., & Huang, W. (2024) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd61410.2024.10580276 · Cited by 9
“Ng et al. [8] used a transfer learning method based on deep convolutional neural network architecture, and then performed fine-tuning in stages to achieve facial expression recognition.”
MethodEnhancing Feature Correlation for Bi-Modal Group Emotion Recognition
Liu, N., Fang, Y., & Guo, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00767-6_3 · Cited by 9
Neural Network-Based Transfer Learning of Manipulator Inverse Displacement Analysis
Tang, H., & Notash, L. (2021) · Journal of Mechanisms and Robotics · doi:10.1115/1.4050622 · Cited by 9
Mustafid, A., Pamuji, M. M., & Helmiyah, S. (2020) · IJID (International Journal on Informatics for Development) · doi:10.14421/ijid.2020.09207 · Cited by 13
Nakamura, F., Murakami, M., Suzuki, K., Fukuoka, M., Masai, K., & Sugimoto, M. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3179766 · Cited by 9
Across the Universe: Biasing Facial Representations Toward Non-Universal Emotions With the Face-STN
Barros, P. V. A., & Sciutti, A. (2022) · IEEE Access · doi:10.1109/access.2022.3210183 · Cited by 9
“facial expression recognition (FER) claim to have addressed the problem of global FER by approaching maximum generalization [8], [49], [63].”
BackgroundEnhancing motor imagery decoding via transfer learning
George, O., Dabas, S., Sikder, A., Smith, R. O., Madiraju, P., Yahyasoltani, N., & Ahamed, S. I. (2022) · Smart Health · doi:10.1016/j.smhl.2022.100339 · Cited by 8
Innovative Method for Face Emotion Recognition using Hybrid Deep Neural Networks
Dimlo, U. M. F., Bhanarkar, P., Jayalakshmi, V., Veenu, V., Sekhar, S. C., & Rastogi, R. (2023) · International Conference on Trends in Electronics and Informatics (ICOEI) · doi:10.1109/icoei56765.2023.10126007 · Cited by 9
“By employing a two-stage supervised fine-tuning on the pre-trained network based on the generic Image Net [19] datasets, [20] adopted a transfer learning strategy for deep CNNs.”
MethodPitfalls of Affective Computing
Cooney, M., Pashami, S., Sant’Anna, A., Fan, Y., & Nowaczyk, S. (2018) · International World Wide Web Conference (WWW) · doi:10.1145/3184558.3191611 · Cited by 9
“, galvanic skin response, brain machine interfaces, and cameras) [14, 20, 22].”
BackgroundReal-time Facial Expression Recognition on Robot for Healthcare
Wang, F., Chen, H., Kong, L. J., & Sheng, W. (2018) · IEEE International Conference on Intelligence and Safety for Robotics (ISR) · doi:10.1109/iisr.2018.8535710 · Cited by 9
Detection of driver fatigue symptoms using transfer learning
Jakubowski, J., & Chmielińska, J. (2018) · Bulletin of the Polish Academy of Sciences Technical Sciences · doi:10.24425/bpas.2018.125934 · Cited by 9
“medical image analysis [21], human action recognition [22], emotion recognition [23] and also driver monitoring systems.”
BackgroundDeep Learning Based Approach of Emotion Detection and Grading System
Sonawane, B., & Sharma, P. (2020) · Pattern Recognition and Image Analysis · doi:10.1134/s1054661820040239 · Cited by 9
“are not sufficient to train the CNN, concept of transfer learning is used in many studies where well known pre-trained models are fine-tuned for FER [9, 10].”
BackgroundDeep Multi-Facial Patches Aggregation Network For Facial Expression RecognitionPreprint
Hazourli, A. R., Djeghri, A., Salam, H., & Othmani, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09298 · Cited by 9
“Several works have been proposed for facial emotions classification from static images in the literature [9,53, 5,21,4,20,51,65,44,56,27,42,46,59,30,43] or from continuous video input [1,2,38].”
BackgroundJoint Deep Cross-Domain Transfer Learning for Emotion RecognitionPreprint
Nguyen, D., Sridharan, S., Nguyen, D. T., Denman, S., Tran, S. N., Zeng, R., & Fookes, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.11136 · Cited by 9
“Toaddress the problem of data scarcity in emotion classification, transfer learning has been widely adopted Zhang et al. (2017); Ng et al. (2015); Jung et al. (2015); Hu et al. (2018).”
BackgroundMethodUsing Self-Supervised Co-Training to Improve Facial Representation.Preprint
Pourmirzaei, M., Esmaili, F., & Montazer, G. A. (2021) · arXiv (Cornell University) · Cited by 9
Facial expression recognition sensing the complexity of testing samples
Chang, T., Li, H., Wen, G., Hu, Y., & Ma, J. (2019) · Applied Intelligence · doi:10.1007/s10489-019-01491-8 · Cited by 9
“Also, [28, 29] employ the model based on transfer features from pre-trained deep CNN, while the ensemble algorithms based on deep learning network [30, 31] have significantly improved recognition performance of FER.”
MethodChen, X., Du, J., Deng, F., & Zhao, F. (2023) · IET Intelligent Transport Systems · doi:10.1049/itr2.12423 · Cited by 8
Group Affect Prediction Using Multimodal Distributions
Shamsi, S., Singh, B. P., & Wadhwa, M. (2018) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw.2018.00015 · Cited by 8
“In [17], a two-stage fine-tuning was applied on deep CNN while doing transfer learning.”
BackgroundMethodReal-time Emotion Recognition for Sales
Naas, S., & Sigg, S. (2020) · International Conference on Mobility, Sensing and Networking (MSN) · doi:10.1109/msn50589.2020.00096 · Cited by 8
Park, S., & Wallraven, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-02375-0_30 · Cited by 8
Deep Convolutional Neural Networks with Transfer Learning for Waterline Detection in Mussel Farms
McLeay, A. J., McGhie, A., Briscoe, D. K., Bi, Y., Xue, B., Vennell, R., & Zhang, M. (2021) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci50451.2021.9659987 · Cited by 8
Your Smartphone Knows you Better than you May Think: Emotional Assessment ‘on the Go’ Via TapSense
Ghosh, S., Löchner, J., Mitra, B., & De, P. (2022) · Health informatics · doi:10.1007/978-3-030-94212-0_10 · Cited by 8
Engagement Emotion Classification through Facial Landmark Using Convolutional Neural Network
Paidja, A. N. R., & Bachtiar, F. A. (2022) · International Conference on Information Technology and Education (ICITE) · doi:10.1109/icite54466.2022.9759546 · Cited by 8
Expression-Guided Deep Joint Learning for Facial Expression Recognition
Fang, B., Zhao, Y., Han, G., & Ju-hou, H. (2023) · Sensors · doi:10.3390/s23167148 · Cited by 10
“The transfer learning method is an effective method for solving the small-sample problem [37].”
MethodHypericons for interpretability: decoding abstract concepts in visual data
Pandiani, D. S. M., Lazzari, N., van Erp, M., & Presutti, V. (2023) · International Journal of Digital Humanities · doi:10.1007/s42803-023-00077-8 · Cited by 11
“Popular datasets such as ImageNet offer little to no coverage of non-concrete concepts, and JFT, an internal dataset at Google, includes non-concrete classes but is not publicly released (Deng et al., 2009; Sun et al., 2017).”
InfluentialEmotion recognition on static images using deep transfer learning and ensembling
Abanoz, H., & Çataltepe, Z. (2018) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu.2018.8404346 · Cited by 8
“Küçük boyutlu görüntüler içermesine rağmen, veri kümesi duygu tanıma çalışmalarında yaygın biçimde kullanılır [1]–[5].”
InfluentialRat Behavior Observation System Based on Transfer Learning
Jin, T., & Duan, F. (2019) · IEEE Access · doi:10.1109/access.2019.2916339 · Cited by 8
“With the development of transfer learning, incredible results can be achieved by fine-tuning existing models which brings great possibility to apply deep learning in other fields [30], [31].”
BackgroundThe State of the Art in Machine Learning-Based Digital Forensics
Oladipo, F. O., Ogbuju, E., Samson, A. F., & Musa, A. E. (2020) · International Journal of Computing and Artificial Intelligence · doi:10.33545/27076571.2021.v2.i1a.21 · Cited by 28
Audio Visual Recognition of Spontaneous Emotions In-the-Wild
Xia, X., Guo, L., Jiang, D., Pei, E., Yang, L., & Hichem Sahli, B. (2016) · Communications in computer and information science · doi:10.1007/978-981-10-3005-5_57 · Cited by 7
“[16] followed a transfer learning approach for deep CNN architectures to overcome the over-fitting problem, by utilizing a two-stage supervised fine-tuning process.”
BackgroundTransfer Learning-Based Independent Component Analysis
Zheng, Z., Zhao, W., Hable, B., Gong, Y., Wang, X., Shannon, R. W., & Liu, K. (2022) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2022.3229294 · Cited by 7
“One of the most widely used parameter-transfer approaches is the fine-tuning of the neural network, which borrows the network weights from the network that is already trained in the source domain and only updates the last few layers based on the data in the target domain [27].”
MethodComparison of the Usability of Apple M2 and M1 Processors for Various Machine Learning Tasks
Kasperek, D., Antonowicz, P., Baranowski, M., Sokołowska, M., & Podpora, M. (2023) · Sensors · doi:10.3390/s23125424 · Cited by 10
“Having that knowledge, training a production-capable model requires a lot of data [15,18,19] (or, alternatively, the use of transfer learning [18–20], which requires the knowledge of where to find such a model, and which one to use).”
BackgroundChouhayebi, H., Mahraz, M. A., Riffi, J., & Tairi, H. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16779-8 · Cited by 9
“For the classification of facial expressions, Ng et al. [39] used AlexNet with five linear layers.”
Facial expression recognition by re-ranking with global and local generic features
Vo, D. M., Sugimoto, A., & Lê, T. H. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7900279 · Cited by 7
“Since the generic feature are also effective for FER [14], [15], we employ the generic feature for our FER method.”
MethodNovel representation for driver emotion recognition in motor vehicle videos
Theagarajan, R., Bhanu, B., Cruz, A. C., Le, B., & Tambo, A. L. (2017) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2017.8296393 · Cited by 7
“While deep learning has been a growing trend in image processing and computer vision, the effects of transfer learning — using expression data from other datasets [5] — are diminished possibly because of various factors [6].”
BackgroundResultInfluentialDeep Modeling of Human Age Guesses for Apparent Age Estimation
Rondeau, J., & Alvarez, M. A. (2018) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489570 · Cited by 7
“Furthermore, in [19], state-of-the-art performance is achieved for facial expression recognition by using transfer learning from the popular ImageNet dataset.”
Chao, G.-Y., Lin, Y.-S., Chang, C.-M., & Lee, C.-C. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2037 · Cited by 7
Deep Learning Model for Facial Emotion Recognition
Pathak, A. R., Bhalsing, S., Desai, S., Gandhi, M., & Patwardhan, P. (2019) · Lecture notes in electrical engineering · doi:10.1007/978-3-030-30577-2_48 · Cited by 7
Deep Domain Adaptation for Facial Expression Analysis
Kalischek, N., Thiam, P., Bellmann, P., & Schwenker, F. (2019) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw.2019.8925055 · Cited by 7
Deep transfer learning in human–robot interaction for cognitive and physical rehabilitation purposes
Ilyas, C. M. A., Rehm, M., Nasrollahi, K., Madadi, Y., Moeslund, T. B., & Seydi, V. (2021) · Pattern Analysis and Applications · doi:10.1007/s10044-021-00988-8 · Cited by 9
Emotions During Covid-19: LSTM Models for Emotion Detection in Tweets
Javed, N., & Muralidhara, B. L. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-16-6407-6_13 · Cited by 7
Classification of Mineral Foam Flotation Conditions Based on Multi-Modality Image Fusion
Jiang, X., Zhao, H., & Liu, J. (2023) · Applied Sciences · doi:10.3390/app13063512 · Cited by 7
Hybrid CNNLBP using Facial Emotion Recognition based on Deep Learning Approach
Depuru, S., Vaishnavi, K., Manogna, B., Sri, K. J., Preethi, A. A., & Priyanka, C. (2023) · International Conference on Artificial Intelligence and Smart Energy (ICAIS) · doi:10.1109/icais56108.2023.10073918 · Cited by 7
“It uses EmotiW database [12].”
Facial Expression Recognition Using Transfer Learning with ResNet50
Hiremath, S. S., Hiremath, J. S., Kulkarni, V. V., Harshit, B., Kumar, S., & Hiremath, M. S. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-99-1624-5_21 · Cited by 7
Self-supervised Cross-modal Pretraining for Speech Emotion Recognition and Sentiment Analysis
Chu, I., Chen, Z., Yu, X., Han, M., Xiao, J., & Chang, P. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.375 · Cited by 7
“With the rapid progress in deep learning (DL), many DL methods have been applied to SER (Ng et al., 2015; Sun et al., 2021b; Chen and Rudnicky, 2021) and sentiment analysis (SA) (Zhang et al., 2018; Devlin et al., 2018).”
Darvishnezhad, M., & Sebt, M. A. (2023) · IET Radar Sonar & Navigation · doi:10.1049/rsn2.12484 · Cited by 7
Hierarchical Transfer Convolutional Neural Networks for Image Classification
Dong, X., Wu, H.-H., Yan, Y., & Qian, L. (2019) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata47090.2019.9005707 · Cited by 7
“…recent approaches based on transfer CNN are to extract shallow layers or the whole pre-trained models, extend the extracted components by adding extra layers, and train the extended model by fine-tuning or frozen the extracted components for the target tasks [23], [24], [25], [26], [27], [22], [28].”
Towards Awareness of Human Relational Strategies in Virtual Agents
Beaver, I., Freeman, C., & Mueen, A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i03.5644 · Cited by 6
“Secondly, promising work has been done with transfer learning for emotion recognition using small datasets in image processing (Ng et al. 2015).”
BackgroundUsing Self-Supervised Auxiliary Tasks to Improve Fine-Grained Facial RepresentationPreprint
Pourmirzaei, M., Montazer, G. A., & Esmaili, F. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4395579 · Cited by 8
“ImageNet Transfer Learning vs training from scratch With the advent of deep learning, transfer learning has become extremely popular and has shown good performance in many datasets [54][55].”
Emotion Recognition using Deep Stacked Autoencoder with Softmax Classifier
Mohana, M., & Subashini, P. (2023) · International Conference on Artificial Intelligence and Smart Energy (ICAIS) · doi:10.1109/icais56108.2023.10073937 · Cited by 6
“Later days transfer learning techniques [10] has introduced to increase the performance of CNN.”
BackgroundLiu, X., Xu, Y., Wang, M., Cai, Y., & Qi, G. H. (2019) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2019.p0883 · Cited by 6
Face Reenactment Based Facial Expression Recognition
Ali, K., & Hughes, C. E. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-64556-4_39 · Cited by 6
“Recently, deep CNN based methods [11], [12], [13], [14], [40] have been employed to increase the robustness of FER to real-world scenarios.”
MethodUllah, A., Wang, J., Anwar, M. S., Whangbo, T. K., & Zhu, Y. (2021) · Journal of Sensors · doi:10.1155/2021/8893661 · Cited by 6
Selitskiy, S., Christou, N., & Selitskaya, N. (2021) · International Conference on Artificial Intelligence and Virtual Reality (AIVR) · doi:10.1145/3480433.3480447 · Cited by 6
“With the Deep Learning (DL) rising, convolutional neural networks were (CNN) employed for solving the facial expression recognition problem [27, 32, 34, 36, 43, 45].”
MethodFused deep learning based Facial Expression Recognition of students in online learning mode
Sumalakshmi, C., & Vasuki, P. (2022) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.7137 · Cited by 7
Variation of deep features analysis for facial expression recognition system
Shabbir, N., & Rout, R. K. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-14054-w · Cited by 6
“With this great advancement of computing power and large scale data, researchers have started to use deep learning in their facial expression recognition models [27, 29].”
BackgroundMethodCIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition
Barros, P. V. A., & Sciutti, A. (2022) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii55700.2022.9953863 · Cited by 6
“No 804388, wHiSPER achieve good results [6].”
Metadata
- Authors
- Ng, H.-W., Nguyen, V. D., Vonikakis, V., & Winkler, S.
- Affiliations
- University of Illinois at Urbana-Champaign (Singapore), Singapore, Singapore
- University of Illinois Urbana-Champaign (US)
- Venue
- Proceedings of the 2015 ACM on International Conference on Multimodal Interaction
- Publisher
- ACM
- Published
- 2015-11-09
- Pages
- 443-449
- OpenAlex
- W2253728219
- Cited by
- 657
- References
- 32
- Citations by year
- 2026: 13 · 2025: 29 · 2024: 37 · 2023: 70 · 2022: 69 · 2021: 106 · 2020: 106 · 2019: 106 · 2018: 63 · 2017: 44 · 2016: 13 · 2015: 1
- FWCI
- 53.84
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
- Funders
- Agency for Science Technology and Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- emotion recognition, facial expression recognition, still images, transfer learning, convolutional neural networks, fine-tuning, ImageNet pre-training, small datasets
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
