The DOI matches its Crossref record; 31 of the 32 references it cites were found in a registry or an index.
Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning
Abstract
This paper presents the techniques employed in our team's submissions to the 2015 Emotion Recognition in the Wild contest, for the sub-challenge of Static Facial Expression Recognition in the Wild. The objective of this sub-challenge is to classify the emotions expressed by the primary human subject in static images extracted from movies. We follow a transfer learning approach for deep Convolutional Neural Network (CNN) architectures. Starting from a network pre-trained on the generic ImageNet dataset, we perform supervised fine-tuning on the network in a two-stage process, first on datasets relevant to facial expressions, followed by the contest's dataset. Experimental results show that this cascading fine-tuning approach achieves better results, compared to a single stage fine-tuning with the combined datasets. Our best submission exhibited an overall accuracy of 48.5% in the validation set and 55.6% in the test set, which compares favorably to the respective 35.96% and 39.13% of the challenge baseline.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Since the number of labeled examples for FER remains relatively limited, large 11 models are likely to overfit the data [106], [107], [89], [108].”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [107]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019
Return of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
Recognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745
Flagged first, then by citations
1 of the 32 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- J. R. S. Widen and A. Brooks . Anger and disgust: Discrete or overlapping categories ? In Proceedings of the 2004 APS Annual Convention , 2004 . J. R. S. Widen and A. Brooks. Anger and disgust: Discrete or overlapping categories? In Proceedings of the 2004 APS Annual Convention, 2004.
Cited by
Sorted by citations
Items 601–655 of 655
Lin, J., Liu, Z., Hung, C., Liu, T., & Chai, Y. (2020)
Neural Network for Bullying Emotion Recognition
Zhou, X., Ye, L., He, C., Seppänen, T., & Alasaarela, E. (2021) · Lecture notes in electrical engineering · doi:10.1007/978-981-15-8411-4_49
Communication Analysis through Visual Analytics: Current Practices, Challenges, and New FrontiersPreprint
Fischer, M. T., Dennig, F. L., Seebacher, D., Keim, D. A., & El‐Assady, M. (2021) · doi:10.36227/techrxiv.15022791.v1 · Cited by 1
A New Dataset and Recognition for Egocentric Microgesture Designed by Ergonomists
Li, G., Liu, Y., Song, W., Wang, C., & Wang, Y. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87358-5_7
Retracted: Survey Analysis on Facial Expression
Yadav, M. R., & Reddy, P. C. S. (2021) · International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC) · doi:10.1109/i-smac52330.2021.9640832
RetractedSaha, R. (2021) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2021.39424
“A few methodologies have been presented for powerful head pivot discovery [53], recognition of facial highlights [64], discourse [19], or even feelings [44].”
BackgroundOptimal Hyperparameter Tuning of Convolutional Neural Networks for Visual Sentiment Analysis
Cheguru, S., & Vijayalata, Y. (2021) · Helix · doi:10.29042/2021-11-5-20-31
ScaleNet: An Unsupervised Representation Learning Method for Limited Information
Huang, H., & Roozbahani, M. M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-92659-5_11
“001] for the ScaleNet to avoid over-fitting [36].”
MethodImplementation of Convolutional Neural Network to Realize a Real Time Emotion based Music Player
(2019) · International Journal of Recent Technology and Engineering (IJRTE) · doi:10.35940/ijrte.b1027.0982s1119
Emotion Detection for Live Video Face Expression – Standard Learning Techniques
Reeja, S. R., & Yaseen, I. (2022) · International Journal of Computer Applications · doi:10.5120/ijca2022922134
Deep Learning in Ecology: Data-driven Methods for Ecosystem AnalysisPreprint
Hart, J. D. A. (2022) · Thesis Commons (OSF Preprints) · doi:10.31237/osf.io/bnm5w
Data mining with deep learning in biomedical data
Singh, K., & Malhotra, J. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-99864-2.00018-4
Unveiling Cellular Antenna Orientations from Large Crowdsourced Datasets: A Deep Learning Approach
Eller, L., Svoboda, P., & Rupp, M. (2022) · IEEE International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob) · doi:10.1109/wimob55322.2022.9941528
“Using pretrained single-sector processing networks in a transfer learning manner further ensures that no complete retraining is required [18].”
BackgroundAgrwal, S. L., Sharma, S. K., & Kant, V. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-28183-9_41
Zhou, Z., Xu, Y., & Li, M. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-2401-1_14
Pandit, D., & Jadhav, S. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-35644-5_23
Kavadi, D. P., Hasan, A., Tanna, R., Singh, R., Lande, J., & Tiwari, M. (2023) · International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE) · doi:10.1109/icacite57410.2023.10182490
Open-Set Recognition for Facial-Expression Recognition
Uchida, M., Orihashi, S., Takashima, A., Yamazaki, Y., & Masumura, R. (2023) · Proceedings - International Conference on Image Processing · doi:10.1109/icip49359.2023.10222488
Improved Lightweight Multi-Target Recognition Model for Live Streaming Scenes
Li, Z., Qiao, K., Chen, J., Li, Z., & Zhang, Y. (2023) · Applied Sciences · doi:10.3390/app131810170
Digital support for visual inspections using deep learning
Engelmann, P. D.-I. H. F., Gerlach, J., & Riedel, A. (2021) · R STAHL Ex-Magazine · doi:10.60048/exm20_36
Facial Expression Recognition Based on Separable Convolution Network and Attention Mechanism
Yengikand, A. K., Afsharyan, M. F., & Nejati, P. (2023) · Sharq. · doi:10.61186/itrc.15.4.25
“Also, some researchers used pre-trained networks [7, 12].”
A Nighttime Driving Detection Model with Adaptive Growth CharacteristicsPreprint
Ma, H., Yang, G., Chen, W., & Hou, J. (2024) · Research Square · doi:10.21203/rs.3.rs-4423745/v1
Pranathi, P., Lakshmi, C., & Manne, S. (2024) · AIP conference proceedings · doi:10.1063/5.0220178
Klasifikasi Ekspresi Wajah Menggunakan Covolutional Neural Network
Akbar, A. T., Saifullah, S., & Prapcoyo, H. (2024) · Jurnal Teknologi Informasi dan Ilmu Komputer · doi:10.25126/jtiik.1168888 · Cited by 6
Veluchamy, S., Pasupuleti, H., & Saravanan, S. (2024) · IEEE Delhi Section Conference (DELCON) · doi:10.1109/delcon64804.2024.10867236
A Detection Method for Fatigue Driving Based on Facial Emotion Recognition
Liu, S. (2025) · Smart innovation, systems and technologies · doi:10.1007/978-981-96-0092-2_20
Facial Expression Recognition via Closed-Loop Transcription
Liu, X., Ma, J., & Wang, Q. (2024) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon55916.2024.10905539
İşler, B., & Kiliç, F. R. (2025) · DÜMF Mühendislik Dergisi · doi:10.24012/dumf.1576811
Parmar, A. (2025) · INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · doi:10.55041/ijsrem47033
Facial Emotion Recognition Using Transfer Learning with Xception Architecture
jallal, M., Wadee, A., Samarah, A., Alrousan, A., Alwedyan, M., & Obeidat, R. (2025) · International Conference on Information and Communication Systems (ICICS) · doi:10.1109/icics65354.2025.11073104
“Nguyen et al. [8] tackled the problem of training deep networks from limited datasets by taking a two-stage fine-tuning strategy.”
Multi-Label Facial Emotion Classification by Efficient Network from Korean Drama Video Clips
Shin, H., Kim, S., Ok, J., & Yang, X. (2025) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss65446.2025.11149907
“While prior works based on CNNs, autoencoders, and ResNets have shown promising progress, they often lack cultural awareness and support for multi-label classification, relying heavily on prototypical and single-label datasets [14, 15].”
Optimizing Emotion Detection with Attention-Enhanced Convolutional Neural Networks
Rehman, I. U., Ali, Z., & Jan, Z. A. (2025) · International Conference on Frontiers of Information Technology (FIT) · doi:10.1109/fit67061.2025.11333646
“In contrast, earlier research, including that by Huang et al. [4] and Sun et al. [5], has demonstrated that attention-based deep learning models enhance the capacity to capture subtle variations in facial expressions.”
Comprehensive Review of Facial Emotion Recognition Datasets and Methodologies
Kayhan, E., Yüksek, A. G., & Ates, M. (2025) · Sivas Cumhuriyet Üniversitesi Mühendislik Fakültesi Dergisi · doi:10.66248/cumfad.1789566
Urvashi, Rishu, & Kukreja, V. (2026) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-026-05044-z
Multi-object detection and tracking in video sequences
Mhalla, A. (2018) · theses.fr (ABES) · doi:10.70675/6d64f9c8ze926z44d0zb444z11cb37d67638
Apprentissage neuronal profond pour l'analyse de contenus multimodaux et temporels
Vielzeuf, V. (2019) · theses.fr (ABES) · doi:10.70675/cd4ec871zb372z492fz94bfz25086c5fda4a
“Bien que l’idée qu’il existe des liens entre les vidéos de sports et les séquences de visages étudiées ne soit pas immédiate, il est possible de faire le parallèle avec l’approche classique de réseaux entraînés sur ImageNet [74] et donnant de bons résultats sur des tâches très différentes, notamment en reconnaissance…”
BackgroundEstimation automatisée sur vidéos de la biodiversité et de l’abondance des poissons coralliens
Villon, S. (2019) · theses.fr (ABES) · doi:10.70675/db0cb978z2664z4be3zaceaz64b91e4d1da1
Cornec, K. L. (2020) · theses.fr (ABES) · doi:10.70675/208af321z8d3bz4b62z99f6z0b296efd57bc
Marre, G. (2020) · theses.fr (ABES) · doi:10.70675/063a8ffaz9404z402bz968ez6b4ca8aef617
“Cette technique a fait ses preuves (Huh et al., 2016) et elle est couramment utilisée, notamment dans le cas de jeux de données de petite taille (Ng et al., 2015).”
MethodZheng, K., Fu, J., Xu, S., He, Y., Jose, J. M., Han, H., & Ge, X. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810710
Spatiotemporal Self-Attentive Graph-TCN Framework for Enhanced Teen Stress Detection
Indumathy, P., & Kumar, R. P. (2026) · International Journal on Smart Sensing and Intelligent Systems · doi:10.2478/ijssis-2026-0033
Facial Expression Recognition using Deep Convolutional Neural Network
Dubey, S., & Dixit, M. (2019) · International Journal of Communication Systems and Network Technologies · doi:10.18486/ijcsnt/8.1.099
“After this a flatten layer is used which flattens the input from ND to 1D Generally the system operates in 80-10-10 ratio for training-validation-test sets respectively[24].”
Impact of Hyperparameters on Learning and Prediction in CNNs for Facial Emotion Recognition
Qathrady, M. A., Alburaiki, F. M., Almasabi, S., Muthanna, A., & Alwadei, S. A. (2026) · International Conference on Artificial Intelligence Systems (AIS) · doi:10.1109/ais69919.2026.11621311
“Ng et al. [1] used ImageNet-pretrained CNNs with multi-stage fine-tuning for in-the-wild emotion recognition, demonstrating improved performance under limited data conditions.”
DCapsNet: a robust multi-feature deep capsule network for speech emotion recognition
Jahangir, R., Alturki, N. M., Hanif, F., & Alreshoodi, M. A. (2026) · Automatika · doi:10.1080/00051144.2026.2727198
Real-Time Facial Expression Recognition via Live Video Using VGG16-BiLSTM and Residual Attention
Sushant, M. K., Pruthvi, R., Patil, A. M., Rajath, H. R., & Aralimatti, A. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-30960-0_40
Hariri, W. (2026) · Soft Computing · doi:10.1007/s00500-026-11527-0
“As an example, a supervised fine-tuning on a small dataset is applied in Ng et al. (2015) for FER.”
Sathe, O., Mahajan, S., Damahe, K., Yadav, G., Kadam, P., & Patil, V. (2026) · International Conference on Computer, Mechatronics, Robotics & Artificial Intelligence (ICCMRAI) · doi:10.1109/iccmrai67732.2026.11723675
“All images undergo the following preprocessing stages before being fed through any model [6], [24]:”
Influential
Metadata
- Authors
- Ng, H.-W., Nguyen, V. D., Vonikakis, V., & Winkler, S.
- Affiliations
- University of Illinois at Urbana-Champaign (Singapore), Singapore, Singapore
- University of Illinois Urbana-Champaign (US)
- Venue
- Proceedings of the 2015 ACM on International Conference on Multimodal Interaction
- Publisher
- ACM
- Published
- 2015-11-09
- Pages
- 443-449
- OpenAlex
- W2253728219
- Cited by
- 657
- References
- 32
- Citations by year
- 2026: 13 · 2025: 29 · 2024: 37 · 2023: 70 · 2022: 69 · 2021: 106 · 2020: 106 · 2019: 106 · 2018: 63 · 2017: 44 · 2016: 13 · 2015: 1
- FWCI
- 53.84
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
- Funders
- Agency for Science Technology and Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- emotion recognition, facial expression recognition, still images, transfer learning, convolutional neural networks, fine-tuning, ImageNet pre-training, small datasets
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
