A Deeper Look at Facial Expression Dataset Bias
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Datasets play an important role in the progress of facial expression recognition algorithms, but they may suffer from obvious biases caused by different cultures and collection conditions. To look deeper into this bias, we first conduct comprehensive experiments on dataset recognition and cross-dataset generalization tasks, and for the first time, explore the intrinsic causes of the dataset discrepancy. The results quantitatively verify that current datasets have a strong build-in bias, and corresponding analyses indicate that the conditional probability distributions between source and target datasets are different. However, previous researches are mainly based on shallow features with limited discriminative ability under the assumption that the conditional distribution remains unchanged across domains. To address these issues, we further propose a novel deep Emotion-Conditional Adaption Network (ECAN) to learn domain-invariant and discriminative feature representations, which can match not only the marginal distribution but also the class-conditional distribution across domains by exploring the underlying label information of the target dataset. Moreover, the largely ignored expression class distribution bias is also addressed so that the training and testing domains can share similar class distribution. Extensive cross-database experiments on both lab-controlled datasets (CK+, JAFFE, MMI, and Oulu-CASIA) and real-world databases (AffectNet, FER2013, RAF-DB 2.0, and SFEW 2.0) demonstrate that our ECAN can yield competitive performances across various cross-dataset facial expression recognition tasks and outperform the state-of-the-art methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · cited 1×
“As related research [31] has suggested, the network becomes more task-aimed as the layer goes deeper and hence it will become difficult to directly transfer the learned feature to the target domain.”
BackgroundLearning Transferable Features with Deep Adaptation Networks
Long, M., Cao, Y., Wang, J., & Jordan, M. I. (2015) · arXiv · doi:10.48550/arxiv.1502.02791 · Cited by 4,505 · cited 1×
“Following the work in [5], we first plug the multi-kernel maximum mean discrepancy (MK-MMD) into the original deep network architecture to effectively measure the divergence between the source and target domain.”
MethodLucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303 · cited 1×
“(2) CK+ Database [34]: The lab-controlled database CK+ contains 593 video sequences from 123 subjects.”
Background(2012) · MPG.PuRe (Max Planck Society) · doi:10.5555/2188385.2188410 · Cited by 3,603
VGGFace2: A Dataset for Recognising Faces across Pose and Age
Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00020 · Cited by 3,032
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338
MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition
Guo, Y., Zhang, L., Hu, Y., He, X., & Gao, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_6 · Cited by 2,122 · cited 1×
“We choose the ResNet-50 Convolutional Neural Network [25] that are first pre-trained on the MS-Celeb-1M face dataset [26] and then fine-tuned on the VGGFace2 face dataset [27] to extract deep features.”
MethodReturn of Frustratingly Easy Domain Adaptation
Sun, B., Feng, J., & Saenko, K. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10306 · Cited by 2,004
Integrating structured biological data by Kernel Maximum Mean Discrepancy
Borgwardt, K. M., Gretton, A., Rasch, M. J., Kriegel, H.-P., Schölkopf, B., & Smola, A. J. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl242 · Cited by 1,674
Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism
Li, Y., Zeng, J., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2886767 · Cited by 916 · cited 1×
“Moreover, as deep learning has been gradually leveraged to deal with challenging variety in the wild [4], [19], we also employ CNN to learn features with more discriminative ability.”
MethodChallenges in representation learning: A report on three machine learning contests
Goodfellow, I. J., Erhan, D., Luc Carrier, P., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D.-H., et al. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.005 · Cited by 697
Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark
Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130508 · Cited by 541 · cited 2×
“(8) SFEW 2.0 Database [3]: The in-the-wild database SFEW contains dynamic images selected from different films with spontaneous expressions, various head pose, age range, occlusions and illuminations.”
BackgroundInfluential
Cited by
A systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 615
Facial Expression Recognition: A Survey
Huang, Y., Chen, F., Lv, S., & Wang, X. (2019) · Symmetry · doi:10.3390/sym11101189 · Cited by 195
Face2Exp: Combating Data Biases for Facial Expression Recognition
Zeng, D., Lin, Z., Yan, X., Liu, Y., Wang, F., & Tang, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01965 · Cited by 141
“ECAN [18] learns domain-invariant and discriminative feature representations by applying maximum mean discrepancy as re-weighted regularization and class-conditional regularization learning.”
MethodUsing an Artificial Intelligence (AI) Agent to Support Teacher Instruction and Student Learning
Dieker, L., Hines, R., Wilkins, I., Hughes, C., Hawkins Scott, K., Smith, S., Ingraham, K., Ali, K., Zaugg, T., & Shah, S. (2024) · Journal of Special Education Preparation · doi:10.33043/d8xb94q7 · Cited by 16
Metadata
- Authors
- Li, S., & Deng, W.
- Venue
- IEEE Transactions on Affective Computing
- Published
- 2022-04-01
- Volume
- 13
- Issue
- 2
- Pages
- 881-893
- ISSN
- 1949-3045 (ISSN-L)
- 2371-9850
- OpenAlex
- W2940983946
- Cited by
- 139
- References
- 14
- FWCI
- 16.19
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.