The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Cross-dataset learning and person-specific normalisation for automatic Action Unit detection
Abstract
Automatic detection of Facial Action Units (AUs) is crucial for facial analysis systems. Due to the large individual differences, performance of AU classifiers depends largely on training data and the ability to estimate facial expressions of a neutral face. In this paper, we present a real-time Facial Action Unit intensity estimation and occurrence detection system based on appearance (Histograms of Oriented Gradients) and geometry features (shape parameters and landmark locations). Our experiments show the benefits of using additional labelled data from different datasets, which demonstrates the generalisability of our approach. This holds both when training for a specific dataset or when a generic model is needed. We also demonstrate the benefits of using a simple and efficient median based feature normalisation technique that accounts for person-specific neutral expressions. Finally, we show that our results outperform the FERA 2015 baselines in all three challenge tasks - AU occurrence detection, fully automatic AU intensity and pre-segmented AU intensity estimation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303 · cited 2×
“To determine such stable points we observed the most stable CLNF detected landmark points under facial expression on the on CK+ dataset [15] .”
BackgroundMethodDlib-ml: A Machine Learning Toolkit
King, D. E. (2009) · doi:10.5555/1577069.1755843 · Cited by 2,777
Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2010) · Image and Vision Computing · doi:10.1016/j.imavis.2009.08.002 · Cited by 1,621
300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge
Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., & Pantic, M. (2013) · 2013 IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.59 · Cited by 1,119
DISFA: A Spontaneous Facial Action Intensity Database
Mavadati, S. M., Mahoor, M. H., Bartlett, K., Trinh, P., & Cohn, J. F. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2013.4 · Cited by 793
BP4D-Spontaneous: a high-resolution spontaneous 3D dynamic facial expression database
Zhang, X., Yin, L., Cohn, J. F., Canavan, S., Reale, M., Horowitz, A., Liu, P., & Girard, J. M. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2014.06.002 · Cited by 745
Feature Detection and Tracking with Constrained Local Models
Cristinacce, D., & Cootes, T. F. (2006) · Procedings of the British Machine Vision Conference 2006 · doi:10.5244/c.20.95 · Cited by 718
Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition
Sariyanidi, E., Gunes, H., & Cavallaro, A. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2366127 · Cited by 685
Bartlett, M. S., Littlewort, G., Fasel, I., & Movellan, J. R. (2003) · 2003 Conference on Computer Vision and Pattern Recognition Workshop · doi:10.1109/cvprw.2003.10057 · Cited by 573
Research works printed but not linked: 1
- ekman, 1977, Manual for the Facial Action Coding System
Cited by
Openface 2.0: facial behavior analysis toolkit
Baltrusaitis, T., Zadeh, A., Lim, Y. C., & Morency, L.-P. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00019 · Cited by 1,861
Self-citationOpenFace: An open source facial behavior analysis toolkit
Baltrusaitis, T., Robinson, P., & Morency, L.-P. (2016) · 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2016.7477553 · Cited by 1,747
Self-citationGANimation: Anatomically-Aware Facial Animation from a Single Image
Pumarola, A., Agudo, A., Martinez, A. M., Sanfeliu, A., & Moreno-Noguer, F. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_50 · Cited by 621
(2019) · Cited by 415
AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head Synthesis
Guo, Y., Chen, K., Liang, S., Liu, Y.-J., Bao, H., & Zhang, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00573 · Cited by 384
Automatic Analysis of Facial Actions: A Survey
Martinez, B., Valstar, M. F., Jiang, B., & Pantic, M. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2731763 · Cited by 350
Island Loss for Learning Discriminative Features in Facial Expression Recognition
Cai, J., Meng, Z., Khan, A. S., Li, Z., OReilly, J., & Tong, Y. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00051 · Cited by 321
Identity-Aware Convolutional Neural Network for Facial Expression Recognition
Meng, Z., Liu, P., Cai, J., Han, S., & Tong, Y. (2017) · 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) · doi:10.1109/fg.2017.140 · Cited by 319
A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face
Lian, H., Lu, C., Li, S., Zhao, Y., Tang, C., & Zong, Y. (2023) · Entropy · doi:10.3390/e25101440 · Cited by 224
HeadGAN: One-shot Neural Head Synthesis and Editing
Doukas, M. C., Zafeiriou, S., & Sharmanska, V. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01413 · Cited by 117
GLOBEM: Cross-Dataset Generalization of Longitudinal Human Behavior Modeling
Xu, X., Liu, X., Zhang, H., Wang, W., Nepal, S., Sefidgar, Y., Seo, W., Kuehn, K. S., Huckins, J. F., Morris, M. E., et al. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3569485 · Cited by 96
“Almost any applied ML area will encounter the cross-dataset domain generalization challenge for real-life deployment, such as object recognition [6, 15, 45, 86], NLP [70, 117], affective computing [20, 42], security [41, 116], and intelligent interaction [102, 105, 106].”
Multimodal End-to-End Sparse Model for Emotion Recognition
Dai, W., Cahyawijaya, S., Liu, Z., & Fung, P. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.417 · Cited by 91
ICface: Interpretable and Controllable Face Reenactment Using GANs
Tripathy, S., Kannala, J., & Rahtu, E. (2020) · 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv45572.2020.9093474 · Cited by 88
“We used a pre-trained network [2] to predict the presence of 17 action units in a yes or no fashion for the images generated by X2face, DAE, ICface and GANimation.”
MethodJiang, Z., Harati, S., Crowell, A., Mayberg, H. S., Nemati, S., & Clifford, G. D. (2021) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2020.3010472 · Cited by 60
Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion
Agarwal, S., Hu, L., Ng, E., Darrell, T., Li, H., & Rohrbach, A. (2023) · 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv56688.2023.00469 · Cited by 28
A Dual-Branch Fusion Model for Deepfake Detection Using Video Frames and Microexpression Features
Petmezas, G., Vanian, V., Rufete, M. P., Almaloglou, E. E. I., & Zarpalas, D. (2025) · Journal of Imaging · doi:10.3390/jimaging11070231 · Cited by 8
“To leverage this, we followed the methodology by Baltrušaitis et al. [22] to perform a feature-based analysis of facial landmarks and AUs, extracting the relevant microexpression features from the preprocessed video frames.”
EA-FUSION: A multimodal emotion recognition model using EEG and facial expression data
Xu, C., Li, W., Li, S., Zhao, P., Xing, F., & Song, A. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108629 · Cited by 6
Metadata
- Authors
- Baltrusaitis, T., Mahmoud, M., & Robinson, P.
- Venue
- 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG)
- Publisher
- IEEE
- Published
- 2015-05
- Pages
- 1-6
- OpenAlex
- W1628791547
- Cited by
- 412
- References
- 27
- FWCI
- 12.89
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.