The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
EmotioNet: An Accurate, Real-Time Algorithm for the Automatic Annotation of a Million Facial Expressions in the Wild
Abstract
Research in face perception and emotion theory requires very large annotated databases of images of facial expressions of emotion. Annotations should include Action Units (AUs) and their intensities as well as emotion category. This goal cannot be readily achieved manually. Herein, we present a novel computer vision algorithm to annotate a large database of one million images of facial expressions of emotion in the wild (i.e., face images downloaded from the Internet). First, we show that this newly proposed algorithm can recognize AUs and their intensities reliably across databases. To our knowledge, this is the first published algorithm to achieve highly-accurate results in the recognition of AUs and their intensities across multiple databases. Our algorithm also runs in real-time (>30 images/second), allowing it to work with large numbers of images and video sequences. Second, we use WordNet to download 1,000,000 images of facial expressions with associated emotion keywords from the Internet. These images are then automatically annotated with AUs, AU intensities and emotion categories by our algorithm. The result is a highly useful database that can be readily queried using semantic descriptions for applications in computer vision, affective computing, social and cognitive psychology and neuroscience, e.g., "show me all the images with happy faces" or "all images with AU 1 at intensity c".
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 7×
“WordNet includes synonyms (i.e., words that have the same or nearly the same meaning), hyponyms (i.e., subordinate nouns or nouns of more specific meaning, which defines a hierarchy of relationships), troponymys (i.e., verbs of more specific meaning, which defines a hierarchy of verbs), and entailments (i.e., deductio…”
MethodInfluentialLucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
LabelMe: A Database and Web-Based Tool for Image Annotation
Russell, B. C., Torralba, A., Murphy, K. P., & Freeman, W. T. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0090-8 · Cited by 4,210
Daugman, J. G. (1985) · Journal of the Optical Society of America A · doi:10.1364/josaa.2.001160 · Cited by 3,219
Face recognition by elastic bunch graph matching
Wiskott, L., Fellous, J.-M., Kuiger, N., & von der Malsburg, C. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598235 · Cited by 2,883
One millisecond face alignment with an ensemble of regression trees
Kazemi, V., & Sullivan, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.241 · Cited by 2,831
Ekman, P., & Rosenberg, E. L. (2005) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195179644.001.0001 · Cited by 1,941
Supervised Descent Method and Its Applications to Face Alignment
Xiong, X., & De la Torre, F. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.75 · Cited by 1,896
Cited by
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338
Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements
Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019) · Psychological Science in the Public Interest · doi:10.1177/1529100619832930 · Cited by 1,748
Self-citationDeep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 1,753
Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild
Li, S., Deng, W., & Du, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.277 · Cited by 1,470
Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition
Wang, K., Peng, X., Yang, J., Meng, D., & Qiao, Y. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2956143 · Cited by 940
Suppressing Uncertainties for Large-Scale Facial Expression Recognition
Wang, K., Peng, X., Yang, J., Lu, S., & Qiao, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00693 · Cited by 772
GANimation: Anatomically-Aware Facial Animation from a Single Image
Pumarola, A., Agudo, A., Martinez, A. M., Sanfeliu, A., & Moreno-Noguer, F. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_50 · Cited by 621
Self-citationWing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks
Feng, Z.-H., Kittler, J., Awais, M., Huber, P., & Wu, X.-J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00238 · Cited by 464
Facial Expression Recognition with Inconsistently Annotated Datasets
Zeng, J., Shan, S., & Chen, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_14 · Cited by 341
Context-Aware Emotion Recognition Networks
Lee, J., Kim, S., Kim, S., Park, J., & Sohn, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01024 · Cited by 323
Facial Expression Recognition Using Enhanced Deep 3D Convolutional Neural Networks
Hasani, B., & Mahoor, M. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.282 · Cited by 287
She, J., Hu, Y., Shi, H., Wang, J., Shen, Q., & Mei, T. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00618 · Cited by 264
Joint Pose and Expression Modeling for Facial Expression Recognition
Zhang, F., Zhang, T., Mao, Q., & Xu, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00354 · Cited by 256
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243
Context Based Emotion Recognition Using EMOTIC Dataset
Kosti, R., Alvarez, J., Recasens, A., & Lapedriza, A. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2916866 · Cited by 211
Facial Expression Recognition: A Survey
Huang, Y., Chen, F., Lv, S., & Wang, X. (2019) · Symmetry · doi:10.3390/sym11101189 · Cited by 195
(2019) · doi:10.1109/wacv.2019.00178 · Cited by 151
Self-supervised learning of a facial attribute embedding from video
Wiles, O., Koepke, A. S., & Zisserman, A. (2018) · arXiv · doi:10.48550/arxiv.1808.06882 · Cited by 111
Srinivasan, R., & Martinez, A. M. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2887267 · Cited by 94
Self-citationLe Mau, T., Hoemann, K., Lyons, S. H., Fugate, J. M. B., Brown, E. N., Gendron, M., & Barrett, L. F. (2021) · Nature Communications · doi:10.1038/s41467-021-25352-6 · Cited by 74
Human Emotion Recognition With Relational Region-Level Analysis
Li, W., Dong, X., & Wang, Y. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3064918 · Cited by 43
Sharifnejad, M., Shahbahrami, A., Akoushideh, A., & Hassanpour, R. Z. (2021) · IET Image Processing · doi:10.1049/ipr2.12037 · Cited by 19
“In order to evaluate the proposed technique, we have used recognition accuracy, computational time and feature vector length criteria [31, 32].”
MethodLearning Gait Emotions Using Affective and Deep Features
Randhavane, T., Bhattacharya, U., Kabra, P., Kapsaskis, K., Gray, K., Manocha, D., & Bera, A. (2022) · Proceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games · doi:10.1145/3561975.3562957 · Cited by 13
Metadata
- Authors
- Benitez-Quiroz, C. F., Srinivasan, R., & Martinez, A. M.
- Venue
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2016-06
- Pages
- 5562-5570
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2436394355
- Cited by
- 607
- References
- 32
- FWCI
- 60.38
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.