The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Recurrent Neural Networks for Emotion Recognition in Video
Abstract
Deep learning based approaches to facial analysis and video analysis have recently demonstrated high performance on a variety of key tasks such as face recognition, emotion recognition and activity recognition. In the case of video, information often must be aggregated across a variable length sequence of frames to produce a classification result. Prior work using convolutional neural networks (CNNs) for emotion recognition in video has relied on temporal averaging and pooling operations reminiscent of widely used approaches for the spatial aggregation of information. Recurrent neural networks (RNNs) have seen an explosion of recent interest as they yield state-of-the-art performance on a variety of sequence analysis tasks. RNNs provide an attractive framework for propagating information over a sequence using a continuous valued hidden layer representation. In this work we present a complete system for the 2015 Emotion Recognition in the Wild (EmotiW) Challenge. We focus our presentation and experimental analysis on a hybrid CNN-RNN architecture for facial expression analysis that can outperform a previously applied CNN approach using temporal averaging for aggregation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893
(2012) · Cited by 8,009
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601
Advances in Neural Information Processing Systems 20
Bonilla, E. V., Chai, K. M. A., & Williams, C. K. I. (2008) · neural information processing systems · Cited by 2,846
Opensmile: the munich versatile and fast open-source audio feature extractor
Eyben, F., Wöllmer, M., & Schuller, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874246 · Cited by 2,642 · cited 1×
“It uses 1582 features extracted with the open-source Emotion and Affect Recognition (openEAR) [12] toolkit which uses openSMILE [11] as backend.”
MethodFacial expression recognition based on Local Binary Patterns: A comprehensive study
Shan, C., Gong, S., & McOwan, P. W. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.08.005 · Cited by 2,300
Deep Convolutional Network Cascade for Facial Point Detection
Sun, Y., Wang, X., & Tang, X. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.446 · Cited by 1,382
Theano: new features and speed improvements
Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I., Bergeron, A., Bouchard, N., Warde-Farley, D., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5590 · Cited by 1,005
Long-term Recurrent Convolutional Networks for Visual Recognition and Description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., & Darrell, T. (2014) · doi:10.21236/ada623249 · Cited by 968
Le, Q. V., Zou, W. Y., Yeung, S. Y., & Ng, A. Y. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995496 · Cited by 960
Sequential Deep Learning for Human Action Recognition
Baccouche, M., Mamalet, F., Wolf, C., Garcia, C., & Baskurt, A. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-25446-8_4 · Cited by 782
Random search for hyper-parameter optimization
BergstraJames, & BengioYoshua (2012) · Journal of Machine Learning Research · doi:10.5555/2188385.2188395 · Cited by 690
Offline Arabic Handwriting Recognition with Multidimensional Recurrent Neural Networks
Graves, A. (2012) · Guide to OCR for Arabic Scripts · doi:10.1007/978-1-4471-4072-6_12 · Cited by 684
Collecting Large, Richly Annotated Facial-Expression Databases from Movies
Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2012) · IEEE MultiMedia · doi:10.1109/mmul.2012.26 · Cited by 663
A Simple Way to Initialize Recurrent Networks of Rectified Linear Units
Le, Q. V., Jaitly, N., & Hinton, G. E. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.00941 · Cited by 513
Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification
Wu, Z., Wang, X., Jiang, Y.-G., Ye, H., & Xue, X. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2806222 · Cited by 479
OpenEAR — Introducing the munich open-source emotion and affect recognition toolkit
Eyben, F., Wollmer, M., & Schuller, B. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349350 · Cited by 418
EmoNets: Multimodal deep learning approaches for emotion recognition in video
Kahou, S. E., Bouthillier, X., Lamblin, P., Gulcehre, C., Michalski, V., Konda, K., Jean, S., Froumenty, P., Dauphin, Y., Boulanger-Lewandowski, N., et al. (2016) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-015-0195-2 · Cited by 412
Self-citationCombining modality specific deep neural networks for emotion recognition in video
Kahou, S. E., Pal, C., Bouthillier, X., Froumenty, P., Gülçehre, Ç., Memisevic, R., Vincent, P., Courville, A., Bengio, Y., Ferrari, R. C., et al. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2531745 · Cited by 391
Self-citationVideo and Image based Emotion Recognition Challenges in the Wild: EmotiW 2015
Dhall, A., Ramana Murthy, O. V., Goecke, R., Joshi, J., & Gedeon, T. (2015) · Proceedings of the 2015 ACM on International Conference on Multimodal Interaction · doi:10.1145/2818346.2829994 · Cited by 305
AVEC 2011–The First International Audio/Visual Emotion Challenge
Schuller, B., Valstar, M., Eyben, F., McKeown, G., Cowie, R., & Pantic, M. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-24571-8_53 · Cited by 271
Emotion Recognition In The Wild Challenge 2014: Baseline, Data and Protocol
Dhall, A., Goecke, R., Joshi, J., Sikka, K., & Gedeon, T. (2014) · Proceedings of the 16th International Conference on Multimodal Interaction · doi:10.1145/2663204.2666275 · Cited by 216
Combining Multiple Kernel Methods on Riemannian Manifold for Emotion Recognition in the Wild
Liu, M., Wang, R., Li, S., Shan, S., Huang, Z., & Chen, X. (2014) · Proceedings of the 16th International Conference on Multimodal Interaction · doi:10.1145/2663204.2666274 · Cited by 202
Emotion recognition in the wild challenge 2013
Dhall, A., Goecke, R., Joshi, J., Wagner, M., & Gedeon, T. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2531739 · Cited by 178
Journal on Multimodal User Interfaces
(2013) · Journal on Multimodal User Interfaces · doi:10.1007/12193.1783-8738 · Cited by 79
Facial Expression Analysis Based on High Dimensional Binary Features
Kahou, S. E., Froumenty, P., & Pal, C. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16181-5_10 · Cited by 45
Self-citationLearning to encode motion using spatio-temporal synchronyPreprint
Konda, K., Memisevic, R., & Michalski, V. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.3162 · Cited by 15
Self-citation
Cited by
Tensor Fusion Network for Multimodal Sentiment Analysis
Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1115 · Cited by 2,102
Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 1,753
End-to-End Multimodal Emotion Recognition using Deep Neural Networks
Tzirakis, P., Trigeorgis, G., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2017) · arXiv · doi:10.48550/arxiv.1704.08619 · Cited by 752
A Brief Review of Facial Emotion Recognition Based on Visual Information
Ko, B. (2018) · Sensors · doi:10.3390/s18020401 · Cited by 724
Speech Emotion Recognition Using Deep Learning Techniques: A Review
Khalil, R. A., Jones, E., Babar, M. I., Jan, T., Zafar, M. H., & Alhussain, T. (2019) · IEEE Access · doi:10.1109/access.2019.2936124 · Cited by 702
Metadata
- Authors
- Ebrahimi Kahou, S., Michalski, V., Konda, K., Memisevic, R., & Pal, C.
- Venue
- Proceedings of the 2015 ACM on International Conference on Multimodal Interaction
- Publisher
- ACM
- Published
- 2015-11-09
- Pages
- 467-474
- OpenAlex
- W2277498883
- Cited by
- 379
- References
- 33
- FWCI
- 40.63
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.