Multi-scale Temporal Modeling for Dimensional Emotion Recognition in Video
The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Understanding nonverbal behaviors in human machine interaction is a complex and challenge task. One of the key aspects is to recognize human emotion states accurately. This paper presents our effort to the Audio/Visual Emotion Challenge (AVEC'14), whose goal is to predict the continuous values of the emotion dimensions arousal, valence and dominance at each moment in time. The proposed method utilizes deep belief network based models to recognize emotion states from audio and visual modalities. Firstly, we employ temporal pooling functions in the deep neutral network to encode dynamic information in the features, which achieves the first time scale temporal modeling. Secondly, we combine the predicted results from different modalities and emotion temporal context information simultaneously. The proposed multimodal-temporal fusion achieves temporal modeling for the emotion states in the second time scale. Experiments results show the efficiency of each key point of the proposed method and competitive results are obtained
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep learning for robust feature generation in audiovisual emotion recognition
Kim, Y., Lee, H., & Provost, E. M. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638346 · Cited by 401
Combining modality specific deep neural networks for emotion recognition in video
Kahou, S. E., Pal, C., Bouthillier, X., Froumenty, P., Gülçehre, Ç., Memisevic, R., Vincent, P., Courville, A., Bengio, Y., Ferrari, R. C., et al. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2531745 · Cited by 391
Neural Correlates of Processing Valence and Arousal in Affective Words
Lewis, P., Critchley, H., Rotshtein, P., & Dolan, R. (2006) · Cerebral Cortex · doi:10.1093/cercor/bhk024 · Cited by 391
LSTM-Modeling of continuous emotions in an audiovisual affect recognition framework
Wöllmer, M., Kaiser, M., Eyben, F., Schuller, B., & Rigoll, G. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.03.001 · Cited by 288 · cited 2×
“Moreover, a number of recent studies [13-17] in affective computing show the importance of temporal modeling for dimensional emotion recognition.”
BackgroundDeep neural networks for acoustic emotion recognition: Raising the benchmarks
Stuhlsatz, A., Meyer, C., Eyben, F., Zielke, T., Meier, G., & Schuller, B. (2011) · 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2011.5947651 · Cited by 288 · cited 2×
“Meanwhile, deep learning techniques have been found successful recently in various communities, including emotion recognition [23-26].”
BackgroundMethodMeng, H., Huang, D., Wang, H., Yang, H., AI-Shuraifi, M., & Wang, Y. (2013) · Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge · doi:10.1145/2512530.2512532 · Cited by 225
Emotion recognition in the wild challenge 2013
Dhall, A., Goecke, R., Joshi, J., Wagner, M., & Gedeon, T. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2531739 · Cited by 178
Soladié, C., Salam, H., Pelachaud, C., Stoiber, N., & Séguier, R. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388782 · Cited by 33 · cited 1×
“Previous studies [19, 35] have demonstrated that there is a delay between recordings and labels.”
Background
Cited by
AFEW-VA database for valence and arousal estimation in-the-wild
Kossaifi, J., Tzimiropoulos, G., Todorovic, S., & Pantic, M. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.02.001 · Cited by 236
“averaging, as in [29], to more complex methods such as linear and multi-linear regression [15,25,26,28,30,37,38], SVR [39], random forests [32] or Kalman filters based [34–36].”
BackgroundMethodFacial Expression Recognition: A Survey
Huang, Y., Chen, F., Lv, S., & Wang, X. (2019) · Symmetry · doi:10.3390/sym11101189 · Cited by 195
Multimodal Multi-task Learning for Dimensional and Continuous Emotion Recognition
Chen, S., Jin, Q., Zhao, J., & Wang, S. (2017) · Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge · doi:10.1145/3133944.3133949 · Cited by 147
“The second place winner [3] in AVEC2014 challenge use Deep Belief Network (DBN) with temporal pooling and multimodaltemporal fusion, which demonstrates that high level temporal fusion can improve performance.”
BackgroundSun, L., Lian, Z., Tao, J., Liu, B., & Niu, M. (2020) · Proceedings of the 1st International on Multimodal Sentiment Analysis in Real-life Media Challenge and Workshop · doi:10.1145/3423327.3423672 · Cited by 81
Self-citationContinuous Multimodal Emotion Prediction Based on Long Short Term Memory Recurrent Neural Network
Huang, J., Li, Y., Tao, J., Lian, Z., Wen, Z., Yang, M., & Yi, J. (2017) · Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge · doi:10.1145/3133944.3133946 · Cited by 54
“Chao et al. utilize temporal pooling function in the forward network for deep belief network [33] and LSTM [24] to reduce the label noise and redundant information.”
BackgroundSelf-citation
Metadata
- Authors
- Chao, L., Tao, J., Yang, M., Li, Y., & Wen, Z.
- Venue
- Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge
- Publisher
- ACM
- Published
- 2014-11-07
- Pages
- 11-18
- OpenAlex
- W2069889204
- Cited by
- 39
- References
- 8
- FWCI
- 8.75
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Multi-scale Temporal Modeling for Dimensional Emotion Recognition in Video. https://app.cochat.ai/papers/10.1145/2661806.2661811
- The paper
- Chao, L., Tao, J., Yang, M., Li, Y., & Wen, Z. (2014). Multi-scale Temporal Modeling for Dimensional Emotion Recognition in Video. Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge. https://doi.org/10.1145/2661806.2661811
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.