An Investigation of Annotation Delay Compensation and Output-Associative Fusion for Multimodal Continuous Emotion Prediction
The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Continuous emotion dimension prediction has increased in popularity over the last few years, as the shift away from discrete classification based tasks has introduced more realism in emotion modeling. However, many questions remain including how best to combine information from several modalities (e.g. audio, video, etc). As part of the AV+EC 2015 Challenge, we investigate annotation delay compensation and propose a range of multimodal systems based on an output-associative fusion framework. The performance of the proposed systems are significantly higher than the challenge baseline, with the strongest performing system yielding 66.7% and 53.9% relative increases in prediction accuracy over the AV+EC 2015 test set arousal and valence baselines respectively. Results also demonstrate the importance of annotation delay compensation for continuous emotion analysis. Of particular interest was the output-associative based fusion framework, which performed very well in a number of significantly different configurations, highlighting that incorporating both affective dimensional dependencies and temporal information is a promising research direction for predicting emotion dimensions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing
Eyben, F., Scherer, K. R., Schuller, B. W., Sundberg, J., Andre, E., Busso, C., Devillers, L. Y., Epps, J., Laukka, P., Narayanan, S. S., et al. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2457417 · Cited by 1,818 · cited 1×
“…provides a set of standard acoustic features to participants: the 102-dimensional Extended Geneva Minimalistic Acoustic Parameter Set ( E G E MAPS) [20]; two types of video descriptors (84-dimensional set of facial based appearance features and 316-dimensional set of facial based geometric…”
BackgroundSelf-citationIntroducing the RECOLA multimodal corpus of remote collaborative and affective interactions
Ringeval, F., Sonderegger, A., Sauer, J., & Lalanne, D. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553805 · Cited by 771
Nicolaou, M. A., Gunes, H., & Pantic, M. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.9 · Cited by 450 · cited 9×
“Also, a combination of the modalities can lead to further improvements [7, 8].”
BackgroundMethodInfluentialGunes, H., & Schuller, B. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.06.016 · Cited by 360 · cited 10×
“A linear kernel was used and C was set to 0.005 and 0.05 for arousal and valence respectively, based on optimizing the performance with delay and smoothing in the development set in the range of [10 -4 ,1].”
BackgroundMethodInfluentialPrediction of asynchronous dimensional emotion ratings from audiovisual and physiological data
Ringeval, F., Eyben, F., Kroupi, E., Yuce, A., Thiran, J.-P., Ebrahimi, T., Lalanne, D., & Schuller, B. (2015) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.11.007 · Cited by 202 · cited 1×
“Common systems for learning decision level fusion weights include either Linear Regression (LR) [9, 12], or SVR [18].”
Background(2015) · doi:10.1145/2808196.2811642 · Cited by 178
Cited by
AFEW-VA database for valence and arousal estimation in-the-wild
Kossaifi, J., Tzimiropoulos, G., Todorovic, S., & Pantic, M. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.02.001 · Cited by 236
“The same approach was recently used in the AVEC challenge by the work [27], which combines early, outputassociative and late fusion.”
MethodMultimodal Multi-task Learning for Dimensional and Continuous Emotion Recognition
Chen, S., Jin, Q., Zhao, J., & Wang, S. (2017) · Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge · doi:10.1145/3133944.3133949 · Cited by 147
“[11] investigate the RML model with the annotation delay compensation and outputassociative fusion.”
MethodMulti-Modal Audio, Video and Physiological Sensor Learning for Continuous Emotion Prediction
Brady, K., Gwon, Y., Khorrami, P., Godoy, E., Campbell, W., Dagli, C., & Huang, T. S. (2016) · Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge · doi:10.1145/2988257.2988264 · Cited by 110
Semi-supervised Multi-modal Emotion Recognition with Cross-Modal Distribution Matching
Liang, J., Li, R., & Jin, Q. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413579 · Cited by 73
Strength modelling for real-worldautomatic continuous affect recognition from audiovisual signals
Han, J., Zhang, Z., Cummins, N., Ringeval, F., & Schuller, B. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2016.11.020 · Cited by 53
Self-citation
Metadata
- Authors
- Huang, Z., Dang, T., Cummins, N., Stasak, B., Le, P., Sethu, V., & Epps, J.
- Venue
- Proceedings of the 5th International Workshop on Audio/Visual Emotion Challenge
- Publisher
- ACM
- Published
- 2015-10-26
- Pages
- 41-48
- OpenAlex
- W1976235033
- Cited by
- 69
- References
- 6
- FWCI
- 13.39
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An Investigation of Annotation Delay Compensation and Output-Associative Fusion for Multimodal Continuous Emotion Prediction. https://app.cochat.ai/papers/10.1145/2808196.2811640
- The paper
- Huang, Z., Dang, T., Cummins, N., Stasak, B., Le, P., Sethu, V., & Epps, J. (2015). An Investigation of Annotation Delay Compensation and Output-Associative Fusion for Multimodal Continuous Emotion Prediction. Proceedings of the 5th International Workshop on Audio/Visual Emotion Challenge. https://doi.org/10.1145/2808196.2811640
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.