Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; the references it cites are not checked yet.

Semi-supervised Multi-modal Emotion Recognition with Cross-Modal Distribution Matching

Open access
Liang, Jingjun, Li, Ruichen, Jin, Qin · 2020 · Proceedings of the 28th ACM International Conference on Multimedia

Abstract

Automatic emotion recognition is an active research topic with wide range of applications. Due to the high manual annotation cost and inevitable label ambiguity, the development of emotion recognition dataset is limited in both scale and quality. Therefore, one of the key challenges is how to build effective models with limited data resource. Previous works have explored different approaches to tackle this challenge including data enhancement, transfer learning, and semi-supervised learning etc. However, the weakness of these existing approaches includes such as training instability, large performance loss during transfer, or marginal improvement. In this work, we propose a novel semi-supervised multi-modal emotion recognition model based on cross-modality distribution matching, which leverages abundant unlabeled data to enhance the model training under the assumption that the inner emotional status is consistent at the utterance level across modalities. We conduct extensive experiments to evaluate the proposed model on two benchmark datasets, IEMOCAP and MELD. The experiment results prove that the proposed semi-supervised learning model can effectively utilize unlabeled data and combine multi-modalities to boost the emotion recognition performance, which outperforms other state-of-the-art approaches under the same condition. The proposed model also achieves competitive capacity compared with existing approaches which take advantage of additional auxiliary information such as speaker and interaction context.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3394171.3413579

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

The citation lists are queued

They are fetched later and appear here then.

References 51 · Cited by 73

Research works printed but not linked: 12

  1. 10.1145/3240508.3240578 · DOI · Paper page
  2. 10.5555/3298023.3298146 · doi:10.5555/3298023.3298146
  3. Eyben Florian, 2010, ACM International Conference on Multimedia. 1459--1462
  4. Ringeval Fabien, 2018, Proceedings of the 8th International Workshop on Audio/Visual Emotion Challenge, AVEC'18
  5. Goodfellow Ian J., 2014, International Conference on Neural Information Processing Systems. 2672--2680
  6. Herve Jegou Matthijs Douze and Cordelia Schmid. 2010b. Product quantization for nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence Vol. 33 1 (2010) 117--128. Herve Jegou Matthijs Douze and Cordelia Schmid. 2010b. Product quantization for nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence Vol. 33 1 (2010) 117--128. · DOI · Paper page
  7. 10.18653/v1/P17-1081 · DOI · Paper page
  8. Rozgić Viktor, 2012, INTERSPEECH
  9. Salimans Tim, 2016, Improved Techniques for Training GANs. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016
  10. Yutaka Sasaki et al. 2007. The truth of the F-measure. Teach Tutor mater Vol. 1 5 (2007) 1--5. Yutaka Sasaki et al. 2007. The truth of the F-measure. Teach Tutor mater Vol. 1 5 (2007) 1--5.
  11. Srivastava Nitish, 2015, International conference on machine learning. 843--852
  12. Zadeh Amir, 2018, Thirty-Second AAAI Conference on Artificial Intelligence.

Metadata

Authors
Liang, J., Li, R., & Jin, Q.
Affiliations
Renmin University of China (CN)
Venue
Proceedings of the 28th ACM International Conference on Multimedia
Publisher
ACM
Published
2020-10-12
Pages
2852-2861
OpenAlex
W3083650259
Cited by
73
References
51
Citations by year
2026: 9 · 2025: 13 · 2024: 15 · 2023: 15 · 2022: 13 · 2021: 8
FWCI
8.78
Type
conference paper
Language
en
Open access
Locations
  • ACM International Conference on Multimedia (ACM MM) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Beijing Natural Science Foundation (4192028)
  • National Natural Science Foundation of China (61772535)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
  • Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
Keywords
emotion recognition, semi-supervised learning, multimodal fusion, unlabeled data, limited annotations, IEMOCAP, MELD

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Semi-supervised Multi-modal Emotion Recognition with Cross-Modal Distribution Matching. https://app.cochat.ai/papers/10.1145/3394171.3413579
The paper
Liang, J., Li, R., & Jin, Q. (2020). Semi-supervised Multi-modal Emotion Recognition with Cross-Modal Distribution Matching. Proceedings of the 28th ACM International Conference on Multimedia. https://doi.org/10.1145/3394171.3413579

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.