NNIME: The NTHU-NTUA Chinese interactive multimodal emotion corpus
The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
The increasing availability of large-scale emotion corpus along with the advancement in emotion recognition algorithms have enabled the emergence of next-generation human-machine interfaces. This paper describes a newly-collected multimodal corpus, i.e., the NTHU-NTUA Chinese Interactive Emotion Corpus (NNIME). The database is a result of the collaborative work between engineers and drama experts. This database includes recordings of 44 subjects engaged in spontaneous dyadic spoken interactions. The multimodal data includes approximately 11-hour worth of audio, video, and electrocardiogram data recorded continuously and synchronously. The database is also completed with a rich set of emotion annotations on discrete and continuous-in-time annotation by a total of 49 annotators. Thees emotion annotations include a diverse perspectives: peer-report, director-report, self-report, and observer-report. This carefully-engineered data collection and annotation processes provide an additional valuable resource to quantify and investigate various aspects of affective phenomenon and human communication. To our best knowledge, the NNIME is one of the few large-scale Chinese affective dyadic interaction database that have been systematically collected, organized, and to be publicly-released to the research community.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Structurally, the NNIME dataset comprises 6701 sentences and includes both discrete and continuous emotion labels.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [29]
In the citing paper
“The NTHU-NTUA Chinese Interactive Emotion Corpus (NNIME) [29] is a groundbreaking initiative aimed at creating an interactive multimodal corpus for Chinese emotional expressions.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [29]
Citations
Cites
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
Praat, a system for doing phonetics by computer
Boersma, P. (2002) · UvA-DARE (University of Amsterdam) · Cited by 4,417
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869 · cited 6×
“Furthermore, due to its similarity in the design with the IEMOCAP and the CreativeIT, the idiosyncratic factor in the database design can be much mitigated.”
MethodSelf-citationAction Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
Exploiting Generative Models in Discriminative Classifiers
Jaakkola, T., & Haussler, D. H. (1998) · Cited by 1,271
The devil is in the details: an evaluation of recent feature encoding methods
Chatfield, K., Lempitsky, V., Vedaldi, A., & Zisserman, A. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.76 · Cited by 834 · cited 1×
“In order to encode the sequence of LLDs to a fixed-length high-dimensional session-level features, we use the GMM based Fishervector encoding [30], [31], i.”
MethodIntroducing the RECOLA multimodal corpus of remote collaborative and affective interactions
Ringeval, F., Sonderegger, A., Sauer, J., & Lalanne, D. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553805 · Cited by 771 · cited 4×
“RECOLA [17] 46 6 Audio, Video, ECG, EDA Dyadic French Spont.”
BackgroundInfluentialFEELTRACE: an instrument for recording perceived emotion in real time
Cowie, R., Douglas‐Cowie, E., Savvidou, S. M., & McMahon, E. R. (2000) · Research Portal (Queen's University Belfast) · Cited by 539
Effects of Dyadic Interaction on Argumentive Reasoning
Kuhn, D., Shaw, V., & Felton, M. (1997) · Cognition and Instruction · doi:10.1207/s1532690xci1503_1 · Cited by 475 · cited 1×
“Dyadic interaction is a basic unit of face-to-face interaction providing an important gateway for humans to convey emotion, communicate information, and foster mutual understanding [4], [5], [6], [7].”
BackgroundMother and infant at play: The dyadic interaction involving facial, vocal, and gaze behaviors.
Stern, D. N. (1974) · Cited by 459
Research works printed but not linked: 5
- busso, 2008, Recording audio-visual emotional databases from actors: a closer look, Second International Workshop on Emotion Corpora for Research on Emotion and Affect International conference on Language Resources and Evaluation (LREC 2008), 17
- knapp, 2013, Nonverbal communication in human interaction Cengage Learning
- 10.1037/0033-2909.89.1.101 · doi:10.1037/0033-2909.89.1.101 · Paper page
- 10.1037/0012-1649.33.3.538 · doi:10.1037/0012-1649.33.3.538 · Paper page
- nogueiras, 2002, Speech emotion recognition using hidden markov models, speech prosody 2002, an International Conference France, 11
Cited by
A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face
Lian, H., Lu, C., Li, S., Zhao, Y., Tang, C., & Zong, Y. (2023) · Entropy · doi:10.3390/e25101440 · Cited by 224
“The NTHU-NTUA Chinese Interactive Emotion Corpus (NNIME) [29] is a ground-breaking initiative aimed at creating an interactive multimodal corpus for Chinese emotional expressions.”
BackgroundSpeech Emotion Recognition Using Deep Neural Network Considering Verbal and Nonverbal Speech Sounds
Huang, K.-Y., Wu, C.-H., Hong, Q.-B., Su, M.-H., & Chen, Y.-H. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8682283 · Cited by 116
“Different from other scripted emotion speech database or segmented audio from dramas, NTHU-NTUA Chinese Interactive Multimodal Emotion Corpus (NNIME) [18], one of the few large-scale Chinese affective dyadic interaction database which contains various emotional nonverbal sounds, such as laughter,”
BackgroundInfluentialMulti-modal emotion recognition using EEG and speech signals
Wang, Q., Wang, M., Yang, Y., & Zhang, X. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105907 · Cited by 100
George, S. M., & Ilyas, P. M. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.127015 · Cited by 125
Speech Emotion Recognition Considering Nonverbal Vocalization in Affective Conversations
Hsu, J.-H., Su, M.-H., Wu, C., & Chen, Y. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3076364 · Cited by 84
Metadata
- Authors
- Chou, H.-C., Lin, W.-C., Chang, L.-C., Li, C.-C., Ma, H.-P., & Lee, C.-C.
- Affiliations
- National Tsing Hua University (TW)
- National Taiwan University of Arts (TW)
- Venue
- 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 292-298
- ISSN
- 2156-8103 (ISSN-L)
- OpenAlex
- W2785952417
- Cited by
- 57
- References
- 29
- Citations by year
- 2026: 4 · 2025: 10 · 2024: 13 · 2023: 5 · 2022: 6 · 2021: 9 · 2020: 1 · 2019: 5 · 2018: 2 · 2017: 2
- FWCI
- 3.14
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Affective Computing and Intelligent Interaction and workshops (publishedVersion)
- Funders
- National Taiwan University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Linguistics and Language › Linguistic Studies and Language Acquisition
- Keywords
- dyadic interaction, spontaneous conversation, emotion annotation, electrocardiogram, human-machine interaction, emotional communication
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: NNIME: The NTHU-NTUA Chinese interactive multimodal emotion corpus. https://app.cochat.ai/papers/10.1109/acii.2017.8273615
- The paper
- Chou, H.-C., Lin, W.-C., Chang, L.-C., Li, C.-C., Ma, H.-P., & Lee, C.-C. (2017). NNIME: The NTHU-NTUA Chinese interactive multimodal emotion corpus. 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII). https://doi.org/10.1109/acii.2017.8273615
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.