The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Multi-attention Recurrent Network for Human Communication Comprehension
Open accessAbstract
Human face-to-face communication is a complex multimodal signal. We use words (language modality), gestures (vision modality) and changes in tone (acoustic modality) to convey our intentions. Humans easily process and understand face-to-face communication, however, comprehending this form of communication remains a significant challenge for Artificial Intelligence (AI). AI must understand each modality and the interactions between them that shape the communication. In this paper, we present a novel neural architecture for understanding human communication called the Multi-attention Recurrent Network (MARN). The main strength of our model comes from discovering interactions between modalities through time using a neural component called the Multi-attention Block (MAB) and storing them in the hybrid memory of a recurrent component called the Long-short Term Hybrid Memory (LSTHM). We perform extensive comparisons on six publicly available datasets for multimodal sentiment analysis, speaker trait recognition and emotion recognition. MARN shows state- of-the-art results performance in all the datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
Statistical Inference for Probabilistic Functions of Finite State Markov Chains
Baum, L. E., & Petrie, T. (1966) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177699147 · Cited by 2,775
Tensor Fusion Network for Multimodal Sentiment Analysis
Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1115 · Cited by 2,102
Self-citationA Survey on Multi-view Learning
Xu, C., Tao, D., & Xu, C. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1304.5634 · Cited by 942
Context-Dependent Sentiment Analysis in User-Generated Videos
Poria, S., Cambria, E., Hazarika, D., Majumder, N., Zadeh, A., & Morency, L.-P. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1081 · Cited by 939
Self-citationMultimodal Sentiment Intensity Analysis in Videos: Facial Gestures and Verbal Messages
Zadeh, A., Zellers, R., Pincus, E., & Morency, L.-P. (2016) · IEEE Intelligent Systems · doi:10.1109/mis.2016.94 · Cited by 852
Self-citationConvolutional MKL Based Multimodal Emotion Recognition and Sentiment Analysis
Poria, S., Chaturvedi, I., Cambria, E., & Hussain, A. (2016) · 2016 IEEE 16th International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2016.0055 · Cited by 595
Self-citationSpeaker identification on the SCOTUS corpus
Yuan, J., & Liberman, M. (2008) · The Journal of the Acoustical Society of America · doi:10.1121/1.2935783 · Cited by 594
Poria, S., Cambria, E., & Gelbukh, A. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1303 · Cited by 533
Self-citationNeural Synchronization during Face-to-Face Communication
Jiang, J., Dai, B., Peng, D., Zhu, C., Liu, L., & Lu, C. (2012) · The Journal of Neuroscience · doi:10.1523/jneurosci.2926-12.2012 · Cited by 489
Hidden Conditional Random Fields
Quattoni, A., Wang, S., Morency, L.-P., Collins, M., & Darrell, T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1124 · Cited by 488
Self-citationTowards multimodal sentiment analysis: harvesting opinions from the web
Morency, L.-P., Mihalcea, R., & Doshi, P. (2011) · Proceedings of the 13th international conference on multimodal interfaces · doi:10.1145/2070481.2070509 · Cited by 470
Self-citationYoutube movie reviews: Sentiment analysis in an audio-visual context
Wollmer, M., Weninger, F., Knaup, T., Schuller, B., Sun, C., Sagae, K., & Morency, L.-P. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.34 · Cited by 403
Self-citationLatent-Dynamic Discriminative Models for Continuous Gesture Recognition
Morency, L.-P., Quattoni, A., & Darrell, T. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383299 · Cited by 374
Self-citationDeep multimodal fusion for persuasiveness prediction
Nojavanasghari, B., Gopinath, D., Koushik, J., Baltrušaitis, T., & Morency, L.-P. (2016) · Proceedings of the 18th ACM International Conference on Multimodal Interaction · doi:10.1145/2993148.2993176 · Cited by 237
Self-citation(2013) · Cited by 236
Sergent, J., & Signoret, J.-L. (1992) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.1992.0007 · Cited by 186
Extending Long Short-Term Memory for Multi-View Structured Learning
Rajagopalan, S. S., Morency, L., Baltrušaitis, T., & Goecke, R. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_21 · Cited by 139
Self-citationAction Recognition by Hierarchical Sequence Summarization
Song, Y., Morency, L., & Davis, R. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.457 · Cited by 97
Self-citationMulti-view latent variable discriminative models for action recognition
Song, Y., Morency, L., & Davis, R. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247918 · Cited by 83
Self-citationImaging first impressions: Distinct neural processing of verbal and nonverbal social information
Kuzmanovic, B., Bente, G., von Cramon, D. Y., Schilbach, L., Tittgemeyer, M., & Vogeley, K. T. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.12.046 · Cited by 54
Multimodal Sentiment Analysis Using Deep Neural Networks
Abburi, H., Prasath, R., Shrivastava, M., & Gangashetty, S. V. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-58130-9_6 · Cited by 37
Wörtwein, T., & Scherer, S. J. (2017) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2017.8273573 · Cited by 33
Wang, H., Meghawat, A., Morency, L., & Xing, E. P. (2016) · arXiv (Cornell University) · Cited by 32
Self-citation
Cited by
Multimodal Transformer for Unaligned Multimodal Language Sequences
Tsai, Y.-H. H., Bai, S., Liang, P. P., Kolter, J. Z., Morency, L.-P., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1656 · Cited by 2,259
Self-citationMultimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Bagher Zadeh, A., Liang, P. P., Poria, S., Cambria, E., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1208 · Cited by 1,433
Self-citationEfficient Low-rank Multimodal Fusion With Modality-Specific Factors
Liu, Z., Shen, Y., Lakshminarasimhan, V. B., Liang, P. P., Bagher Zadeh, A., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1209 · Cited by 1,170
Self-citationMISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis
Hazarika, D., Zimmermann, R., & Poria, S. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413678 · Cited by 1,073
Self-citationDialogueRNN: An Attentive RNN for Emotion Detection in Conversations
Majumder, N., Poria, S., Hazarika, D., Mihalcea, R., Gelbukh, A., & Cambria, E. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016818 · Cited by 860
Self-citation
Metadata
- Authors
- Zadeh, A., Liang, P. P., Poria, S., Vij, P., Cambria, E., & Morency, L.-P.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2018-04-27
- Volume
- 32
- Issue
- 1
- ISSN
- 2374-3468
- 2159-5399
- OpenAlex
- W2963710346
- Cited by
- 505
- References
- 28
- FWCI
- 23.55
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.