The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Memory Fusion Network for Multi-view Sequential Learning
Open accessAbstract
Multi-view sequential learning is a fundamental problem in machine learning dealing with multi-view sequences. In a multi-view sequence, there exists two forms of interactions between different views: view-specific interactions and cross-view interactions. In this paper, we present a new neural architecture for multi-view sequential learning called the Memory Fusion Network (MFN) that explicitly accounts for both interactions in a neural architecture and continuously models them through time. The first component of the MFN is called the System of LSTMs, where view-specific interactions are learned in isolation through assigning an LSTM function to each view. The cross-view interactions are then identified using a special attention mechanism called the Delta-memory Attention Network (DMAN) and summarized through time with a Multi-view Gated Memory. Through extensive experimentation, MFN is compared to various proposed approaches for multi-view sequential learning on multiple publicly available benchmark datasets. MFN outperforms all the multi-view approaches. Furthermore, MFN outperforms all current state-of-the-art models, setting new state-of-the-art results for all three multi-view datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
An argument for basic emotions
Ekman, P. (1992) · Cognition and Emotion · doi:10.1080/02699939208411068 · Cited by 9,646
The random subspace method for constructing decision forests
Ho, T. K. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.709601 · Cited by 7,005
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
Statistical Inference for Probabilistic Functions of Finite State Markov Chains
Baum, L. E., & Petrie, T. (1966) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177699147 · Cited by 2,775
Tensor Fusion Network for Multimodal Sentiment Analysis
Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1115 · Cited by 2,102
Self-citationMultiple kernel learning, conic duality, and the SMO algorithm
Bach, F. R., Lanckriet, G. R. G., & Jordan, M. I. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015424 · Cited by 1,453
Fast Human Detection Using a Cascade of Histograms of Oriented Gradients
Zhu, Q., Yeh, M.-C., Cheng, K.-T., & Avidan, S. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.119 · Cited by 1,399
A Survey on Multi-view LearningPreprint
Xu, C., Tao, D., & Xu, C. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1304.5634 · Cited by 1,252
Context-Dependent Sentiment Analysis in User-Generated Videos
Poria, S., Cambria, E., Hazarika, D., Majumder, N., Zadeh, A., & Morency, L.-P. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1081 · Cited by 939
Self-citationMultimodal Sentiment Intensity Analysis in Videos: Facial Gestures and Verbal Messages
Zadeh, A., Zellers, R., Pincus, E., & Morency, L.-P. (2016) · IEEE Intelligent Systems · doi:10.1109/mis.2016.94 · Cited by 852
Self-citationCOVAREP — A collaborative voice analysis repository for speech technologies
Degottex, G., Kane, J., Drugman, T., Raitio, T., & Scherer, S. (2014) · 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2014.6853739 · Cited by 695
Speaker identification on the SCOTUS corpus
Yuan, J., & Liberman, M. (2008) · The Journal of the Acoustical Society of America · doi:10.1121/1.2935783 · Cited by 610
Poria, S., Cambria, E., & Gelbukh, A. F. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1303 · Cited by 559
Self-citationHidden Conditional Random Fields
Quattoni, A., Wang, S., Morency, L., Collins, M., & Darrell, T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1124 · Cited by 489
Self-citationTowards multimodal sentiment analysis: harvesting opinions from the web
Morency, L.-P., Mihalcea, R., & Doshi, P. (2011) · Proceedings of the 13th international conference on multimodal interfaces · doi:10.1145/2070481.2070509 · Cited by 470
Self-citationVocal quality factors: Analysis, synthesis, and perception
Childers, D. G., & Lee, C. K. (1991) · The Journal of the Acoustical Society of America · doi:10.1121/1.402044 · Cited by 461
Automatic, Dimensional and Continuous Emotion Recognition
Güneş, H., & Pantić, M. (2010) · International Journal of Synthetic Emotions · doi:10.4018/jse.2010101605 · Cited by 442
Youtube movie reviews: Sentiment analysis in an audio-visual context
Wollmer, M., Weninger, F., Knaup, T., Schuller, B., Sun, C., Sagae, K., & Morency, L.-P. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.34 · Cited by 403
Self-citationLatent-Dynamic Discriminative Models for Continuous Gesture Recognition
Morency, L., Quattoni, A., & Darrell, T. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383299 · Cited by 376
Self-citationNormalized amplitude quotient for parametrization of the glottal flow.
Alku, P., Bäckström, T., & Vilkman, E. (2002) · The Journal of the Acoustical Society of America · doi:10.1121/1.1490365 · Cited by 287
(2009) · Springer eBooks · doi:10.1007/978-0-387-73003-5_2171 · Cited by 269
Deep multimodal fusion for persuasiveness prediction
Nojavanasghari, B., Gopinath, D., Koushik, J., Baltrušaitis, T., & Morency, L. (2016) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2993148.2993176 · Cited by 247
Self-citationDetection of Glottal Closure Instants From Speech Signals: A Quantitative Review
Thomas Drugman, A., Thomas, M. R., Guðnason, J., Naylor, P. A., & Dutoit, T. (2011) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2011.2170835 · Cited by 236
(2013) · Cited by 236
Joint robust voicing detection and pitch estimation based on residual harmonics
Drugman, T., & Alwan, A. A. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-519 · Cited by 228
Select-additive learning: Improving generalization in multimodal sentiment analysis
Wang, H., Meghawat, A., Morency, L., & Xing, E. P. (2017) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2017.8019301 · Cited by 224
Self-citationExtending Long Short-Term Memory for Multi-View Structured Learning
Rajagopalan, S. S., Morency, L., Baltrušaitis, T., & Goecke, R. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_21 · Cited by 139
Self-citationComputational Analysis of Persuasiveness in Social Multimedia
Park, S., Shim, H. S., Chatterjee, M., Sagae, K., & Morency, L. (2014) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2663204.2663260 · Cited by 127
Self-citationAction Recognition by Hierarchical Sequence Summarization
Song, Y., Morency, L., & Davis, R. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.457 · Cited by 97
Self-citationWavelet Maxima Dispersion for Breathy to Tense Voice Discrimination
Kane, J., & Gobl, C. (2013) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2013.2245653 · Cited by 85
Multi-view latent variable discriminative models for action recognition
Song, Y., Morency, L., & Davis, R. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247918 · Cited by 81
Self-citationGlottal wave analysis with pitch synchronous iterative adaptive inverse filtering
Alku, P. (1991) · European Conference on Speech Communication and Technology (Eurospeech) · doi:10.21437/eurospeech.1991-257 · Cited by 43
Wang, H., Meghawat, A., Morency, L., & Xing, E. P. (2016) · arXiv (Cornell University) · Cited by 29
Self-citationAutomatic, Dimensional and Continuous Emotion Recognition
Güneş, H., & Pantić, M. (2012) · IGI Global eBooks · doi:10.4018/978-1-4666-1595-3.ch005 · Cited by 18
Select-Additive Learning: Improving Generalization in Multimodal Sentiment AnalysisPreprint
Wang, H., Meghawat, A., Morency, L., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.05244 · Cited by 17
Self-citation
Research works printed but not linked: 3
- Alku, P.; Strik, H.; and Vilkman, E. 1997. Parabolic spectral parametera new method for quantification of the glottal flow. Speech Communication 22(1):67-79.
- Degottex, G.; Kane, J.; Drugman, T.; Raitio, T.; and Scherer, S. 2014. Covarepa collaborative voice analysis repository for speech technologies. In Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on , 960-964. IEEE.
- Perez-Rosas, V.; Mihalcea, R.; and Morency, L.-P. 2013. UtteranceLevel Multimodal Sentiment Analysis. In Association for Computational Linguistics (ACL) .
Cited by
Sorted by citations
Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
Bagher Zadeh, A., Liang, P. P., Poria, S., Cambria, E., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1208 · Cited by 1,433
Self-citationEfficient Low-rank Multimodal Fusion With Modality-Specific Factors
Liu, Z., Shen, Y., Lakshminarasimhan, V. B., Liang, P. P., Bagher Zadeh, A., & Morency, L.-P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1209 · Cited by 1,170
Self-citationMISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis
Hazarika, D., Zimmermann, R., & Poria, S. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413678 · Cited by 1,073
Self-citationYu, W., Xu, H., Yuan, Z., & Wu, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17289 · Cited by 816
Gandhi, A., Adhvaryu, K., Poria, S., Cambria, E., & Hussain, A. (2023) · Information Fusion · doi:10.1016/j.inffus.2022.09.025 · Cited by 783
Self-citation
Metadata
- Authors
- Zadeh, A., Liang, P. P., Mazumder, N., Poria, S., Cambria, E., & Morency, L.-P.
- Affiliations
- Carnegie Mellon University (US)
- Instituto Politécnico Nacional (MX)
- Nanyang Technological University (SG)
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2018-04-27
- Volume
- 32
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2787581402
- Cited by
- 855
- References
- 36
- Citations by year
- 2026: 100 · 2025: 169 · 2024: 163 · 2023: 125 · 2022: 118 · 2021: 87 · 2020: 48 · 2019: 29 · 2018: 16
- FWCI
- 13.57
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- LSTM, attention mechanism, temporal modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
