The DOI matches its Crossref record; 120 of the 120 references it cites were found in a registry or an index.
A Survey of Multi-View Representation Learning
Open accessAbstract
Recently, multi-view representation learning has become a rapidly growing direction in machine learning and data mining areas. This paper introduces two categories for multi-view representation learning: multi-view representation alignment and multi-view representation fusion. Consequently, we first review the representative methods and theories of multi-view representation learning based on the perspective of alignment, such as correlation-based alignment. Representative examples are canonical correlation analysis (CCA) and its several extensions. Then, from the perspective of representation fusion, we investigate the advancement of multi-view representation learning that ranges from generative methods including multi-modal topic learning, multi-view sparse coding, and multi-view latent space Markov networks, to neural network-based methods including multi-modal autoencoders, multi-view convolutional neural networks, and multi-modal recurrent neural networks. Further, we also investigate several important applications of multi-view representation learning. Overall, this survey aims to provide an insightful overview of theoretical foundation and state-of-the-art developments in the field of multi-view representation learning and to help researchers find the most appropriate tools for particular applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Schölkopf, B., & Smola, A. J. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/4175.001.0001 · Cited by 11,703
PLS-regression: a basic tool of chemometrics
Wold, S., Sjöström, M., & Eriksson, L. (2001) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/s0169-7439(01)00155-1 · Cited by 10,084
Research works printed but not linked: 66
- sutskever, 2014, Sequence to sequence learning with neural networks, Proc Conf Neural Inf Process Syst, 3104
- 10.1145/1873951.1874183 · doi:10.1145/1873951.1874183
- frome, 2013, Devise: A deep visual-semantic embedding model, Proc Advances Neural Inf Process Syst, 2121
- krizhevsky, 2012, Imagenet classification with deep convolutional neural networks, Proc 25th Int Conf Neural Inf Process Syst, 1097
- gunasekar, 2015, Consistent collective matrix completion under joint low rank structure, Proc Conf Artif Intell Statist, 306
- 10.3115/v1/D14-1005 · DOI · Paper page
- collell, 2017, Imagined visual representations as multimodal embeddings, Proc 31st AAAI Conf Artif Intell, 4378
- wang, 2015, On deep multi-view representation learning, Proc Int Conf Int Conf Mach Learn, 1083
- mcwilliams, 2013, Correlated random features for fast semi-supervised learning, Proc 20th Int Conf Neural Inf Process Syst, 440
- xing, 2002, Distance metric learning with application to clustering with side-information, Proc Advances Neural Inf Process Syst, 505
- hinton, 2009, Replicated softmax: An undirected topic model, Proc Advances Neural Inf Process Syst, 1607
- welling, 2004, Exponential family harmoniums with an application to information retrieval, Proc 17th Int Conf Neural Inf Process Syst, 1481
- xie, 2013, Multi-modal distance metric learning, Proc 23rd Int Joint Conf Artif Intell, 1806
- zhuang, 2013, Supervised coupled dictionary learning with group structures for multi-modal retrieval, Proc AAAI Conf Artif Intell, 1070
- 10.2307/2334380 · DOI
- zhang, 2011, Multi-label output codes using canonical correlation analysis, Proc 14th Int Conf Artificial Intell, 873
- dhillon, 2012, Two Step CCA: A new spectral method for estimating vector models of words, Proc 29th Int Conf Mach Learn
- dhillon, 2011, Multi-view learning of word embeddings via CCA, Proc Advances Neural Inf Process Syst, 199
- salakhutdinov, 2009, Deep boltzmann machines, Proc Int Conf Artif Intell Statist, 1
- andrew, 2013, Deep canonical correlation analysis, Mach Learn Res, 1247
- hu, 2013, Multimodal DBN for predicting high-quality answers in CQA portals, Proc Annual Meeting of the Assoc Computational Linguistics, 843
- cohn, 2000, The missing link - A probabilistic model of document content and hypertext connectivity, Proc Advances Neural Inf Process Syst, 430
- avron, 2013, Efficient dimensionality reduction for canonical correlation analysis, Proc 30th Int Conf Mach Learn, 28, 347
- 10.1145/1390156.1390285 · doi:10.1145/1390156.1390285
- chen, 2012, Structured sparse canonical correlation analysis, Proc Conf Artif Intell Statist, 199
- akaho, 2001, A kernel method for canonical correlation analysis, Proc Int Meeting Psychometric Soc
- 10.1137/130919222 · doi:10.1137/130919222
- lu, 2014, large scale canonical correlation analysis with iterative least squares, Proc 27th Int Conf Neural Inf Process Syst, 91
- belkin, 2006, Manifold regularization: A geometric framework for learning from labeled and unlabeled examples, J Mach Learn Res, 7, 2399
- torres, 2007, Finding musically meaningful words by sparse CCA, Proc Adv Neural Inf Process Syst
- graves, 2014, Towards end-to-end speech recognition with recurrent neural networks, Proc 31st Int Conf Int Conf Mach Learn, 1764
- serban, 2016, Building end-to-end dialogue systems using generative hierarchical neural network models, Proc 13th AAAI Conf Artif Intell, 16, 3776
- wang, 2015, Large-scale approximate kernel canonical correlation analysis, CoRR, abs 1511 4773
- roller, 2013, A multimodal lda model integrating textual, cognitive and visual modalities, Proc Conf Empirical Methods Natural Language Process, 1146
- le, 2013, Fastfood - computing hilbert space expansions in loglinear time, Proc Int Conf Mach Learn, 244
- yang, 2012, Nyström method versus random fourier features: A theoretical and empirical comparison, Proc Advances Neural Inf Process Syst, 485
- williams, 2000, Using the nyström method to speed up kernel machines, Proc 13th Int Conf Neural Inf Process Syst, 682
- lai, 1998, Canonical correlation analysis using artificial neural networks, Proc Eur Symp Artif Neural Netw, 363
- arora, 2012, Kernel CCA for multi-view learning of acoustic features using articulatory measurements, Proc Symp Mach Learning Speech Lang Process, 34
- gretton, 2005, Kernel methods for measuring independence, J Mach Learn Res, 6, 2075
- 10.1145/1871985.1871989 · doi:10.1145/1871985.1871989
- brand, 2002, Incremental singular value decomposition of uncertain data with missing values, Proc Eur Conf Comput Vis, 707
- fukumizu, 2007, Statistical consistency of kernel canonical correlation analysis, J Mach Learn Res, 8, 361
- 10.1007/s11425-011-4245-2 · doi:10.1007/s11425-011-4245-2
- dong, 2017, A hybrid collaborative filtering model with deep structure for recommender systems, Proc 31st AAAI Conf Artif Intell, 1309
- weiss, 2008, Spectral hashing, Proc Advances Neural Inf Process Syst, 1753
- huang, 2013, Learning deep structured semantic models for web search using clickthrough data, Proc ACM Int Conf Inf Knowl Manag, 2333
- kiros, 2014, Multimodal neural language models, Proc 31st Int Conf Mach Learn, 14, 595
- dean, 2012, Large scale distributed deep networks, Proc Advances Neural Inf Process Syst, 1223
- xu, 2015, Jointly modeling deep video and compositional text to bridge vision and language in a unified framework, Proc AAAI Conf Artif Intell, 2346
- blei, 2003, Latent dirichlet allocation, J Mach Learn Res, 3, 993
- wang, 2009, Simultaneous image classification and annotation, Proc IEEE Conf Comput Vis Pattern Recognit, 1903
- bach, 2002, Kernel independent component analysis, J Mach Learn Res, 3, 1
- bishop, 2006, Pattern Recognition and Machine Learning
- 10.1109/Allerton.2012.6483308 · doi:10.1109/allerton.2012.6483308
- rosipal, 2006, Overview and recent advances in partial least squares, 34
- wold, 1982, Soft modeling: the basic design and some extensions, Systems under Indirect Observation, 2, 589
- wang, 2012, Kernel cross-modal factor analysis for information fusion with application to bimodal emotion recognition, IEEE Trans Multimedia, 14, 3
- jia, 2010, Factorized latent spaces with structured sparsity, Proc Advances in Neural Inf Process Syst, 982
- xing, 2005, Mining associated text and images with dual-wing harmoniums, Proc 21st Conf Uncertainty Artif Intell, 633
- chen, 2010, Predictive subspace learning for multi-view data: A large margin approach, Proc 23rd Int Conf Neural Inf Process Syst, 361
- srivastava, 2012, Multimodal learning with deep boltzmann machines, Proc Int Conf Neural Inf Process, 2231
- ngiam, 2011, Multimodal deep learning, Proc Int Conf Mach Learn, 689
- collins, 2002, New ranking algorithms for parsing and tagging: Kernels over discrete structures, and the voted perceptron, Proc Annual Meeting of the Assoc Computational Linguistics, 263
- kumar, 2011, Learning hash functions for cross-view similarity search, Proc 20nd Int Joint Conf Artif Intell, 1360
- zhen, 2012, Co-regularized hashing for multimodal data, Proc Advances Neural Inf Process Syst, 1385
Cited by
Tian, Y., Krishnan, D., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_45 · Cited by 1,997
A survey on deep learning for multimodal data fusion
Gao, J., Li, P., Chen, Z., & Zhang, J. (2020) · Neural Computation · doi:10.1162/neco_a_01273 · Cited by 821
Robust Multi-View Clustering With Incomplete Information
Yang, M., Li, Y., Hu, P., Bai, J., Lv, J., & Peng, X. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3155499 · Cited by 332
Self-citationDeep multi-view learning methods: A review
Yan, X., Hu, S., Mao, Y., Ye, Y., & Yu, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.03.090 · Cited by 322
A survey of multimodal deep generative models
Suzuki, M., & Matsuo, Y. (2022) · Advanced Robotics · doi:10.1080/01691864.2022.2035253 · Cited by 109
“The first purpose is sometimes referred to as multimodal representation learning [50], and besides generative approaches, there are studies based on discriminative approaches such as contrastive learning [1, 97, 100].”
Metadata
- Authors
- Li, Y., Yang, M., & Zhang, Z.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2019-10-01
- Volume
- 31
- Issue
- 10
- Pages
- 1863-1883
- ISSN
- 1041-4347 (ISSN-L)
- 1558-2191
- 2326-3865
- OpenAlex
- W2892816441
- Cited by
- 524
- References
- 120
- FWCI
- 13.96
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- National Natural Science Foundation of China (61702448, 61672456)
- Fundamental Research Funds for the Central Universities (2017QNA5008, 2017FZA5007)
- Zhejiang University
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Survey of Multi-View Representation Learning. https://app.cochat.ai/papers/10.1109/tkde.2018.2872063
- The paper
- Li, Y., Yang, M., & Zhang, Z. (2019). A Survey of Multi-View Representation Learning. IEEE Transactions on Knowledge and Data Engineering. https://doi.org/10.1109/tkde.2018.2872063
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.