Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 524

The DOI matches its Crossref record; 120 of the 120 references it cites were found in a registry or an index.

A Survey of Multi-View Representation Learning

Open access
Li, Yingming, Yang, Ming, Zhang, Zhongfei·2019·IEEE Transactions on Knowledge and Data Engineering

Abstract

Recently, multi-view representation learning has become a rapidly growing direction in machine learning and data mining areas. This paper introduces two categories for multi-view representation learning: multi-view representation alignment and multi-view representation fusion. Consequently, we first review the representative methods and theories of multi-view representation learning based on the perspective of alignment, such as correlation-based alignment. Representative examples are canonical correlation analysis (CCA) and its several extensions. Then, from the perspective of representation fusion, we investigate the advancement of multi-view representation learning that ranges from generative methods including multi-modal topic learning, multi-view sparse coding, and multi-view latent space Markov networks, to neural network-based methods including multi-modal autoencoders, multi-view convolutional neural networks, and multi-modal recurrent neural networks. Further, we also investigate several important applications of multi-view representation learning. Overall, this survey aims to provide an insightful overview of theoretical foundation and state-of-the-art developments in the field of multi-view representation learning and to help researchers find the most appropriate tools for particular applications.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tkde.2018.2872063

Cites

120 of 120 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 66

  1. sutskever, 2014, Sequence to sequence learning with neural networks, Proc Conf Neural Inf Process Syst, 3104
  2. 10.1145/1873951.1874183 · doi:10.1145/1873951.1874183
  3. frome, 2013, Devise: A deep visual-semantic embedding model, Proc Advances Neural Inf Process Syst, 2121
  4. krizhevsky, 2012, Imagenet classification with deep convolutional neural networks, Proc 25th Int Conf Neural Inf Process Syst, 1097
  5. gunasekar, 2015, Consistent collective matrix completion under joint low rank structure, Proc Conf Artif Intell Statist, 306
  6. 10.3115/v1/D14-1005 · DOI · Paper page
  7. collell, 2017, Imagined visual representations as multimodal embeddings, Proc 31st AAAI Conf Artif Intell, 4378
  8. wang, 2015, On deep multi-view representation learning, Proc Int Conf Int Conf Mach Learn, 1083
  9. mcwilliams, 2013, Correlated random features for fast semi-supervised learning, Proc 20th Int Conf Neural Inf Process Syst, 440
  10. xing, 2002, Distance metric learning with application to clustering with side-information, Proc Advances Neural Inf Process Syst, 505
  11. hinton, 2009, Replicated softmax: An undirected topic model, Proc Advances Neural Inf Process Syst, 1607
  12. welling, 2004, Exponential family harmoniums with an application to information retrieval, Proc 17th Int Conf Neural Inf Process Syst, 1481
  13. xie, 2013, Multi-modal distance metric learning, Proc 23rd Int Joint Conf Artif Intell, 1806
  14. zhuang, 2013, Supervised coupled dictionary learning with group structures for multi-modal retrieval, Proc AAAI Conf Artif Intell, 1070
  15. 10.2307/2334380 · DOI
  16. zhang, 2011, Multi-label output codes using canonical correlation analysis, Proc 14th Int Conf Artificial Intell, 873
  17. dhillon, 2012, Two Step CCA: A new spectral method for estimating vector models of words, Proc 29th Int Conf Mach Learn
  18. dhillon, 2011, Multi-view learning of word embeddings via CCA, Proc Advances Neural Inf Process Syst, 199
  19. salakhutdinov, 2009, Deep boltzmann machines, Proc Int Conf Artif Intell Statist, 1
  20. andrew, 2013, Deep canonical correlation analysis, Mach Learn Res, 1247
  21. hu, 2013, Multimodal DBN for predicting high-quality answers in CQA portals, Proc Annual Meeting of the Assoc Computational Linguistics, 843
  22. cohn, 2000, The missing link - A probabilistic model of document content and hypertext connectivity, Proc Advances Neural Inf Process Syst, 430
  23. avron, 2013, Efficient dimensionality reduction for canonical correlation analysis, Proc 30th Int Conf Mach Learn, 28, 347
  24. 10.1145/1390156.1390285 · doi:10.1145/1390156.1390285
  25. chen, 2012, Structured sparse canonical correlation analysis, Proc Conf Artif Intell Statist, 199
  26. akaho, 2001, A kernel method for canonical correlation analysis, Proc Int Meeting Psychometric Soc
  27. 10.1137/130919222 · doi:10.1137/130919222
  28. lu, 2014, large scale canonical correlation analysis with iterative least squares, Proc 27th Int Conf Neural Inf Process Syst, 91
  29. belkin, 2006, Manifold regularization: A geometric framework for learning from labeled and unlabeled examples, J Mach Learn Res, 7, 2399
  30. torres, 2007, Finding musically meaningful words by sparse CCA, Proc Adv Neural Inf Process Syst
  31. graves, 2014, Towards end-to-end speech recognition with recurrent neural networks, Proc 31st Int Conf Int Conf Mach Learn, 1764
  32. serban, 2016, Building end-to-end dialogue systems using generative hierarchical neural network models, Proc 13th AAAI Conf Artif Intell, 16, 3776
  33. wang, 2015, Large-scale approximate kernel canonical correlation analysis, CoRR, abs 1511 4773
  34. roller, 2013, A multimodal lda model integrating textual, cognitive and visual modalities, Proc Conf Empirical Methods Natural Language Process, 1146
  35. le, 2013, Fastfood - computing hilbert space expansions in loglinear time, Proc Int Conf Mach Learn, 244
  36. yang, 2012, Nyström method versus random fourier features: A theoretical and empirical comparison, Proc Advances Neural Inf Process Syst, 485
  37. williams, 2000, Using the nyström method to speed up kernel machines, Proc 13th Int Conf Neural Inf Process Syst, 682
  38. lai, 1998, Canonical correlation analysis using artificial neural networks, Proc Eur Symp Artif Neural Netw, 363
  39. arora, 2012, Kernel CCA for multi-view learning of acoustic features using articulatory measurements, Proc Symp Mach Learning Speech Lang Process, 34
  40. gretton, 2005, Kernel methods for measuring independence, J Mach Learn Res, 6, 2075
  41. 10.1145/1871985.1871989 · doi:10.1145/1871985.1871989
  42. brand, 2002, Incremental singular value decomposition of uncertain data with missing values, Proc Eur Conf Comput Vis, 707
  43. fukumizu, 2007, Statistical consistency of kernel canonical correlation analysis, J Mach Learn Res, 8, 361
  44. 10.1007/s11425-011-4245-2 · doi:10.1007/s11425-011-4245-2
  45. dong, 2017, A hybrid collaborative filtering model with deep structure for recommender systems, Proc 31st AAAI Conf Artif Intell, 1309
  46. weiss, 2008, Spectral hashing, Proc Advances Neural Inf Process Syst, 1753
  47. huang, 2013, Learning deep structured semantic models for web search using clickthrough data, Proc ACM Int Conf Inf Knowl Manag, 2333
  48. kiros, 2014, Multimodal neural language models, Proc 31st Int Conf Mach Learn, 14, 595
  49. dean, 2012, Large scale distributed deep networks, Proc Advances Neural Inf Process Syst, 1223
  50. xu, 2015, Jointly modeling deep video and compositional text to bridge vision and language in a unified framework, Proc AAAI Conf Artif Intell, 2346
  51. blei, 2003, Latent dirichlet allocation, J Mach Learn Res, 3, 993
  52. wang, 2009, Simultaneous image classification and annotation, Proc IEEE Conf Comput Vis Pattern Recognit, 1903
  53. bach, 2002, Kernel independent component analysis, J Mach Learn Res, 3, 1
  54. bishop, 2006, Pattern Recognition and Machine Learning
  55. 10.1109/Allerton.2012.6483308 · doi:10.1109/allerton.2012.6483308
  56. rosipal, 2006, Overview and recent advances in partial least squares, 34
  57. wold, 1982, Soft modeling: the basic design and some extensions, Systems under Indirect Observation, 2, 589
  58. wang, 2012, Kernel cross-modal factor analysis for information fusion with application to bimodal emotion recognition, IEEE Trans Multimedia, 14, 3
  59. jia, 2010, Factorized latent spaces with structured sparsity, Proc Advances in Neural Inf Process Syst, 982
  60. xing, 2005, Mining associated text and images with dual-wing harmoniums, Proc 21st Conf Uncertainty Artif Intell, 633
  61. chen, 2010, Predictive subspace learning for multi-view data: A large margin approach, Proc 23rd Int Conf Neural Inf Process Syst, 361
  62. srivastava, 2012, Multimodal learning with deep boltzmann machines, Proc Int Conf Neural Inf Process, 2231
  63. ngiam, 2011, Multimodal deep learning, Proc Int Conf Mach Learn, 689
  64. collins, 2002, New ranking algorithms for parsing and tagging: Kernels over discrete structures, and the voted perceptron, Proc Annual Meeting of the Assoc Computational Linguistics, 263
  65. kumar, 2011, Learning hash functions for cross-view similarity search, Proc 20nd Int Joint Conf Artif Intell, 1360
  66. zhen, 2012, Co-regularized hashing for multimodal data, Proc Advances Neural Inf Process Syst, 1385

Cited by

See all 524

Metadata

Authors
Li, Y., Yang, M., & Zhang, Z.
Venue
IEEE Transactions on Knowledge and Data Engineering
Published
2019-10-01
Volume
31
Issue
10
Pages
1863-1883
ISSN
  • 1041-4347 (ISSN-L)
  • 1558-2191
  • 2326-3865
OpenAlex
W2892816441
Cited by
524
References
120
FWCI
13.96
Type
journal article
Language
en
Open access
green
Funders
  • National Natural Science Foundation of China (61702448, 61672456)
  • Fundamental Research Funds for the Central Universities (2017QNA5008, 2017FZA5007)
  • Zhejiang University

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.