The DOI matches its Crossref record; 120 of the 120 references it cites were found in a registry or an index.
A Survey of Multi-View Representation Learning
Open accessAbstract
Recently, multi-view representation learning has become a rapidly growing direction in machine learning and data mining areas. This paper introduces two categories for multi-view representation learning: multi-view representation alignment and multi-view representation fusion. Consequently, we first review the representative methods and theories of multi-view representation learning based on the perspective of alignment, such as correlation-based alignment. Representative examples are canonical correlation analysis (CCA) and its several extensions. Then, from the perspective of representation fusion, we investigate the advancement of multi-view representation learning that ranges from generative methods including multi-modal topic learning, multi-view sparse coding, and multi-view latent space Markov networks, to neural network-based methods including multi-modal autoencoders, multi-view convolutional neural networks, and multi-modal recurrent neural networks. Further, we also investigate several important applications of multi-view representation learning. Overall, this survey aims to provide an insightful overview of theoretical foundation and state-of-the-art developments in the field of multi-view representation learning and to help researchers find the most appropriate tools for particular applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Schölkopf, B., & Smola, A. J. (2001) · The MIT Press eBooks · doi:10.7551/mitpress/4175.001.0001 · Cited by 11,703
PLS-regression: a basic tool of chemometrics
Wold, S., Sjöström, M., & Eriksson, L. (2001) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/s0169-7439(01)00155-1 · Cited by 10,084
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Extracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
Combining labeled and unlabeled data with co-training
Blum, A., & Mitchell, T. (1998) · Proceedings of the eleventh annual conference on Computational learning theory · doi:10.1145/279943.279962 · Cited by 5,495
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 1×
“Undirected graphical models, also called Markov random fields that have many special cases, including the exponential family Harmonium [125] and restricted Boltzmann machine [126].”
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Multi-view Convolutional Neural Networks for 3D Shape Recognition
Su, H., Maji, S., Kalogerakis, E., & Learned-Miller, E. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.114 · Cited by 3,480
Canonical Correlation Analysis: An Overview with Application to Learning Methods
Hardoon, D. R., Szedmak, S., & Shawe-Taylor, J. (2004) · Neural Computation · doi:10.1162/0899766042321814 · Cited by 3,202
Partial least squares for discrimination
Barker, M., & Rayens, W. (2003) · Journal of Chemometrics · doi:10.1002/cem.785 · Cited by 2,841
Learning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102363 · Cited by 2,836
Convolutional Two-Stream Network Fusion for Video Action Recognition
Feichtenhofer, C., Pinz, A., & Zisserman, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.213 · Cited by 2,684
A Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1044 · Cited by 2,656
Convolutional Neural Networks for Speech Recognition
Abdel-Hamid, O., Mohamed, A.-R., Jiang, H., Deng, L., Penn, G., & Yu, D. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2339736 · Cited by 2,262
Beyond short snippets: Deep networks for video classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299101 · Cited by 2,062
Bleiholder, J., & Naumann, F. (2009) · ACM Computing Surveys · doi:10.1145/1456650.1456651 · Cited by 1,838
Collaborative Deep Learning for Recommender Systems
Wang, H., Wang, N., & Yeung, D.-Y. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783273 · Cited by 1,657
Relations Between Two Sets of Variates
Hotelling, H. (1936) · Biometrika · doi:10.2307/2333955 · Cited by 1,645
Witten, D. M., Tibshirani, R., & Hastie, T. (2009) · Biostatistics · doi:10.1093/biostatistics/kxp008 · Cited by 1,535
Collaborative topic modeling for recommending scientific articles
Wang, C., & Blei, D. M. (2011) · Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2020408.2020480 · Cited by 1,527
Incorporating Copying Mechanism in Sequence-to-Sequence Learning
Gu, J., Lu, Z., Li, H., & Li, V. O. K. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1154 · Cited by 1,446
Vinyals, O., & Le, Q. (2015) · arXiv · doi:10.48550/arxiv.1506.05869 · Cited by 1,400
Sequence to Sequence -- Video to Text
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R., Darrell, T., & Saenko, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.515 · Cited by 1,352
A new approach to cross-modal multimedia retrieval
Rasiwasia, N., Costa Pereira, J., Coviello, E., Doyle, G., Lanckriet, G. R. G., Levy, R., & Vasconcelos, N. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1873987 · Cited by 1,334
Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics
Hodosh, M., Young, P., & Hockenmaier, J. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3994 · Cited by 1,314
Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects
Lahat, D., Adali, T., & Jutten, C. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2460697 · Cited by 1,296
From captions to visual concepts and back
Fang, H., Gupta, S., Iandola, F., Srivastava, R. K., Deng, L., Dollar, P., Gao, J., He, X., Mitchell, M., Platt, J. C., et al. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298754 · Cited by 1,287
Multimodal fusion for multimedia analysis: a survey
Atrey, P. K., Hossain, M. A., El Saddik, A., & Kankanhalli, M. S. (2010) · Multimedia Systems · doi:10.1007/s00530-010-0182-0 · Cited by 1,279
An improved deep learning architecture for person re-identification
Ahmed, E., Jones, M., & Marks, T. K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299016 · Cited by 1,248
Relational learning via collective matrix factorization
Singh, A. P., & Gordon, G. J. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401969 · Cited by 1,204
Large Margin Rank Boundaries for Ordinal Regression
Herbrich, R., Graepel, T., & Obermayer, K. (2000) · Advances in Large-Margin Classifiers · doi:10.7551/mitpress/1113.003.0010 · Cited by 1,074
Iterative quantization: A procrustean approach to learning binary codes
Gong, Y., & Lazebnik, S. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995432 · Cited by 1,038
Multimodal Distributional Semantics
Bruni, E., Tran, N. K., & Baroni, M. (2014) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4135 · Cited by 908
Canonical analysis of several sets of variables
KETTENRING, J. R. (1971) · Biometrika · doi:10.1093/biomet/58.3.433 · Cited by 896
A survey of multi-view machine learning
Sun, S. (2013) · Neural Computing and Applications · doi:10.1007/s00521-013-1362-6 · Cited by 878
Multi-view clustering via canonical correlation analysis
Chaudhuri, K., Kakade, S. M., Livescu, K., & Sridharan, K. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553391 · Cited by 786
A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems
Elkahky, A. M., Song, Y., & He, X. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741667 · Cited by 704
A Direct Formulation for Sparse PCA Using Semidefinite Programming
d'Aspremont, A., El Ghaoui, L., Jordan, M. I., & Lanckriet, G. R. G. (2007) · SIAM Review · doi:10.1137/050645506 · Cited by 678
Inter-media hashing for large-scale retrieval from heterogeneous data sources
Song, J., Yang, Y., Yang, Y., Huang, Z., & Shen, H. T. (2013) · Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2463676.2465274 · Cited by 620
Palangi, H., Deng, L., Shen, Y., Gao, J., He, X., Chen, J., Song, X., & Ward, R. (2016) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2016.2520371 · Cited by 618
Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations
Kim, T.-K., Kittler, J., & Cipolla, R. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1037 · Cited by 590
End-to-End Learning of Action Detection from Frame Glimpses in Videos
Yeung, S., Russakovsky, O., Mori, G., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.293 · Cited by 585
Recurrent Convolutional Network for Video-Based Person Re-identification
McLaughlin, N., Martinez del Rincon, J., & Miller, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.148 · Cited by 564
Cross-modal Retrieval with Correspondence Autoencoder
Feng, F., Wang, X., & Li, R. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654902 · Cited by 560
Click Prediction for Web Image Reranking Using Multimodal Sparse Coding
Yu, J., Rui, Y., & Tao, D. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2311377 · Cited by 550
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics
Gong, Y., Ke, Q., Isard, M., & Lazebnik, S. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-013-0658-4 · Cited by 540
KERNEL AND NONLINEAR CANONICAL CORRELATION ANALYSIS
LAI, P. L., & FYFE, C. (2000) · International Journal of Neural Systems · doi:10.1142/s012906570000034x · Cited by 509
Kanezaki, A., Matsushita, Y., & Nishida, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00526 · Cited by 504
Self-organizing neural network that discovers surfaces in random-dot stereograms
Becker, S., & Hinton, G. E. (1992) · Nature · doi:10.1038/355161a0 · Cited by 495
Human detection using partial least squares analysis
Schwartz, W. R., Kembhavi, A., Harwood, D., & Davis, L. S. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459205 · Cited by 493
Barnard, K., Duygulu, P., Forsyth, D. A., de Freitas, N., Blei, D. M., Jordan, M. I., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2003) · Cited by 459
Data fusion through cross-modality metric learning using similarity-sensitive hashing
Bronstein, M. M., Bronstein, A. M., Michel, F., & Paragios, N. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539928 · Cited by 455
On the Role of Correlation and Abstraction in Cross-Modal Multimedia Retrieval
Costa Pereira, J., Coviello, E., Doyle, G., Rasiwasia, N., Lanckriet, G. R. G., Levy, R., & Vasconcelos, N. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.142 · Cited by 435
Large scale image annotation: learning to rank with joint word-image embeddings
Weston, J., Bengio, S., & Usunier, N. (2010) · Machine Learning · doi:10.1007/s10994-010-5198-3 · Cited by 399
Deep correlation for matching images and text
Yan, F., & Mikolajczyk, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298966 · Cited by 395
Joint Learning of Single-Image and Cross-Image Representations for Person Re-identification
Wang, F., Zuo, W., Lin, L., Zhang, D., & Zhang, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.144 · Cited by 389
Cross-Modal Retrieval With CNN Visual Features: A New Baseline
Wei, Y., Zhao, Y., Lu, C., Wei, S., Liu, L., Zhu, Z., & Yan, S. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2519449 · Cited by 370
Cao, L., & Fei-Fei, L. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408965 · Cited by 360
Blei, D. M., & Jordan, M. I. (2003) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/860458.860460 · Cited by 359
A Discriminative Kernel-Based Approach to Rank Images from Text Queries
Grangier, D., & Bengio, S. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70791 · Cited by 318
Links between perceptrons, MLPs and SVMs
Collobert, R., & Bengio, S. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015415 · Cited by 302
Linear cross-modal hashing for efficient multimedia search
Zhu, X., Huang, Z., Shen, H. T., & Zhao, X. (2013) · Proceedings of the 21st ACM international conference on Multimedia · doi:10.1145/2502081.2502107 · Cited by 282
Combining Language and Vision with a Multimodal Skip-gram Model
Lazaridou, A., Pham, N. T., & Baroni, M. (2015) · Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.3115/v1/n15-1016 · Cited by 280
Correlational spectral clustering
Blaschko, M. B., & Lampert, C. H. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587353 · Cited by 236
Learning Image Embeddings using Convolutional Neural Networks for Improved Multi-Modal Semantics
Kiela, D., & Bottou, L. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1005 · Cited by 224
Socher, R., & Fei-Fei, L. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540112 · Cited by 217
Learning Grounded Meaning Representations with Autoencoders
Silberer, C., & Lapata, M. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1068 · Cited by 217
Multiview Hessian discriminative sparse coding for image annotation
Liu, W., Tao, D., Cheng, J., & Tang, Y. (2014) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2013.03.007 · Cited by 212
Temporal Action Localization with Pyramid of Score Distribution Features
Yuan, J., Ni, B., Yang, X., & Kassim, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.337 · Cited by 193
Multi-view Regression Via Canonical Correlation Analysis
Kakade, S. M., & Foster, D. P. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-72927-3_8 · Cited by 184
Audio-visual deep learning for noise robust speech recognition
Huang, J., & Kingsbury, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639140 · Cited by 179
Waaijenborg, S., Verselewel de Witt Hamer, P. C., & Zwinderman, A. H. (2008) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1329 · Cited by 177
Kidron, E., Schechner, Y. Y., & Elad, M. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.274 · Cited by 173
Discriminative coupled dictionary hashing for fast cross-media retrieval
Zhou, Y., Wu, F. Q., Yang, Y., Tian, Q., Luo, J., & Zhuang, Y. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609563 · Cited by 144
Nonlinear canonical correlation analysis by neural networks
Hsieh, W. W. (2000) · Neural Networks · doi:10.1016/s0893-6080(00)00067-8 · Cited by 143
Mutual information maximization: models of cortical self-organization
Becker, S. (1996) · Network Computation in Neural Systems · doi:10.1080/0954898x.1996.11978653 · Cited by 141
Unsupervised learning of acoustic features via deep canonical correlation analysis
Wang, W., Arora, R., Livescu, K., & Bilmes, J. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178840 · Cited by 140
Joint estimation of age, gender and ethnicity: CCA vs. PLS
Guo, G., & Mu, G. (2013) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2013.6553737 · Cited by 135
Deep Multilingual Correlation for Improved Word Embeddings
Lu, A., Wang, W., Bansal, M., Gimpel, K., & Livescu, K. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1028 · Cited by 134
Ramanishka, V., Das, A., Park, D. H., Venugopalan, S., Hendricks, L. A., Rohrbach, M., & Saenko, K. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2984066 · Cited by 132
Wu, F. Q., Zhou, Y., Yang, Y., Tang, S., Zhang, Y., & Zhuang, Y. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2291214 · Cited by 131
Learning the Relative Importance of Objects from Tagged Images for Retrieval and Cross-Modal Search
Hwang, S. J., & Grauman, K. L. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-011-0494-3 · Cited by 128
A probabilistic framework for relational clustering
Long, B., Zhang, Z. M., & Yu, P. S. (2007) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1281192.1281244 · Cited by 121
Self-citationLearning to rank with (a lot of) word features
Bai, B., Weston, J., Grangier, D., Collobert, R., Sadamasa, K., Qi, Y., Chapelle, O., & Weinberger, K. Q. (2009) · Information Retrieval · doi:10.1007/s10791-009-9117-9 · Cited by 118
Yamanishi, Y., Vert, J., Nakaya, A., & Kanehisa, M. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg1045 · Cited by 118
Randomized Nonlinear Component AnalysisPreprint
López-Paz, D., Sra, S., Smola, A. J., Ghahramani, Z., & Schölkopf, B. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.0119 · Cited by 112
Multimedia content processing through cross-modal association
Li, D., Dimitrova, N., Li, M., & Sethi, I. K. (2003) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/957142.957143 · Cited by 106
A neural implementation of canonical correlation analysis
Lai, P., & Fyfe, C. (1999) · Neural Networks · doi:10.1016/s0893-6080(99)00075-1 · Cited by 103
Multi-modal Auto-Encoders as Joint Estimators for Robotics Scene Understanding
Cadena, C., Dick, A., & D. Reid, I. (2016) · Robotics: Science and Systems XII · doi:10.15607/rss.2016.xii.041 · Cited by 93 · cited 1×
“[160] apply this insight to fuse information available from cameras and depth sensors and reconstruct other missing data for scene understanding problems.”
Sparse Unsupervised Dimensionality Reduction for Multiple View Data
Han, Y., Wu, F. Q., Tao, D., Shao, J., Zhuang, Y., & Jiang, J. (2012) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2012.2202075 · Cited by 84
Multi-view CCA-based acoustic features for phonetic recognition across speakers and domains
Arora, R., & Livescu, K. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639047 · Cited by 78
Convergence analysis of kernel Canonical Correlation Analysis: theory and practice
Hardoon, D. R., & Shawe-Taylor, J. S. (2008) · Machine Learning · doi:10.1007/s10994-008-5085-3 · Cited by 76
Spectral Multimodal Hashing and Its Application to Multimedia Retrieval
Zhen, Y., Gao, Y., Yeung, D., Hongyuan Zha, B., & Li, X. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2392052 · Cited by 74
Stochastic optimization for deep CCA via nonlinear orthogonal iterations
Wang, W., Arora, R., Livescu, K., & Srebro, N. (2015) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2015.7447071 · Cited by 62
Full regularization path for sparse principal component analysis
d’Aspremont, A., Bach, F., & Ghaoui, L. E. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273519 · Cited by 55
Deep Compositional Cross-modal Learning to Rank via Local-Global Alignment
Jiang, X., Wu, F. Q., Li, X., Zhao, Z., Lü, W., Tang, S., & Zhuang, Y. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806240 · Cited by 51
Notice of Removal: MDL-CW: A Multimodal Deep Learning Framework with CrossWeights
Rastegar, S., Baghshah, M. S., Rabiee, H. R., & Shojaee, S. M. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.285 · Cited by 49
Semi-supervised kernel canonical correlation analysis with application to human fMRI
Blaschko, M. B., Shelton, J. A., Bartels, A., Lampert, C. H., & Gretton, A. (2011) · Pattern Recognition Letters · doi:10.1016/j.patrec.2011.02.011 · Cited by 46
Learning of Multimodal Representations With Random Walks on the Click Graph
Wu, F. Q., Lu, X., Song, J., Yan, S., Zhang, Z. M., Rui, Y., & Zhuang, Y. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2507401 · Cited by 46
Self-citationMulti-source deep learning for information trustworthiness estimation
Ge, L., Gao, J., Xiao-yi, L., & Zhang, A. (2013) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2487575.2487612 · Cited by 44
A scalable two-stage approach for a class of dimensionality reduction techniques
Sun, L., Ceran, B., & Ye, J. (2010) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1835804.1835846 · Cited by 43
Mutlimodal Learning with Deep Boltzmann Machine for Emotion Prediction in User Generated Videos
Pang, L., & Ngo, C. (2015) · International Conference on Multimedia Retrieval · doi:10.1145/2671188.2749400 · Cited by 34
Discriminant Learning Through Multiple Principal Angles for Visual Recognition
Su, Y., Fu, Y., Gao, X., & Tian, Q. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2169972 · Cited by 28
Correlation Clustering for Learning Mixtures of Canonical Correlation Models
Fern, X. Z., Brodley, C. E., & Friedl, M. A. (2005) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972757.39 · Cited by 19
Robust Multimodal Dictionary Learning
Cao, T., Jojic, V., Modla, S., Powell, D., Czymmek, K. J., & Niethammer, M. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40811-3_33 · Cited by 18
Learning Cross Space Mapping via DNN Using Large Scale Click-Through Logs
Yu, W., Yang, K., Bai, Y., Yao, H., & Rui, Y. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2480340 · Cited by 8
Research works printed but not linked: 66
- sutskever, 2014, Sequence to sequence learning with neural networks, Proc Conf Neural Inf Process Syst, 3104
- 10.1145/1873951.1874183 · doi:10.1145/1873951.1874183
- frome, 2013, Devise: A deep visual-semantic embedding model, Proc Advances Neural Inf Process Syst, 2121
- krizhevsky, 2012, Imagenet classification with deep convolutional neural networks, Proc 25th Int Conf Neural Inf Process Syst, 1097
- gunasekar, 2015, Consistent collective matrix completion under joint low rank structure, Proc Conf Artif Intell Statist, 306
- 10.3115/v1/D14-1005 · DOI · Paper page
- collell, 2017, Imagined visual representations as multimodal embeddings, Proc 31st AAAI Conf Artif Intell, 4378
- wang, 2015, On deep multi-view representation learning, Proc Int Conf Int Conf Mach Learn, 1083
- mcwilliams, 2013, Correlated random features for fast semi-supervised learning, Proc 20th Int Conf Neural Inf Process Syst, 440
- xing, 2002, Distance metric learning with application to clustering with side-information, Proc Advances Neural Inf Process Syst, 505
- hinton, 2009, Replicated softmax: An undirected topic model, Proc Advances Neural Inf Process Syst, 1607
- welling, 2004, Exponential family harmoniums with an application to information retrieval, Proc 17th Int Conf Neural Inf Process Syst, 1481
- xie, 2013, Multi-modal distance metric learning, Proc 23rd Int Joint Conf Artif Intell, 1806
- zhuang, 2013, Supervised coupled dictionary learning with group structures for multi-modal retrieval, Proc AAAI Conf Artif Intell, 1070
- 10.2307/2334380 · DOI
- zhang, 2011, Multi-label output codes using canonical correlation analysis, Proc 14th Int Conf Artificial Intell, 873
- dhillon, 2012, Two Step CCA: A new spectral method for estimating vector models of words, Proc 29th Int Conf Mach Learn
- dhillon, 2011, Multi-view learning of word embeddings via CCA, Proc Advances Neural Inf Process Syst, 199
- salakhutdinov, 2009, Deep boltzmann machines, Proc Int Conf Artif Intell Statist, 1
- andrew, 2013, Deep canonical correlation analysis, Mach Learn Res, 1247
- hu, 2013, Multimodal DBN for predicting high-quality answers in CQA portals, Proc Annual Meeting of the Assoc Computational Linguistics, 843
- cohn, 2000, The missing link - A probabilistic model of document content and hypertext connectivity, Proc Advances Neural Inf Process Syst, 430
- avron, 2013, Efficient dimensionality reduction for canonical correlation analysis, Proc 30th Int Conf Mach Learn, 28, 347
- 10.1145/1390156.1390285 · doi:10.1145/1390156.1390285
- chen, 2012, Structured sparse canonical correlation analysis, Proc Conf Artif Intell Statist, 199
- akaho, 2001, A kernel method for canonical correlation analysis, Proc Int Meeting Psychometric Soc
- 10.1137/130919222 · doi:10.1137/130919222
- lu, 2014, large scale canonical correlation analysis with iterative least squares, Proc 27th Int Conf Neural Inf Process Syst, 91
- belkin, 2006, Manifold regularization: A geometric framework for learning from labeled and unlabeled examples, J Mach Learn Res, 7, 2399
- torres, 2007, Finding musically meaningful words by sparse CCA, Proc Adv Neural Inf Process Syst
- graves, 2014, Towards end-to-end speech recognition with recurrent neural networks, Proc 31st Int Conf Int Conf Mach Learn, 1764
- serban, 2016, Building end-to-end dialogue systems using generative hierarchical neural network models, Proc 13th AAAI Conf Artif Intell, 16, 3776
- wang, 2015, Large-scale approximate kernel canonical correlation analysis, CoRR, abs 1511 4773
- roller, 2013, A multimodal lda model integrating textual, cognitive and visual modalities, Proc Conf Empirical Methods Natural Language Process, 1146
- le, 2013, Fastfood - computing hilbert space expansions in loglinear time, Proc Int Conf Mach Learn, 244
- yang, 2012, Nyström method versus random fourier features: A theoretical and empirical comparison, Proc Advances Neural Inf Process Syst, 485
- williams, 2000, Using the nyström method to speed up kernel machines, Proc 13th Int Conf Neural Inf Process Syst, 682
- lai, 1998, Canonical correlation analysis using artificial neural networks, Proc Eur Symp Artif Neural Netw, 363
- arora, 2012, Kernel CCA for multi-view learning of acoustic features using articulatory measurements, Proc Symp Mach Learning Speech Lang Process, 34
- gretton, 2005, Kernel methods for measuring independence, J Mach Learn Res, 6, 2075
- 10.1145/1871985.1871989 · doi:10.1145/1871985.1871989
- brand, 2002, Incremental singular value decomposition of uncertain data with missing values, Proc Eur Conf Comput Vis, 707
- fukumizu, 2007, Statistical consistency of kernel canonical correlation analysis, J Mach Learn Res, 8, 361
- 10.1007/s11425-011-4245-2 · doi:10.1007/s11425-011-4245-2
- dong, 2017, A hybrid collaborative filtering model with deep structure for recommender systems, Proc 31st AAAI Conf Artif Intell, 1309
- weiss, 2008, Spectral hashing, Proc Advances Neural Inf Process Syst, 1753
- huang, 2013, Learning deep structured semantic models for web search using clickthrough data, Proc ACM Int Conf Inf Knowl Manag, 2333
- kiros, 2014, Multimodal neural language models, Proc 31st Int Conf Mach Learn, 14, 595
- dean, 2012, Large scale distributed deep networks, Proc Advances Neural Inf Process Syst, 1223
- xu, 2015, Jointly modeling deep video and compositional text to bridge vision and language in a unified framework, Proc AAAI Conf Artif Intell, 2346
- blei, 2003, Latent dirichlet allocation, J Mach Learn Res, 3, 993
- wang, 2009, Simultaneous image classification and annotation, Proc IEEE Conf Comput Vis Pattern Recognit, 1903
- bach, 2002, Kernel independent component analysis, J Mach Learn Res, 3, 1
- bishop, 2006, Pattern Recognition and Machine Learning
- 10.1109/Allerton.2012.6483308 · doi:10.1109/allerton.2012.6483308
- rosipal, 2006, Overview and recent advances in partial least squares, 34
- wold, 1982, Soft modeling: the basic design and some extensions, Systems under Indirect Observation, 2, 589
- wang, 2012, Kernel cross-modal factor analysis for information fusion with application to bimodal emotion recognition, IEEE Trans Multimedia, 14, 3
- jia, 2010, Factorized latent spaces with structured sparsity, Proc Advances in Neural Inf Process Syst, 982
- xing, 2005, Mining associated text and images with dual-wing harmoniums, Proc 21st Conf Uncertainty Artif Intell, 633
- chen, 2010, Predictive subspace learning for multi-view data: A large margin approach, Proc 23rd Int Conf Neural Inf Process Syst, 361
- srivastava, 2012, Multimodal learning with deep boltzmann machines, Proc Int Conf Neural Inf Process, 2231
- ngiam, 2011, Multimodal deep learning, Proc Int Conf Mach Learn, 689
- collins, 2002, New ranking algorithms for parsing and tagging: Kernels over discrete structures, and the voted perceptron, Proc Annual Meeting of the Assoc Computational Linguistics, 263
- kumar, 2011, Learning hash functions for cross-view similarity search, Proc 20nd Int Joint Conf Artif Intell, 1360
- zhen, 2012, Co-regularized hashing for multimodal data, Proc Advances Neural Inf Process Syst, 1385
Cited by
Tian, Y., Krishnan, D., & Isola, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58621-8_45 · Cited by 1,997
A survey on deep learning for multimodal data fusion
Gao, J., Li, P., Chen, Z., & Zhang, J. (2020) · Neural Computation · doi:10.1162/neco_a_01273 · Cited by 821
Robust Multi-View Clustering With Incomplete Information
Yang, M., Li, Y., Hu, P., Bai, J., Lv, J., & Peng, X. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3155499 · Cited by 332
Self-citationDeep multi-view learning methods: A review
Yan, X., Hu, S., Mao, Y., Ye, Y., & Yu, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.03.090 · Cited by 322
A survey of multimodal deep generative models
Suzuki, M., & Matsuo, Y. (2022) · Advanced Robotics · doi:10.1080/01691864.2022.2035253 · Cited by 109
“The first purpose is sometimes referred to as multimodal representation learning [50], and besides generative approaches, there are studies based on discriminative approaches such as contrastive learning [1, 97, 100].”
Metadata
- Authors
- Li, Y., Yang, M., & Zhang, Z.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2019-10-01
- Volume
- 31
- Issue
- 10
- Pages
- 1863-1883
- ISSN
- 1041-4347 (ISSN-L)
- 1558-2191
- 2326-3865
- OpenAlex
- W2892816441
- Cited by
- 524
- References
- 120
- FWCI
- 13.96
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- National Natural Science Foundation of China (61702448, 61672456)
- Fundamental Research Funds for the Central Universities (2017QNA5008, 2017FZA5007)
- Zhejiang University
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.