The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Joint Feature Selection and Subspace Learning for Cross-Modal Retrieval
Abstract
Cross-modal retrieval has recently drawn much attention due to the widespread existence of multimodal data. It takes one type of data as the query to retrieve relevant data objects of another type, and generally involves two basic problems: the measure of relevance and coupled feature selection. Most previous methods just focus on solving the first problem. In this paper, we aim to deal with both problems in a novel joint learning framework. To address the first problem, we learn projection matrices to map multimodal data into a common subspace, in which the similarity between different modalities of data can be measured. In the learning procedure, the l21-norm penalties are imposed on the projection matrices separately to solve the second problem, which selects relevant and discriminative features from different feature spaces simultaneously. A multimodal graph regularization term is further imposed on the projected data,which preserves the inter-modality and intra-modality similarity relationships.An iterative algorithm is presented to solve the proposed joint learning problem, along with its convergence analysis. Experimental results on cross-modal retrieval tasks demonstrate that the proposed method outperforms the state-of-the-art subspace approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Roweis, S. T., & Saul, L. K. (2000) · Science · doi:10.1126/science.290.5500.2323 · Cited by 14,965
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,223
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Canonical Correlation Analysis: An Overview with Application to Learning Methods
Hardoon, D. R., Szedmak, S., & Shawe-Taylor, J. (2004) · Neural Computation · doi:10.1162/0899766042321814 · Cited by 3,202
NUS-WIDE: a real-world web image database from National University of Singapore
Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., & Zheng, Y. (2009) · Proceedings of the ACM International Conference on Image and Video Retrieval · doi:10.1145/1646396.1646452 · Cited by 3,025
DeViSE: A Deep Visual-Semantic Embedding Model
Frome, A., Corrado, G. S., Shlens, J., Samy Bengio, A., Dean, J., Ranzato, M., & Mikolov, T. (2013) · Cited by 2,308
Multimodal Learning with Deep Boltzmann Machines
Srivastava, N., & Salakhutdinov, R. (2012) · Journal of Machine Learning Research · doi:10.5555/2627435.2697059 · Cited by 1,659
Efficient and Robust Feature Selection via Joint ℓ2,1-Norms Minimization
Nie, F., Huang, H., Cai, X., & Ding, C. H. Q. (2010) · Cited by 1,522
A new approach to cross-modal multimedia retrieval
Rasiwasia, N., Costa Pereira, J., Coviello, E., Doyle, G., Lanckriet, G. R. G., Levy, R., & Vasconcelos, N. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1873987 · Cited by 1,334
Multimodal fusion for multimedia analysis: a survey
Atrey, P. K., Hossain, M. A., El Saddik, A., & Kankanhalli, M. S. (2010) · Multimedia Systems · doi:10.1007/s00530-010-0182-0 · Cited by 1,279
Overview and Recent Advances in Partial Least Squares
Rosipal, R., & Krämer, N. (2006) · Lecture Notes in Computer Science · doi:10.1007/11752790_2 · Cited by 1,091
Blei, D. M., & Jordan, M. I. (2003) · Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval · doi:10.1145/860435.860460 · Cited by 1,054
Separating Style and Content with Bilinear Models
Tenenbaum, J. B., & Freeman, W. T. (2000) · Neural Computation · doi:10.1162/089976600300015349 · Cited by 840 · cited 2×
“Recently, several approaches for establishing inter-modal relationships between data from different modalities generally rely on subspace learning, such as Canonical Correlation Analysis [25], [26], Partial Least Squares (PLS) [27] and Bilinear Model (BLM) [28], [29].”
BackgroundMethodGeneralized Multiview Analysis: A discriminative latent space
Sharma, A., Kumar, A., Daume, H., & Jacobs, D. W. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247923 · Cited by 714
Collective Matrix Factorization Hashing for Multimodal Data
Ding, G., Guo, Y., & Zhou, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.267 · Cited by 680
Wsabie: Scaling up to large vocabulary image annotation
Weston, J., Samy Bengio, A., & Usunier, N. (2011) · Cited by 677
Maximum Correntropy Criterion for Robust Face Recognition
He, R., Zheng, W.-S., & Hu, B.-G. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.220 · Cited by 670
Self-citationDiscriminative Learning and Recognition of Image Set Classes Using Canonical Correlations
Kim, T.-K., Kittler, J., & Cipolla, R. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1037 · Cited by 590
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics
Gong, Y., Ke, Q., Isard, M., & Lazebnik, S. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-013-0658-4 · Cited by 540
Analysis of Half-Quadratic Minimization Methods for Signal and Image Recovery
Nikolova, M., & Ng, M. K. (2005) · SIAM Journal on Scientific Computing · doi:10.1137/030600862 · Cited by 478
Data fusion through cross-modality metric learning using similarity-sensitive hashing
Bronstein, M. M., Bronstein, A. M., Michel, F., & Paragios, N. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539928 · Cited by 455
On the Role of Correlation and Abstraction in Cross-Modal Multimedia Retrieval
Costa Pereira, J., Coviello, E., Doyle, G., Rasiwasia, N., Lanckriet, G. R. G., Levy, R., & Vasconcelos, N. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.142 · Cited by 435
Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch
Sharma, A., & Jacobs, D. W. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995350 · Cited by 376
Blei, D. M., & Jordan, M. I. (2003) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/860458.860460 · Cited by 359
Spectral Regression for Efficient Regularized Subspace Learning
Cai, D., He, X., & Han, J. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408855 · Cited by 355
Label Embedding Trees for Large Multi-Class Tasks
Samy Bengio, A., Weston, J., & Grangier, D. (2010) · Cited by 298
Learning Cross-Media Joint Representation With Sparse and Semisupervised Regularization
Zhai, X., Peng, Y., & Xiao, J. (2014) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2013.2276704 · Cited by 294
Multimedia content processing through cross-modal association
Li, D., Dimitrova, N., Li, M., & Sethi, I. K. (2003) · Proceedings of the eleventh ACM international conference on Multimedia · doi:10.1145/957013.957143 · Cited by 265
Large-Margin Multi-ViewInformation Bottleneck
Xu, C., Tao, D., & Xu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.2296528 · Cited by 257
Bridging the Gap: Query by Semantic Example
Rasiwasia, N., Moreno, P. J., & Vasconcelos, N. (2007) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2007.900138 · Cited by 247
Learning Coupled Feature Spaces for Cross-Modal Matching
Wang, K., He, R., Wang, W., Wang, L., & Tan, T. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.261 · Cited by 235
Self-citationInter-modality Face Recognition
Lin, D., & Tang, X. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744085_2 · Cited by 204
l2, 1 Regularized correntropy for robust feature selection
He, R., Tan, T. N., Wang, L., & Zheng, W. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247966 · Cited by 202
Self-citationCo-Regularized Hashing for Multimodal Data
Zhen, Y., & Yeung, D. (2012) · neural information processing systems · Cited by 202
Topic regression multi-modal Latent Dirichlet Allocation for image annotation
Putthividhy, D., Attias, H. T., & Nagarajan, S. S. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540000 · Cited by 192
Supervised Coupled Dictionary Learning with Group Structures for Multi-modal Retrieval
Zhuang, Y., Wang, Y., Wu, F., Zhang, Y., & Lu, W. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8603 · Cited by 191
Learning cross-modality similarity for multinomial data
Jia, Y., Salzmann, M., & Darrell, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126524 · Cited by 181
Multimodal Similarity-Preserving Hashing
Masci, J., Bronstein, M. M., Bronstein, A. M., & Schmidhuber, J. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.225 · Cited by 179
Ranking with local regression and global alignment for cross media retrieval
Yang, Y., Xu, D., Nie, F., Luo, J., & Zhuang, Y. (2009) · Proceedings of the 17th ACM international conference on Multimedia · doi:10.1145/1631272.1631298 · Cited by 179
Mining Semantic Correlation of Heterogeneous Multimedia Data for Cross-Media Retrieval
Zhuang, Y., Yang, Y., & Wu, F. Q. (2008) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2007.911822 · Cited by 149
Discriminative coupled dictionary hashing for fast cross-media retrieval
Zhou, Y., Wu, F. Q., Yang, Y., Tian, Q., Luo, J., & Zhuang, Y. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609563 · Cited by 144
Learning Multimodal Latent Attributes
Fu, Y., Hospedales, T. M., Xiang, T., & Gong, S. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.128 · Cited by 142
Heterogeneous Metric Learning with Joint Graph Regularization for Cross-Media Retrieval
Zhai, X., Peng, Y., & Xiao, J. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8464 · Cited by 127
Learning Multi-View Neighborhood Preserving Projections
Quadrianto, N., & Lampert, C. H. (2011) · Figshare · Cited by 117
Multimedia content processing through cross-modal association
Li, D., Dimitrova, N., Li, M., & Sethi, I. K. (2003) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/957142.957143 · Cited by 106
Lu, Z., & Peng, Y. (2012) · International Journal of Computer Vision · doi:10.1007/s11263-012-0602-z · Cited by 92
Reading between the Lines: Object Localization Using Implicit Cues from Image Tags
Hwang, S. J., & Grauman, K. L. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.190 · Cited by 91
Parallel field alignment for cross media retrieval
Mao, X., Lin, B., Cai, D., et al. (2013) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2502081.2502087 · Cited by 84
Cross-media semantic representation via bi-directional learning to rank
Wu, F. Q., Lu, X., Zhang, Z. M., et al. (2013) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2502081.2502097 · Cited by 82
Large Margin Taxonomy Embedding for Document Categorization
Weinberger, K. Q., & Chapelle, O. (2008) · neural information processing systems · Cited by 77
Multi-modal Clustering for Multimedia Collections
Bekkerman, R., & Jeon, J. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383223 · Cited by 75
CMML: a New Metric Learning Approach for Cross Modal Matching
Mignon, A., & Jurie, F. (2012) · Cited by 66
A low rank structural large margin method for cross-modal ranking
Lu, X., Wu, F. Q., Tang, S., Zhang, Z. M., He, X., & Zhuang, Y. (2013) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2484028.2484039 · Cited by 50
Heterogeneous feature selection by group lasso with logistic regression
Wu, F. Q., Yuan, Y., & Zhuang, Y. (2010) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1873951.1874129 · Cited by 34
Maximum Covariance Unfolding : Manifold Learning for Bimodal Data
Mahadevan, V., Wong, C. W., Pereira, J. C., et al. (2011) · Cited by 26
Wu, F. Q., Han, Y., Liu, X., Shao, J., Zhuang, Y., & Zhang, Z. M. (2012) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-012-0001-9 · Cited by 24
Face recognition based on non-corresponding region matching
Li, A., Shan, S., Chen, X., & Gao, W. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126352 · Cited by 22
Unified Constraint Propagation on Multi-View Data
Lu, Z., & Peng, Y. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8638 · Cited by 17
A Multi-View Embedding Space for Internet Images, Tags, and Their Semantics
Ke, Q., & Isard, M. (2012) · International Journal of Computer Vision · Cited by 13
Cited by
Adversarial Cross-Modal Retrieval
Wang, B., Yang, Y., Xu, X., Hanjalic, A., & Shen, H. T. (2017) · Proceedings of the 25th ACM international conference on Multimedia · doi:10.1145/3123266.3123326 · Cited by 779
Review the state-of-the-art technologies of semantic segmentation based on deep learning
Mo, Y., Wu, Y., Yang, X., Liu, F., & Liao, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.01.005 · Cited by 632
Hierarchical Deep Click Feature Prediction for Fine-Grained Image Recognition
Yu, J., Tan, M., Zhang, H., Rui, Y., & Tao, D. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2932058 · Cited by 498
Triplet-Based Deep Hashing Network for Cross-Modal Retrieval
Deng, C., Chen, Z., Liu, X., Gao, X., & Tao, D. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2821921 · Cited by 400
An Overview of Cross-Media Retrieval: Concepts, Methodologies, Benchmarks, and Challenges
Peng, Y., Huang, X., & Zhao, Y. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2705068 · Cited by 341
Metadata
- Authors
- Wang, K., He, R., Wang, L., Wang, W., & Tan, T.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2016-10-01
- Volume
- 38
- Issue
- 10
- Pages
- 2010-2023
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2211092169
- Cited by
- 318
- References
- 61
- FWCI
- 21.37
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.