The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index.
Cross-Modal Retrieval With CNN Visual Features: A New Baseline
Abstract
Recently, convolutional neural network (CNN) visual features have demonstrated their powerful ability as a universal representation for various recognition tasks. In this paper, cross-modal retrieval with CNN visual features is implemented with several classic methods. Specifically, off-the-shelf CNN visual features are extracted from the CNN model, which is pretrained on ImageNet with more than one million images from 1000 object categories, as a generic image representation to tackle cross-modal retrieval. To further enhance the representational ability of CNN visual features, based on the pretrained CNN model on ImageNet, a fine-tuning step is performed by using the open source Caffe CNN library for each target data set. Besides, we propose a deep semantic matching method to address the cross-modal retrieval problem with respect to samples which are annotated with one or multiple labels. Extensive experiments on five popular publicly available data sets well demonstrate the superiority of CNN visual features for cross-modal retrieval.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope
Oliva, A., & Torralba, A. (2001) · International Journal of Computer Vision · doi:10.1023/a:1011139631724 · Cited by 6,319
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · cited 2×
“Some recent articles [8], [12], [36], [40], [41], [55] demonstrated”
BackgroundMethodCanonical Correlation Analysis: An Overview with Application to Learning Methods
Hardoon, D. R., Szedmak, S., & Shawe-Taylor, J. (2004) · Neural Computation · doi:10.1162/0899766042321814 · Cited by 3,202
Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.222 · Cited by 3,150 · cited 1×
“Some recent articles [8], [12], [36], [40], [41], [55] demonstrated”
BackgroundLocality-constrained Linear Coding for image classification
Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., & Gong, Y. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540018 · Cited by 3,100 · cited 9×
“We first extract SIFT interest points for each image and then encode them with LLC or VLAD to generate the feature representation.”
BackgroundInfluentialNUS-WIDE: a real-world web image database from National University of Singapore
Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., & Zheng, Y. (2009) · Proceedings of the ACM International Conference on Image and Video Retrieval · doi:10.1145/1646396.1646452 · Cited by 3,025
(2011) · Cited by 2,752
Aggregating local descriptors into a compact image representation
Jegou, H., Douze, M., Schmid, C., & Perez, P. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540039 · Cited by 2,692
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680
Deep Canonical Correlation Analysis
Andrew, G., Arora, R., Bilmes, J., & Livescu, K. (2013) · International Conference on Machine Learning · Cited by 2,401
DeViSE: A Deep Visual-Semantic Embedding Model
Frome, A., Corrado, G. S., Shlens, J., Samy Bengio, A., Dean, J., Ranzato, M., & Mikolov, T. (2013) · Cited by 2,308
Convolutional networks and applications in vision
LeCun, Y., Kavukcuoglu, K., & Farabet, C. (2010) · Proceedings of 2010 IEEE International Symposium on Circuits and Systems · doi:10.1109/iscas.2010.5537907 · Cited by 2,213
A new approach to cross-modal multimedia retrieval
Rasiwasia, N., Costa Pereira, J., Coviello, E., Doyle, G., Lanckriet, G. R. G., Levy, R., & Vasconcelos, N. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1873987 · Cited by 1,334
CNN: Single-label to Multi-label
Wei, Y., Xia, W., Huang, J., Ni, B., Dong, J., Zhao, Y., & Yan, S. (2014) · arXiv · doi:10.48550/arxiv.1406.5726 · Cited by 784
Self-citationHCP: A Flexible CNN Framework for Multi-Label Image Classification
Wei, Y., Xia, W., Lin, M., Huang, J., Ni, B., Dong, J., Zhao, Y., & Yan, S. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2491929 · Cited by 784
Self-citationGeneralized Multiview Analysis: A discriminative latent space
Sharma, A., Kumar, A., Daume, H., & Jacobs, D. W. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247923 · Cited by 714
Inter-media hashing for large-scale retrieval from heterogeneous data sources
Song, J., Yang, Y., Yang, Y., Huang, Z., & Shen, H. T. (2013) · Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2463676.2465274 · Cited by 620
Large-Scale Supervised Multimodal Hashing with Semantic Correlation Maximization
Zhang, D., & Li, W.-J. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v28i1.8995 · Cited by 616
Learning Hash Functions for Cross-View Similarity Search
Kumar, S., & Udupa, R. (2012) · Cited by 614
Collecting Image Annotations Using Amazon's Mechanical Turk
Rashtchian, C., Young, P., Hodosh, M., & Hockenmaier, J. (2010) · Cited by 541
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics
Gong, Y., Ke, Q., Isard, M., & Lazebnik, S. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-013-0658-4 · Cited by 540
Data fusion through cross-modality metric learning using similarity-sensitive hashing
Bronstein, M. M., Bronstein, A. M., Michel, F., & Paragios, N. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539928 · Cited by 455
Unified Video Annotation via Multigraph Learning
Wang, M., Hua, X.-S., Hong, R., Tang, J., Qi, G.-J., & Song, Y. (2009) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2009.2017400 · Cited by 441
On the Role of Correlation and Abstraction in Cross-Modal Multimedia Retrieval
Costa Pereira, J., Coviello, E., Doyle, G., Rasiwasia, N., Lanckriet, G. R. G., Levy, R., & Vasconcelos, N. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.142 · Cited by 435
Multimodal Graph-Based Reranking for Web Image Search
Wang, M., Li, H., Tao, D., Lu, K., & Wu, X. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2207397 · Cited by 344
LIBLINEAR: A Library for Large Linear Classification
FanRong-En, ChangKai-Wei, HsiehCho-Jui, WangXiang-Rui, & LinChih-Jen (2008) · Journal of Machine Learning Research · doi:10.5555/1390681.1442794 · Cited by 328
Linear cross-modal hashing for efficient multimedia search
Zhu, X., Huang, Z., Shen, H. T., & Zhao, X. (2013) · Proceedings of the 21st ACM international conference on Multimedia · doi:10.1145/2502081.2502107 · Cited by 282
Towards a Relevant and Diverse Search of Social Images
Wang, M., Yang, K., Hua, X., & Zhang, H.-J. (2010) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2010.2055045 · Cited by 228
Yang, Y., Zhuang, Y.-T., Wu, F., & Pan, Y.-H. (2008) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2008.917359 · Cited by 213
Feature Selection for Multimedia Analysis by Sharing Information Among Multiple Tasks
Yang, Y., Ma, Z., Hauptmann, A. G., & Sebe, N. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2012.2237023 · Cited by 202
Supervised Coupled Dictionary Learning with Group Structures for Multi-modal Retrieval
Zhuang, Y., Wang, Y., Wu, F., Zhang, Y., & Lu, W. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8603 · Cited by 191
Learning cross-modality similarity for multinomial data
Jia, Y., Salzmann, M., & Darrell, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126524 · Cited by 181
Ranking with local regression and global alignment for cross media retrieval
Yang, Y., Xu, D., Nie, F., Luo, J., & Zhuang, Y. (2009) · Proceedings of the 17th ACM international conference on Multimedia · doi:10.1145/1631272.1631298 · Cited by 179
Cluster Canonical Correlation Analysis
Rasiwasia, N., Mahajan, D. K., Mahadevan, V., & Aggarwal, G. (2014) · Cited by 175
Multi-Feature Fusion via Hierarchical Regression for Multimedia Analysis
Yang, Y., Song, J., Huang, Z., Ma, Z., Sebe, N., & Hauptmann, A. G. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2012.2234731 · Cited by 171
Large-Margin Multi-Modal Deep Learning for RGB-D Object Recognition
Wang, A., Lu, J., Cai, J., Cham, T.-J., & Wang, G. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2476655 · Cited by 153
Discriminative coupled dictionary hashing for fast cross-media retrieval
Zhou, Y., Wu, F. Q., Yang, Y., Tian, Q., Luo, J., & Zhuang, Y. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609563 · Cited by 152
Effective multi-modal retrieval based on stacked auto-encoders
Wang, W., Ooi, B. C., Yang, X., Zhang, D., & Zhuang, Y. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2732296.2732301 · Cited by 151
Feature Correlation Hypergraph: Exploiting High-order Potentials for Multimodal Recognition
Zhang, L., Gao, Y., Hong, C., Feng, Y., Zhu, J., & Cai, D. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2285219 · Cited by 143
Wu, F. Q., Zhou, Y., Yang, Y., Tang, S., Zhang, Y., & Zhuang, Y. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2291214 · Cited by 139
Learning the Relative Importance of Objects from Tagged Images for Retrieval and Cross-Modal Search
Hwang, S. J., & Grauman, K. L. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-011-0494-3 · Cited by 128
Improving web image search results using query-relative classifiers
Krapac, J., Allan, M., Verbeek, J. J., & Jurie, F. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540092 · Cited by 125
Modality-Dependent Cross-Media Retrieval
Wei, Y., Zhao, Y., Zhu, Z., Wei, S., Xiao, Y., Feng, J., & Yan, S. (2016) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2775109 · Cited by 90
Self-citationParallel field alignment for cross media retrieval
Mao, X., Lin, B., Cai, D., et al. (2013) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2502081.2502087 · Cited by 84
Cross-media semantic representation via bi-directional learning to rank
Wu, F. Q., Lu, X., Zhang, Z. M., et al. (2013) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2502081.2502097 · Cited by 82
Robust Hashing With Local Models for Approximate Similarity Search
Song, J., Yang, Y., Li, X., Huang, Z. H., & Yang, Y. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2289351 · Cited by 78
Accounting for the Relative Importance of Objects in Image Retrieval
Hwang, S. J., & Grauman, K. L. (2010) · British Machine Vision Conference (BMVC) · doi:10.5244/c.24.58 · Cited by 71
Large-Scale Unsupervised Hashing with Shared Structure Learning
Liu, X., Mu, Y., Zhang, D., Lang, B., & Li, X. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2360856 · Cited by 62
Im2Text and Text2Im: Associating Images and Texts for Cross-Modal Retrieval
Verma, Y., & Jawahar, C. V. (2014) · British Machine Vision Conference (BMVC) · doi:10.5244/c.28.97 · Cited by 54
Facilitating Image Search With a Scalable and Compact Semantic Mapping
Wang, M., Li, W., Liu, D., Ni, B., Shen, J., & Yan, S. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2356136 · Cited by 50
Self-citationA low rank structural large margin method for cross-modal ranking
Lu, X., Wu, F. Q., Tang, S., Zhang, Z. M., He, X., & Zhuang, Y. (2013) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2484028.2484039 · Cited by 50
Query-Oriented Multi-Document Summarization via Unsupervised Deep Learning
Liu, Y., Zhong, S.-H., & Li, W. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v26i1.8352 · Cited by 43
A Cross-Modal Approach for Extracting Semantic Relationships Between Concepts Using Tagged Images
Katsurai, M., Ogawa, T., & Haseyama, M. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2306655 · Cited by 35
Learning a mid-level feature space for cross-media regularization
Wei, Y., Zhao, Y., Zhu, Z., et al. (2014) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2014.6890289 · Cited by 8
Self-citation
Cited by
An End-to-End Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features
He, Y., Song, K., Meng, Q., & Yan, Y. (2020) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2019.2915404 · Cited by 1,341
“Unsurprisingly, the performance of the methods based on convolutional features is strongly higher than the methods without CNN [37].”
MethodRGB-Infrared Cross-Modality Person Re-identification
Wu, A., Zheng, W.-S., Yu, H.-X., Gong, S., & Lai, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.575 · Cited by 862
Dual-path Convolutional Image-Text Embeddings with Instance Loss
Zheng, Z., Zheng, L., Garrett, M., Yang, Y., Xu, M., & Shen, Y.-D. (2020) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3383184 · Cited by 589
A Survey of Multi-View Representation Learning
Li, Y., Yang, M., & Zhang, Z. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2872063 · Cited by 524
Deep Supervised Cross-Modal Retrieval
Zhen, L., Hu, P., Wang, X., & Peng, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01064 · Cited by 478
“Inspired by the great success of deep neural networks in representation learning [14], a large number of deep learning-based approaches [2, 33, 19, 36, 21, 25, 7] have been proposed to learn a common presentation space for multimedia data.”
Background
Metadata
- Authors
- Wei, Y., Zhao, Y., Lu, C., Wei, S., Liu, L., Zhu, Z., & Yan, S.
- Venue
- IEEE Transactions on Cybernetics
- Published
- 2016
- Volume
- 47
- Issue
- 2
- Pages
- 1-12
- ISSN
- 2168-2267 (ISSN-L)
- 2168-2275
- OpenAlex
- W2326180695
- Cited by
- 370
- References
- 66
- FWCI
- 33.77
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.