The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking
Abstract
How do we retrieve images accurately? Also, how do we rank a group of images precisely and efficiently for specific queries? These problems are critical for researchers and engineers to generate a novel image searching engine. First, it is important to obtain an appropriate description that effectively represent the images. In this paper, multimodal features are considered for describing images. The images unique properties are reflected by visual features, which are correlated to each other. However, semantic gaps always exist between images visual features and semantics. Therefore, we utilize click feature to reduce the semantic gap. The second key issue is learning an appropriate distance metric to combine these multimodal features. This paper develops a novel deep multimodal distance metric learning (Deep-MDML) method. A structured ranking model is adopted to utilize both visual and click features in distance metric learning (DML). Specifically, images and their related ranking results are first collected to form the training set. Multimodal features, including click and visual features, are collected with these images. Next, a group of autoencoders is applied to obtain initially a distance metric in different visual spaces, and an MDML method is used to assign optimal weights for different modalities. Next, we conduct alternating optimization to train the ranking model, which is used for the ranking of new queries with click features. Compared with existing image ranking methods, the proposed method adopts a new ranking model to use multimodal features, including click features and visual features in DML. We operated experiments to analyze the proposed Deep-MDML in two benchmark data sets, and the results validate the effects of the method.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Object recognition from local scale-invariant features
Lowe, D. G. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790410 · Cited by 16,472
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
Content-based image retrieval at the end of the early years
Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895972 · Cited by 6,056
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Greedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · cited 1×
“Autoencoders (AEs) [35], [36] can efficiently map the input data onto suitable outputs.”
BackgroundCumulated gain-based evaluation of IR techniques
Järvelin, K., & Kekäläinen, J. (2002) · ACM Transactions on Information Systems · doi:10.1145/582415.582418 · Cited by 4,841
Optimizing search engines using clickthrough data
Joachims, T. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775067 · Cited by 3,873
Learning with Local and Global Consistency
Zhou, D., Bousquet, O., Lal, T. N., Weston, J., & Schölkopf, B. (2003) · MPG.PuRe (Max Planck Society) · Cited by 3,864
Query by image and video content: the QBIC system
Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., et al. (1995) · Computer · doi:10.1109/2.410146 · Cited by 3,060
SIGNATURE VERIFICATION USING A “SIAMESE” TIME DELAY NEURAL NETWORK
BROMLEY, J., BENTZ, J. W., BOTTOU, L., GUYON, I., LECUN, Y., MOORE, C., SÄCKINGER, E., & SHAH, R. (1994) · Series in Machine Perception and Artificial Intelligence · doi:10.1142/9789812797926_0003 · Cited by 2,350
Large Margin Methods for Structured and Interdependent Output Variables
Tsochantaridis, I., Joachims, T., Hofmann, T., & Altün, Y. (2005) · Journal of Machine Learning Research · doi:10.5555/1046920.1088722 · Cited by 2,225
Distance Metric Learning for Large Margin Nearest Neighbor Classification
Weinberger, K. Q., Blitzer, J. C., & Saul, L. K. (2005) · neural information processing systems · Cited by 2,132
Training linear SVMs in linear time
Joachims, T. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150429 · Cited by 1,920
Learning to rank for information retrieval
Liu, T.-Y. (2010) · Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1835449.1835676 · Cited by 1,856
Neighbourhood Components Analysis
Goldberger, J., Hinton, G. E., Roweis, S. T., & Salakhutdinov, R. (2004) · Cited by 1,855
Learning to Rank for Information Retrieval
Liu, T.-Y. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000016 · Cited by 1,568
Photobook: Content-based manipulation of image databases
Pentland, A., Picard, R. W., & Sclaroff, S. (1996) · International Journal of Computer Vision · doi:10.1007/bf00123143 · Cited by 1,562
Content-based multimedia information retrieval: State of the art and challenges
Lew, M. S., Sebe, N., Djeraba, C., & Jain, R. (2006) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/1126004.1126005 · Cited by 1,528
Efficient object localization using Convolutional Networks
Tompson, J., Goroshin, R., Jain, A., LeCun, Y., & Bregler, C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298664 · Cited by 1,365
Learning with Hypergraphs: Clustering, Classification, and Embedding
Zhou, D., Huang, J., & Schölkopf, B. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0205 · Cited by 1,283
Support vector machine learning for interdependent and structured output spaces
Tsochantaridis, I., Hofmann, T., Joachims, T., & Altun, Y. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015341 · Cited by 1,243
Virage image search engine: an open framework for image management:
Bach, J. R., Fuller, C., Gupta, A., Hampapur, A., Horowitz, B., Humphrey, R., Jain, R. C., & Shu, C.-F. (1996) · SPIE Proceedings · doi:10.1117/12.234785 · Cited by 776
Discriminative Deep Metric Learning for Face Verification in the Wild
Hu, J., Lu, J., & Tan, Y.-P. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.242 · Cited by 682
Content-based image retrieval systems: A survey
Veltkamp, R. C., & Tănase, M. (2000) · Utrecht University Repository (Utrecht University) · Cited by 480
Random walks on the click graph
Craswell, N., & Szummer, M. (2007) · Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1277741.1277784 · Cited by 441
DeepID-Net: Deformable deep convolutional neural networks for object detection
Ouyang, W., Wang, X., Zeng, X., Qiu, S., Luo, P., Tian, Y., Li, H., Yang, S., Wang, Z., Loy, C.-C., et al. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298854 · Cited by 430
Multimodal semi-supervised learning for image classification
Guillaumin, M., Verbeek, J., & Schmid, C. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540120 · Cited by 418
Stacked Convolutional Denoising Auto-Encoders for Feature Representation
Du, B., Xiong, W., Wu, J., Zhang, L., Zhang, L., & Tao, D. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2536638 · Cited by 416
Self-citationMultimodal Graph-Based Reranking for Web Image Search
Wang, M., Li, H., Tao, D., Lu, K., & Wu, X. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2207397 · Cited by 344
Self-citationBlock-Row Sparse Multiview Multilabel Learning for Image Classification
Zhu, X., Li, X., & Zhang, S. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2403356 · Cited by 324
Learning Distance Metrics with Contextual Constraints for Image Retrieval
Hoi, S. C. H., Liu, W., Lyu, M. R., & Ma, W.-Y. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.167 · Cited by 307
Image tag refinement towards low-rank, content-tag prior and error sparsity
Zhu, G., Yan, S., & Ma, Y. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874028 · Cited by 243
Semantics-Preserving Bag-of-Words Models and Applications
Wu, L., Hoi, S. C. H., & Yu, N. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2045169 · Cited by 191
Yang, L., Jin, R., Mummert, L., Sukthankar, R., Goode, A., Zheng, B., Hoi, S. C. H., & Satyanarayanan, M. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.273 · Cited by 184
Mei, J., Liu, M., Wang, Y.-F., & Gao, H. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2426723 · Cited by 179
Localized Content-Based Image Retrieval
Rahmani, R., Goldman, S. A., Zhang, H., Cholleti, S. R., & Fritts, J. E. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.112 · Cited by 168
Statistical Analysis of Bayes Optimal Subset Ranking
Cossock, D., & Zhang, T. (2008) · IEEE Transactions on Information Theory · doi:10.1109/tit.2008.929939 · Cited by 160
A model to estimate intrinsic document relevance from the clickthrough logs of a web search engine
Dupret, G., & Liao, C. Y. (2010) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/1718487.1718510 · Cited by 146
Learning to re-rank: query-dependent image re-ranking using click data
Jain, V., & Varma, M. (2011) · International World Wide Web Conference (WWW) · doi:10.1145/1963405.1963447 · Cited by 131
Low-Rank Discriminant Embedding for Multiview Learning
Li, J., Wu, Y., Zhao, J., & Lü, K. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2565898 · Cited by 121
Discriminative Object Tracking via Sparse Representation and Online Dictionary Learning
Xie, Y., Zhang, W., Li, C., Lin, S., Qu, Y., & Zhang, Y. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2259230 · Cited by 101
Content-Based Visual Landmark Search via Multimodal Hypergraph Learning
Zhu, L., Shen, J., Jin, H., Zheng, R., & Xie, L. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2383389 · Cited by 97
Distance metric learning from uncertain side information with application to automated photo tagging
Wu, L., Hoi, S. C. H., Jin, R., Zhu, J., & Yu, N. (2009) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1631272.1631293 · Cited by 90
Tag-based web photo retrieval improved by batch mode re-tagging
Chen, L., Xu, D., Tsang, I. W.-H., & Luo, J. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5539988 · Cited by 84
Are click-through data adequate for learning web search rankings?
Dou, Z., Song, R., Yuan, X., & Wen, J.-R. (2008) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/1458082.1458095 · Cited by 82
Spectral Multimodal Hashing and Its Application to Multimedia Retrieval
Zhen, Y., Gao, Y., Yeung, D., Hongyuan Zha, B., & Li, X. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2392052 · Cited by 74
Click-through-based cross-view learning for image search
Pan, Y., Yao, T., Mei, T., et al. (2014) · doi:10.1145/2600428.2609568 · Cited by 74
Textual Query of Personal Photos Facilitated by Large-Scale Web Data
Liu, Y., Xu, D., Tsang, I. W.-H., & Luo, J. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.142 · Cited by 72
Positive and Unlabeled Multi-Graph Learning
Wu, J. X., Pan, S., Xingquan Zhu, A., Chengqi Zhang, B., & Wu, X. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2527239 · Cited by 65
Content-aware Ranking for visual search
Geng, B., Yang, L., Xu, C., & Hua, X. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540003 · Cited by 47
Wen, Z., Hou, Z., & Jiao, L. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2581861 · Cited by 41
Learning a mixture of sparse distance metrics for classification and dimensionality reduction
Hong, Y., Li, Q., Jiang, J., & Zhuowen Tu, A. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126332 · Cited by 40
Wang, S., Jiang, S., Huang, Q., & Tian, Q. (2010) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1873951.1873975 · Cited by 21
A Ranking Approach on Large-Scale Graph With Multidimensional Heterogeneous Information
Wei, W., Gao, B., Liu, T., Wang, T., Li, G., & Li, H. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2418233 · Cited by 15
Visually Interpreting Names as Demographic Attributes by Exploiting Click-Through Data
Chen, Y., Kuo, Y.-H., Wu, C.-C., & Hsu, W. H. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9184 · Cited by 3
Cited by
Deep Metric Learning: A Survey
KAYA, M., & BİLGE, H. Ş. (2019) · Symmetry · doi:10.3390/sym11091066 · Cited by 703
“However, these approaches are not successful enough to reveal nonlinear knowledge of the data [21].”
BackgroundA model-based gait recognition method with body pose and human prior knowledge
Liao, R., Yu, S., An, W., & Huang, Y. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2019.107069 · Cited by 509
Revisiting multiple instance neural networks
Wang, X., Yan, Y., Tang, P., Bai, X., & Liu, W. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.08.026 · Cited by 498
Multi-view low-rank sparse subspace clustering
Brbić, M., & Kopriva, I. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.08.024 · Cited by 483
A Decade Survey of Content Based Image Retrieval Using Deep Learning
Dubey, S. R. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3080920 · Cited by 356
Metadata
- Authors
- Yu, J., Yang, X., Gao, F., & Tao, D.
- Venue
- IEEE Transactions on Cybernetics
- Published
- 2017-12
- Volume
- 47
- Issue
- 12
- Pages
- 4014-4024
- ISSN
- 2168-2267 (ISSN-L)
- 2168-2275
- OpenAlex
- W2497173630
- Cited by
- 364
- References
- 58
- FWCI
- 56.59
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.