The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Click Prediction for Web Image Reranking Using Multimodal Sparse Coding
Abstract
Image reranking is effective for improving the performance of a text-based image search. However, existing reranking algorithms are limited for two main reasons: 1) the textual meta-data associated with images is often mismatched with their actual visual content and 2) the extracted visual features do not accurately describe the semantic similarities between images. Recently, user click information has been used in image reranking, because clicks have been shown to more accurately describe the relevance of retrieved images to search queries. However, a critical problem for click-based methods is the lack of click data, since only a small number of web images have actually been clicked on by users. Therefore, we aim to solve this problem by predicting image clicks. We propose a multimodal hypergraph learning-based sparse coding method for image click prediction, and apply the obtained click data to the reranking of images. We adopt a hypergraph to build a group of manifolds, which explore the complementarity of different features through a group of weights. Unlike a graph that has an edge between two vertices, a hyperedge in a hypergraph connects a set of vertices, and helps preserve the local smoothness of the constructed sparse codes. An alternating optimization procedure is then performed, and the weights of different modalities and the sparse codes are simultaneously obtained. Finally, a voting strategy is used to describe the predicted click as a binary event (click or no click), from the images' corresponding sparse codes. Thorough empirical studies on a large-scale database including nearly 330 K images demonstrate the effectiveness of our approach for click prediction when compared with several other methods. Additional image reranking experiments on real-world data show the use of click prediction is beneficial to improving the performance of prominent graph-based image reranking algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482
Robust Face Recognition via Sparse Representation
Wright, J., Yang, A. Y., Ganesh, A., Sastry, S. S., & Ma, Y. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.79 · Cited by 9,388
Efron, B., Hastie, T., Johnstone, I., & Tibshirani, R. (2004) · The Annals of Statistics · doi:10.1214/009053604000000067 · Cited by 9,357
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
Belkin, M., & Niyogi, P. (2003) · Neural Computation · doi:10.1162/089976603321780317 · Cited by 7,621
Stable signal recovery from incomplete and inaccurate measurements
Candès, E. J., Romberg, J. K., & Tao, T. (2006) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20124 · Cited by 7,036
Atomic Decomposition by Basis Pursuit
Chen, S. S., Donoho, D. L., & Saunders, M. A. (1998) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827596304010 · Cited by 6,845
Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope
Oliva, A., & Torralba, A. (2001) · International Journal of Computer Vision · doi:10.1023/a:1011139631724 · Cited by 6,319
Cumulated gain-based evaluation of IR techniques
Järvelin, K., & Kekäläinen, J. (2002) · ACM Transactions on Information Systems · doi:10.1145/582415.582418 · Cited by 4,841
Learning with Local and Global Consistency
Zhou, D., Bousquet, O., Lal, T. N., Weston, J., & Schölkopf, B. (2003) · MPG.PuRe (Max Planck Society) · Cited by 3,864
Locality-constrained Linear Coding for image classification
Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., & Gong, Y. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540018 · Cited by 3,100
Linear spatial pyramid matching using sparse coding for image classification
Yang, J., Yu, K., Gong, Y., & Huang, T. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206757 · Cited by 2,828
Efficient sparse coding algorithms
Lee, H., Battle, A., Raina, R., & Ng, A. Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0105 · Cited by 2,394
Non-local sparse models for image restoration
Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459452 · Cited by 1,666
Learning with Hypergraphs: Clustering, Classification, and Embedding
Zhou, D., Huang, J., & Schölkopf, B. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0205 · Cited by 1,283
A Survey on Multi-view Learning
Xu, C., Tao, D., & Xu, C. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1304.5634 · Cited by 942
Self-citationEarly versus late fusion in semantic video analysis
Snoek, C. G. M., Worring, M., & Smeulders, A. W. M. (2005) · Proceedings of the 13th annual ACM international conference on Multimedia · doi:10.1145/1101149.1101236 · Cited by 879
A survey of multi-view machine learning
Sun, S. (2013) · Neural Computing and Applications · doi:10.1007/s00521-013-1362-6 · Cited by 878
Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search
Joachims, T., Granka, L., Pan, B., Hembrooke, H., Radlinski, F., & Gay, G. (2007) · ACM Transactions on Information Systems · doi:10.1145/1229179.1229181 · Cited by 677
Linear spatial pyramid matching using sparse coding for image classification
Yang, J., Yu, K., Gong, Y., & Huang, T. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvprw.2009.5206757 · Cited by 496
Unified Video Annotation via Multigraph Learning
Wang, M., Hua, X.-S., Hong, R., Tang, J., Qi, G.-J., & Song, Y. (2009) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2009.2017400 · Cited by 441
Random walks on the click graph
Craswell, N., & Szummer, M. (2007) · Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1277741.1277784 · Cited by 441
Probabilistic graph and hypergraph matching
Zass, R., & Shashua, A. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587500 · Cited by 369
Laplacian Sparse Coding, Hypergraph Laplacian Sparse Coding, and Applications
Gao, S., Tsang, I. W.-H., & Chia, L.-T. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.63 · Cited by 357
Kernel Sparse Representation for Image Classification and Face Recognition
Gao, S., Tsang, I. W.-H., & Chia, L.-T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_1 · Cited by 345
Image retrieval via probabilistic hypergraph ranking
Huang, Y., Liu, Q., Zhang, S., & Metaxas, D. N. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540012 · Cited by 293
Hypergraph spectral learning for multi-label classification
Sun, L., Ji, S., & Ye, J. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401971 · Cited by 258
Video search reranking through random walk over document-level context graph
Hsu, W. H., Kennedy, L. S., & Chang, S.-F. (2007) · Proceedings of the 15th ACM international conference on Multimedia · doi:10.1145/1291233.1291446 · Cited by 226
Video search reranking via information bottleneck principle
Hsu, W. H., Kennedy, L. S., & Chang, S.-F. (2006) · Proceedings of the 14th ACM international conference on Multimedia · doi:10.1145/1180639.1180654 · Cited by 215
Positional relevance model for pseudo-relevance feedback
Lv, Y., & Zhai, C. (2010) · Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1835449.1835546 · Cited by 212
]Video object segmentation by hypergraph cut
Huang, Y., Liu, Q., & Metaxas, D. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206795 · Cited by 209
Online feature selection using grafting
Perkins, S. J., & Theiler, J. P. (2003) · Cited by 206
Eriksson, A., & van den Hengel, A. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540139 · Cited by 190
Learning an image manifold for retrieval
He, X., Ma, W.-Y., & Zhang, H.-J. (2004) · Proceedings of the 12th annual ACM international conference on Multimedia · doi:10.1145/1027527.1027532 · Cited by 179
A model to estimate intrinsic document relevance from the clickthrough logs of a web search engine
Dupret, G., & Liao, C. Y. (2010) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/1718487.1718510 · Cited by 146
Tian, Z., Hwang, T.-H., & Kuang, R. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp467 · Cited by 131
Learning to re-rank: query-dependent image re-ranking using click data
Jain, V., & Varma, M. (2011) · International World Wide Web Conference (WWW) · doi:10.1145/1963405.1963447 · Cited by 131
Evaluating Search Engines by Modeling the Relationship Between Relevance and Clicks
Carterette, B., & Jones, R. (2007) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · Cited by 122
Eriksson, A. P., & Hengel, A. V. D. (2010) · Cited by 97
Content-aware Ranking for visual search
Geng, B., Yang, L., Xu, C., & Hua, X. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540003 · Cited by 47
Tian, X., Yang, L., Wang, J., Wu, X., & Hua, X. (2011) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2011.2111363 · Cited by 46
Ensemble manifold regularization
Geng, B., Xu, C., Tao, D., Yang, L., & Hua, X. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206695 · Cited by 39
Self-citation
Cited by
A Survey of Multi-View Representation Learning
Li, Y., Yang, M., & Zhang, Z. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2872063 · Cited by 524
Hierarchical Deep Click Feature Prediction for Fine-Grained Image Recognition
Yu, J., Tan, M., Zhang, H., Rui, Y., & Tao, D. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2932058 · Cited by 498
Self-citationMulti-view low-rank sparse subspace clustering
Brbić, M., & Kopriva, I. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.08.024 · Cited by 483
Learning to Rank Using User Clicks and Visual Features for Image Retrieval
Yu, J., Tao, D., Wang, M., & Rui, Y. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2336697 · Cited by 445
Self-citationMulti-View Intact Space Learning
Xu, C., Tao, D., & Xu, C. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2417578 · Cited by 395
Self-citation
Metadata
- Authors
- Yu, J., Rui, Y., & Tao, D.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2014-05
- Volume
- 23
- Issue
- 5
- Pages
- 2019-2032
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W1972702299
- Cited by
- 550
- References
- 43
- FWCI
- 64.69
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.