Learning Social Tag Relevance by Neighbor Voting
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Social image analysis and retrieval is important for helping people organize and access the increasing amount of user tagged multimedia. Since user tagging is known to be uncontrolled, ambiguous, and overly personalized, a fundamental problem is how to interpret the relevance of a user-contributed tag with respect to the visual content the tag is describing. Intuitively, if different persons label visually similar images using the same tags, these tags are likely to reflect objective aspects of the visual content. Starting from this intuition, we propose in this paper a neighbor voting algorithm which accurately and efficiently learns tag relevance by accumulating votes from visual neighbors. Under a set of well-defined and realistic assumptions, we prove that our algorithm is a good tag relevance measurement for both image ranking and tag ranking. Three experiments on 3.5 million Flickr photos demonstrate the general applicability of our algorithm in both social image retrieval and image tag suggestion. Our tag relevance learning algorithm substantially improves upon baselines for all the experiments. The results suggest that the proposed algorithm is promising for real-world applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Content-based image retrieval at the end of the early years
Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895972 · Cited by 6,056
Self-citationMultivariate Density Estimation: Theory, Practice, and Visualization
Scott, D. W. (1992) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316849 · Cited by 3,696
Image retrieval: Ideas, influences, and trends of the new age
Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2008) · ACM Computing Surveys · doi:10.1145/1348246.1348248 · Cited by 3,040
Multivariate Density Estimation, Theory, Practice, and Visualbation
Ramsay, P. H. (1993) · Technometrics · doi:10.1080/00401706.1993.10485361 · Cited by 2,789
80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition
Torralba, A., Fergus, R., & Freeman, W. T. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.128 · Cited by 1,921
Usage patterns of collaborative tagging systems
Golder, S. A., & Huberman, B. A. (2006) · Journal of Information Science · doi:10.1177/0165551506062337 · Cited by 1,817
Image indexing using color correlograms
Huang, J., Kumar, S. R., Mitra, M., Zhu, W.-J., & Zabih, R. (2002) · Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1997.609412 · Cited by 1,646
I tube, you tube, everybody tubes: analyzing the world's largest user generated content video system
Cha, M., Kwak, H., Rodriguez, P., Ahn, Y.-Y., & Moon, S. (2007) · Proceedings of the 7th ACM SIGCOMM conference on Internet measurement · doi:10.1145/1298306.1298309 · Cited by 1,576
A probabilistic model of information retrieval: development and comparative experiments
Sparck Jones, K., Walker, S., & Robertson, S. E. (2000) · Information Processing & Management · doi:10.1016/s0306-4573(00)00015-7 · Cited by 942
Flickr tag recommendation based on collective knowledge
Sigurbjörnsson, B., & van Zwol, R. (2008) · Proceedings of the 17th international conference on World Wide Web · doi:10.1145/1367497.1367542 · Cited by 922
Supervised Learning of Semantic Classes for Image Annotation and Retrieval
Carneiro, G., Chan, A. B., Moreno, P. J., & Vasconcelos, N. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.61 · Cited by 859
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244303322533214 · Cited by 615
A study on tolerable waiting time: how long are Web users willing to wait?
Nah, F. F.-H. (2004) · Behaviour & Information Technology · doi:10.1080/01449290410001669914 · Cited by 593
Real-Time Computerized Annotation of Pictures
Li, J., & Wang, J. Z. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70847 · Cited by 506
CBSA: content-based soft annotation for multimodal image retrieval using bayes point machines
Chang, E., Goh, K., Sychay, G., & Wu, G. (2003) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2002.808079 · Cited by 450
VisualRank: Applying PageRank to Large-Scale Image Search
Jing, Y., & Baluja, S. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.121 · Cited by 441
Automated Tag Clustering: Improving search and exploration in the tag space
Begelman, G., Keller, P., & Smadja, F. (2006) · Cited by 419
Kennedy, L., Naaman, M., Ahern, S., Nair, R., & Rattenbury, T. (2007) · Proceedings of the 15th ACM international conference on Multimedia · doi:10.1145/1291233.1291384 · Cited by 385
A Discriminative Kernel-Based Approach to Rank Images from Text Queries
Grangier, D., & Bengio, S. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70791 · Cited by 318
A probabilistic model of information retrieval: development and comparative experiments
Sparck Jones, K., Walker, S., & Robertson, S. E. (2000) · Information Processing & Management · doi:10.1016/s0306-4573(00)00016-9 · Cited by 277
A Visual Category Filter for Google Images
Fergus, R., Perona, P., & Zisserman, A. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24670-1_19 · Cited by 258
Cusano, C., Ciocca, G., & Schettini, R. (2003) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.526746 · Cited by 258
Image annotations by combining multiple evidence & wordNet
Jin, Y., Khan, L., Wang, L., & Awad, M. (2005) · Proceedings of the 13th annual ACM international conference on Multimedia · doi:10.1145/1101149.1101305 · Cited by 240
Video search reranking via information bottleneck principle
Hsu, W. H., Kennedy, L. S., & Chang, S.-F. (2006) · Proceedings of the 14th ACM international conference on Multimedia · doi:10.1145/1180639.1180654 · Cited by 215
Adding Semantics to Detectors for Video Retrieval
Snoek, C. G. M., Huurnink, B., Hollink, L., de Rijke, M., Schreiber, G., & Worring, M. (2007) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2007.900156 · Cited by 195
Self-citationAnnotating Images by Mining Image Search Results
Wang, X.-J., Zhang, L., Li, X., & Ma, W.-Y. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.127 · Cited by 194
Self-citationCan High-Level Concepts Fill the Semantic Gap in Video Retrieval? A Case Study With Broadcast News
Hauptmann, A., Yan, R., Lin, W.-H., Christel, M., & Wactlar, H. (2007) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2007.900150 · Cited by 190
Quelhas, P., Monay, F., Odobez, J.-M., Gatica-Perez, D., & Tuytelaars, T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1155 · Cited by 185
Semantic concept-based query expansion and re-ranking for multimedia retrieval
Natsev, A. (., Haubold, A., Tešić, J., Xie, L., & Yan, R. (2007) · Proceedings of the 15th ACM international conference on Multimedia · doi:10.1145/1291233.1291448 · Cited by 183
Weinberger, K. Q., Slaney, M., & van Zwol, R. (2008) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1459359.1459375 · Cited by 126
Image retrieval on large-scale image databases
Hörster, E., Lienhart, R. W., & Slaney, M. (2007) · ACM International Conference on Image and Video Retrieval (CIVR) · doi:10.1145/1282280.1282283 · Cited by 92
Questioning query expansion: an examination of behaviour and parameters
Billerbeck, B., & Zobel, J. (2004) · Cited by 83
Watch what I watch: using community activity to understand content
Shamma, D. A., Shaw, R. B., Shafton, P. L., & Liu, Y. (2007) · Multimedia Information Retrieval · doi:10.1145/1290082.1290120 · Cited by 80
Toward Bridging the Annotation-Retrieval Gap in Image Search
Datta, R., Ge, W., Li, J., & Wang, J. Z. (2007) · IEEE Multimedia · doi:10.1109/mmul.2007.67 · Cited by 51
Scalable search-based image annotation
Wang, C., Jing, F., Zhang, L., & Zhang, H.-J. (2008) · Multimedia Systems · doi:10.1007/s00530-008-0128-y · Cited by 44
Cited by
Visual-Textual Joint Relevance Learning for Tag-Based Social Image Search
Gao, Y., Wang, M., Zha, Z.-J., Shen, J., Li, X., & Wu, X. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2202676 · Cited by 438
Self-citationDeep Collaborative Embedding for Social Image Understanding
Li, Z., Tang, J., & Mei, T. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2852750 · Cited by 404
Semantic Image Analysis on Social Networks and Data Processing: Review and Future Directions
Alzboon, M. S., Alqaraleh, M. K., Aljarrah, E. M., & Alomari, S. A. (2022) · Advances in Business Information Systems and Analytics · doi:10.4018/978-1-7998-9016-4.ch009 · Cited by 1
Metadata
- Authors
- Li, X., Snoek, C. G. M., & Worring, M.
- Venue
- IEEE Transactions on Multimedia
- Published
- 2009-11
- Volume
- 11
- Issue
- 7
- Pages
- 1310-1322
- ISSN
- 1520-9210 (ISSN-L)
- 1941-0077
- OpenAlex
- W2161258050
- Cited by
- 378
- References
- 35
- FWCI
- 35.55
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.