Descriptive visual words and visual phrases for image applications
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
The Bag-of-visual Words (BoW) image representation has been applied for various problems in the fields of multimedia and computer vision. The basic idea is to represent images as visual documents composed of repeatable and distinctive visual elements, which are comparable to the words in texts. However, massive experiments show that the commonly used visual words are not as expressive as the text words, which is not desirable because it hinders their effectiveness in various applications. In this paper, Descriptive Visual Words (DVWs) and Descriptive Visual Phrases (DVPs) are proposed as the visual correspondences to text words and phrases, where visual phrases refer to the frequently co-occurring visual word pairs. Since images are the carriers of visual objects and scenes, novel descriptive visual element set can be composed by the visual words and their combinations which are effective in representing certain visual objects or scenes. Based on this idea, a general framework is proposed for generating DVWs and DVPs from classic visual words for various applications. In a large-scale image database containing 1506 object and scene categories, the visual words and visual word pairs descriptive to certain scenes or objects are identified as the DVWs and DVPs. Experiments show that the DVWs and DVPs are compact and descriptive, thus are more comparable with the text words than the classic visual words. We apply the identified DVWs and DVPs in several applications including image retrieval, image re-ranking, and object recognition. The DVW and DVP combination outperforms the classic visual words by 19.5% and 80% in image retrieval and object recognition tasks, respectively. The DVW and DVP based image re-ranking algorithm: DWPRank outperforms the state-of-the-art VisualRank by 12.4% in accuracy and about 11 times faster in efficiency.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 3×
“Similarly, large-scale labeled web image database: ImageNet is collected and released in [3].”
BackgroundMethodDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 3×
“(1). where, Scale i is the scale of the interest point [14] from which the instance of visual word i is computed, and P d is a parameter controlling the constraint of co-occurrence.”
BackgroundMethodInfluentialThe anatomy of a large-scale hypertextual Web search engine
Brin, S., & Page, L. (1998) · Computer Networks and ISDN Systems · doi:10.1016/s0169-7552(98)00110-x · Cited by 16,004 · cited 7×
“2) The idea of PageRank [2] and VisualRank [8] is leveraged in VisualWordRank for DVW selection.”
BackgroundMethodInfluentialRobust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
WordNet: An Electronic Lexical Database
Kilgarriff, A., & Fellbaum, C. (2000) · Language · doi:10.2307/417141 · Cited by 11,464
Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories
Lazebnik, S., Schmid, C., & Ponce, J. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.68 · Cited by 7,916 · cited 3×
“Bag-of-visual Words (BoW) image representation has been utilized for many multimedia and vision problems, including video event detection [27, 30, 34], object recognition [11, 12, 15, 18, 20, 21, 26], image segmentation [28, 31], and large-scale image retrieval [17, 22, 29], etc .”
BackgroundClustering by Passing Messages Between Data Points
Frey, B. J., & Dueck, D. (2007) · Science · doi:10.1126/science.1136800 · Cited by 6,758 · cited 1×
“Though some other clustering methods such as one-step K-means, Affinity Propagation [5] or some recent visual vocabulary generation methods [9, 10, 16, 19], could also be adopted, they are in general less efficient, in terms of either time or space complexity.”
MethodVideo Google: a text retrieval approach to object matching in videos
Sivic, & Zisserman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238663 · Cited by 6,367
WordNet: An Electronic Lexical Database
(1998) · The MIT Press eBooks · doi:10.7551/mitpress/7287.001.0001 · Cited by 4,857
Scalable Recognition with a Vocabulary Tree
Nister, D., & Stewenius, H. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.264 · Cited by 3,579 · cited 10×
“2) The performance will be saturated when the number of visual words reaches certain levels [12, 17, 20].”
BackgroundMethodResultInfluential(1999) · Cited by 3,186
80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition
Torralba, A., Fergus, R., & Freeman, W. T. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.128 · Cited by 1,921
Fisher Kernels on Visual Vocabularies for Image Categorization
Perronnin, F., & Dance, C. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383266 · Cited by 1,641
Object categorization by learned universal visual dictionary
Winn, J., Criminisi, A., & Minka, T. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.171 · Cited by 838
Creating efficient codebooks for visual recognition
Jurie, F., & Triggs, B. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.66 · Cited by 713 · cited 3×
“Though some other clustering methods such as one-step K-means, Affinity Propagation [5] or some recent visual vocabulary generation methods [9, 10, 16, 19], could also be adopted, they are in general less efficient, in terms of either time or space complexity.”
BackgroundMethodSimultaneous image classification and annotation
Chong, W., Blei, D., & Li, F.-F. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206800 · Cited by 510 · cited 3×
“Bag-of-visual Words (BoW) image representation has been utilized for many multimedia and vision problems, including video event detection [27, 30, 34], object recognition [11, 12, 15, 18, 20, 21, 26], image segmentation [28, 31], and large-scale image retrieval [17, 22, 29], etc .”
BackgroundVisualRank: Applying PageRank to Large-Scale Image Search
Jing, Y., & Baluja, S. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.121 · Cited by 441 · cited 6×
“2) The idea of PageRank [2] and VisualRank [8] is leveraged in VisualWordRank for DVW selection.”
BackgroundMethodInfluentialBundling features for large scale partial-duplicate web image search
Wu, Z., Ke, Q., Isard, M., & Sun, J. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206566 · Cited by 392 · cited 5×
“In a recent work [29], visual words are bundled for large-scale near-duplicate image retrieval.”
BackgroundMethodInfluentialRandomized Clustering Forests for Image Classification
Moosmann, F., Nowak, E., & Jurie, F. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70822 · Cited by 287 · cited 3×
“Though some other clustering methods such as one-step K-means, Affinity Propagation [5] or some recent visual vocabulary generation methods [9, 10, 16, 19], could also be adopted, they are in general less efficient, in terms of either time or space complexity.”
BackgroundMethodDiscovery of Collocation Patterns: from Visual Words to Visual Phrases
Yuan, J., Wu, Y. N., & Yang, M. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383222 · Cited by 260 · cited 1×
“Aiming at the first problem, lots of works are conducted to combine multiple visual words with spatial information [1, 12, 15, 20, 27, 29, 31-33].”
BackgroundVideo search reranking through random walk over document-level context graph
Hsu, W. H., Kennedy, L. S., & Chang, S.-F. (2007) · Proceedings of the 15th ACM international conference on Multimedia · doi:10.1145/1291233.1291446 · Cited by 226 · cited 1×
“Image re-ranking is a research problem catching more and more attentions in recent years [7, 8, 13, 23].”
BackgroundSupervised Learning of Quantizer Codebooks by Information Loss Minimization
Lazebnik, S., & Raginsky, M. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.138 · Cited by 221 · cited 4×
“Though some other clustering methods such as one-step K-means, Affinity Propagation [5] or some recent visual vocabulary generation methods [9, 10 , 16, 19], could also be adopted, they are in general less efficient, in terms of either time or space complexity.”
BackgroundMethodInfluentialDiscriminative Object Class Models of Appearance and Shape by Correlatons
Savarese, S., Winn, J., & Criminisi, A. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.102 · Cited by 197 · cited 9×
“2) Single visual word shows limited descriptive ability, no matter how fine-scaled it is [12, 17, 20].”
BackgroundInfluentialUniversal and Adapted Vocabularies for Generic Visual Categorization
Perronnin, F. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.70755 · Cited by 185
Spatial Weighting for Bag-of-Features
Marszaek, M., & Schmid, C. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.288 · Cited by 168 · cited 4×
“In previous works [12, 15, 17, 18, 20, 22, 27, 28, 30, 31, 34], various numbers of visual words are generated for different tasks.”
BackgroundResultInfluentialIntegrated feature selection and higher-order spatial feature extraction for object categorization
Liu, D., Hua, G., Viola, P. A., & Chen, T. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587403 · Cited by 151 · cited 10×
“2) Single visual word shows limited descriptive ability, no matter how fine-scaled it is [12, 17, 20].”
BackgroundInfluentialSelf-citationBayesian video search reranking
Tian, X., Yang, L., Wang, J., Yang, Y., Wu, X., & Hua, X. (2008) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1459359.1459378 · Cited by 134 · cited 1×
“Image re-ranking is a research problem catching more and more attentions in recent years [7, 8, 13, 23].”
BackgroundVideo Event Recognition Using Kernel Methods with Multilevel Temporal Alignment
Xu, D., & Chang, S. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.129 · Cited by 127 · cited 2×
“In previous works [12, 15, 17, 18, 20, 22, 27, 28, 30, 31, 34], various numbers of visual words are generated for different tasks.”
BackgroundMultiple Class Segmentation Using A Unified Framework over Mean-Shift Patches
Yang, L., Meer, P., & Foran, D. J. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383229 · Cited by 122 · cited 3×
“In previous works [12, 15, 17, 18, 20, 22, 27, 28, 30, 31, 34], various numbers of visual words are generated for different tasks.”
BackgroundSIFT-Bag kernel for video event analysis
Zhou, X., Zhuang, X., Yan, S., Chang, S.-F., Hasegawa-Johnson, M., & Huang, T. S. (2008) · Proceedings of the 16th ACM international conference on Multimedia · doi:10.1145/1459359.1459391 · Cited by 113 · cited 2×
“In previous works [12, 15, 17, 18, 20, 22, 27, 28, 30, 31, 34], various numbers of visual words are generated for different tasks.”
BackgroundVideo event detection using motion relativity and visual relatedness
Wang, F., Jiang, Y., & Ngo, C. (2008) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1459359.1459392 · Cited by 95 · cited 3×
“In previous works [12, 15, 17, 18, 20, 22, 27, 28, 30, 31, 34], various numbers of visual words are generated for different tasks.”
BackgroundVisual Synset: Towards a higher-level visual representation
Zheng, Y.-T., Zhao, M., Neo, S.-Y., Chua, T., & Tian, Q. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587611 · Cited by 84
Self-citationVideo search re-ranking via multi-graph propagation
Liu, J., Lai, W., Hua, X., Huang, Y., & Li, S. (2007) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1291233.1291279 · Cited by 71 · cited 1×
“Image re-ranking is a research problem catching more and more attentions in recent years [7, 8, 13, 23].”
BackgroundSelf-citationSpatial Hierarchy of Textons Distributions for Scene Classification
Battiato, S., Farinella, G. M., Gallo, G., & Ravì, D. (2009) · Lecture notes in computer science · doi:10.1007/978-3-540-92892-8_35 · Cited by 24
Learning mixed templates for object recognition
Si, Z., Gong, H., Wu, Y., & Zhu, S. C. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206770 · Cited by 22 · cited 3×
“Bag-of-visual Words (BoW) image representation has been utilized for many multimedia and vision problems, including video event detection [27, 30, 34], object recognition [11, 12, 15, 18, 20, 21, 26], image segmentation [28, 31], and large-scale image retrieval [17, 22, 29], etc .”
Background
Cited by
Scalable Person Re-identification: A Benchmark
Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., & Tian, Q. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.133 · Cited by 4,506
Self-citationSIFT Meets CNN: A Decade Survey of Instance Retrieval
Zheng, L., Yang, Y., & Tian, Q. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2709749 · Cited by 752
Self-citation
Metadata
- Authors
- Zhang, S., Tian, Q., Hua, G., Huang, Q., & Li, S.
- Venue
- Proceedings of the 17th ACM international conference on Multimedia
- Publisher
- ACM
- Published
- 2009-10-19
- Pages
- 75-84
- OpenAlex
- W2133671054
- Cited by
- 234
- References
- 35
- FWCI
- 23.17
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.