The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Deep Collaborative Embedding for Social Image Understanding
Abstract
In this work, we investigate the problem of learning knowledge from the massive community-contributed images with rich weakly-supervised context information, which can benefit multiple image understanding tasks simultaneously, such as social image tag refinement and assignment, content-based image retrieval, tag-based image retrieval and tag expansion. Towards this end, we propose a Deep Collaborative Embedding (DCE) model to uncover a unified latent space for images and tags. The proposed method incorporates the end-to-end learning and collaborative factor analysis in one unified framework for the optimal compatibility of representation learning and latent space discovery. A nonnegative and discrete refined tagging matrix is learned to guide the end-to-end learning. To collaboratively explore the rich context information of social images, the proposed method integrates the weakly-supervised image-tag correlation, image correlation and tag correlation simultaneously and seamlessly. The proposed model is also extended to embed new tags in the uncovered space. To verify the effectiveness of the proposed method, extensive experiments are conducted on two widely-used social image benchmarks for multiple social image understanding tasks. The encouraging performance of the proposed method over the state-of-the-art approaches demonstrates its superiority.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Learning the parts of objects by non-negative matrix factorization
Lee, D. D., & Seung, H. S. (1999) · Nature · doi:10.1038/44565 · Cited by 13,868
Content-based image retrieval at the end of the early years
Smeulders, A. W. M., Worring, M., Santini, S., Gupta, A., & Jain, R. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895972 · Cited by 6,056
Probabilistic Matrix Factorization
Mnih, A., & Salakhutdinov, R. (2007) · neural information processing systems · Cited by 3,798
Image retrieval: Ideas, influences, and trends of the new age
Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2008) · ACM Computing Surveys · doi:10.1145/1348246.1348248 · Cited by 3,040
NUS-WIDE: a real-world web image database from National University of Singapore
Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., & Zheng, Y. (2009) · Proceedings of the ACM International Conference on Image and Video Retrieval · doi:10.1145/1646396.1646452 · Cited by 3,025
Self-citationGraph Regularized Nonnegative Matrix Factorization for Data Representation
Cai, D., He, X., Han, J., & Huang, T. S. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.231 · Cited by 2,451
The MIR flickr retrieval evaluation
Huiskes, M. J., & Lew, M. S. (2008) · Proceedings of the 1st ACM international conference on Multimedia information retrieval · doi:10.1145/1460096.1460104 · Cited by 1,836
Image Captioning with Semantic Attention
You, Q., Jin, H., Wang, Z., Fang, C., & Luo, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.503 · Cited by 1,803
The Google Similarity Distance
Cilibrasi, R. L., & Vitanyi, P. M. B. (2007) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.48 · Cited by 1,760
Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary
Duygulu, P., Barnard, K., de Freitas, J. F. G., & Forsyth, D. A. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-47979-1_7 · Cited by 1,608
Deep Canonical Correlation Analysis
Andrew, G., Arora, R., Bilmes, J., & Livescu, K. (2013) · International Conference on Machine Learning · Cited by 1,602
Bayesian probabilistic matrix factorization using Markov chain Monte Carlo
Salakhutdinov, R., & Mnih, A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390267 · Cited by 1,413
A new approach to cross-modal multimedia retrieval
Rasiwasia, N., Costa Pereira, J., Coviello, E., Doyle, G., Lanckriet, G. R. G., Levy, R., & Vasconcelos, N. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1873987 · Cited by 1,334
Probabilistic Matrix Factorization
Salakhutdinov, R., & Mnih, A. (2007) · Cited by 1,075
Deep Visual-Semantic Alignments for Generating Image Descriptions
Karpathy, A., & Fei-Fei, L. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2598339 · Cited by 1,048
Supervised Learning of Semantic Classes for Image Annotation and Retrieval
Carneiro, G., Chan, A. B., Moreno, P. J., & Vasconcelos, N. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.61 · Cited by 859
Variational Autoencoder for Deep Learning of Images, Labels and Captions
Pu, Y., Gan, Z., Henao, R., Yuan, X., Li, C., Stevens, A., & Carin, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.08976 · Cited by 562
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics
Gong, Y., Ke, Q., Isard, M., & Lazebnik, S. (2014) · International Journal of Computer Vision · doi:10.1007/s11263-013-0658-4 · Cited by 540
Constrained Nonnegative Matrix Factorization for Image Representation
Liu, H., Wu, Z., Cai, D., & Huang, T. S. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.217 · Cited by 520
Barnard, K., Duygulu, P., Forsyth, D. A., de Freitas, N., Blei, D. M., Jordan, M. I., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2003) · Cited by 459
Harvey, D., & Moran, S. (2009) · Cited by 439
On the Role of Correlation and Abstraction in Cross-Modal Multimedia Retrieval
Costa Pereira, J., Coviello, E., Doyle, G., Rasiwasia, N., Lanckriet, G. R. G., Levy, R., & Vasconcelos, N. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.142 · Cited by 435
Learning Social Tag Relevance by Neighbor Voting
Li, X., Snoek, C. G. M., & Worring, M. (2009) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2009.2030598 · Cited by 378
Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation
Shin, H.-C., Roberts, K., Lu, L., Demner-Fushman, D., Yao, J., & Summers, R. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.274 · Cited by 373
A Deep Matrix Factorization Method for Learning Attribute Representations
Trigeorgis, G., Bousmalis, K., Zafeiriou, S., & Schuller, B. W. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2554555 · Cited by 356
Robust Structured Subspace Learning for Data Representation
Li, Z., Liu, J., Tang, J., & Lu, H. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2400461 · Cited by 353
Self-citationSun, L., Ji, S., & Ye, J. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.160 · Cited by 299
Robust Structured Nonnegative Matrix Factorization for Image Representation
Li, Z., Tang, J., & He, X. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2691725 · Cited by 257
Self-citationImage tag refinement towards low-rank, content-tag prior and error sparsity
Zhu, G., Yan, S., & Ma, Y. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874028 · Cited by 243
Tag Completion for Image Retrieval
Wu, L., Jin, R., & Jain, A. K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.124 · Cited by 238
Weakly Supervised Deep Matrix Factorization for Social Image Understanding
Li, Z., & Tang, J. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2624140 · Cited by 211
Self-citationWeakly Supervised Deep Metric Learning for Community-Contributed Image Retrieval
Li, Z., & Tang, J. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2477035 · Cited by 197
Self-citationZha, Z.-J., Yang, L., Mei, T., Wang, M., & Wang, Z. (2009) · Proceedings of the 17th ACM international conference on Multimedia · doi:10.1145/1631272.1631278 · Cited by 179
Self-citationLi, X., Uricchio, T., Ballan, L., Bertini, M., Snoek, C. G. M., & Bimbo, A. D. (2017) · ACM Computing Surveys · doi:10.1145/2906152 · Cited by 173
Matrix Completion for Weakly-Supervised Multi-Label Image Classification
Cabral, R., De la Torre, F., Costeira, J. P., & Bernardino, A. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2343234 · Cited by 173
Exploring Context and Content Links in Social Media: A Latent Space Method
Qi, G.-J., Aggarwal, C. C., Tian, Q., Ji, H., & Huang, T. S. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.191 · Cited by 158
Learning the Relative Importance of Objects from Tagged Images for Retrieval and Cross-Modal Search
Hwang, S. J., & Grauman, K. L. (2011) · International Journal of Computer Vision · doi:10.1007/s11263-011-0494-3 · Cited by 128
Weinberger, K. Q., Slaney, M., & van Zwol, R. (2008) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1459359.1459375 · Cited by 126
Love Thy Neighbors: Image Annotation by Exploiting Image Metadata
Johnson, J., Ballan, L., & Fei-Fei, L. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.525 · Cited by 117
Learning from Weak and Noisy Labels for Semantic Segmentation
Lu, Z., Fu, Z., Xiang, T., Han, P., Wang, L., & Gao, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2552172 · Cited by 115
Liu, D., Hua, X., Wang, M., & Zhang, H.-J. (2010) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1873951.1874031 · Cited by 110
Nonnegative shared subspace learning and its application to social media retrieval
Gupta, S., Phung, D., Adams, B., Tran, T., & Venkatesh, S. (2010) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1835804.1835951 · Cited by 100
Image Tag Completion via Image-Specific and Tag-Specific Linear Sparse Reconstructions
Lin, Z., Ding, G., Hu, M., Wang, J., & Ye, X. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.212 · Cited by 93
Multi-Label Dictionary Learning for Image Annotation
Jing, X., Wu, F., Li, Z., Hu, R., & Zhang, D. D. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2549459 · Cited by 93
Self-citationImage Retagging Using Collaborative Tag Propagation
Liu, D., Yan, S., Hua, X., & Zhang, H.-J. (2011) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2011.2134078 · Cited by 74
Semantic Concept Co-Occurrence Patterns for Image Annotation and Retrieval
Feng, L., & Bhanu, B. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2469281 · Cited by 65
Tag-Based Image Retrieval Improved by Augmented Features and Group-Based Refinement
Lin, C., Xu, D., Tsang, I. W.-H., & Luo, J. (2012) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2012.2187435 · Cited by 62
Learning Visual Contexts for Image Annotation From Flickr Groups
Ulges, A., Worring, M., & Breuel, T. M. (2010) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2010.2101051 · Cited by 54
Image Tag Completion by Noisy Matrix Recovery
Feng, Z., Feng, S., Jin, R., & Jain, A. K. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10584-0_28 · Cited by 46
Joint Image Clustering and Labeling by Matrix Factorization
Hong, S., Choi, J., Feyereisl, J., Han, B., & Davis, L. S. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2487982 · Cited by 42
Scalable Nonlinear Embeddings for Semantic Category-Based Image Retrieval
Sharma, G., & Schiele, B. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.153 · Cited by 19
Cited by
Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey
Jelodar, H., Wang, Y., Yuan, C., Feng, X., Jiang, X., Li, Y., & Zhao, L. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6894-4 · Cited by 1,961
A Bottom-Up Clustering Approach to Unsupervised Person Re-Identification
Lin, Y., Dong, X., Zheng, L., Yan, Y., & Yang, Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018738 · Cited by 564
A Survey of Community Detection Approaches: From Statistical Modeling to Deep Learning
Jin, D., Yu, Z., Jiao, P., Pan, S., He, D., Wu, J., Yu, P., & Zhang, W. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3104155 · Cited by 397
Deep Learning-Based Parkinson’s Disease Classification Using Vocal Feature Sets
Gunduz, H. (2019) · IEEE Access · doi:10.1109/access.2019.2936564 · Cited by 360
Content‐Based Image Retrieval and Feature Extraction: A Comprehensive Review
Latif, A., Rasheed, A., Sajid, U., Ahmed, J., Ali, N., Ratyal, N. I., Zafar, B., Dar, S. H., Sajid, M., & Khalil, T. (2019) · Mathematical Problems in Engineering · doi:10.1155/2019/9658350 · Cited by 347
Metadata
- Authors
- Li, Z., Tang, J., & Mei, T.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2019-09-01
- Volume
- 41
- Issue
- 9
- Pages
- 2070-2083
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2832876791
- Cited by
- 404
- References
- 53
- FWCI
- 22.68
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.