Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-Modal Pretraining
Open accessAbstract
Nowadays, customer’s demands for E-commerce are more diversified, which introduces more complications to the product retrieval industry. Previous methods are either subject to single-modal input or perform supervised image-level product retrieval, thus fail to accommodate real-life scenarios where enormous weakly annotated multi-modal data are present. In this paper, we investigate a more realistic setting that aims to perform weakly-supervised multi-modal instance-level product retrieval among fine-grained product categories. To promote the study of this challenging task, we contribute Product1M, one of the largest multi-modal cosmetic datasets for real-world instance-level retrieval. Notably, Product1M contains over 1 million image-caption pairs and consists of two sample types, i.e., single-product and multi-product samples, which encompass a wide variety of cosmetics brands. In addition to the great diversity, Product1M enjoys several appealing characteristics including fine-grained categories, complex combinations, and fuzzy correspondence that well mimic the real-world scenes. Moreover, we propose a novel model named Cross-modal contrAstive Product Transformer for instance-level prodUct REtrieval (CAPTURE), that excels in capturing the potential synergy between multi-modal inputs via a hybrid-stream transformer in a self-supervised manner. CAPTURE generates discriminative instance features via masked multi-modal learning as well as cross-modal contrastive pretraining and it outperforms several SOTA cross-modal baselines. Extensive ablation studies well demonstrate the effectiveness and the generalization capacity of our model. Dataset and codes are available at https: //github.com/zhanxlin/Product1M.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recently, Transformer based alignment has led to a surge of leveraging large quantities of web data for vision and language tasks.
“Recently, Transformer based alignment [9], [119], [260], [261], [262] has led to a surge of leveraging large quantities of web data (e.g., image-text pairs) for vision and language tasks.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [119]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 56,035 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857 · see citing works
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107 · see citing works
Places: A 10 Million Image Database for Scene Recognition
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., & Torralba, A. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2723009 · Cited by 4,042 · see citing works
LXMERT: Learning Cross-Modality Encoder Representations from Transformers
Tan, H., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1514 · Cited by 2,319 · see citing works
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109 · see citing works
UNITER: UNiversal Image-TExt Representation Learning
Chen, Y.-C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58577-8_7 · Cited by 2,055 · see citing works
VisualBERT: A Simple and Performant Baseline for Vision and Language
Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019) · arXiv · doi:10.48550/arxiv.1908.03557 · Cited by 1,198 · see citing works
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J. (2019) · arXiv · doi:10.48550/arxiv.1908.08530 · Cited by 950 · see citing works
Large-Scale Image Retrieval with Attentive Deep Local Features
Noh, H., Araújo, A. D. M., Sim, J., Weyand, T., & Han, B. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.374 · Cited by 851 · see citing works
Adversarial Cross-Modal Retrieval
Wang, B., Yang, Y., Xu, X., Hanjalic, A., & Shen, H. T. (2017) · Proceedings of the 25th ACM international conference on Multimedia · doi:10.1145/3123266.3123326 · Cited by 779 · see citing works
Cut, Paste and Learn: Surprisingly Easy Synthesis for Instance Detection
Dwibedi, D., Misra, I., & Hebert, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.146 · Cited by 687 · see citing works
Cross-modal Retrieval with Correspondence Autoencoder
Feng, F., Wang, X., & Li, R. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2654902 · Cited by 572 · see citing works
Medical image retrieval using deep convolutional neural network
Qayyum, A., Anwar, S. M., Awais, M., & Majid, M. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.05.025 · Cited by 433 · see citing works
PCL: Proposal Cluster Learning for Weakly Supervised Object Detection
Tang, P., Wang, X., Bai, S., Shen, W., Bai, X., Liu, W., & Yuille, A. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2876304 · Cited by 406 · see citing works
Triplet-Based Deep Hashing Network for Cross-Modal Retrieval
Deng, C. D., Chen, Z., Liu, X., Gao, X., & Tao, D. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2821921 · Cited by 402 · see citing works
Cross-Modal Retrieval With CNN Visual Features: A New Baseline
Wei, Y., Zhao, Y., Lu, C., Wei, S., Liu, L., Zhu, Z., & Yan, S. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2519449 · Cited by 376 · see citing works
Self-citationJoint Feature Selection and Subspace Learning for Cross-Modal Retrieval
Wang, K., He, R., Wang, L., Wang, W., & Tan, T. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2505311 · Cited by 318 · see citing works
Google Landmarks Dataset v2 – A Large-Scale Benchmark for Instance-Level Recognition and Retrieval
Weyand, T., Araújo, A. D. M., Cao, B., & Sim, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00265 · Cited by 309 · see citing works
Visually-Aware Fashion Recommendation and Design with Generative Image Models
Kang, W.-C., Chen, F., Wang, Z., & McAuley, J. (2017) · Proceedings · doi:10.1109/icdm.2017.30 · Cited by 294 · see citing works
Deep Visual-Semantic Hashing for Cross-Modal Retrieval
Cao, Y., Long, M., Wang, J., Yang, Q., & Yu, P. S. (2016) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2939672.2939812 · Cited by 259 · see citing works
A Comprehensive Survey on Cross-modal RetrievalPreprint
Wang, K., Yin, Q., Wang, W., Wu, S., & Wang, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.06215 · Cited by 230 · see citing works
Sketch-based image retrieval via Siamese convolutional neural network
Qi, Y., Song, Y.-Z., Zhang, H., & Liu, J. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7532801 · Cited by 224 · see citing works
Information Retrieval: Implementing and Evaluating Search Engines
(2011) · The Electronic Library · doi:10.1108/02640471111188088 · Cited by 192 · see citing works
Vogler, S., Vitry, A. I., & Babar, Z. (2015) · The Lancet Oncology · doi:10.1016/s1470-2045(15)00449-0 · Cited by 175 · see citing works
Improving web image search results using query-relative classifiers
Krapac, J., Allan, M., Verbeek, J. J., & Jurie, F. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540092 · Cited by 125 · see citing works
Effective deep learning-based multi-modal retrieval
Wang, W., Yang, X., Ooi, B. C., Zhang, D., & Zhuang, Y. (2015) · The VLDB Journal · doi:10.1007/s00778-015-0391-4 · Cited by 121 · see citing works
Optimization of deep convolutional neural network for large scale image retrieval
Bai, C., Huang, L., Pan, X., Zheng, J., & Chen, S. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.04.034 · Cited by 117 · see citing works
RPC: A Large-Scale Retail Product Checkout DatasetPreprint
Wei, X.-S., Cui, Q., Yang, L., Wang, P., & Liu, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.07249 · Cited by 111 · see citing works
Fashion Retrieval via Graph Reasoning Networks on a Similarity Pyramid
Kuang, Z., Gao, Y., Li, G., Luo, P., Chen, Y., Lin, L., & Zhang, W. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00316 · Cited by 96 · see citing works
Leveraging Weakly Annotated Data for Fashion Image Retrieval and Label Prediction
Corbière, C., Ben-younes, H., Ramé, A., & Ollion, C. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.266 · Cited by 93 · see citing works
Document Retrieval Model Through Semantic Linking
Ensan, F., & Bagheri, E. (2017) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3018661.3018692 · Cited by 83 · see citing works
Complete the Look: Scene-Based Complementary Product Recommendation
Kang, W.-C., Kim, E. S., Leskovec, J., Rosenberg, C., & McAuley, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01078 · Cited by 81 · see citing works
Generative Adversarial Learning Towards Fast Weakly Supervised Detection
Shen, Y., Ji, R., Zhang, S., Zuo, W., & Wang, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00604 · Cited by 79 · see citing works
Cross-Domain Image Retrieval with Attention Modeling
Ji, X., Wang, W., Zhang, M., & Yang, Y. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123429 · Cited by 78 · see citing works
Twitter100k: A Real-World Dataset for Weakly Supervised Cross-Media Retrieval
Hu, Y., Zheng, L., Yang, Y., & Huang, Y. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2760101 · Cited by 71 · see citing works
Cap2Det: Learning to Amplify Weak Caption Supervision for Object Detection
Ye, K., Zhang, M., Kovashka, A., Li, W., Qin, D., & Berent, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00978 · Cited by 46 · see citing works
Multi-Modal and Multi-Domain Embedding Learning for Fashion Retrieval and Analysis
Gu, X., Wong, Y., Shou, L., Peng, P., Chen, G., & Kankanhalli, M. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2876822 · Cited by 45 · see citing works
Information Retrieval: The Early Years
Harman, D. (2019) · Foundations and Trends in Information Retrieval · doi:10.1561/1500000065 · Cited by 45 · see citing works
Product retrieval for grocery stores
Nurmi, P., Lagerspetz, E., Buntine, W., Floréen, P., & Kukkonen, J. (2008) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1390334.1390501 · Cited by 27 · see citing works
Lin, Z., Yang, Z., Huang, F., & Chen, J. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3266436 · Cited by 27 · see citing works
A Systematic Review of Recommender System for e-Portfolio Domain
Rahayu, P., Sensuse, D. I., Purwandari, B., Budi, I. D., Khalid, F., & Zulkarnaim, N. (2017) · International Conference on Information and Education Technology (ICIET) · doi:10.1145/3029387.3029420 · Cited by 25 · see citing works
A Revisit on Deep Hashings for Large-scale Content Based Image RetrievalPreprint
Cai, D., Gu, X., & Wang, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.06016 · Cited by 13 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072 · see citing works
Large-scale Multi-modal Pre-trained Models: A Comprehensive Survey
Wang, X., Chen, G., Qian, G., Gao, P., Wei, X.-Y., Wang, Y., Tian, Y., & Gao, W. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1410-8 · Cited by 245 · see citing works
DepMSTAT: Multimodal Spatio-Temporal Attentional Transformer for Depression Detection
Tao, Y., Yang, M., Li, H., Wu, Y., & Hu, B. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3350071 · Cited by 114 · see citing works
Self-citationMultimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications
Manzoor, M. A., Albarri, S., Xian, Z., Meng, Z., Nakov, P., & Liang, S. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3617833 · Cited by 76 · see citing works
EI-CLIP: Entity-aware Interventional Contrastive Learning for E-commerce Cross-modal Retrieval
Ma, H., Zhao, H., Lin, Z., Kale, A., Wang, Z., Yu, T., Gu, J., Choudhary, S., & Xie, X. S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01752 · Cited by 66 · see citing works
Cross-Modal Learning for Anomaly Detection in Complex Industrial Process: Methodology and Benchmark
Wu, G., Zhang, Y., Deng, L., Zhang, J., & Chai, T. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3491865 · Cited by 49 · see citing works
Self-citationEspina-Romero, L. C., Sánchez, J. G. N., Hurtado, H. G., Conde, H. O. D., Castro, Y. S., Cajo, L. E. C., & Corredoira, J. R. (2023) · Sustainability · doi:10.3390/su151612176 · Cited by 45 · see citing works
Zhu, H., Wei, Y., Liang, X., Zhang, C., & Zhao, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02034 · Cited by 31 · see citing works
Self-citationM5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretraining
Dong, X. M., Zhan, X., Wu, Y., Wei, Y., Kampffmeyer, M., Wei, X.-Y., Lu, M., Wang, Y., & Liang, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02057 · Cited by 33 · see citing works
Self-citationHe, X., Fang, Y., Xiang, N., Li, Z., Wang, Q., Yang, C., Wang, H. H., & Pan, Y. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129532 · Cited by 27 · see citing works
A Feature-space Multimodal Data Augmentation Technique for Text-video Retrieval
Falcon, A., Serra, G., & Lanz, O. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548365 · Cited by 25 · see citing works
Comprehensive Relationship Reasoning for Composed Query Based Image Retrieval
Zhang, F., Yan, M., Zhang, J., & Xu, C. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548126 · Cited by 21 · see citing works
Ticino: A multi-modal remote sensing dataset for semantic segmentation
Barbato, M. P., Piccoli, F., & Napoletano, P. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123600 · Cited by 21 · see citing works
Yu, X., Elazab, A., Ge, R., Zhu, J., Zhang, L., Jia, G., Wu, Q., Wan, X., Li, L., & Wang, C. (2025) · Neural Networks · doi:10.1016/j.neunet.2024.107096 · Cited by 18 · see citing works
Token Embeddings Alignment for Cross-Modal Retrieval
Xie, C.-W., Wu, J., Zheng, Y. P., Pan, P., & Hua, X. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548107 · Cited by 17 · see citing works
Multimodal false information detection method based on Text-CNN and SE module
Liang, Y., Tohti, T., & Hamdulla, A. (2022) · PLoS ONE · doi:10.1371/journal.pone.0277463 · Cited by 17 · see citing works
Cross-modal hashing with missing labels
Ni, H., Zhang, J., Kang, P., Fang, X., Sun, W., Xie, S., & Han, N. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.05.035 · Cited by 16 · see citing works
Real20M: A Large-scale E-commerce Dataset for Cross-domain Retrieval
Chen, Y., Zhong, H., He, X., Peng, Y., & Cheng, L. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612408 · Cited by 16 · see citing works
Modality-aware Transformer for Financial Time series Forecasting
Emami-Gohari, H., Dang, X. H., Shah, S. Y., & Zerfos, P. (2024) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3677052.3698654 · Cited by 16 · see citing works
SPIRIT: Style-guided Patch Interaction for Fashion Image Retrieval with Text Feedback
Chen, Y., Zhou, J., & Peng, Y. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3640345 · Cited by 15 · see citing works
MiC: Image-text Matching in Circles with cross-modal generative knowledge enhancement
Pu, X., Chen, Y., Yuan, L., Zhang, Y., Li, H., Jing, L., & Gao, X. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111503 · Cited by 14 · see citing works
Self-citationDeep Multigraph Hierarchical Enhanced Semantic Representation for Cross-Modal Retrieval
Zhu, L., Zhang, C., Song, J., Zhang, S., Tian, C., & Zhu, X. (2022) · IEEE Multimedia · doi:10.1109/mmul.2022.3144138 · Cited by 14 · see citing works
Entity-Graph Enhanced Cross-Modal Pretraining for Instance-level Product Retrieval
Dong, X. M., Zhan, X., Wei, Y., Wei, X.-Y., Wang, Y., Lu, M., Cao, X., & Liang, X. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3291237 · Cited by 13 · see citing works
Self-citationMultimodal Pre-Training with Self-Distillation for Product Understanding in E-Commerce
Liu, S., Li, L., Song, J., Yang, Y., & Zeng, X. (2023) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3539597.3570423 · Cited by 13 · see citing works
Automatic Check-Out via Prototype-Based Classifier Learning from Single-Product Exemplars
Chen, H., Wei, X.-S., Zhang, F., Shen, Y., Xu, H., & Xiao, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19806-9_16 · Cited by 7 · see citing works
Self-citationILIAS: Instance-Level Image retrieval At Scale
Kordopatis-Zilos, G., Stojnić, V., Manko, A., Šuma, P., Ypsilantis, N.-A., Efthymiadis, N., Laskar, Z., Matas, J., Chum, O., & Tolias, G. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01377 · Cited by 6 · see citing works
M5Product: A Multi-modal Pretraining Benchmark for E-commercial Product Downstream TasksPreprint
Dong, X. M., Zhan, X., Wu, Y., Wei, Y., & Liang, X. (2021) · arXiv (Cornell University) · doi:10.13140/rg.2.2.35620.81281 · Cited by 6 · see citing works
Self-citationPrototype Learning for Automatic Check-Out
Chen, H., Wei, X.-S., & Xiao, L. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3247219 · Cited by 5 · see citing works
Adaptive Multi-view Rule Discovery for Weakly-Supervised Compatible Products Prediction
Zhang, R., West, R., Cui, X., & Zhang, C. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539208 · Cited by 6 · see citing works
Cross-Domain Product Representation Learning for Rich-Content E-Commerce
Bai, X., Li, Y., Cheng, Y., Wenjie, Y., Chen, Q., & Li, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00524 · Cited by 4 · see citing works
Enhancing Weakly Supervised Multimodal Video Anomaly Detection Through Text Guidance
Sun, S., Hua, J., Feng, J., & Gong, X. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3668927 · Cited by 3 · see citing works
VERD: Emergence of Product-Based Video E-Commerce Retrieval Dataset from User’s Perspective
Lee, G., Jo, W., & Choi, Y. (2023) · Sensors · doi:10.3390/s23010513 · Cited by 3 · see citing works
Multi-label adversarial fine-grained cross-modal retrieval
Sun, C., Zhang, H., Liu, L., Liu, D., & Wang, L. (2023) · Signal Processing Image Communication · doi:10.1016/j.image.2023.117018 · Cited by 3 · see citing works
Semi-supervised Prototype Semantic Association Learning for Robust Cross-modal Retrieval
Wang, J., Gong, T., & Yan, Y. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657756 · Cited by 3 · see citing works
Scaling Object-centric Robotic Manipulation with Multimodal Object Identification
Mitash, C., Hussein, M., Vanbaar, J., Terhuja, V., & Katyal, K. D. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611181 · Cited by 3 · see citing works
Hierarchical Fusion Transformer for Multimodal Ground-Based Cloud Type Classification
Liu, S., Yu, Z., Zhang, Z., Shi, C., & Xiao, B. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3614756 · Cited by 2 · see citing works
Advances in the applications of machine learning for cosmetic formulation development
Jiang, W., Yao, L., Meng, Q., & Zhang, Q. (2026) · Chemical Papers · doi:10.1007/s11696-026-04696-7 · Cited by 2 · see citing works
Multi-modal Multi-emotion Emotional Support Conversation
Liu, G., Dong, X. M., Wang, M., Yu, J., Gan, M., Liu, W., & Yin, J. J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-46661-8_20 · Cited by 2 · see citing works
Self-citationDe-noised Vision-language Fusion Guided by Visual Cues for E-commerce Product Search
Hu, Z., Li, S., Du, M., Dhua, A., & Gray, D. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00204 · Cited by 2 · see citing works
NecklaceFIR: A Large Volume Benchmarked Necklace Dataset for Fashion Image Retrieval
Islam, S. M., Joardar, S., & Sekh, A. A. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-22485-0_17 · Cited by 2 · see citing works
MOON: Generative MLLM-based Multimodal Representation Learning for E-commerce Product Understanding
Zhang, D., Fu, C., Nie, Z., Liu, J., Guan, W., Gao, Y., Song, J., Wang, P., Xu, J., & Zheng, B. (2026) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3773966.3777958 · Cited by 1 · see citing works
A Comprehensive Survey of Cross-Modal Retrieval: Breaking Through Modal Barriers
Liu, X., Yu, Z., Yao, Y., Yang, K., Zeng, H., & Chen, C. L. P. (2026) · ACM Computing Surveys · doi:10.1145/3829354 · Cited by 1 · see citing works
Challenges of Self-Supervised Learning for Unified, Multi-Modal, Multi-Task Transformer Models
Annett, G., Andersen, T., & Annett, R. D. (2022) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci58124.2022.00056
Adapting Video Instance Segmentation for Instance Search
Nguyen, A. T. (2023) · International Workshop on Content-Based Multimedia Indexing (CBMI) · doi:10.1145/3617233.3617249
PDTW150K: A Dataset for Patent Drawing Retrieval
Hsu, C.-M., Lin, T.-H., Chen, Y., & Chiu, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56435-2_5
MVItem: A Benchmark for Multi-View Cross-Modal Item Retrieval
Li, B., Zhu, J., Dai, L., Jing, H., Huang, Z., & Sui, Y. (2024) · IEEE Access · doi:10.1109/access.2024.3447872
Stimulating conversation-style emergencies of multi-modal LMs
湯浅, 順., Liu, B., Sun, C., Xu, Z., & Wang, B. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103047
Liu, W., Zou, J., Zhang, S., Luo, N., & Liu, H. (2025) · Scientific Reports · doi:10.1038/s41598-025-07043-0
Edge-Embedded Multi-Feature Fusion Network for Automatic Checkout
Li, J., Zhu, M., & Ren, H. (2025) · Journal of Imaging · doi:10.3390/jimaging11100337
A KAN-Enhanced Siamese Transformer Pyramid Network for Multi-MSFA Demosaicing
Song, B., Wang, Z., Li, J., Du, W., Mao, N., Yang, F., Wang, X., Bao, Q., & Sun, W. (2025) · IEEE Sensors Journal · doi:10.1109/jsen.2025.3645135
Multimodal fusion and knowledge enhancement for accurate video captioning
Zhong, R., Zhang, Q., Li, H., & Zuo, M. (2026) · The Journal of Supercomputing · doi:10.1007/s11227-026-08284-0
Semi-solid metal oxide slag morphology detection based on cross-modal learning
Wu, J., & Xu, D. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114380
Multimodal deep learning for audiovisual production
Ouenniche, K. (2023) · theses.fr (ABES) · doi:10.70675/a01becf4z4ed9z4955z940cz49c821c1b8a8
Indexing Multimodal Language Models for Large-Scale Image Retrieval
Tharwat, B., Kordopatis-Zilos, G., Šuma, P., Reid, I. D., & Tolias, G. (2026) · doi:10.1109/cvprf71909.2026.00647
- Authors
- Zhan, X., Wu, Y., Dong, X., Wei, Y., Lu, M., Zhang, Y., Xu, H., & Liang, X.
- Affiliations
- Sun Yat-sen University (CN)
- Beijing Jiaotong University (CN)
- Alibaba Group (China) (CN)
- Huawei Technologies (China) (CN)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 11762-11771
- OpenAlex
- W3202384916
- Cited by
- 54
- References
- 52
- Citations by year
- 2026: 7 · 2025: 8 · 2024: 11 · 2023: 16 · 2022: 11 · 2021: 1
- FWCI
- 3.19
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Natural Science Foundation of China
- Shenzhen Fundamental Research Program
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- cross-modal retrieval, e-commerce product search
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
