Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformers
Open accessAbstract
Our objective is language-based search of large-scale image and video datasets. For this task, the approach that consists of independently mapping text and vision to a joint embedding space, a.k.a. dual encoders, is attractive as retrieval scales and is efficient for billions of images using approximate nearest neighbour search. An alternative approach of using vision-text transformers with cross-attention gives considerable improvements in accuracy over the joint embeddings, but is often inapplicable in practice for large-scale retrieval given the cost of the cross-attention mechanisms required for each sample at test time. This work combines the best of both worlds. We make the following three contributions. First, we equip transformer-based models with a new fine-grained cross-attention architecture, providing significant improvements in retrieval accuracy whilst preserving scalability. Second, we introduce a generic approach for combining a Fast dual encoder model with our Slow but accurate transformer-based model via distillation and reranking. Finally, we validate our approach on the Flickr30K image dataset where we show an increase in inference speed by several orders of magnitude while having results competitive to the state of the art. We also extend our method to the video domain, improving the state of the art on the VATEX dataset.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
This has been studied for more challenging and fine-grained tasks by imposing region level alignment.
“This has been studied for more challenging and fine-grained tasks (e.g., object detection [269], visual question answering [103], [106], [112], [263], and instance retrieval [222], [263]) by imposing region (semantic parts such as objects) level alignment.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [269]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631 · see citing works
(2011) · Cited by 14,317 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741 · see citing works
EfficientDet: Scalable and Efficient Object Detection
Tan, M., Pang, R., & Le, Q. V. (2019) · arXiv · Cited by 9,841 · cited 2× · see citing works
“We choose a lightweight architecture for this upsampling process inspired by recent advances in efficient object detection [69].”
MethodVideo Google: a text retrieval approach to object matching in videos
Sivic, & Zisserman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238663 · Cited by 6,367 · see citing works
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634 · see citing works
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309 · see citing works
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228 · see citing works
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902 · see citing works
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653 · see citing works
GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering
Hudson, D. A., & Manning, C. D. (2019) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2019.00686 · Cited by 3,703 · see citing works
Audio Set: An ontology and human-labeled dataset for audio events
Gemmeke, J. F., Ellis, D. P. W., Freedman, D., Jansen, A., Lawrence, W., Moore, R. C., Plakal, M., & Ritter, M. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952261 · Cited by 3,134 · see citing works
Locality-sensitive hashing scheme based on p-stable distributions
Datar, M., Immorlica, N., Indyk, P., & Mirrokni, V. (2004) · Annual Symposium on Computational Geometry (SoCG) · doi:10.1145/997817.997857 · Cited by 2,971 · see citing works
Product Quantization for Nearest Neighbor Search
Jégou, H., Douze, M., & Schmid, C. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.57 · Cited by 2,956 · see citing works
Object retrieval with large vocabularies and fast spatial matching
Philbin, J. F., Chum, O., Isard, M., Šivic, J., & Zisserman, A. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383172 · Cited by 2,931 · see citing works
Self-citationFAST APPROXIMATE NEAREST NEIGHBORS WITH AUTOMATIC ALGORITHM CONFIGURATION
(2009) · Proceedings of the Fourth International Conference on Computer Vision Theory and Applications · doi:10.5220/0001787803310340 · Cited by 2,640 · see citing works
Self-Training With Noisy Student Improves ImageNet Classification
Xie, Q., Luong, M.-T., Hovy, E., & Le, Q. V. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01070 · Cited by 2,393 · see citing works
LXMERT: Learning Cross-Modality Encoder Representations from Transformers
Tan, H., & Bansal, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1514 · Cited by 2,319 · see citing works
TSM: Temporal Shift Module for Efficient Video Understanding
Lin, J., Gan, C., & Han, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00718 · Cited by 2,152 · see citing works
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109 · see citing works
Buciluǎ, C., Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150464 · Cited by 2,068 · see citing works
Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search
Jegou, H., Douze, M., & Schmid, C. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_24 · Cited by 1,901 · see citing works
Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning
Sharma, P., Ding, N., Goodman, S., & Soricut, R. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1238 · Cited by 1,879 · see citing works
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Li, X., Yin, X. P., Li, C., Zhang, P., Hu, X., Zhang, L., Wang, L., Hu, H., Li, D., Wei, F., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58577-8_8 · Cited by 1,736 · see citing works
Microsoft COCO Captions: Data Collection and Evaluation Server
Chen, X., Fang, H., Lin, T.-Y., Vedantam, R., Gupta, S., Dollar, P., & Zitnick, C. L. (2015) · arXiv · doi:10.48550/arxiv.1504.00325 · Cited by 1,614 · see citing works
Noise-contrastive estimation: A new estimation principle for unnormalized statistical models
Gutmann, M. U., & Hyvärinen, A. (2010) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 1,588 · see citing works
Hypercolumns for object segmentation and fine-grained localization
Hariharan, B., Arbelaez, P., Girshick, R., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298642 · Cited by 1,511 · see citing works
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models
Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J., & Lazebnik, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.303 · Cited by 1,444 · see citing works
Exploring the Limits of Language ModelingPreprint
Józefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., & Wu, Y.-H. (2016) · arXiv.org · doi:10.48550/arxiv.1602.02410 · Cited by 1,202 · see citing works
VisualBERT: A Simple and Performant Baseline for Vision and Language
Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., & Chang, K.-W. (2019) · arXiv · doi:10.48550/arxiv.1908.03557 · Cited by 1,198 · see citing works
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Miech, A., Zhukov, D., Alayrac, J.-B., Tapaswi, M., Laptev, I., & Sivic, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00272 · Cited by 986 · see citing works
Self-citationVL-BERT: Pre-training of Generic Visual-Linguistic Representations
Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J. (2019) · arXiv · doi:10.48550/arxiv.1908.08530 · Cited by 950 · see citing works
Sampling Matters in Deep Embedding Learning
Wu, C.-Y., Manmatha, R., Smola, A. J., & Krähenbühl, P. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.309 · Cited by 880 · see citing works
Unified Vision-Language Pre-Training for Image Captioning and VQA
Zhou, L., Palangi, H., Zhang, L., Hu, H., Corso, J., & Gao, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.7005 · Cited by 864 · see citing works
Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval
Chum, O., Philbin, J. F., Šivic, J., Isard, M., & Zisserman, A. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4408891 · Cited by 861 · see citing works
Self-citationUnicoder-VL: A Universal Encoder for Vision and Language by Cross-Modal Pre-Training
Li, G., Duan, N., Fang, Y., Gong, M., & Jiang, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6795 · Cited by 795 · see citing works
Learning Deep Structure-Preserving Image-Text Embeddings
Wang, L., Li, Y., & Lazebnik, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.541 · Cited by 788 · see citing works
Structured Knowledge Distillation for Semantic Segmentation
Liu, Y., Chen, K., Liu, C., Qin, Z., Luo, Z., & Wang, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00271 · Cited by 711 · see citing works
End-to-End Learning of Visual Representations From Uncurated Instructional Videos
Miech, A., Alayrac, J.-B., Smaira, L., Laptev, I., Šivic, J., & Zisserman, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00990 · Cited by 622 · see citing works
Self-citationJointly Modeling Embedding and Translation to Bridge Video and Language
Pan, Y., Mei, T., Yao, T., Li, H., & Rui, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.497 · Cited by 573 · see citing works
Learning from Noisy Labels with Distillation
Li, Y., Yang, J., Song, Y., Cao, L., Luo, J., & Li, L.-J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.211 · Cited by 557 · see citing works
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics
Gong, Y., Ke, Q., Isard, M., & Lazebnik, S. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0658-4 · Cited by 539 · see citing works
Learning Two-Branch Neural Networks for Image-Text Matching Tasks
Wang, L., Li, Y., Huang, J. L., & Lazebnik, S. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2797921 · Cited by 538 · see citing works
Cross Modal Distillation for Supervision Transfer
Gupta, S., Hoffman, J., & Malik, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.309 · Cited by 530 · cited 1× · see citing works
“ing [41, 61], depth estimation [22], action recognition [66], semantic segmentation [44], self-supervised learning [58] or self-training [76].”
BackgroundVaTeX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research
Wang, X., Wu, J., Chen, J., Li, L., Wang, Y., & Wang, W. Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00468 · Cited by 506 · see citing works
Data Distillation: Towards Omni-Supervised Learning
Radosavovic, I., Dollár, P., Girshick, R., Gkioxari, G., & He, K. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00433 · Cited by 430 · see citing works
ActBERT: Learning Global-Local Video-Text Representations
Zhu, L., & Yang, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00877 · Cited by 414 · see citing works
12-in-1: Multi-Task Vision and Language Representation Learning
Lu, J., Goswami, V., Rohrbach, M., Parikh, D., & Lee, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01045 · Cited by 410 · see citing works
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
Li, L., Chen, Y.-C., Cheng, Y., Gan, Z., Yu, L., & Liu, J. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.161 · Cited by 408 · see citing works
VirTex: Learning Visual Representations from Textual Annotations
Desai, K., & Johnson, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01101 · Cited by 350 · see citing works
Fine-Grained Video-Text Retrieval With Hierarchical Graph Reasoning
Chen, S., Zhao, Y., Jin, Q., & Wu, Q. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01065 · Cited by 345 · see citing works
Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal TransformersPreprint
Huang, Z., Zeng, Z., Liu, B., Fu, D., & Fu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00849 · Cited by 344 · see citing works
Xu, R., Xiong, C., Chen, W. R., & Corso, J. J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9512 · Cited by 309 · see citing works
Dual Encoding for Zero-Example Video Retrieval
Dong, J., Li, X., Xu, C., Ji, S., He, Y., Yang, G., & Wang, X. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00957 · Cited by 292 · see citing works
Associating neural word embeddings with deep image representations using Fisher Vectors
Klein, B., Lev, G., Sadeh, G., & Wolf, L. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299073 · Cited by 290 · see citing works
Improving Image-Sentence Embeddings Using Large Weakly Annotated Photo Collections
Gong, Y., Wang, L., Hodosh, M., Hockenmaier, J., & Lazebnik, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10593-2_35 · Cited by 286 · see citing works
UNITER: Learning UNiversal Image-TExt Representations
Chen, Y.-C., Li, L., Yu, L., Kholy, A. E., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2019) · Cited by 280 · see citing works
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J. (2020) · International Conference on Learning Representations · Cited by 276 · see citing works
A Short Note about Kinetics-600
Carreira, J., Noland, E., Banki-Horvath, A., Hillier, C., & Zisserman, A. (2018) · arXiv · doi:10.48550/arxiv.1808.01340 · Cited by 270 · see citing works
Self-citationSelf-Supervised MultiModal Versatile NetworksPreprint
Alayrac, J.-B., Recasens, A., Schneider, R. G., Arandjelović, R., Ramapuram, J., Fauw, J. D., Smaira, L., Dieleman, S., & Zisserman, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.16228 · Cited by 247 · see citing works
Self-citationLearning a Text-Video Embedding from Incomplete and Heterogeneous DataPreprint
Miech, A., Laptev, I., & Šivic, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.02516 · Cited by 217 · see citing works
Self-citationD3D: Distilled 3D Networks for Video Action Recognition
Stroud, J., Ross, D. A., Sun, C., Deng, J., & Sukthankar, R. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093274 · Cited by 194 · see citing works
Support-set bottlenecks for video-text representation learning
Patrick, M., Huang, P.-Y., Asano, Y. M., Metze, F., Hauptmann, A. G., Henriques, J., & Vedaldi, A. (2020) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.2010.02824 · Cited by 192 · see citing works
Evolving Losses for Unsupervised Video Representation Learning
Piergiovanni, A., Angelova, A., & Ryoo, M. S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00021 · Cited by 134 · see citing works
Contrastive Learning for Weakly Supervised Phrase Grounding
Gupta, T., Vahdat, A., Chechik, G., Yang, X., Kautz, J., & Hoiem, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_44 · Cited by 121 · see citing works
Enhancing Video Summarization via Vision-Language Embedding
Plummer, B. A., Brown, M., & Lazebnik, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.118 · Cited by 112 · see citing works
Temporal Localization of Moments in Video Collections with Natural Language.Preprint
Escorcia, V. A., Soldan, M., Šivic, J., Ghanem, B., & Russell, B. C. (2019) · arXiv (Cornell University) · Cited by 65 · see citing works
Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval
Geigle, G., Pfeiffer, J., Reimers, N., Vulić, I., & Gurevych, I. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00473 · Cited by 56 · see citing works
Attention-Based Query Expansion Learning
Gordo, A., Radenović, F., & Berg, T. L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58604-1_11 · Cited by 35 · see citing works
Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal RetrievalPreprint
Geigle, G., Pfeiffer, J., Reimers, N., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.11920 · Cited by 1 · see citing works
Flagged first, then by citations
7 of the 85 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Guobin Chen, Wongun Choi, Xiang Yu, Tony Han, and Man- mohan Chandraker. Learning efficient object detection mod- els with knowledge distillation. In NIPS, 2017. 3
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072 · see citing works
Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval
Jiang, D., & Ye, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00273 · Cited by 348 · see citing works
Perceiver IO: A General Architecture for Structured Inputs & OutputsPreprint
Jaegle, A., Borgeaud, S., Alayrac, J.-B., Doersch, C., Ionescu, C., Ding, X. D., Koppula, S., Zoran, D., Brock, A., Shelhamer, E., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.14795 · Cited by 357 · see citing works
Self-citationX-Pool: Cross-Modal Language-Video Attention for Text-Video Retrieval
Gorti, S. K., Vouitsis, N., Ma, J., Golestan, K., Volkovs, M., Garg, A., & Yu, G. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00495 · Cited by 242 · see citing works
Selva, J., Johansen, A. S., Escalera, S., Nasrollahi, K., Moeslund, T. B., & Clapés, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3243465 · Cited by 177 · see citing works
Image-text Retrieval: A Survey on Recent Research and Development
Cao, M., Li, S., Li, J., Nie, L., & Zhang, M. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/759 · Cited by 125 · see citing works
Local-Global Context Aware Transformer for Language-Guided Video Segmentation
Liang, C., Wang, W., Zhou, T., Miao, J., Luo, Y., & Yang, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3262578 · Cited by 106 · see citing works
Cross Modal Retrieval with Querybank Normalisation
Bogolin, S.-V., Croitoru, I., Jin, H., Liu, Y., & Albanie, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00513 · Cited by 107 · see citing works
TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis
Petrovich, M., Black, M. J., & Varol, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00870 · Cited by 108 · see citing works
Vision Transformer Hashing for Image Retrieval
Dubey, S. R., Singh, S. K., & Chu, W.-T. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859900 · Cited by 92 · see citing works
ViSTA: Vision and Scene Text Aggregation for Cross-Modal Retrieval
Cheng, M., Sun, Y., Wang, L., Zhu, X., Yao, K., Chen, J., Song, G., Han, J., Liu, J., Ding, E., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00512 · Cited by 87 · see citing works
Align and Tell: Boosting Text-Video Retrieval With Local Alignment and Fine-Grained Supervision
Wang, X., Zhu, L., Zheng, Z., Xu, M., & Yang, Y. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3204444 · Cited by 86 · see citing works
Improving Cross-Modal Retrieval with Set of Diverse Embeddings
Kim, D.-W., Kim, N., & Kwak, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02243 · Cited by 80 · see citing works
Temporal Alignment Networks for Long-term Video
Han, T., Xie, W., & Zisserman, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00292 · Cited by 72 · see citing works
Self-citationA survey of transformer-based multimodal pre-trained modals
Han, X., Wang, Y.-T., Feng, J.-L., Deng, C., Chen, Z.-H., Huang, Y.-A., Su, H., Hu, L., & Hu, P.-W. (2023) · Neurocomputing · doi:10.1016/j.neucom.2022.09.136 · Cited by 59 · see citing works
Efficient Token-Guided Image-Text Retrieval With Consistent Multimodal Contrastive Training
Liu, C., Zhang, Y., Wang, H., Chen, W., Wang, F., Huang, Y., Shen, Y.-D., & Wang, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3286710 · Cited by 57 · see citing works
Mitosis detection, fast and slow: Robust and efficient detection of mitotic figures
Jahanifar, M., Shephard, A., Tajeddin, N. Z., Graham, S., Raza, S. E. A., Minhas, F., & Rajpoot, N. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103132 · Cited by 53 · see citing works
ESA: External Space Attention Aggregation for Image-Text Retrieval
Zhu, H., Zhang, C., Wei, Y., Huang, S., & Zhao, Y. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3253548 · Cited by 49 · see citing works
xGQA: Cross-Lingual Visual Question Answering
Pfeiffer, J., Geigle, G., Kamath, A., Steitz, J.-M. O., Roth, S., Vulić, I., & Gurevych, I. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.196 · Cited by 51 · see citing works
Cross-modal adapter for vision–language retrieval
Jiang, H., Zhang, J., Huang, R., Ge, C., Ni, Z., Song, S., & Huang, G. M. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111144 · Cited by 54 · see citing works
X-DETR: A Versatile Architecture for Instance-wise Vision-Language Tasks
Cai, Z., Kwon, G., Ravichandran, A., Bas, E., Tu, Z., Bhotika, R., & Soatto, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_17 · Cited by 40 · see citing works
Li, Z., Zhang, L., Zhang, K., Zhang, Y., & Mao, Z. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3358411 · Cited by 39 · see citing works
Transformer Decoders with MultiModal Regularization for Cross-Modal Food Retrieval
Shukor, M., Couairon, G., Grechka, A., & Cord, M. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00503 · Cited by 36 · see citing works
LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Sparse Retrieval
Luo, Z., Zhao, P., Xu, C., Geng, X., Shen, T., Tao, C., Ma, J., Lin, Q., & Jiang, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01029 · Cited by 34 · see citing works
Yu, L., Chen, J., Sinha, A., Wang, M., Chen, Y., Berg, T. L., & Zhang, N. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539151 · Cited by 38 · see citing works
Unsupervised Hashing Retrieval via Efficient Correlation Distillation
Zhang, X., Xiu, W., & Cheng, P. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3234037 · Cited by 32 · see citing works
Efficient text-image semantic search: A multi-modal vision-language approach for fashion retrieval
Moro, G., Salvatori, S., & Frisoni, G. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.03.057 · Cited by 32 · see citing works
Partially Does It: Towards Scene-Level FG-SBIR with Partial Input
Chowdhury, P. N., Bhunia, A. K., Gajjala, V. R., Sain, A., Xiang, T., & Song, Y.-Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00243 · Cited by 31 · see citing works
Dual Learning with Dynamic Knowledge Distillation for Partially Relevant Video Retrieval
Dong, J., Zhang, M., Zhang, Z., Chen, X., Liu, D., Qu, X., Wang, X., & Liu, B. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01038 · Cited by 30 · see citing works
Cross-Lingual Cross-Modal Retrieval with Noise-Robust Learning
Wang, Y., Dong, J., Liang, T., Zhang, M., Cai, R., & Wang, X. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548003 · Cited by 27 · see citing works
Fast-Slow Transformer for Visually Grounding Speech
Peng, P., & Harwath, D. F. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747103 · Cited by 26 · see citing works
Distilled Dual-Encoder Model for Vision-Language Understanding
Wang, Z., Wang, W., Zhu, H., Liu, M., Qin, B., & Wei, F. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.608 · Cited by 30 · see citing works
Localized Vision-Language Matching for Open-vocabulary Object Detection
Bravo, M. A., Mittal, S., & Brox, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16788-1_24 · Cited by 24 · see citing works
Towards Fast and Accurate Image-Text Retrieval With Self-Supervised Fine-Grained Alignment
Zhuang, J., Yu, J., Ding, Y., Qu, X., & Hu, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3280734 · Cited by 24 · see citing works
Cross-Modal Alternating Learning With Task-Aware Representations for Continual Learning
Li, W., Gao, B.-B., Xia, B., Wang, J., Liu, J., Liu, Y., Wang, C., & Zheng, F. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3340551 · Cited by 26 · see citing works
X-volution: On the unification of convolution and self-attentionPreprint
Chen, X., Wang, H., & Ni, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02253 · Cited by 24 · see citing works
A Proposal for a Language Model Based Cognitive Architecture
Knowles, K., Witbrock, M., Dobbie, G., & Yogarajan, V. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v2i1.27691 · Cited by 23 · see citing works
Fine-grained Semantics-aware Representation Learning for Text-based Person Retrieval
Wang, D., Yan, F., Wang, Y., Zhao, L., Xiao, L., Zhong, H., & Zhang, R. (2024) · International Conference on Multimedia Retrieval · doi:10.1145/3652583.3658054 · Cited by 21 · see citing works
LamRA: Large Multimodal Model as Your Advanced Retrieval Assistant
Liu, Y., Zhang, Y., Cai, J., Jiang, X., Hu, Y. G., Yao, J., Wang, Y., & Xie, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00380 · Cited by 21 · see citing works
Noise Is Also Useful: Negative Correlation-Steered Latent Contrastive Learning
Yan, J., Luo, L., Xu, C., Deng, C. D., & Huang, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00013 · Cited by 21 · see citing works
Image Retrieval from Contextual Descriptions
Krojer, B., Adlakha, V., Vineet, V., Goyal, Y., Ponti, E. M., & Reddy, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.241 · Cited by 20 · see citing works
Decoupled Cross-Modal Phrase-Attention Network for Image-Sentence Matching
Shi, Z., Zhang, T., Wei, X., Wu, F. H., & Zhang, Y. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3197972 · Cited by 20 · see citing works
Video Question Answering with Iterative Video-Text Co-tokenization
Piergiovanni, A., Morton, K., Kuo, W., Ryoo, M. S., & Angelova, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_5 · Cited by 18 · see citing works
Weakly Supervised Grounding for VQA in Vision-Language Transformers
Khan, A. U., Kuehne, H., Gan, C., Lobo, N. D. V., & Shah, M. A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_38 · Cited by 16 · see citing works
Cross-Modality Pyramid Alignment for Visual Intention Understanding
Ye, M., Shi, Q., Su, K., & Du, B. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3261743 · Cited by 17 · see citing works
Query-Oriented Micro-Video Summarization
Jia, M., Wei, Y., Song, X., Sun, T., Zhang, M., & Nie, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3355402 · Cited by 16 · see citing works
MetaViewer: Towards A Unified Multi-View Representation
Wang, R., Sun, H., Ma, Y., Xi, X., & Yin, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01115 · Cited by 15 · see citing works
Inflate and Shrink:Enriching and Reducing Interactions for Fast Text-Image Retrieval
Liu, H., Yu, T., & Li, P. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.772 · Cited by 15 · see citing works
Visual context learning based on textual knowledge for image–text retrieval
Qin, Y., Gu, X., & Tan, Z. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.05.008 · Cited by 15 · see citing works
Kedia, A., Zaidi, M. A., & Lee, H. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.285 · Cited by 14 · see citing works
Multi-modal Dictionary BERT for Cross-modal Video Search in Baidu Advertising
Yu, T., Yang, Y., Li, Y., Liu, L., Sun, M., & Li, P. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3481937 · Cited by 13 · see citing works
Collaborative group: Composed image retrieval via consensus learning from noisy annotations
Zhang, X., Zheng, Z., Zhu, L., & Yang, Y. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112135 · Cited by 13 · see citing works
Learning Text-to-Video Retrieval from Image Captioning
Ventura, L., Schmid, C., & Varol, G. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02202-8 · Cited by 12 · see citing works
SoFTNet: A concept-controlled deep learning architecture for interpretable image classification
Zia, T., Bashir, N., Ullah, M. A., & Murtaza, S. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.108066 · Cited by 12 · see citing works
FeatInter: Exploring fine-grained object features for video-text retrieval
Liu, B., Zheng, Q., Wang, Y., Zhang, M., Dong, J., & Wang, X. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.01.094 · Cited by 12 · see citing works
Show, tell and rectify: Boost image caption generation via an output rectifier
Ge, G., Han, Y., Hao, L., Hao, K., Wei, B., & Tang, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127651 · Cited by 12 · see citing works
Text-guided Image Restoration and Semantic Enhancement for Text-to-Image Person Retrieval
Liu, D., Li, H., Zhao, Z., & Dong, Y. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.107028 · Cited by 12 · see citing works
Uncertainty-aware coarse-to-fine alignment for text-image person retrieval
Deng, Y., Chen, Z., Li, C., & Tang, J. (2025) · Visual Intelligence · doi:10.1007/s44267-025-00078-x · Cited by 12 · see citing works
Animating Images to Transfer CLIP for Video-Text Retrieval
Liu, Y., Chen, H., Huang, L., Chen, D., Wang, B., Pan, P., & Wang, L. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531776 · Cited by 12 · see citing works
Transformers in speech processing: Overcoming challenges and paving the future
Latif, S., Zaidi, S. A. M., Cuayáhuitl, H., Shamshad, F., Shoukat, M., Usama, M., & Qadir, J. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100768 · Cited by 68 · see citing works
Filling the Information Gap between Video and Query for Language-Driven Moment Retrieval
Liu, D., Qu, X., Dong, J., Nan, G., Zhou, P., Xu, Z., Chen, L., Yan, H., & Cheng, Y. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612038 · Cited by 11 · see citing works
Train/Test-Time Adaptation with Retrieval
Zancato, L., Achille, A., Liu, T. Y., Trager, M., Perera, P., & Soatto, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01527 · Cited by 10 · see citing works
Balance Act: Mitigating Hubness in Cross-Modal Retrieval with Query and Gallery Banks
Wang, Y., Jian, X., & Xue, B. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.652 · Cited by 11 · see citing works
Scene-Text Aware Image and Text Retrieval with Dual-Encoder
Miyawaki, S., Hasegawa, T., Nishida, K., Kato, T., & Suzuki, J. (2022) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/2022.acl-srw.34 · Cited by 9 · see citing works
Assorted Attention Network for Cross-Lingual Language-to-Vision Retrieval
Yu, T., Yang, Y., Fei, H., Li, Y., Chen, X., & Li, P. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482233 · Cited by 8 · see citing works
VideoCLIP: A Cross-Attention Model for Fast Video-Text Retrieval Task with Image CLIP
Li, Y., Hsiao, J.-H., & Ho, C. (2022) · International Conference on Multimedia Retrieval · doi:10.1145/3512527.3531429 · Cited by 8 · see citing works
Hi-ToM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models
Wu, Y., He, Y., Jia, Y., Mihalcea, R. F., Chen, Y., & Deng, N. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.717 · Cited by 8 · see citing works
Cross-Probe BERT for Fast Cross-Modal Search
Yu, T., Fei, H., & Li, P. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531826 · Cited by 8 · see citing works
Audio-Visual Mismatch-Aware Video Retrieval via Association and Adjustment
Lee, S., Park, S., & Ro, Y. M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19781-9_29 · Cited by 8 · see citing works
Egocentric Biochemical Video-and-Language Dataset
Nishimura, T., Sakoda, K., Hashimoto, A., Ushiku, Y., Tanaka, N., Ono, F., Kameko, H., & Mori, S. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00348 · Cited by 8 · see citing works
Cascaded Fast and Slow Models for Efficient Semantic Code SearchPreprint
Gotmare, A. D., Li, J., Joty, S., & Hoi, S. C. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07811 · Cited by 10 · see citing works
Accurate and Lightweight Learning for Specific Domain Image-Text Retrieval
Yang, R., Wang, S., Tao, J., Han, Y., Lin, Q., Guo, Y., Hou, B., & Jiao, L. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681280 · Cited by 8 · see citing works
Cofopose: Conditional 2D Pose Estimation with Transformers
Aidoo, E., Wang, X., Liu, Z., Tenagyei, E. K., Owusu-Agyemang, K., Kodjiku, S. L., Ejianya, V. N., & Aggrey, E. S. E. B. (2022) · Sensors · doi:10.3390/s22186821 · Cited by 7 · see citing works
Efficient Image-Text Retrieval via Keyword-Guided Pre-Screening
Cao, M., Bai, Y., Cao, Z., Nie, L., & Zhang, M. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3339489 · Cited by 7 · see citing works
How to Make Cross Encoder a Good Teacher for Efficient Image-Text Retrieval?
Chen, Y., Ma, Z., Zhang, Z., Qi, Z., Yuan, C., Li, B., Pu, J., Shan, Y., Qi, X., & Hu, W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02549 · Cited by 7 · see citing works
Wang, S., Zhou, F., Yang, M., Shi, L., & Tan, C. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3524297 · Cited by 7 · see citing works
Enhancing Text-Video Retrieval Performance With Low-Salient but Discriminative Objects
Zheng, Y., Huang, B., Chen, Z., & Yu, D. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3527369 · Cited by 6 · see citing works
Deep Vision-Language Model for Efficient Multi-modal Similarity Search in Fashion Retrieval
Moro, G., & Salvatori, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17849-8_4 · Cited by 6 · see citing works
From Within to Between: Knowledge Distillation for Cross Modality Retrieval
Tran, V., Balasubramanian, N., & Hoai, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26316-3_36 · Cited by 5 · see citing works
Efficient Text-to-Code Retrieval with Cascaded Fast and Slow Transformer Models
Gotmare, A. D., Li, J., Joty, S., & Hoi, S. C. H. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616369 · Cited by 5 · see citing works
Huang, X. Q., Liu, J., Zhang, Z., Xie, Y., Tang, Y., Zhang, W., & Cui, X. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3384672 · Cited by 5 · see citing works
Multi-scale Multi-modal Dictionary BERT For Effective Text-image Retrieval in Multimedia Advertising
Yu, T., Liu, J., Jin, Z., Yang, Y., Fei, H., & Li, P. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557653 · Cited by 5 · see citing works
Endoscopy Image Retrieval by Mixer Multi-Layer Perceptron
Trinh, Q.-H., & Nguyen, M.-V. (2021) · Annals of Computer Science and Information Systems · doi:10.15439/2021f145 · Cited by 4 · see citing works
Cycle mapping with adversarial event classification network for fake news detection
Wu, F., Zhou, H., Feng, Y., Gao, G., Ji, Y., & Jing, X. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18499-z · Cited by 4 · see citing works
ITContrast: contrastive learning with hard negative synthesis for image-text matching
Wu, F., Wang, Q., Wang, Z., Yu, S., Li, Y., Zhang, B., & Lim, E. G. (2024) · The Visual Computer · doi:10.1007/s00371-024-03274-w · Cited by 4 · see citing works
Vision-Language Models Learn Super Images for Efficient Partially Relevant Video Retrieval
Nishimura, T., Nakada, S., & Kondo, M. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3708349 · Cited by 6 · see citing works
Learning Linguistic Association Towards Efficient Text-Video Retrieval
Fang, S., Wang, S., Zhuo, J., Han, X., & Huang, Q. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_15 · Cited by 4 · see citing works
HADA: A Graph-Based Amalgamation Framework in Image-text Retrieval
Nguyen, M.-D., Nguyen, B. T., & Gurrin, C. G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28244-7_45 · Cited by 4 · see citing works
HiVLP: Hierarchical Interactive Video-Language Pre-Training
Shao, B., Liu, J., Pei, R., Xu, S., Dai, P., Lu, J., Li, W., & Yan, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01265 · Cited by 4 · see citing works
MixBERT for Image-Ad Relevance Scoring in Advertising
Yu, T., Li, X., Xie, J., Yin, R., Xu, Q. Y., & Li, P. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482143 · Cited by 4 · see citing works
Chen, N., Xu, T., Sun, M., Yao, C., & Yang, D. (2025) · Intelligent Computing · doi:10.34133/icomputing.0143 · Cited by 5 · see citing works
Beyond One and Two Tower: Cross-Modal Consensus Learning for Image-Text Retrieval
Shi, Z., Ding, Y., Dong, J., & Zhang, T. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3605958 · Cited by 3 · see citing works
Deepfake Video Detection Based on the Decomposition of Spatial-Temporal Attention Mechanism in ViViT
Sun, G., & Lian, Z. (2024) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa63168.2024.00221 · Cited by 4 · see citing works
Linking Surface Facts to Large-Scale Knowledge Graphs
Radevski, G., Gashteovski, K., Hung, C.-C., Lawrence, C., & Glavaš, G. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.445 · Cited by 3 · see citing works
MMTF: Multi-Modal Temporal Fusion for Commonsense Video Question Answering
Ahmad, M., Park, G., Park, D., & Park, S. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00502 · Cited by 3 · see citing works
Dual-path temporal map optimization for make-up temporal video grounding
Li, J., Li, K., Li, J., Chen, G., Wang, M., & Guo, D. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01340-w · Cited by 3 · see citing works
Generating Illustrated Instructions
Menon, S., Misra, I., & Girdhar, R. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00600 · Cited by 3 · see citing works
Ullah, M. A., Zia, T., Kim, J.-E., & Kadry, S. (2024) · PLoS ONE · doi:10.1371/journal.pone.0311879 · Cited by 3 · see citing works
RetractedDisentangled OCR: A More Granular Information for “Text”-to-Image Retrieval
Zhou, X., Li, S., Chen, H., & Zhu, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-18907-4_40 · Cited by 3 · see citing works
Vo, T.-P., Duong, Q.-B., Nguyen, Q.-T., Mai, D., & Ly, N.-K. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-4291-5_19 · Cited by 2 · see citing works
- Authors
- Miech, A., Alayrac, J.-B., Laptev, I., Sivic, J., & Zisserman, A.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 9821-9831
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3141362964
- Cited by
- 149
- References
- 85
- Citations by year
- 2026: 7 · 2025: 21 · 2024: 31 · 2023: 39 · 2022: 39 · 2021: 12
- FWCI
- 9.88
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion)
- arXiv (Cornell University)
- Funders
- European Regional Development Fund
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- text-to-image retrieval, video-text retrieval, cross-attention, dual-encoder models, approximate nearest neighbor search, knowledge distillation, re-ranking, inference efficiency, large-scale image retrieval, Flickr30k
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
