The DOI matches its Crossref record; 79 of the 85 references it cites were found in a registry or an index.
Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformers
Open accessAbstract
Our objective is language-based search of large-scale image and video datasets. For this task, the approach that consists of independently mapping text and vision to a joint embedding space, a.k.a. dual encoders, is attractive as retrieval scales and is efficient for billions of images using approximate nearest neighbour search. An alternative approach of using vision-text transformers with cross-attention gives considerable improvements in accuracy over the joint embeddings, but is often inapplicable in practice for large-scale retrieval given the cost of the cross-attention mechanisms required for each sample at test time. This work combines the best of both worlds. We make the following three contributions. First, we equip transformer-based models with a new fine-grained cross-attention architecture, providing significant improvements in retrieval accuracy whilst preserving scalability. Second, we introduce a generic approach for combining a Fast dual encoder model with our Slow but accurate transformer-based model via distillation and reranking. Finally, we validate our approach on the Flickr30K image dataset where we show an increase in inference speed by several orders of magnitude while having results competitive to the state of the art. We also extend our method to the video domain, improving the state of the art on the VATEX dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This has been studied for more challenging and fine-grained tasks (e.g., object detection [269], visual question answering [103], [106], [112], [263], and instance retrieval [222], [263]) by imposing region (semantic parts such as objects) level alignment.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [269]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
(2011) · Cited by 14,317
Research works printed but not linked: 4
- 10.1109/ICCV.2019.00054 · DOI
- 10.1109/CVPR42600.2020.01079 · DOI · Paper page
- daniel, 2017, Thinking Fast and Slow
- chen, 2017, Learning efficient object detection models with knowledge distillation, NIPS
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072
Cross-Modal Implicit Relation Reasoning and Aligning for Text-to-Image Person Retrieval
Jiang, D., & Ye, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00273 · Cited by 367
Perceiver IO: A General Architecture for Structured Inputs & OutputsPreprint
Jaegle, A., Borgeaud, S., Alayrac, J.-B., Doersch, C., Ionescu, C., Ding, X. D., Koppula, S., Zoran, D., Brock, A., Shelhamer, E., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.14795 · Cited by 641
Self-citationX-Pool: Cross-Modal Language-Video Attention for Text-Video Retrieval
Gorti, S. K., Vouitsis, N., Ma, J., Golestan, K., Volkovs, M., Garg, A., & Yu, G. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00495 · Cited by 261
Selva, J., Johansen, A. S., Escalera, S., Nasrollahi, K., Moeslund, T. B., & Clapés, A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3243465 · Cited by 192
Image-text Retrieval: A Survey on Recent Research and Development
Cao, M., Li, S., Li, J., Nie, L., & Zhang, M. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/759 · Cited by 125
Local-Global Context Aware Transformer for Language-Guided Video Segmentation
Liang, C., Wang, W., Zhou, T., Miao, J., Luo, Y., & Yang, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3262578 · Cited by 114
Cross Modal Retrieval with Querybank Normalisation
Bogolin, S.-V., Croitoru, I., Jin, H., Liu, Y., & Albanie, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00513 · Cited by 118
TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis
Petrovich, M., Black, M. J., & Varol, G. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00870 · Cited by 139
Vision Transformer Hashing for Image Retrieval
Dubey, S. R., Singh, S. K., & Chu, W.-T. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859900 · Cited by 98
ViSTA: Vision and Scene Text Aggregation for Cross-Modal Retrieval
Cheng, M., Sun, Y., Wang, L., Zhu, X., Yao, K., Chen, J., Song, G., Han, J., Liu, J., Ding, E., et al. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00512 · Cited by 87
Align and Tell: Boosting Text-Video Retrieval With Local Alignment and Fine-Grained Supervision
Wang, X., Zhu, L., Zheng, Z., Xu, M., & Yang, Y. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3204444 · Cited by 86
Improving Cross-Modal Retrieval with Set of Diverse Embeddings
Kim, D.-W., Kim, N., & Kwak, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02243 · Cited by 81
Temporal Alignment Networks for Long-term Video
Han, T., Xie, W., & Zisserman, A. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00292 · Cited by 72
Self-citationA survey of transformer-based multimodal pre-trained modals
Han, X., Wang, Y.-T., Feng, J.-L., Deng, C., Chen, Z.-H., Huang, Y.-A., Su, H., Hu, L., & Hu, P.-W. (2023) · Neurocomputing · doi:10.1016/j.neucom.2022.09.136 · Cited by 59
Efficient Token-Guided Image-Text Retrieval With Consistent Multimodal Contrastive Training
Liu, C., Zhang, Y., Wang, H., Chen, W., Wang, F., Huang, Y., Shen, Y.-D., & Wang, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3286710 · Cited by 58
Mitosis detection, fast and slow: Robust and efficient detection of mitotic figures
Jahanifar, M., Shephard, A., Tajeddin, N. Z., Graham, S., Raza, S. E. A., Minhas, F., & Rajpoot, N. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103132 · Cited by 57
ESA: External Space Attention Aggregation for Image-Text Retrieval
Zhu, H., Zhang, C., Wei, Y., Huang, S., & Zhao, Y. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3253548 · Cited by 49
xGQA: Cross-Lingual Visual Question Answering
Pfeiffer, J., Geigle, G., Kamath, A., Steitz, J.-M. O., Roth, S., Vulić, I., & Gurevych, I. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.196 · Cited by 67
Cross-modal adapter for vision–language retrieval
Jiang, H., Zhang, J., Huang, R., Ge, C., Ni, Z., Song, S., & Huang, G. M. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111144 · Cited by 72
X-DETR: A Versatile Architecture for Instance-wise Vision-Language Tasks
Cai, Z., Kwon, G., Ravichandran, A., Bas, E., Tu, Z., Bhotika, R., & Soatto, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_17 · Cited by 48
Li, Z., Zhang, L., Zhang, K., Zhang, Y., & Mao, Z. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3358411 · Cited by 40
Transformer Decoders with MultiModal Regularization for Cross-Modal Food Retrieval
Shukor, M., Couairon, G., Grechka, A., & Cord, M. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00503 · Cited by 36
LexLIP: Lexicon-Bottlenecked Language-Image Pre-Training for Large-Scale Image-Text Sparse Retrieval
Luo, Z., Zhao, P., Xu, C., Geng, X., Shen, T., Tao, C., Ma, J., Lin, Q., & Jiang, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01029 · Cited by 43
Yu, L., Chen, J., Sinha, A., Wang, M., Chen, Y., Berg, T. L., & Zhang, N. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539151 · Cited by 43
Unsupervised Hashing Retrieval via Efficient Correlation Distillation
Zhang, X., Xiu, W., & Cheng, P. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3234037 · Cited by 32
Efficient text-image semantic search: A multi-modal vision-language approach for fashion retrieval
Moro, G., Salvatori, S., & Frisoni, G. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.03.057 · Cited by 32
Partially Does It: Towards Scene-Level FG-SBIR with Partial Input
Chowdhury, P. N., Bhunia, A. K., Gajjala, V. R., Sain, A., Xiang, T., & Song, Y.-Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00243 · Cited by 31
Dual Learning with Dynamic Knowledge Distillation for Partially Relevant Video Retrieval
Dong, J., Zhang, M., Zhang, Z., Chen, X., Liu, D., Qu, X., Wang, X., & Liu, B. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01038 · Cited by 30
Cross-Lingual Cross-Modal Retrieval with Noise-Robust Learning
Wang, Y., Dong, J., Liang, T., Zhang, M., Cai, R., & Wang, X. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548003 · Cited by 32
Fast-Slow Transformer for Visually Grounding Speech
Peng, P., & Harwath, D. F. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747103 · Cited by 27
Distilled Dual-Encoder Model for Vision-Language Understanding
Wang, Z., Wang, W., Zhu, H., Liu, M., Qin, B., & Wei, F. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.608 · Cited by 34
Localized Vision-Language Matching for Open-vocabulary Object Detection
Bravo, M. A., Mittal, S., & Brox, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16788-1_24 · Cited by 25
Towards Fast and Accurate Image-Text Retrieval With Self-Supervised Fine-Grained Alignment
Zhuang, J., Yu, J., Ding, Y., Qu, X., & Hu, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3280734 · Cited by 25
Cross-Modal Alternating Learning With Task-Aware Representations for Continual Learning
Li, W., Gao, B.-B., Xia, B., Wang, J., Liu, J., Liu, Y., Wang, C., & Zheng, F. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3340551 · Cited by 26
X-volution: On the unification of convolution and self-attentionPreprint
Chen, X., Wang, H., & Ni, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02253 · Cited by 30
A Proposal for a Language Model Based Cognitive Architecture
Knowles, K., Witbrock, M., Dobbie, G., & Yogarajan, V. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v2i1.27691 · Cited by 23
Fine-grained Semantics-aware Representation Learning for Text-based Person Retrieval
Wang, D., Yan, F., Wang, Y., Zhao, L., Xiao, L., Zhong, H., & Zhang, R. (2024) · International Conference on Multimedia Retrieval · doi:10.1145/3652583.3658054 · Cited by 25
LamRA: Large Multimodal Model as Your Advanced Retrieval Assistant
Liu, Y., Zhang, Y., Cai, J., Jiang, X., Hu, Y. G., Yao, J., Wang, Y., & Xie, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00380 · Cited by 54
Noise Is Also Useful: Negative Correlation-Steered Latent Contrastive Learning
Yan, J., Luo, L., Xu, C., Deng, C. D., & Huang, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00013 · Cited by 22
Image Retrieval from Contextual Descriptions
Krojer, B., Adlakha, V., Vineet, V., Goyal, Y., Ponti, E. M., & Reddy, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.241 · Cited by 38
Decoupled Cross-Modal Phrase-Attention Network for Image-Sentence Matching
Shi, Z., Zhang, T., Wei, X., Wu, F. H., & Zhang, Y. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3197972 · Cited by 20
Video Question Answering with Iterative Video-Text Co-tokenization
Piergiovanni, A., Morton, K., Kuo, W., Ryoo, M. S., & Angelova, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_5 · Cited by 18
Weakly Supervised Grounding for VQA in Vision-Language Transformers
Khan, A. U., Kuehne, H., Gan, C., Lobo, N. D. V., & Shah, M. A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_38 · Cited by 16
Cross-Modality Pyramid Alignment for Visual Intention Understanding
Ye, M., Shi, Q., Su, K., & Du, B. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3261743 · Cited by 19
Query-Oriented Micro-Video Summarization
Jia, M., Wei, Y., Song, X., Sun, T., Zhang, M., & Nie, L. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3355402 · Cited by 17
MetaViewer: Towards A Unified Multi-View Representation
Wang, R., Sun, H., Ma, Y., Xi, X., & Yin, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01115 · Cited by 15
Inflate and Shrink:Enriching and Reducing Interactions for Fast Text-Image Retrieval
Liu, H., Yu, T., & Li, P. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.772 · Cited by 16
Visual context learning based on textual knowledge for image–text retrieval
Qin, Y., Gu, X., & Tan, Z. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.05.008 · Cited by 15
Kedia, A., Zaidi, M. A., & Lee, H. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.285 · Cited by 14
Multi-modal Dictionary BERT for Cross-modal Video Search in Baidu Advertising
Yu, T., Yang, Y., Li, Y., Liu, L., Sun, M., & Li, P. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3481937 · Cited by 13
Collaborative group: Composed image retrieval via consensus learning from noisy annotations
Zhang, X., Zheng, Z., Zhu, L., & Yang, Y. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112135 · Cited by 18
Learning Text-to-Video Retrieval from Image Captioning
Ventura, L., Schmid, C., & Varol, G. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02202-8 · Cited by 13
SoFTNet: A concept-controlled deep learning architecture for interpretable image classification
Zia, T., Bashir, N., Ullah, M. A., & Murtaza, S. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.108066 · Cited by 12
FeatInter: Exploring fine-grained object features for video-text retrieval
Liu, B., Zheng, Q., Wang, Y., Zhang, M., Dong, J., & Wang, X. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.01.094 · Cited by 12
Show, tell and rectify: Boost image caption generation via an output rectifier
Ge, G., Han, Y., Hao, L., Hao, K., Wei, B., & Tang, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127651 · Cited by 12
Text-guided Image Restoration and Semantic Enhancement for Text-to-Image Person Retrieval
Liu, D., Li, H., Zhao, Z., & Dong, Y. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.107028 · Cited by 13
Uncertainty-aware coarse-to-fine alignment for text-image person retrieval
Deng, Y., Chen, Z., Li, C., & Tang, J. (2025) · Visual Intelligence · doi:10.1007/s44267-025-00078-x · Cited by 12
Animating Images to Transfer CLIP for Video-Text Retrieval
Liu, Y., Chen, H., Huang, L., Chen, D., Wang, B., Pan, P., & Wang, L. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531776 · Cited by 13
Transformers in speech processing: Overcoming challenges and paving the future
Latif, S., Zaidi, S. A. M., Cuayáhuitl, H., Shamshad, F., Shoukat, M., Usama, M., & Qadir, J. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100768 · Cited by 116
Filling the Information Gap between Video and Query for Language-Driven Moment Retrieval
Liu, D., Qu, X., Dong, J., Nan, G., Zhou, P., Xu, Z., Chen, L., Yan, H., & Cheng, Y. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612038 · Cited by 11
Train/Test-Time Adaptation with Retrieval
Zancato, L., Achille, A., Liu, T. Y., Trager, M., Perera, P., & Soatto, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01527 · Cited by 11
Balance Act: Mitigating Hubness in Cross-Modal Retrieval with Query and Gallery Banks
Wang, Y., Jian, X., & Xue, B. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.652 · Cited by 15
Scene-Text Aware Image and Text Retrieval with Dual-Encoder
Miyawaki, S., Hasegawa, T., Nishida, K., Kato, T., & Suzuki, J. (2022) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/2022.acl-srw.34 · Cited by 9
Assorted Attention Network for Cross-Lingual Language-to-Vision Retrieval
Yu, T., Yang, Y., Fei, H., Li, Y., Chen, X., & Li, P. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482233 · Cited by 8
VideoCLIP: A Cross-Attention Model for Fast Video-Text Retrieval Task with Image CLIP
Li, Y., Hsiao, J.-H., & Ho, C. (2022) · International Conference on Multimedia Retrieval · doi:10.1145/3512527.3531429 · Cited by 8
Hi-ToM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models
Wu, Y., He, Y., Jia, Y., Mihalcea, R. F., Chen, Y., & Deng, N. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.717 · Cited by 33
Cross-Probe BERT for Fast Cross-Modal Search
Yu, T., Fei, H., & Li, P. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531826 · Cited by 8
Audio-Visual Mismatch-Aware Video Retrieval via Association and Adjustment
Lee, S., Park, S., & Ro, Y. M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19781-9_29 · Cited by 8
Egocentric Biochemical Video-and-Language Dataset
Nishimura, T., Sakoda, K., Hashimoto, A., Ushiku, Y., Tanaka, N., Ono, F., Kameko, H., & Mori, S. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00348 · Cited by 8
Cascaded Fast and Slow Models for Efficient Semantic Code SearchPreprint
Gotmare, A. D., Li, J., Joty, S., & Hoi, S. C. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07811 · Cited by 10
Accurate and Lightweight Learning for Specific Domain Image-Text Retrieval
Yang, R., Wang, S., Tao, J., Han, Y., Lin, Q., Guo, Y., Hou, B., & Jiao, L. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681280 · Cited by 8
Cofopose: Conditional 2D Pose Estimation with Transformers
Aidoo, E., Wang, X., Liu, Z., Tenagyei, E. K., Owusu-Agyemang, K., Kodjiku, S. L., Ejianya, V. N., & Aggrey, E. S. E. B. (2022) · Sensors · doi:10.3390/s22186821 · Cited by 7
Efficient Image-Text Retrieval via Keyword-Guided Pre-Screening
Cao, M., Bai, Y., Cao, Z., Nie, L., & Zhang, M. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3339489 · Cited by 7
How to Make Cross Encoder a Good Teacher for Efficient Image-Text Retrieval?
Chen, Y., Ma, Z., Zhang, Z., Qi, Z., Yuan, C., Li, B., Pu, J., Shan, Y., Qi, X., & Hu, W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02549 · Cited by 7
Wang, S., Zhou, F., Yang, M., Shi, L., & Tan, C. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3524297 · Cited by 7
Enhancing Text-Video Retrieval Performance With Low-Salient but Discriminative Objects
Zheng, Y., Huang, B., Chen, Z., & Yu, D. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3527369 · Cited by 6
Deep Vision-Language Model for Efficient Multi-modal Similarity Search in Fashion Retrieval
Moro, G., & Salvatori, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17849-8_4 · Cited by 6
From Within to Between: Knowledge Distillation for Cross Modality Retrieval
Tran, V., Balasubramanian, N., & Hoai, M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26316-3_36 · Cited by 5
Efficient Text-to-Code Retrieval with Cascaded Fast and Slow Transformer Models
Gotmare, A. D., Li, J., Joty, S., & Hoi, S. C. H. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616369 · Cited by 7
Huang, X. Q., Liu, J., Zhang, Z., Xie, Y., Tang, Y., Zhang, W., & Cui, X. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3384672 · Cited by 6
Multi-scale Multi-modal Dictionary BERT For Effective Text-image Retrieval in Multimedia Advertising
Yu, T., Liu, J., Jin, Z., Yang, Y., Fei, H., & Li, P. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557653 · Cited by 5
Endoscopy Image Retrieval by Mixer Multi-Layer Perceptron
Trinh, Q.-H., & Nguyen, M.-V. (2021) · Annals of Computer Science and Information Systems · doi:10.15439/2021f145 · Cited by 4
Cycle mapping with adversarial event classification network for fake news detection
Wu, F., Zhou, H., Feng, Y., Gao, G., Ji, Y., & Jing, X. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18499-z · Cited by 4
ITContrast: contrastive learning with hard negative synthesis for image-text matching
Wu, F., Wang, Q., Wang, Z., Yu, S., Li, Y., Zhang, B., & Lim, E. G. (2024) · The Visual Computer · doi:10.1007/s00371-024-03274-w · Cited by 4
Vision-Language Models Learn Super Images for Efficient Partially Relevant Video Retrieval
Nishimura, T., Nakada, S., & Kondo, M. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3708349 · Cited by 8
Learning Linguistic Association Towards Efficient Text-Video Retrieval
Fang, S., Wang, S., Zhuo, J., Han, X., & Huang, Q. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_15 · Cited by 4
HADA: A Graph-Based Amalgamation Framework in Image-text Retrieval
Nguyen, M.-D., Nguyen, B. T., & Gurrin, C. G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28244-7_45 · Cited by 4
HiVLP: Hierarchical Interactive Video-Language Pre-Training
Shao, B., Liu, J., Pei, R., Xu, S., Dai, P., Lu, J., Li, W., & Yan, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01265 · Cited by 4
MixBERT for Image-Ad Relevance Scoring in Advertising
Yu, T., Li, X., Xie, J., Yin, R., Xu, Q. Y., & Li, P. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482143 · Cited by 4
Chen, N., Xu, T., Sun, M., Yao, C., & Yang, D. (2025) · Intelligent Computing · doi:10.34133/icomputing.0143 · Cited by 5
Beyond One and Two Tower: Cross-Modal Consensus Learning for Image-Text Retrieval
Shi, Z., Ding, Y., Dong, J., & Zhang, T. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3605958 · Cited by 3
Deepfake Video Detection Based on the Decomposition of Spatial-Temporal Attention Mechanism in ViViT
Sun, G., & Lian, Z. (2024) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa63168.2024.00221 · Cited by 5
Linking Surface Facts to Large-Scale Knowledge Graphs
Radevski, G., Gashteovski, K., Hung, C.-C., Lawrence, C., & Glavaš, G. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.445 · Cited by 4
MMTF: Multi-Modal Temporal Fusion for Commonsense Video Question Answering
Ahmad, M., Park, G., Park, D., & Park, S. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00502 · Cited by 3
Dual-path temporal map optimization for make-up temporal video grounding
Li, J., Li, K., Li, J., Chen, G., Wang, M., & Guo, D. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01340-w · Cited by 3
Generating Illustrated Instructions
Menon, S., Misra, I., & Girdhar, R. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00600 · Cited by 3
Ullah, M. A., Zia, T., Kim, J.-E., & Kadry, S. (2024) · PLoS ONE · doi:10.1371/journal.pone.0311879 · Cited by 3
RetractedDisentangled OCR: A More Granular Information for “Text”-to-Image Retrieval
Zhou, X., Li, S., Chen, H., & Zhu, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-18907-4_40 · Cited by 3
Vo, T.-P., Duong, Q.-B., Nguyen, Q.-T., Mai, D., & Ly, N.-K. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-4291-5_19 · Cited by 2
Metadata
- Authors
- Miech, A., Alayrac, J.-B., Laptev, I., Sivic, J., & Zisserman, A.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 9821-9831
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3141362964
- Cited by
- 149
- References
- 85
- Citations by year
- 2026: 7 · 2025: 21 · 2024: 31 · 2023: 39 · 2022: 39 · 2021: 12
- FWCI
- 9.88
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion)
- arXiv (Cornell University)
- Funders
- European Regional Development Fund
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- text-to-image retrieval, video-text retrieval, cross-attention, dual-encoder models, approximate nearest neighbor search, knowledge distillation, re-ranking, inference efficiency, large-scale image retrieval, Flickr30k
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
