The DOI matches its Crossref record; 79 of the 85 references it cites were found in a registry or an index.
Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformers
Open accessAbstract
Our objective is language-based search of large-scale image and video datasets. For this task, the approach that consists of independently mapping text and vision to a joint embedding space, a.k.a. dual encoders, is attractive as retrieval scales and is efficient for billions of images using approximate nearest neighbour search. An alternative approach of using vision-text transformers with cross-attention gives considerable improvements in accuracy over the joint embeddings, but is often inapplicable in practice for large-scale retrieval given the cost of the cross-attention mechanisms required for each sample at test time. This work combines the best of both worlds. We make the following three contributions. First, we equip transformer-based models with a new fine-grained cross-attention architecture, providing significant improvements in retrieval accuracy whilst preserving scalability. Second, we introduce a generic approach for combining a Fast dual encoder model with our Slow but accurate transformer-based model via distillation and reranking. Finally, we validate our approach on the Flickr30K image dataset where we show an increase in inference speed by several orders of magnitude while having results competitive to the state of the art. We also extend our method to the video domain, improving the state of the art on the VATEX dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This has been studied for more challenging and fine-grained tasks (e.g., object detection [269], visual question answering [103], [106], [112], [263], and instance retrieval [222], [263]) by imposing region (semantic parts such as objects) level alignment.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [269]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
(2011) · Cited by 14,317
Research works printed but not linked: 4
- 10.1109/ICCV.2019.00054 · DOI
- 10.1109/CVPR42600.2020.01079 · DOI · Paper page
- daniel, 2017, Thinking Fast and Slow
- chen, 2017, Learning efficient object detection models with knowledge distillation, NIPS
Cited by
Sorted by citations
Items 101–149 of 149
Cross-Modal Alignment Enhancement Network for Text-to-Image Person Re-Identification
He, D., Zhu, X., Li, B., Yue, S., & Zhang, Z. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3621864 · Cited by 2
ELIP: Enhanced Visual-Language Foundation Models for Image Retrieval
Zhan, G., Liu, Y., Han, K., Xie, W., & Zisserman, A. (2025) · International Workshop on Content-Based Multimedia Indexing (CBMI) · doi:10.1109/cbmi66578.2025.11339290 · Cited by 5
Self-citationDSADF: Thinking Fast and Slow for Decision Making
Dou, Z., Cui, D., Yan, J., Wang, W., Chen, B., Wang, H., Xie, Z., & Zhang, S. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02866-4 · Cited by 11
CroMIC-QA: The Cross-Modal Information Complementation Based Question Answering
湯浅, 順., Liu, B., Sun, C., Xu, Z. X., Ma, L., & Wang, B. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3326616 · Cited by 3
Fine-Grained Cross-Modal Retrieval with Triple-Streamed Memory Fusion Transformer Encoder
Guo, W., Huang, H., Kong, X., & He, R. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859738 · Cited by 2
ConTra: (Con)text (Tra)nsformer for Cross-Modal Video Retrieval
Fragomeni, A., Wray, M., & Damen, D. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26316-3_27 · Cited by 2
InvGC: Robust Cross-Modal Retrieval by Inverse Graph Convolution
Jian, X., & Wang, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.60 · Cited by 2
Gazing After Glancing: Edge Information Guided Perception Network for Video Moment Retrieval
Huang, Z., Ji, Y., Li, Y., & Liu, C. (2024) · IEEE Signal Processing Letters · doi:10.1109/lsp.2024.3403533 · Cited by 1
Gong, Z., Mai, C., & Huang, Y. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10687993 · Cited by 2
Huang, S., Li, Z., Ling, H., Xu, L., & Tu, L. (2025) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2025.3582434 · Cited by 1
Measures of type-2 fuzzy numbers: Balancing fast and slow thinking in decision analysis
Zhu, B., Xu, Y., Yu, X., Yu, K., & Yu, K. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129197 · Cited by 1
Purified Zero-Shot Sketch-Based Image Retrieval
Zhou, Y., Yang, J., Wang, J., Huang, K., Lu, G., & He, S. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3632682 · Cited by 1
Adaptive Pseudo Text Augmentation for Noise-Robust Text-to-Image Person Re-Identification
Xiong, L. Y., Li, W., Chen, H., & Feng, Y. (2025) · Sensors · doi:10.3390/s25237157 · Cited by 1
Yu, L., Xiong, L. Y., Li, W., & Feng, Y. (2025) · Electronics · doi:10.3390/electronics14234619 · Cited by 1
Fu, Y., Gao, D., Liu, T., Zheng, H., Hao, D., & Pan, Z. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3312910 · Cited by 2
RETRACTED ARTICLE: Post hoc visual interpretation using a deep learning-based smooth feature network
Abbasi, I. N., Madni, T. M., Sohail, M. K., Janjua, U. I., & Nasir, J. A. (2023) · Soft Computing · doi:10.1007/s00500-023-09430-z · Cited by 1
RetractedTMMM: Transformer in Multimodal Sentiment Analysis under Missing Modalities
Wu, H., Li, S., & Wu, D. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651217 · Cited by 1
Guo, W., Kong, X., & Huang, H. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129003 · Cited by 1
End-to-End Large-Scale Image Retrieval Network with Convolution and Vision Transformers
Zhang, Q., Bao, F., Su, X., Wang, W., & Gao, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-15937-4_52 · Cited by 1
(2021) · Annals of Computer Science and Information Systems · doi:10.15439/978-83-959183-9-1 · Cited by 1
Natural language control for 3D human motion synthesis
Petrovich, M. (2024) · theses.fr (ABES) · doi:10.70675/a4b835c3z7030z4699zbeb2zb5faccaee2de · Cited by 1
HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models
He, Y., Wu, Y., Jia, Y., Mihalcea, R., Chen, Y., & Deng, N. (2023) · arXiv · doi:10.48550/arxiv.2310.16755 · Cited by 1
SwAMP: Swapped Assignment of Multi-Modal Pairs for Cross-Modal RetrievalPreprint
Kim, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.05814 · Cited by 1
Transformer-Enhanced Visual-Semantic Representation for Text-Image Retrieval
Zhang, M., Wu, W., & Zhang, H. (2022) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc55256.2022.10033832
Triplet Vision Transformer for Word Image Retrieval in Historical Arabic DocumentsPreprint
Fathallah, A., El‐Yacoubi, M. A., & Amara, N. E. B. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4577874
SAN: Structure-aware attention network for dyadic human relation recognition in images
Kogashi, K., Nobuhara, S., & Nishino, K. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17229-1
Unveiling the power of transfer learning towards efficient artificial intelligence
Qin, C. (2023) · Digital Repository Service (Northeastern University) · doi:10.17760/d20581917
Text-Guided Image Restoration and Semantic Enhancement for Text-to-Image Person RetrievalPreprint
Liu, D., Li, H., Zhao, Z., Dong, Y., & Boulgouris, N. V. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4648005
Boot and Switch: Alternating Distillation for Zero-Shot Dense Retrieval
Jiang, F., Xu, Q., Drummond, T. W., & Cohn, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.65 · Cited by 3
Realizing Efficient On-Device Language-based Image Retrieval
Hu, Z., Kemertas, M., Xiao, L., Phillips, C., Mohomed, I., & Fazly, A. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3649896
Image-Text Matching for Large-Scale Book Collections
Llabrés, A., Dey, A. U., Karatzas, D., & Valveny, E. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70442-0_6
Alignment-Enhanced Network for Temporal Language Grounding in Videos
Yu, H., Zhang, Y., Liu, Y., Li, H., & Liu, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72338-4_13
Cross-modal Retrieval with Attention Knowledge Distillation
Yin, H. (2024) · Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC) · doi:10.1109/ipec61310.2024.00078
Enhancing Text-Image Person Re-identification via Intra-Class Relevance Learning
He, W., Gao, Y., Liu, W., Chaomurilige, C., Liu, Z., & Guo, A. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-1528-5_12
Robust Text‐Based Person Search via Noisy Pair Identification and Pseudo‐Text Augmentation
Xiong, L., Li, W., & Chen, H. (2025) · Electronics Letters · doi:10.1049/ell2.70437
Li, X., Tong, J., Hong, J., Rolland, V., & Petersson, L. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755446 · Cited by 2
FACE: A Dual-Template and Adaptive Curriculum Framework for Unsupervised Text-Based Person Search
Mu, X., Tang, H., Jiang, H., Liang, T., Zheng, Q., & Zhu, J. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755315
Text-Oriented Image Query Representation for Zero-Shot Composed Image Retrieval
Rachabathuni, P. K., Ciamarra, A., Caldelli, R., & Bertini, M. (2025) · International Workshop on Content-Based Multimedia Indexing (CBMI) · doi:10.1109/cbmi66578.2025.11339329
Zhou, X., Luo, S., Song, L., & Hou, H. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11402549
Chaffin, A. (2023) · theses.fr (ABES) · doi:10.70675/5e4cd6f0z8697z4a0cz921dzd8592affa0a9
Unsupervised Compact Word Representations and Image Sequences Learning
Luka, N. (2024) · theses.fr (ABES) · doi:10.70675/b6c77eedz729az4c9az8e4cze41f6d07b560
Répondre aux questions visuelles à propos d'entités nommées
Lerner, P. J. (2023) · theses.fr (ABES) · doi:10.70675/78c96766zfc95z4da8zb67aze67555da6c75
Heterogeneous Quad-Modality Learning for Visible-Infrared Person Re-Identification
He, Q., & Hu, H. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3685543
Text-to-motion retrieval by text-to-motion generation
Pan, H., Wang, Q., Zhu, G., & Chen, Y. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.113983
Parameter-efficient transfer for CLIP-based text-to-person retrieval
Min, H., Zhan, G., Fan, C., ZHAO, Y., & Jia, W. (2026) · Signal Processing Image Communication · doi:10.1016/j.image.2026.117598 · Cited by 1
Robust Image Matching Driven by Mobile Embedded Sensors for Efficient Tourist Landmark Recognition
Li, L., Wang, G., Song, Z., & Ying, C. (2026) · IEEE Sensors Journal · doi:10.1109/jsen.2026.3721138
Dynamic Uncertainty Learning with Noisy Correspondence for Text-based Person Retrieval
Xie, Z., Wang, C., Cai, S., Feng, F., Luo, G., Zhang, Q., Chen, J., & Jin, T. (2026) · ACM Transactions on Information Systems · doi:10.1145/3831683
FIRE-CIR: Fine-grained Reasoning for Composed Fashion Image Retrieval
Gardères, F., Gauthier, C.-S., Ponce, J., & Chen, S. (2026) · doi:10.1109/cvprf71909.2026.00547
Metadata
- Authors
- Miech, A., Alayrac, J.-B., Laptev, I., Sivic, J., & Zisserman, A.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 9821-9831
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3141362964
- Cited by
- 149
- References
- 85
- Citations by year
- 2026: 7 · 2025: 21 · 2024: 31 · 2023: 39 · 2022: 39 · 2021: 12
- FWCI
- 9.88
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- HAL (submittedVersion)
- arXiv (Cornell University)
- Funders
- European Regional Development Fund
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- text-to-image retrieval, video-text retrieval, cross-attention, dual-encoder models, approximate nearest neighbor search, knowledge distillation, re-ranking, inference efficiency, large-scale image retrieval, Flickr30k
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
