The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models
Open accessAbstract
We extend the task of composed image retrieval, where an input query consists of an image and short textual description of how to modify the image. Existing methods have only been applied to non-complex images within narrow domains, such as fashion products, thereby limiting the scope of study on in-depth visual reasoning in rich image and language contexts. To address this issue, we collect the Compose Image Retrieval on Real-life images (CIRR) dataset, which consists of over 36,000 pairs of crowd-sourced, open-domain images with human-generated modifying text. To extend current methods to the open-domain, we propose CIRPLANT, a transformer based model that leverages rich pre-trained vision-and-language (V&L) knowledge for modifying visual features conditioned on natural language. Retrieval is then done by nearest neighbor lookup on the modified features. We demonstrate that with a relatively simple architecture, CIRPLANT outperforms existing methods on open-domain images, while matching state-of-the-art accuracy on the existing narrow datasets, such as fashion. Together with the release of CIRR, we believe this work will inspire further research on composed image retrieval. Our dataset, code and pre-trained models are available at https://cuberick-orion.github.io/CIRR/.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We note that the simple prediction-based late fusion [247], [248] is less adopted in MML Transformers.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [247]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267
Cited by
Sorted by citations
Items 201–238 of 238
DART: A Dual-Modality Adaptive Representation with Divergence Training Framework for ZS-CIR
Liu, S.-S., Wang, Y., Lin, J., Wen, Y., & Yuan, C. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462656
Relate: Enhance Composed Video Retrieval Via Minimal-Redundancy Hierarchical Collaboration
Zhang, S., Chen, Z., Li, Z., Fu, Z., Wang, W., Nie, J., Wei, Y., & Hu, Y. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11461500
MELT: Improve Composed Image Retrieval Via the Modification Frequentation-Rarity Balance Network
Qiu, G., Chen, Z., Li, Z., Huang, Q., Fu, Z., Song, X., & Hu, Y. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463370
MA-CIR: A Multimodal Arithmetic Benchmark for Composed Image Retrieval
Byun, J., Jang, Y. K., Jeong, S., Kim, D., & Moon, T. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01982
Zero-Shot Composed Image Retrieval via Dual-Stream Instruction-Aware Distillation
Zhong, W., Barton, R., An, W., Jiang, F., Ma, H., Guo, Y., Dan, A., Sam, S., Bouyarmane, K., & Huang, J. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02063
Training-free Conditional Image Embedding Framework Leveraging Large Vision Language Models
Kawarada, M., Yamada, K. C., Tejero-de-Pablos, A., & Inoue, N. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00737
COMBINER: Composed Image Retrieval Guided by Attribute-Based Neighbor Relations
Li, Z., Hu, Y., Chen, Z., Wen, H., Song, X., & Nie, L. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3694139
Enhancing Representation Inversion and Alignment for Zero-Shot Composed Image Retrieval
Zhang, F., Chen, J., Zhou, H., & Xu, C. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3702526
Query-based digital forensic triage using vision language models and explainable AI
Philip, A. O., U., S. M., Raju, M., P., A. A., & Swathi, R. (2026) · Cluster Computing · doi:10.1007/s10586-026-06138-1
Zhu, H., Wang, S., Huang, J.-H., Shen, Y., Rudinac, S., & Kanoulas, E. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810694
Reranking-based Analysis on Visual and Textual Query Contribution to Composed Image Retrieval
Zhang, H., Togo, R., Ogawa, T., & Haseyama, M. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810811
Jing, G., Gao, P. F., Lee, Y., & Zhang, H. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810768
IMAGINE: Adaptive Schema-Imagery Enhanced Composition for Composed Video Retrieval
Huang, J., Li, Z., Chen, Z., Fu, Z., Wang, C., & Hu, Y. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810601
RankVR: Low-Rank Structure Perception and Value Recalibration for Robust Composed Image Retrieval
Huang, J., Li, Z., Fu, Z., Chen, Z., Huang, Q., & Hu, Y. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810604
MEDIA-BTS: Medical Image Retrieval Using Butterfly Mating Optimization for Brain Stroke Diagnosis
Yagnasri, A., & Patil, A. (2026) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-026-01303-1
Jung, G., Park, J., Kim, S. B., & Lee, S.-W. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114690
FlowCIR: Semantic Transport via Flow Matching for Zero-Shot Composed Image Retrieval
He, Z., Jiang, Z., Liu, Y., Wang, Y., Wang, T., & Chen, L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37038-9_10
Selective Pseudo Word Inversion with MLLM Reasoning for Zero-Shot Composed Image Retrieval
Yu, J., Ru, Z., Gan, M., & Yue, Z. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2856-0_41
Semantic Composition via Optimal Transport for Composed Image Retrieval
Wang, Y., Huang, W., & Yuan, C. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3703728
Beyond Vector Search: MLLM-Driven Generative Composed Image Retrieval
Xiong, R., Gengqi, Y., & Wu, D. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3432-5_30
Fairness in vision-language text-to-image retrieval: Natural biases and adversarial threats
Lazzaro, D., Ciná, A. E., Roli, F., Anguita, D., Biggio, B., Laurita, G., & Oneto, L. (2026) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2026.111385
A Sketch+Text Composed Image Retrieval Dataset for Thangka
Xu, J., Sun, Y., Zhang, J., Xie, Q., Ji, D., Bao, Z., Li, J., Ma, Y., & Liu, Y. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808581
ADaFuSE: Adaptive Diffusion-generated Image and Text Fusion for Interactive Text-to-Image Retrieval
Zhang, Z., Zhang, X., Liang, K., Li, G., McCreadie, R., & Long, Z. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809836
Controlling Embedding Spaces with Text-Conditioned Transformations
Fioresi, J., Heilbron, F. C., Nathani, P., Shah, M., & Kafle, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37041-9_30
Blocking Visual Leakage: Visually-Agnostic Text Decomposition for Composed Video Retrieval
Hwang, J., Ma, M., Pham, T. X., & Kim, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31930-2_45
Negation in Vision-Language Models: A Survey
Pokhrel, A., Ghimire, B., Paudel, P. M., & Sheshappanavar, S. V. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31927-2_17
CoCo-IR: Contextual Composed Image Retrieval
Cao, S., Dabral, T. S., Ding, Z., Shanbhogue, M., Chen, K., Li, Z., Seyedhosseini, M., Wang, Y.-X., & Gui, L.-Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37556-8_20
PhotoBench: Beyond Visual Matching Towards Personalized Intent-Driven Photo Retrieval
Xu, T., Shan, R., Wu, J., Zhu, J., Huang, J., Wang, T., Chen, W., Tu, M., Dou, Q., Wang, Z., et al. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817511
MM-BRIGHT: A Multi-Task Multimodal Benchmark for Reasoning-Intensive Retrieval
Abdallah, A., Mounis, M. D., Abdalla, M. I., Kasem, M. S., Senussi, M. F., Mahmoud, M., Abdelgwad, M. M., Jatowt, A., & Kang, H. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817457 · Cited by 12
Exploring Multi-modal Large Language Models and Two-Stage Fine-Tuning for Fashion Image Retrieval
Cao, H.-N., Bui, L.-H., Vo, H. N., & Le, T.-N. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-2600-9_19
Lu, Z., Singh, U. V., & Hu, Y. (2026) · ASCE Construction Research Congress (CRC) · doi:10.1061/9780784486979.077
OCTOPUS: Multi-agentic Universal Compositional Visual Retrieval
Cheng, Z., Zheng, B., Zhong, T., & Zhou, F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37023-5_29
Fu, B., Liu, N., Shu, Y., & Zhang, Y. (2026) · International Conference on Computer Perception and Neural Networks (CPNN) · doi:10.1109/cpnn71220.2026.11706452
Multimodal Image Retrieval Through Image-Text Query Fusion Using CLIP
Pathak, K., Kamal, A. K., Rao, A. S. K., Subrahmanyam, B., & Neeraj (2026) · International Conference on Emerging Techniques in Computational Intelligence (ICETCI) · doi:10.1109/icetci71772.2026.11710514
Optimizing Certified Radius of Zero-Shot Composed Image Retrieval Via Text Guidance
Chen, J., Ni, H., & Lai, H. (2026) · doi:10.1109/cvprf71909.2026.00083
Towards Text-Guided Attribute-Disentangled Multimodal Representation Learning
Wei, Y., Katakol, S., Brack, M., Lin, J., Bai, H., Li, Y.-T., Zhang, R., Shechtman, E., Ravi, H., & Kale, A. (2026) · doi:10.1109/cvprf71909.2026.00185
Fan, Z., Huo, Y., He, L., & Zheng, J. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729836
One Source, One Vote: Quotienting Multiplicity in Multimodal Retrieval
Yan, Q., & Hou, Z. (2026) · doi:10.1145/3840749.3841571
Metadata
- Authors
- Liu, Z., Rodriguez-Opazo, C., Teney, D., & Gould, S.
- Affiliations
- Australian National University (AU)
- University of Adelaide,Australian Institute for Machine Learning
- The University of Adelaide (AU)
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 2105-2114
- OpenAlex
- W3203247393
- Cited by
- 238
- References
- 43
- Citations by year
- 2026: 47 · 2025: 91 · 2024: 55 · 2023: 28 · 2022: 16 · 2021: 1
- FWCI
- 8.18
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- composed image retrieval, image modification, vision-language models, pre-trained language models, visual reasoning, nearest neighbor search
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
