The DOI matches its Crossref record; 78 of the 78 references it cites were found in a registry or an index.
Towards Unsupervised Image Captioning With Shared Multimodal Embeddings
Open accessAbstract
Understanding images without explicit supervision has become an important problem in computer vision. In this paper, we address image captioning by generating language descriptions of scenes without learning from annotated pairs of images and their captions. The core component of our approach is a shared latent space that is structured by visual concepts. In this space, the two modalities should be indistinguishable. A language model is first trained to encode sentences into semantically structured embeddings. Image features that are translated into this embedding space can be decoded into descriptions through the same language model, similarly to sentence embeddings. This translation is learned from weakly paired images and text using a loss robust to noisy assignments and a conditional adversarial component. Our approach allows to exploit large text corpora outside the annotated distributions of image/caption data. Our experiments show that the proposed domain alignment learns a semantically meaningful representation which outperforms previous work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The authors of [128] presented an unsupervised image captioning framework based on a new alignment method that allows the simultaneous integration of visual and textual streams through semantic learning of multimodal embeddings of the language and vision domains.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [128]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
(2008) · Journal of Machine Learning Research · Cited by 33,051
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
(2004) · Cited by 9,718
Flagged first, then by citations
Cited by
Sorted by citations
Items 101–110 of 110
Incorporating additional knowledge into image captioners
Yang, X. (2021) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/151726
Object-Centric Unsupervised Image CaptioningPreprint
Meng, Z., Yang, D. H.-Y., Cao, X., Shah, A. K., & Lim, S.-N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.00969
Removing Partial Mismatches in Unsupervised Image Captioning
Honda, U., Hashimoto, A., Watanabe, T., & Matsumoto, Y. (2022) · Transactions of the Japanese Society for Artificial Intelligence · doi:10.1527/tjsai.37-2_h-l82
Shared Embedding of X-ray & Enose Networks for Lung Cancer Classification
Liao, H.-J., Hsieh, Y.-C., Chiu, S.-W., Lee, M., Tang, K., & Sun, M. (2023) · International Conference on Biomedical Imaging, Signal Processing (ICBSP) · doi:10.1145/3634875.3634877
Semi-Supervised Image Captioning by Adversarially Propagating Labeled DataPreprint
Kim, D.-J., Oh, T.-H., Choi, J., & Kweon, I. S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4757425
SINTESIS TEKS KE GAMBAR: TINJAUAN ATAS DATASET
Arifin, S. R. (2024) · Jurnal EEICT (Electric Electronic Instrumentation Control Telecommunication) · doi:10.31602/eeict.v7i1.13066 · Cited by 1
CLIP-Prefix for Image Captioning and an Experiment on Blind Image Guessing
Huynh, T. M., Nguyen, D., Nguyen, T. T., Vu, T.-D. T., Dang-Ngoc, H., & Dang, D. N. M. (2024) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-67357-3_14
G-DPPD: Gated Data-dependent Prior Probability Distribution for Unsupervised Image Captioning
Fan, X., Tao, Q., & Tang, Y. (2024) · International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc61627.2024.10837133
Mishra, S. K., Chakraborty, S., Saha, S., & Bhattacharyya, P. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3749642
How Vision-Language Tasks Benefit From Large Pre-Trained Models: A Survey
Qi, Y., Hong-xi, L., Song, Y., Wu, X., & Luo, J. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3632653 · Cited by 5
Metadata
- Authors
- Laina, I., Rupprecht, C., & Navab, N.
- Affiliations
- Technical University of Munich (DE)
- University of Oxford (GB)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 7413-7423
- OpenAlex
- W2969295463
- Cited by
- 111
- References
- 77
- Citations by year
- 2025: 6 · 2024: 24 · 2023: 26 · 2022: 20 · 2021: 24 · 2020: 11
- FWCI
- 4.48
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- H2020 European Research Council (638009)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- visual concepts, domain alignment, sentence embeddings, language models, cross-modal translation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
