Towards Unsupervised Image Captioning With Shared Multimodal Embeddings
Open accessAbstract
Understanding images without explicit supervision has become an important problem in computer vision. In this paper, we address image captioning by generating language descriptions of scenes without learning from annotated pairs of images and their captions. The core component of our approach is a shared latent space that is structured by visual concepts. In this space, the two modalities should be indistinguishable. A language model is first trained to encode sentences into semantically structured embeddings. Image features that are translated into this embedding space can be decoded into descriptions through the same language model, similarly to sentence embeddings. This translation is learned from weakly paired images and text using a loss robust to noisy assignments and a conditional adversarial component. Our approach allows to exploit large text corpora outside the annotated distributions of image/caption data. Our experiments show that the proposed domain alignment learns a semantically meaningful representation which outperforms previous work.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An unsupervised image captioning framework was presented that uses an alignment method to integrate visual and textual streams through semantic learning of multimodal embeddings.
“The authors of [128] presented an unsupervised image captioning framework based on a new alignment method that allows the simultaneous integration of visual and textual streams through semantic learning of multimodal embeddings of the language and vision domains.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [128]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
(2004) · Cited by 9,718 · see citing works
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · see citing works
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C. D., Surdeanu, M., Bauer, J., Finkel, J. R., Bethard, S. J., & McClosky, D. (2014) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.3115/v1/p14-5010 · Cited by 7,171 · see citing works
Improved training of Wasserstein GANs
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017) · arXiv · doi:10.48550/arxiv.1704.00028 · Cited by 6,800 · see citing works
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318 · see citing works
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228 · see citing works
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00636 · Cited by 5,117 · see citing works
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107 · see citing works
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902 · see citing works
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · see citing works
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487 · see citing works
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317 · see citing works
Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors
Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., Guadarrama, S., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.351 · Cited by 2,743 · see citing works
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01316-z · Cited by 2,460 · see citing works
Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00982 · Cited by 2,194 · see citing works
Self-Critical Sequence Training for Image Captioning
Rennie, S. J., Marcheret, E., Mroueh, Y., Ross, J., & Goel, V. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.131 · Cited by 2,177 · see citing works
Meteor Universal: Language Specific Translation Evaluation for Any Target Language
Denkowski, M., & Lavie, A. (2014) · WMT@ACL · doi:10.3115/v1/w14-3348 · Cited by 2,108 · see citing works
SPICE: Semantic Propositional Image Caption Evaluation
Anderson, P., Fernando, B., Johnson, M., & Gould, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_24 · Cited by 1,986 · see citing works
Adversarial Feature LearningPreprint
Donahue, J., Krähenbühl, P., & Darrell, T. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1605.09782 · Cited by 1,982 · see citing works
Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning
Sharma, P., Ding, N., Goodman, S., & Soricut, R. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1238 · Cited by 1,879 · see citing works
Image Captioning with Semantic Attention
You, Q., Jin, H., Wang, Z., Chen, F., & Luo, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.503 · Cited by 1,859 · see citing works
Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning
Lu, J., Xiong, C., Parikh, D., & Socher, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.345 · Cited by 1,724 · see citing works
From Word Embeddings To Document Distances
Kusner, M. J., Sun, Y., Kolkin, N., & Weinberger, K. Q. (2015) · PolyPublie (École Polytechnique de Montréal) · Cited by 1,695 · see citing works
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models
Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J., & Lazebnik, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.303 · Cited by 1,444 · see citing works
Unifying Visual-Semantic Embeddings with Multimodal Neural Language ModelsPreprint
Kiros, R., Salakhutdinov, R., & Zemel, R. S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.2539 · Cited by 1,386 · see citing works
From captions to visual concepts and back
Fang, H., Gupta, S., Iandola, F., Srivastava, R. K., Deng, L., Dollár, P., Gao, J., He, X., Mitchell, M., Platt, J. C., et al. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298754 · Cited by 1,305 · see citing works
DenseCap: Fully Convolutional Localization Networks for Dense Captioning
Johnson, J., Karpathy, A., & Fei-Fei, L. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.494 · Cited by 1,161 · see citing works
Long-term Recurrent Convolutional Networks for Visual Recognition and Description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., & Darrell, T. (2014) · doi:10.21236/ada623249 · Cited by 1,081 · see citing works
VSE++: Improving Visual-Semantic Embeddings with Hard NegativesPreprint
Faghri, F., Fleet, D. J., Kiros, J., & Fidler, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05612 · Cited by 1,002 · see citing works
Exploring Visual Relationship for Image Captioning
Yao, T., Pan, Y., Li, Y., & Mei, T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_42 · Cited by 962 · see citing works
Boosting Image Captioning with Attributes
Yao, T., Pan, Y., Li, Y., Qiu, Z., & Mei, T. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.524 · Cited by 752 · see citing works
Phrase-Based & Neural Unsupervised Machine Translation
Lample, G., Ott, M., Conneau, A., Denoyer, L., & Ranzato, M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1549 · Cited by 647 · see citing works
VizWiz Grand Challenge: Answering Visual Questions from Blind People
Gurari, D., Li, Q., Stangl, A. J., Guo, A., Lin, C., Grauman, K., Luo, J., & Bigham, J. P. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00380 · Cited by 621 · cited 1× · see citing works
“Assisting visually impaired people [23, 62] and human-robot interaction [12, 39] are some examples of the importance of image captioning.”
BackgroundUnsupervised Machine Translation Using Monolingual Corpora Only
Lample, G., Conneau, A., Denoyer, L., & Ranzato, M. (2017) · arXiv · doi:10.48550/arxiv.1711.00043 · Cited by 584 · see citing works
Jointly Modeling Embedding and Translation to Bridge Video and Language
Pan, Y., Mei, T., Yao, T., Li, H., & Rui, Y. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.497 · Cited by 573 · see citing works
Towards Diverse and Natural Image Descriptions via a Conditional GAN
Dai, B., Fidler, S., Urtasun, R., & Lin, D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.323 · Cited by 483 · see citing works
Das, A., Kottur, S., Gupta, K., Singh, A., Yadav, D., Moura, J. M. F., Parikh, D., & Batra, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.121 · Cited by 445 · see citing works
Das, A., Kottur, S., Gupta, K., Singh, A., Jain, D., Moura, J. M. F., Parikh, D., & Batra, D. (2017) · Cited by 432 · see citing works
Improved Image Captioning via Policy Gradient optimization of SPIDEr
Liu, S., Zhu, Z., Ye, N., Guadarrama, S., & Murphy, K. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.100 · Cited by 430 · see citing works
Lu, J., Yang, J., Batra, D., & Parikh, D. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00754 · Cited by 412 · see citing works
Women Also Snowboard: Overcoming Bias in Captioning Models
Hendricks, L. A., Burns, K., Saenko, K., Darrell, T., & Rohrbach, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_47 · Cited by 404 · see citing works
StyleNet: Generating Attractive Visual Captions with Styles
Gan, C., Gan, Z., He, X., Gao, J., & Deng, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.108 · Cited by 287 · see citing works
Deep Compositional Captioning: Describing Novel Object Categories without Paired Training Data
Hendricks, L. A., Venugopalan, S., Rohrbach, M., Mooney, R. J., Saenko, K., & Darrell, T. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.8 · Cited by 280 · see citing works
(2017) · doi:10.1145/2998181.2998364 · Cited by 258 · see citing works
Feng, Y., Ma, L., Liu, W., & Luo, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00425 · Cited by 234 · see citing works
Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
Shetty, R., Rohrbach, M., Hendricks, L. A., Fritz, M., & Schiele, B. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.445 · Cited by 231 · see citing works
Review Networks for Caption GenerationPreprint
Yang, Z., Yuan, Y., Wu, Y., Salakhutdinov, R., & Cohen, W. W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07912 · Cited by 230 · see citing works
Guided Open Vocabulary Image Captioning with Constrained Beam Search
Anderson, P., Fernando, B., Johnson, M., & Gould, S. J. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1098 · Cited by 200 · see citing works
Deforming Autoencoders: Unsupervised Disentangling of Shape and Appearance
Shu, Z., Sahasrabudhe, M., Güler, R. A., Samaras, D., Nikos Paragios, A., & Kokkinos, I. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_40 · Cited by 183 · see citing works
Captioning Images with Diverse Objects
Venugopalan, S., Hendricks, L. A., Rohrbach, M., Mooney, R. J., Darrell, T., & Saenko, K. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.130 · Cited by 178 · see citing works
Engaging Image Captioning via Personality
Shuster, K., Humeau, S., Hu, H., Bordes, A., & Weston, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01280 · Cited by 160 · see citing works
Show, Adapt and Tell: Adversarial Training of Cross-Domain Image Captioner
Chen, T.-H., Liao, Y.-H., Chuang, C.-Y., Hsu, W.-T., Fu, J., & Sun, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.64 · Cited by 158 · see citing works
Incorporating Copying Mechanism in Image Captioning for Learning Novel Objects
Yao, T., Pan, Y., Li, Y., & Mei, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.559 · Cited by 157 · see citing works
Learning Like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images
Mao, J., Xu, W., Yang, Y., Wang, J., Huang, Z., & Yuille, A. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.291 · Cited by 147 · see citing works
Wang, L., Schwing, A. G., & Lazebnik, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.07068 · Cited by 143 · see citing works
SemStyle: Learning to Generate Stylised Image Captions Using Unaligned Text
Mathews, A., Xie, L., & He, X. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00896 · Cited by 127 · see citing works
MSCap: Multi-Style Image Captioning With Unpaired Stylized Text
Guo, L., Liu, J., Yao, P., Li, J., & Lu, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00433 · Cited by 120 · see citing works
Women also Snowboard: Overcoming Bias in Captioning ModelsPreprint
Burns, K., Hendricks, L. A., Saenko, K., Darrell, T., & Rohrbach, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09797 · Cited by 113 · see citing works
Adversarial Text Generation via Feature-Mover's DistancePreprint
Chen, L., Dai, S., Tao, C., Shen, D., Gan, Z., Zhang, H., Zhang, Y., & Carin, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.06297 · Cited by 91 · see citing works
Unpaired Image Captioning by Language Pivoting
Gu, J., Joty, S., Cai, J., & Wang, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_31 · Cited by 91 · see citing works
Deforming Autoencoders: Unsupervised Disentangling of Shape and AppearancePreprint
Shu, Z., Sahasrabudhe, M., Guler, A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06503 · Cited by 88 · see citing works
Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of ImagesPreprint
Mao, J., Xu, W., Yang, Y. P., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.06692 · Cited by 63 · see citing works
Generating Diverse and Accurate Visual Captions by Comparative Adversarial LearningPreprint
Li, D., Huang, Q., He, X.-D., Zhang, L., & Sun, M.-T. (2018) · arXiv.org · doi:10.48550/arxiv.1804.00861 · Cited by 50 · see citing works
Dual Learning for Cross-domain Image Captioning
Zhao, W., Xu, W. H., Yang, M., Ye, J., Zhao, Z., Feng, Y., & Qiao, Y. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3132920 · Cited by 49 · see citing works
Teaching Machines to Describe Images with Natural Language Feedback
Ling, H., & Fidler, S. (2017) · neural information processing systems · Cited by 38 · see citing works
Towards Text Generation with Adversarially Learned Neural Outlines
Subramanian, S., Mudumba, S. R., Sordoni, A., Trischler, A., Courville, A. C., & Pal, C. (2018) · PolyPublie (École Polytechnique de Montréal) · doi:10.5555/3327757.3327855 · Cited by 37 · see citing works
Teaching Machines to Describe Images via Natural Language FeedbackPreprint
Ling, H., & Fidler, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.00130 · Cited by 28 · see citing works
Partially-Supervised Image CaptioningPreprint
Anderson, P., Gould, S. J., & Johnson, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06004 · Cited by 28 · see citing works
Unsupervised Image CaptioningPreprint
Feng, Y., Ma, L., Liu, W., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.10787 · Cited by 10 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456 · see citing works
From Show to Tell: A Survey on Deep Learning-Based Image Captioning
Stefanini, M., Cornia, M., Baraldi, L., Cascianelli, S., Fiameni, G., & Cucchiara, R. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3148210 · Cited by 373 · see citing works
Rahate, A., Walambe, R., Ramanna, S., & Kotecha, K. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.12.003 · Cited by 214 · see citing works
ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic
Tewel, Y., Shalev, Y., Schwartz, I., & Wolf, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01739 · Cited by 166 · see citing works
Learning Video Moment Retrieval Without a Single Annotated Video
Gao, J., & Xu, C. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3075470 · Cited by 117 · see citing works
Video Captioning Using Global-Local Representation
Yan, L., Ma, S.-Q., Wang, Q., Chen, Y., Zhang, X., Savakis, A., & Liu, D. (2022) · IEEE transactions on circuits and systems for video technology (Print) · doi:10.1109/tcsvt.2022.3177320 · Cited by 106 · see citing works
Multimodal research in vision and language: A review of current and emerging trends
Uppal, S., Bhagat, S., Hazarika, D., Majumder, N., Poria, S., Zimmermann, R., & Zadeh, A. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.07.009 · Cited by 112 · see citing works
Text-Only Training for Image Captioning using Noise-Injected CLIP
Nukrai, D., Mokady, R., & Globerson, A. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.299 · Cited by 89 · see citing works
Deep image captioning: A review of methods, trends and future challenges
Xu, L., Tang, Q., Lv, J., Zheng, B., Zeng, X., & Li, W. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126287 · Cited by 78 · see citing works
Visuals to Text: A Comprehensive Review on Automatic Image Captioning
Ming, Y., Hu, N., Fan, C., Feng, F., Zhou, J., & Yu, H. L. (2022) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.105734 · Cited by 80 · see citing works
Transferable Decoding with Visual Entities for Zero-Shot Image Captioning
Fei, J., Wang, T., Zhang, J., He, Z., Wang, C., & Zheng, F. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00291 · Cited by 72 · see citing works
Pseudo-Q: Generating Pseudo Language Queries for Visual Grounding
Jiang, H., Lin, Y., Han, D., Song, S., & Huang, G. M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01507 · Cited by 69 · see citing works
Auto-encoding and Distilling Scene Graphs for Image Captioning
Yang, X., Zhang, H., & Cai, J. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3042192 · Cited by 65 · see citing works
Cross-Domain Image Captioning via Cross-Modal Retrieval and Model Adaptation
Zhao, W., Wu, X., & Luo, J. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3042086 · Cited by 65 · see citing works
Neural attention for image captioning: review of outstanding methods
Zohourianshahzadi, Z., & Kalita, J. K. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10092-2 · Cited by 61 · see citing works
Wang, Z., Liang, X., Li, M., Li, S., Liu, J., & Zheng, L. (2024) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2024.100759 · Cited by 62 · see citing works
Prompt-Based Learning for Unpaired Image Captioning
Zhu, P., Wang, X., Zhu, L., Sun, Z., Zheng, W., Wang, Y., & Chen, C. W. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3265842 · Cited by 52 · see citing works
Unpaired Image Captioning With semantic-Constrained Self-Learning
Ben, H., Pan, Y., Li, Y., Yao, T., Hong, R., Wang, M., & Mei, T. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3060948 · Cited by 51 · see citing works
Image captioning by diffusion models: a survey
Daneshfar, F., Bartani, A., & Lotfi, P. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109288 · Cited by 49 · see citing works
Unsupervised Temporal Video Grounding with Deep Semantic Clustering
Liu, D., Qu, X., Wang, Y., Di, X., Zou, K., Cheng, Y., Xu, Z., & Zhou, P. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20060 · Cited by 49 · see citing works
Memorial GAN With Joint Semantic Optimization for Unpaired Image Captioning
Song, P., Guo, D., Zhou, J., Xu, M., & Wang, M. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3175012 · Cited by 41 · see citing works
Deep Vision Multimodal Learning: Methodology, Benchmark, and Trend
Chai, W., & Wang, G. (2022) · Applied Sciences · doi:10.3390/app12136588 · Cited by 43 · see citing works
Unsupervised Vision-and-Language Pre-training Without Parallel Images and Captions
Li, L. H., You, H., Wang, Z., Zareian, A., Chang, S., & Chang, K.-W. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.420 · Cited by 45 · see citing works
Graph neural networks in vision-language image understanding: a survey
Senior, H., Slabaugh, G., Yuan, S., & Rossi, L. (2024) · The Visual Computer · doi:10.1007/s00371-024-03343-0 · Cited by 36 · see citing works
Recurrent Relational Memory Network for Unsupervised Image Captioning
Guo, D., Wang, Y., Song, P., & Wang, M. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/128 · Cited by 35 · see citing works
Interactions Guided Generative Adversarial Network for unsupervised image captioning
Cao, S., An, G., Zheng, Z., & Ruan, Q. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.08.019 · Cited by 33 · see citing works
An Overview of Deep Neural Networks for Few-Shot Learning
Zhao, J., Kong, L., & Lv, J. (2024) · Big Data Mining and Analytics · doi:10.26599/bdma.2024.9020049 · Cited by 32 · see citing works
From Show to Tell: A Survey on Image CaptioningPreprint
Stefanini, M., Cornia, M., Baraldi, L., Cascianelli, S., Fiameni, G., & Cucchiara, R. (2021) · arXiv (Cornell University) · Cited by 31 · see citing works
Cross-modal text and visual generation: A systematic review. Part 1: Image to text
Żelaszczyk, M., & Mańdziuk, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.01.008 · Cited by 29 · see citing works
SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Text
Chowdhury, P. N., Bhunia, A. K., Sain, A., Koley, S., Xiang, T., & Song, Y.-Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01056 · Cited by 30 · see citing works
Text-based Person Search without Parallel Image-Text Data
Bai, Y., Wang, J., Cao, M., Chen, C., Cao, Z., Nie, L., & Zhang, M. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612285 · Cited by 28 · see citing works
Language-free Training for Zero-shot Video Grounding
Kim, D., Park, J., Lee, J., Park, S., & Sohn, K. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00257 · Cited by 28 · see citing works
CgT-GAN: CLIP-guided Text GAN for Image Captioning
Yu, J., Li, H., Hao, Y., Zhu, B., Xu, T., & He, X. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611891 · Cited by 27 · see citing works
Zero-Shot Video Grounding With Pseudo Query Lookup and Verification
Lu, Y., Quan, R., Zhu, L., & Yang, Y. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3365249 · Cited by 26 · see citing works
AERMNet: Attention-enhanced relational memory network for medical image report generation
Zeng, X., Liao, T., Xu, L., & Wang, Z. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107979 · Cited by 24 · see citing works
Image Captioning with Multi-Context Synthetic Data
Ma, F., Zhou, Y., Rao, F., Zhang, Y., & Sun, X. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28203 · Cited by 25 · see citing works
Improving Cross-Modal Alignment with Synthetic Pairs for Text-Only Image Captioning
Liu, Z., Liu, J., & Ma, F. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i4.28178 · Cited by 22 · see citing works
Aligning Source Visual and Target Language Domains for Unpaired Video Captioning
Liu, F., Wu, X., You, C., Ge, S., Zou, Y., & Sun, X. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3132229 · Cited by 22 · see citing works
Wang, B., Huang, S., Liang, B., Tu, G., Yang, M., & Xu, R. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111778 · Cited by 20 · see citing works
Zhang, K., Li, P., & Wang, J. (2024) · Remote Sensing · doi:10.3390/rs16214113 · Cited by 23 · see citing works
Image Captioning With Positional and Geometrical Semantics
Haque, A. U., Ghani, S., & Saeed, M. (2021) · IEEE Access · doi:10.1109/access.2021.3131343 · Cited by 19 · see citing works
Image captioning based on scene graphs: A survey
Jia, J., Ding, X., Pang, S., Gao, X., Xin, X., Hu, R., & Nie, J. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120698 · Cited by 19 · see citing works
Multimodal Data Augmentation for Image Captioning using Diffusion Models
Xiao, C., Xu, S. X., & Zhang, K. (2023) · doi:10.1145/3607827.3616839 · Cited by 23 · see citing works
Self-Distillation for Few-Shot Image Captioning
Chen, X., Jiang, M., & Zhao, Q. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00059 · Cited by 18 · see citing works
Self-Learning for Few-Shot Remote Sensing Image Captioning
Zhou, H., Du, X., Xia, L., & Li, S. (2022) · Remote Sensing · doi:10.3390/rs14184606 · Cited by 18 · see citing works
Yang, B., Liu, F., Zou, Y., Wu, X., Wang, Y., & Clifton, D. A. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3371376 · Cited by 20 · see citing works
Text-Guided Human Image Manipulation via Image-Text Shared Space
Xu, X., Chen, Y.-C., Tao, X., & Jia, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3085339 · Cited by 16 · see citing works
Fusing Pre-Trained Language Models with Multimodal Prompts through Reinforcement Learning
Yu, Y., Chung, J., Yun, H., Hessel, J., Park, J. S., Lu, X., Zellers, R., Ammanabrolu, P., Bras, R. J. L., Kim, G., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01044 · Cited by 16 · see citing works
Honda, U., Ushiku, Y., Hashimoto, A., Watanabe, T., & Matsumoto, Y. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.323 · Cited by 16 · see citing works
MultiCapCLIP: Auto-Encoding Prompts for Zero-Shot Multilingual Visual Captioning
Yang, B., Liu, F., Wu, X., Wang, Y., Sun, X., & Zou, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.664 · Cited by 16 · see citing works
Background-Insensitive Scene Text Recognition with Text Semantic Segmentation
Zhao, L., Wu, Z., Wu, X., Wilsbacher, G., & Wang, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19806-9_10 · Cited by 15 · see citing works
Assessing Multilingual Fairness in Pre-trained Multimodal Representations
Wang, J., Liu, Y., & Wang, X. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.211 · Cited by 14 · see citing works
Object-Centric Unsupervised Image Captioning
Meng, Z., Yang, D. H.-Y., Cao, X., Shah, A. K., & Lim, S.-N. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_13 · Cited by 14 · see citing works
From Association to Generation: Text-only Captioning by Unsupervised Cross-modal Mapping
Wang, J., Yan, M., Zhang, Y., & Sang, J. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/481 · Cited by 14 · see citing works
Unpaired Image Captioning by Image-Level Weakly-Supervised Visual Concept Recognition
Zhu, P., Wang, X., Luo, Y., Sun, Z., Zheng, W., Wang, Y., & Chen, C. W. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3214090 · Cited by 14 · see citing works
Modal-specific Pseudo Query Generation for Video Corpus Moment Retrieval
Jung, M., Choi, S., Kim, J. C., Kim, J.-H., & Zhang, B. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.530 · Cited by 15 · see citing works
A Survey on Enhancing Image Captioning with Advanced Strategies and Techniques
Zou, B., Thobhani, A., Kui, X., Abdussalam, A., Asim, M., Shah, S., & ELAffendi, M. A. (2025) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2025.059192 · Cited by 15 · see citing works
EntroCap: Zero-shot image captioning with entropy-based retrieval
Yan, J., Xie, Y., Zou, S., Wei, Y., & Luan, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128666 · Cited by 12 · see citing works
Sharma, A., & Aggarwal, M. (2025) · SN Computer Science · doi:10.1007/s42979-025-03719-6 · Cited by 12 · see citing works
UNISON: Unpaired Cross-Lingual Image Captioning
Gao, J., Zhou, Y. X., Yu, P. L. H., Joty, S., & Gu, J. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21310 · Cited by 12 · see citing works
Automated Medical Image Captioning with Soft Attention-Based LSTM Model Utilizing YOLOv4 Algorithm
Ravinder, P., & Srinivasan, S. (2023) · Journal of Computer Science · doi:10.3844/jcssp.2024.52.68 · Cited by 11 · see citing works
Style-Enhanced Transformer for Image Captioning in Construction Scenes
Song, K., Chen, L., & Wang, H. (2024) · Entropy · doi:10.3390/e26030224 · Cited by 11 · see citing works
Mining core information by evaluating semantic importance for unpaired image captioning
Wei, J., Li, Z., Zhang, C., & Ma, H. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106519 · Cited by 11 · see citing works
Multimodal Unlearnable Examples: Protecting Data against Multimodal Contrastive Learning
Liu, X., Jia, X., Xun, Y., Liang, S., & Cao, X. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680708 · Cited by 11 · see citing works
Unsupervised Cross-lingual Image Captioning.Preprint
Gao, J., Zhou, Y., Yu, P. L. H., & Gu, J. (2020) · arXiv (Cornell University) · Cited by 9 · see citing works
Towards Unsupervised Referring Expression Comprehension with Visual Semantic Parsing
Wang, Y., Ji, Z., Wang, D., Pang, Y., & Li, X. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111318 · Cited by 8 · see citing works
GazeXplain: Learning to Predict Natural Language Explanations of Visual Scanpaths
Chen, X., Jiang, M. X., & Zhao, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73242-3_18 · Cited by 7 · see citing works
A comprehensive survey on deep-learning-based visual captioning
Xin, B., Xu, N., Zhai, Y., Zhang, T., Lu, Z., Liu, J., Nie, W., Li, X., & Liu, A.-A. (2023) · Multimedia Systems · doi:10.1007/s00530-023-01175-x · Cited by 7 · see citing works
CroCaps: A CLIP-assisted cross-domain video captioner
Xu, W., Xu, Y., Miao, Z., Cen, Y., Wan, L., & Ma, X. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.126296 · Cited by 7 · see citing works
Thai-IC: Thai Image Captioning based on CNN-RNN Architecture
Mookdarsanit, L., & Mookdarsanit, P. (2020) · Cited by 7 · see citing works
Liu, Z., Liu, J., Ling, X., Huang, Q., & Wang, J. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113274 · Cited by 6 · see citing works
Perturb, Predict & Paraphrase: Semi-Supervised Learning using Noisy Student for Image Captioning
Jain, A., Samala, P. R., Jyothi, P., Mittal, D., & Singh, M. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/105 · Cited by 6 · see citing works
Generative adversarial network for semi-supervised image captioning
Liang, 徐. 良. X., Li, C., & Tian, L. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104199 · Cited by 6 · see citing works
Evaluation metrics for video captioning: A survey
Inácio, A. D. S., & Lopes, H. S. (2023) · Machine Learning with Applications · doi:10.1016/j.mlwa.2023.100488 · Cited by 8 · see citing works
Unsupervised Vision-and-Language Pre-training Without Parallel Images and CaptionsPreprint
Li, L. H., You, H., Wang, Z., Zareian, A., Chang, S., & Chang, K.-W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12831 · Cited by 6 · see citing works
Berger, U., Stanovsky, G., Abend, O., & Frermann, L. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.52 · Cited by 5 · see citing works
ZeroGen: Zero-Shot Multimodal Controllable Text Generation with Multiple Oracles
Tu, H., Yang, B., & Zhao, X. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44696-2_39 · Cited by 5 · see citing works
Generating diverse chinese poetry from images via unsupervised method
Wang, J., Li, H., Wu, C., Gong, F., & Wang, L. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.04.024 · Cited by 5 · see citing works
Semantic-Enhanced Cross-Modal Fusion for Improved Unsupervised Image Captioning
Xiang, N., Chen, L., Liang, L., Rao, X., & Gong, Z. (2023) · Electronics · doi:10.3390/electronics12173549 · Cited by 6 · see citing works
Pseudo Content Hallucination for Unpaired Image Captioning
Ben, H., Wang, S., Wang, M., & Hong, R. (2024) · International Conference on Multimedia Retrieval · doi:10.1145/3652583.3658080 · Cited by 5 · see citing works
Multimodal End-to-End Group Emotion Recognition using Cross-Modal AttentionPreprint
Evtodienko, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.05890 · Cited by 12 · see citing works
Incorporating semantic consistency for improved semi-supervised image captioning
Wu, B., & Wo, Y. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17577-y · Cited by 4 · see citing works
Semi-Supervised Learning for Video Captioning
Lin, K., Gan, Z., & Wang, L. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.98 · Cited by 4 · see citing works
A Scientometric Visualization Analysis of Image Captioning Research From 2010 to 2020
Liu, W., Wu, H., Hu, K., Luo, Q., & Cheng, X. (2021) · IEEE Access · doi:10.1109/access.2021.3129782 · Cited by 4 · see citing works
Detection and Description of Change in Visual StreamsPreprint
Gilton, D., Luo, R., Willett, R. M., & Shakhnarovich, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.12633 · Cited by 4 · see citing works
Deep Image Captioning Survey: A Resource Availability Perspective
AlSulaimi, M., Ahmad, I., & Jeragh, M. (2021) · Conference of Open Innovations Association (FRUCT) · doi:10.23919/fruct52173.2021.9435534 · Cited by 3 · see citing works
Data-Driven In-Crisis Community Identification for Disaster Response and Management
Tao, Y., Jiang, R., Coltey, E., Yang, C., Song, X., Shibasaki, R., Shyu, M., & Chen, S. (2021) · IEEE International Conference on Collaboration and Internet Computing (CIC) · doi:10.1109/cic52973.2021.00021 · Cited by 3 · see citing works
Emerging Trends of Multimodal Research in Vision and Language.Preprint
Uppal, S., Bhagat, S., Hazarika, D., Majumder, N., Poria, S., Zimmermann, R., & Zadeh, A. (2020) · arXiv (Cornell University) · Cited by 3 · see citing works
Exploring Semantic Relationships for Unpaired Image CaptioningPreprint
Liu, F., Gao, M., Zhang, T., & Zou, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.10658 · Cited by 3 · see citing works
Zero-shot Natural Language Video Localization
Nam, J., Ahn, D., Kang, D., Ha, S. J., & Choi, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00150 · Cited by 2 · see citing works
KENGIC: KEyword-driven and N-Gram Graph based Image Captioning
Birmingham, B., & Muscat, A. (2022) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta56598.2022.10034584 · Cited by 2 · see citing works
Text-to-Text Pre-Training with Paraphrasing for Improving Transformer-Based Image Captioning
Masumura, R., Makishima, N., Ihori, M., Takashima, A., Tanaka, T., & Orihashi, S. (2023) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco58844.2023.10289992 · Cited by 2 · see citing works
MedRAT: Unpaired Medical Report Generation via Auxiliary Tasks
Hirsch, E., Dawidowicz, G., & Tal, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73209-6_2 · Cited by 2 · see citing works
Towards Few-shot Image Captioning with Cycle-based Compositional Semantic Enhancement Framework
Zhang, P., Bai, Y., Su, J., Huang, Y., & Long, Y. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191558 · Cited by 2 · see citing works
Learning modular language-conditioned robot policies through attention
Zhou, Y., Sonawani, S. D., Phielipp, M., Amor, H. B., & Stepputtis, S. (2023) · Autonomous Robots · doi:10.1007/s10514-023-10129-1 · Cited by 7 · see citing works
Semi-Supervised Image Captioning by Adversarially Propagating Labeled DataPreprint
Kim, D.-J., Oh, T.-H., Choi, J., & Kweon, I. S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4583222 · Cited by 2 · see citing works
Visual Relation-Aware Unsupervised Video Captioning
Ji, P., Cao, M., & Zou, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-15934-3_41 · Cited by 1 · see citing works
Exploring Annotation-free Image Captioning with Retrieval-augmented Pseudo Sentence Generation
Li, Z., Liu, D., Wang, H., Zhang, C., & Cai, W. (2024) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3696409.3700223 · Cited by 1 · see citing works
Video Captioning Using Weak AnnotationPreprint
Hou, J., Jia, Y., Wu, X., & Qi, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.01067 · Cited by 1 · see citing works
Honda, U., Ushiku, Y., Hashimoto, A., Watanabe, T., & Matsumoto, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.13872 · Cited by 1 · see citing works
- Authors
- Laina, I., Rupprecht, C., & Navab, N.
- Affiliations
- Technical University of Munich (DE)
- University of Oxford (GB)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 7413-7423
- OpenAlex
- W2969295463
- Cited by
- 111
- References
- 77
- Citations by year
- 2025: 6 · 2024: 24 · 2023: 26 · 2022: 20 · 2021: 24 · 2020: 11
- FWCI
- 4.48
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- H2020 European Research Council (638009)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- visual concepts, domain alignment, sentence embeddings, language models, cross-modal translation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
