The DOI matches its Crossref record; 37 of the 40 references it cites were found in a registry or an index.
Show and tell: A neural image caption generator
Abstract
Automatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing. In this paper, we present a generative model based on a deep recurrent architecture that combines recent advances in computer vision and machine translation and that can be used to generate natural sentences describing an image. The model is trained to maximize the likelihood of the target description sentence given the training image. Experiments on several datasets show the accuracy of the model and the fluency of the language it learns solely from image descriptions. Our model is often quite accurate, which we verify both qualitatively and quantitatively. For instance, while the current state-of-the-art BLEU-1 score (the higher the better) on the Pascal dataset is 25, our approach yields 59, to be compared to human performance around 69. We also show BLEU-1 score improvements on Flickr30k, from 56 to 66, and on SBU, from 19 to 28. Lastly, on the newly released COCO dataset, we achieve a BLEU-4 of 27.7, which is the current state-of-the-art.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“They coupled a convolutional neural network for encoding of the image, with a recurrent neural network - specifically a long-short term memory net (LSTM) (Hochreiter and Schmidhuber (1997)) - for textual encoding.”
From Explainable artificial intelligence (XAI) in deep learning-based medical image analysis (2022) · cites this paper as Vinyals et al. (2015)
In the citing paper
“They used human-generated sentences as ground truth for training, and used the bilingual evaluation understudy (BLEU) metric for evaluation.”
From Explainable artificial intelligence (XAI) in deep learning-based medical image analysis (2022) · cites this paper as Vinyals et al. (2015)
In the citing paper
“Vinyals et al. (2015) provided textual explanation for images using an end-to-end image captioning framework.”
From Explainable artificial intelligence (XAI) in deep learning-based medical image analysis (2022) · cites this paper as Vinyals et al. (2015)
In the citing paper
“Similarly, an image and a natural language caption can capture complimentary but converging information about a scene [32, 35].”
From Multimodal Generative Models for Scalable Weakly-Supervised Learning (2018) · cites this paper as [32]
In the citing paper
“Dependence on complex datasets including images and textual annotations”
From Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches (2023) · cites this paper as [43]
In the citing paper
“Semantic relationship between images and textual descriptions is intuitive, even for nonexperts”
From Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches (2023) · cites this paper as [43]
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Self-citationNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
Flagged first, then by citations
3 of the 40 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- kuznetsova, 2014, Treetalk: Composition and compression of trees for image descriptions, ACL, 2
Cited by
Sorted by citations
Items 201–300 of 1,039
GuessWhat?! Visual Object Discovery through Multi-modal Dialogue
de Vries, H., Strub, F., Chandar, S., Pietquin, O., Larochelle, H., & Courville, A. C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.475 · Cited by 385
Task-Adaptive Attention for Image Captioning
Yan, C. C., Hao, Y., Li, L., Yin, J., Liu, A.-A., Mao, Z., Chen, Z., & Gao, X. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3067449 · Cited by 369
Cross-modal Memory Networks for Radiology Report Generation
Chen, Z., Shen, Y., Song, Y., & Wan, X. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.459 · Cited by 369
“…studies adopt a conventional encoder-decoder architecture, with convolutional neural networks (CNNs) as the encoder and recurrent (e.g., LSTM/GRU) or non-recurrent networks (e.g., Transformer) as the decoder following the image captioning paradigm (Vinyals et al., 2015; Anderson et al., 2018).”
BackgroundMethodInfluentialViewpoint: Human-in-the-loop Artificial Intelligence
Zanzotto, F. M. (2019) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11345 · Cited by 380
Image Captioning: Transforming Objects into WordsPreprint
Herdade, S., Kappeler, A., Boakye, K., & Soares, J. C. V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05963 · Cited by 407
Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
Das, A., Kottur, S., Moura, J. M. F., Lee, S., & Batra, D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.321 · Cited by 386
Ask Me Anything: Free-Form Visual Question Answering Based on Knowledge from External Sources
Wu, Q., Wang, P., Shen, C., Dick, A., & van den Hengel, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.500 · Cited by 368
Travel time prediction with LSTM neural network
Duan, Y., Yisheng, L. V., & Wang, F. (2016) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2016.7795686 · Cited by 372
MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network
Zhang, Z., Xie, Y., Xing, F., McGough, M., & Yang, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.378 · Cited by 354
ClipCap: CLIP Prefix for Image CaptioningPreprint
Mokady, R., Hertz, A., & Bermano, A. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.09734 · Cited by 929
“Early works have used LSTM variants [8, 38, 39], while recent works [16, 26] adopted the improved trans-former architecture [36].”
Reconstruction Network for Video Captioning
Wang, B., Ma, L., Zhang, W., & Liu, W. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00795 · Cited by 353
VirTex: Learning Visual Representations from Textual Annotations
Desai, K., & Johnson, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01101 · Cited by 376
Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures
Bernardi, R., Çakıcı, R., Elliott, D., Erdem, A., Erdem, E., İkizler-Cinbiş, N., Keller, F., Muscat, A., & Plank, B. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4900 · Cited by 378
Inferring Semantic Layout for Hierarchical Text-to-Image Synthesis
Hong, S., Yang, D., Choi, J., & Lee, H. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00833 · Cited by 350
Sharpness-Aware Low-Dose CT Denoising Using Conditional Generative Adversarial Network
Xin, Y., & Babyn, P. (2018) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-018-0056-0 · Cited by 344
Mnemonic Descent Method: A Recurrent Process Applied for End-to-End Face Alignment
Trigeorgis, G., Snape, P., Nicolaou, M. A., Antonakos, E., & Zafeiriou, S. P. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.453 · Cited by 347
Multimodal Fusion with Co-Attention Networks for Fake News Detection
Wu, Y., Zhan, P., Zhang, Y., Wang, L., & Xu, Z. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.226 · Cited by 368
An Overview of Cross-Media Retrieval: Concepts, Methodologies, Benchmarks, and Challenges
Peng, Y., Huang, X., & Zhao, Y. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2705068 · Cited by 345
Mapping the emotional face. How individual face parts contribute to successful emotion recognition
Wegrzyn, M., Vogt, M., Kireclioglu, B., Schneider, J. A., & Kissler, J. M. (2017) · PLoS ONE · doi:10.1371/journal.pone.0177239 · Cited by 365
Deep Reinforcement Learning-Based Image Captioning with Embedding Reward
Ren, Z., Wang, X., Zhang, N., Lv, X., & Li, L.-J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.128 · Cited by 345
Generating Images from Captions with AttentionPreprint
Mansimov, E., Parisotto, E., Ba, J. L., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.02793 · Cited by 479
Multimodal Convolutional Neural Networks for Matching Image and Sentence
Ma, L., Lu, Z., Shang, L., & Li, H. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.301 · Cited by 334
Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images
Marín, J., Biswas, A., Ofli, F., Hynes, N., Salvador, A., Aytar, Y., Weber, I. G., & Torralba, A. B. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2927476 · Cited by 350
Zhou, Z., Huang, Y., Wang, W., Wang, L., & Tan, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.717 · Cited by 332
Interpreting Deep Visual Representations via Network Dissection
Zhou, B., Bau, D., Oliva, A., & Torralba, A. B. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858759 · Cited by 347
Deep semantic understanding of high resolution remote sensing image
Qu, B., Li, X., Tao, D., & Lu, X. (2016) · 2016 International Conference on Computer, Information and Telecommunication Systems (CITS) · doi:10.1109/cits.2016.7546397 · Cited by 350
Recurrent Instance Segmentation
Romera‐Paredes, B., & Torr, P. H. S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46466-4_19 · Cited by 333
RNN-Stega: Linguistic Steganography Based on Recurrent Neural Networks
Yang, Z., Guo, X., Chen, Z.-M., Huang, Y., & Zhang, Y. (2018) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2018.2871746 · Cited by 329
Dual-level Collaborative Transformer for Image Captioning
Luo, Y., Ji, J., Sun, X., Cao, L., Wu, Y., Huang, F., Lin, C.-W., & Ji, R. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i3.16328 · Cited by 343
A Survey on Multimodal Large Language Models for Autonomous Driving
Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., Chen, J., Lu, J., Yang, Z., Liao, K.-D., et al. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00106 · Cited by 369
“This early attempt bridged vision (understanding block configurations) with language (interpreting and executing instructions), even though it was not based on deep learning.”
BackgroundATRank: An Attention-Based User Behavior Modeling Framework for Recommendation
Zhou, C., Bai, J., Song, J., Liu, X., Zhao, Z., Chen, X., & Gao, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11618 · Cited by 330
Knowledge-Driven Encode, Retrieve, Paraphrase for Medical Image Report Generation
Li, C. Y., Liang, X., Hu, Z., & Xing, E. P. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016666 · Cited by 316
An Overview of Deep-Learning-Based Audio-Visual Speech Enhancement and Separation
Michelsanti, D., Tan, Z., Zhang, S.-X., Xu, Y., Yu, M., Yu, D., & Jensen, J. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3066303 · Cited by 344
Multilevel Language and Vision Integration for Text-to-Clip Retrieval
Xu, H., He, K., Plummer, B. A., Sigal, L., Sclaroff, S., & Saenko, K. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019062 · Cited by 324
ScanRefer: 3D Object Localization in RGB-D Scans Using Natural Language
Chen, D. Z., Chang, A. X., & Nießner, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58565-5_13 · Cited by 360
Yin and Yang: Balancing and Answering Binary Visual Questions
Zhang, P., Goyal, Y., Summers-Stay, D. A., Batra, D., & Parikh, D. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.542 · Cited by 315
Generative Adversarial Networks for face generation: A survey
Kammoun, A., Slama, R., Tabia, H., Ouni, T., & Abid, M. (2022) · ACM Computing Surveys · doi:10.1145/1122445.1122456 · Cited by 370
Artificial Intelligence in Business: From Research and Innovation to Market Deployment
Soni, N. R., Sharma, E. K., Singh, N., & Kapoor, A. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.03.272 · Cited by 400
From Show to Tell: A Survey on Deep Learning-Based Image Captioning
Stefanini, M., Cornia, M., Baraldi, L., Cascianelli, S., Fiameni, G., & Cucchiara, R. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3148210 · Cited by 373
Early-stage malware prediction using recurrent neural networks
Rhode, M., Burnap, P., & Jones, K. (2018) · Computers & Security · doi:10.1016/j.cose.2018.05.010 · Cited by 310
Foreground segmentation using convolutional neural networks for multiscale feature encoding
Lim, L. A., & Keleş, H. Y. (2018) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.08.002 · Cited by 310
A Neural Model for Generating Natural Language Summaries of Program Subroutines
LeClair, A., Jiang, S., & McMillan, C. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00087 · Cited by 319
Faes, L., Wagner, S. K., Fu, D. J., Liu, X., Korot, E., Ledsam, J. R., Back, T., Chopra, R., Pontikos, N., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30108-6 · Cited by 317
Advances in Adversarial Attacks and Defenses in Computer Vision: A Survey
Akhtar, N., Mian, A. S., Kardan, N., & Shah, M. (2021) · IEEE Access · doi:10.1109/access.2021.3127960 · Cited by 355
14.2 DNPU: An 8.1TOPS/W reconfigurable CNN-RNN processor for general-purpose deep neural networks
Shin, D., Lee, J., Lee, J., & Yoo, H. (2017) · Digest of technical papers/Digest of technical papers - IEEE International Solid-State Circuits Conference · doi:10.1109/isscc.2017.7870350 · Cited by 310
Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors
Li, F., Shirahama, K., Nisar, M., Köping, L., & Grzegorzek, M. (2018) · Sensors · doi:10.3390/s18020679 · Cited by 302
Le Gallo, M., Khaddam-Aljameh, R., Stanisavljevic, M., Vasilopoulos, A., Kersting, B., Dazzi, M., Karunaratne, G., Brändli, M., Singh, A., Müller, S. M., et al. (2023) · Nature Electronics · doi:10.1038/s41928-023-01010-1 · Cited by 310
Recurrent Fusion Network for Image Captioning
Jiang, W., Ma, L., Jiang, Y., Liu, W., & Zhang, T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01216-8_31 · Cited by 306
Unsupervised Domain Adaptation with Similarity Learning
Pinheiro, P. O. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00835 · Cited by 305
Rohrbach, A., Torabi, A., Rohrbach, M., Tandon, N., Pal, C., Larochelle, H., Courville, A. C., & Schiele, B. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0987-1 · Cited by 340
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models
Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J., & Lazebnik, S. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0965-7 · Cited by 429
Sequence-to-Sequence Models Can Directly Translate Foreign Speech
Weiss, R. J., Chorowski, J., Jaitly, N., Wu, Y., & Chen, Z. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-503 · Cited by 343
Attention, please! A survey of neural attention models in deep learning
Correia, A. D. S., & Colombini, E. L. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10148-x · Cited by 365
GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models
Chen, D., Yu, N., Zhang, Y., & Fritz, M. (2020) · Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3372297.3417238 · Cited by 311
Image Captioning with Deep Bidirectional LSTMs
Wang, C., Yang, H., Bartz, C., & Meinel, C. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2964299 · Cited by 303
Working Memory Connections for LSTM
Landi, F., Baraldi, L., Cornia, M., & Cucchiara, R. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.08.030 · Cited by 316
RSTNet: Captioning with Adaptive Attention on Visual and Non-Visual Words
Zhang, X., Sun, X., Luo, Y., Ji, J., Zhou, Y., Wu, Y., Huang, F., & Ji, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01521 · Cited by 303
Long Short-Term Memory Over Recursive Structures
Zhu, X., Sobihani, P., & Guo, H. (2015) · Cited by 292
Hanson, J., Yang, Y., Paliwal, K. K., & Zhou, Y. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw678 · Cited by 295
StyleNet: Generating Attractive Visual Captions with Styles
Gan, C., Gan, Z., He, X., Gao, J., & Deng, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.108 · Cited by 291
Wang, T., Huang, J., Zhang, H., & Sun, Q. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01077 · Cited by 291
Generating Natural Questions About an Image
Mostafazadeh, N., Misra, I., Devlin, J., Mitchell, M., He, X., & Vanderwende, L. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1170 · Cited by 270
Associating neural word embeddings with deep image representations using Fisher Vectors
Klein, B., Lev, G., Sadeh, G., & Wolf, L. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299073 · Cited by 292
DSSL: Deep Surroundings-person Separation Learning for Text-based Person Retrieval
Zhu, A., Wang, Z., Li, Y., Wan, X., Jin, J., Wang, T., Hu, F., & Hua, G. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475369 · Cited by 292
Deep Compositional Captioning: Describing Novel Object Categories without Paired Training Data
Hendricks, L. A., Venugopalan, S., Rohrbach, M., Mooney, R. J., Saenko, K., & Darrell, T. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.8 · Cited by 283
Densely Residual Laplacian Super-Resolution
Anwar, S., & Barnes, N. M. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3021088 · Cited by 293
Identity-Aware Textual-Visual Matching with Latent Co-attention
Li, S., Xiao, T., Li, H., Yang, W. P., & Wang, X. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.209 · Cited by 291
Beyond Temporal Pooling: Recurrence and Temporal Convolutions for Gesture Recognition in Video
Pigou, L., van den Oord, A., Dieleman, S., Van Herreweghe, M., & Dambre, J. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-016-0957-7 · Cited by 290
Cui, H., Zhang, H., Ganger, G. R., Gibbons, P. B., & Xing, E. P. (2016) · European Conference on Computer Systems · doi:10.1145/2901318.2901323 · Cited by 293
Ma, Y., Xiang, Z., Du, Q., & Fan, W. (2017) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2017.12.008 · Cited by 293
Referring Image Segmentation via Recurrent Refinement Networks
Li, R., Li, K., Kuo, Y.-C., Shu, M., Qi, X., Shen, X., & Jia, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00602 · Cited by 297
VLP: A Survey on Vision-language Pre-training
Chen, F., Zhang, D., Han, M., Chen, X.-Y., Shi, J. C., Xu, S., & Xu, B. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1369-5 · Cited by 330
“To better adapt to downstream tasks, VLP models sometimes use the training objects of some downstream tasks, such as visual question answering (VQA) [37, 38, 12], and visual captioning (VC) [39, 40], as pre-training objectives.”
MethodSelf-Supervised Video Hashing With Hierarchical Binary Auto-Encoder
Song, J., Zhang, H., Li, X., Gao, L., Wang, M., & Hong, R. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2814344 · Cited by 283
Vision Transformers in medical computer vision—A contemplative retrospection
Parvaiz, A., Khalid, M. A., Zafar, R., Ameer, H., Ali, M. U., & Fraz, M. M. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106126 · Cited by 302
Controllable Abstractive Summarization
Fan, A., Grangier, D., & Auli, M. (2018) · Proceedings of the 2nd Workshop on Neural Machine Translation and Generation · doi:10.18653/v1/w18-2706 · Cited by 281
“…et al., 2014) have been applied to abstractive summarization (Nallapati et al., 2016; See et al., 2017; Paulus et al., 2017) following their success in both machine translation (Bahdanau et al., 2015; Luong et al., 2015b), parsing (Luong et al., 2015a) and image captioning (Vinyals et al., 2015b).”
BackgroundMethodMachine Learning in Genomic Medicine: A Review of Computational Problems and Data Sets
Leung, M. K. K., Delong, A., Alipanahi, B., & Frey, B. J. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2494198 · Cited by 284
Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal TransformersPreprint
Huang, Z., Zeng, Z., Liu, B., Fu, D., & Fu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00849 · Cited by 433
Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback
Wu, H., Gao, Y., Guo, X., Al-Halah, Z., Rennie, S. J., Grauman, K. L., & Feris, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01115 · Cited by 297
Human Trajectory Forecasting in Crowds: A Deep Learning Perspective
Kothari, P., Kreiss, S., & Alahi, A. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3069362 · Cited by 297
Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
Li, C. Y., Liang, X., Hu, Z., & Xing, E. P. (2018) · arXiv · doi:10.48550/arxiv.1805.08298 · Cited by 271
Describing Video With Attention-Based Bidirectional LSTM
Bin, Y., Yang, Y., Shen, F., Xie, N., Shen, H. T., & Li, X. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2831447 · Cited by 272
Interpretable Basis Decomposition for Visual Explanation
Zhou, B., Sun, Y., Bau, D., & Torralba, A. B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_8 · Cited by 275
Machine learning for hydrologic sciences: An introductory overview
Xu, T., & Liang, F. (2021) · Wiley Interdisciplinary Reviews Water · doi:10.1002/wat2.1533 · Cited by 292
Yang, Q., Zhang, Y., Dai, W., & Pan, S. J. (2020) · Cambridge University Press eBooks · doi:10.1017/9781139061773 · Cited by 286
Say As You Wish: Fine-Grained Control of Image Caption Generation With Abstract Scene Graphs
Chen, S., Jin, Q., Wang, P., & Wu, Q. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00998 · Cited by 272
Simple Baseline for Visual Question AnsweringPreprint
Zhou, B., Tian, Y., Sukhbaatar, S., Szlam, A. D., & Fergus, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.02167 · Cited by 266
Beyond Frame-level CNN: Saliency-Aware 3-D CNN With LSTM for Video Action Recognition
Wang, X., Gao, L., Song, J., & Shen, H. T. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2611485 · Cited by 264
ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question AnsweringPreprint
Chen, K., Wang, J., Chen, L.-C., Gao, H., Xu, W., & Nevatia, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05960 · Cited by 316
Normalized and Geometry-Aware Self-Attention Network for Image Captioning
Guo, L., Liu, J., Zhu, X., Yao, P., Lu, S., & Lu, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01034 · Cited by 261
Cross-Modality Binary Code Learning via Fusion Similarity Hashing
Liu, H., Ji, R., Wu, Y., Huang, F., & Zhang, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.672 · Cited by 260
Zhang, J., Du, J., Zhang, S. S., Liu, D., Hu, Y., Hu, J., Wei, S., & Dai, L.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.06.017 · Cited by 259
Adversarial Representation Learning for Text-to-Image Matching
Sarafianos, N., Xu, X., & Kakadiaris, I. A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00591 · Cited by 261
Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257
Learning to Answer Questions from Image Using Convolutional Neural Network
Ma, L., Lu, Z., & Li, H. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10442 · Cited by 260
Personalized Long- and Short-term Preference Learning for Next POI Recommendation
Wu, Y., Li, K., Zhao, G., & Qian, X. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.3002531 · Cited by 268
ReferIt3D: Neural Listeners for Fine-Grained 3D Object Identification in Real-World Scenes
Achlioptas, P., Abdelreheem, A., Xia, F., Elhoseiny, M., & Guibas, L. J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58452-8_25 · Cited by 282
Memory-Attended Recurrent Network for Video Captioning
Pei, W., Zhang, J., Wang, X., Ke, L., Shen, X., & Tai, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00854 · Cited by 257
Huang, T.-H., Ferraro, F., Mostafazadeh, N., Misra, I., Agrawal, A., Devlin, J., Girshick, R., He, X., Kohli, P., Batra, D., et al. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1147 · Cited by 254
Yang, K., Qinami, K., Fei-Fei, L., Deng, J., & Russakovsky, O. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375709 · Cited by 254
Adversarial Attacks on Deep Learning Models in Natural Language Processing: A SurveyPreprint
Zhang, W. E., Sheng, Q. Z., Alhazmi, A., & Li, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.06796 · Cited by 253
Metadata
- Authors
- Vinyals, O., Toshev, A., Bengio, S., & Erhan, D.
- Affiliations
- Google (United States) (US)
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 3156-3164
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1895577753
- Cited by
- 6,318
- References
- 40
- Citations by year
- 2026: 228 · 2025: 464 · 2024: 547 · 2023: 598 · 2022: 589 · 2021: 747 · 2020: 748 · 2019: 770 · 2018: 656 · 2017: 471 · 2016: 329 · 2015: 158
- FWCI
- 220.64
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- image captioning, recurrent neural networks, generative models, machine translation, image description generation, PASCAL dataset, Flickr30k, COCO dataset, BLEU evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
