The DOI matches its Crossref record; 37 of the 40 references it cites were found in a registry or an index.
Show and tell: A neural image caption generator
Abstract
Automatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing. In this paper, we present a generative model based on a deep recurrent architecture that combines recent advances in computer vision and machine translation and that can be used to generate natural sentences describing an image. The model is trained to maximize the likelihood of the target description sentence given the training image. Experiments on several datasets show the accuracy of the model and the fluency of the language it learns solely from image descriptions. Our model is often quite accurate, which we verify both qualitatively and quantitatively. For instance, while the current state-of-the-art BLEU-1 score (the higher the better) on the Pascal dataset is 25, our approach yields 59, to be compared to human performance around 69. We also show BLEU-1 score improvements on Flickr30k, from 56 to 66, and on SBU, from 19 to 28. Lastly, on the newly released COCO dataset, we achieve a BLEU-4 of 27.7, which is the current state-of-the-art.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“They coupled a convolutional neural network for encoding of the image, with a recurrent neural network - specifically a long-short term memory net (LSTM) (Hochreiter and Schmidhuber (1997)) - for textual encoding.”
From Explainable artificial intelligence (XAI) in deep learning-based medical image analysis (2022) · cites this paper as Vinyals et al. (2015)
In the citing paper
“They used human-generated sentences as ground truth for training, and used the bilingual evaluation understudy (BLEU) metric for evaluation.”
From Explainable artificial intelligence (XAI) in deep learning-based medical image analysis (2022) · cites this paper as Vinyals et al. (2015)
In the citing paper
“Vinyals et al. (2015) provided textual explanation for images using an end-to-end image captioning framework.”
From Explainable artificial intelligence (XAI) in deep learning-based medical image analysis (2022) · cites this paper as Vinyals et al. (2015)
In the citing paper
“Similarly, an image and a natural language caption can capture complimentary but converging information about a scene [32, 35].”
From Multimodal Generative Models for Scalable Weakly-Supervised Learning (2018) · cites this paper as [32]
In the citing paper
“Dependence on complex datasets including images and textual annotations”
From Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches (2023) · cites this paper as [43]
In the citing paper
“Semantic relationship between images and textual descriptions is intuitive, even for nonexperts”
From Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches (2023) · cites this paper as [43]
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Self-citationNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
Flagged first, then by citations
3 of the 40 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- kuznetsova, 2014, Treetalk: Composition and compression of trees for image descriptions, ACL, 2
Cited by
Sorted by citations
Items 601–700 of 1,039
Prediction of Pseudoprogression versus Progression using Machine Learning Algorithm in Glioblastoma
Jang, B., Jeon, S. H., Kim, I. H., & Kim, I. A. (2018) · Scientific Reports · doi:10.1038/s41598-018-31007-2 · Cited by 125
MSCap: Multi-Style Image Captioning With Unpaired Stylized Text
Guo, L., Liu, J., Yao, P., Li, J., & Lu, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00433 · Cited by 120
Deep Semantic Multimodal Hashing Network for Scalable Image-Text and Video-Text Retrievals
Jin, L., Li, Z., & Tang, J. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2997020 · Cited by 128
Hybrid approaches based on deep whole-sky-image learning to photovoltaic generation forecasting
Kong, W., Jia, Y., Dong, Z. Y., Meng, K., & Chai, S. (2020) · Applied Energy · doi:10.1016/j.apenergy.2020.115875 · Cited by 118
Minimalistic CNN-based ensemble model for gender prediction from face images
Antipov, G., Berrani, S.-A., & Dugelay, J. (2015) · Pattern Recognition Letters · doi:10.1016/j.patrec.2015.11.011 · Cited by 120
A Review of Deep Learning with Recurrent Neural Network
Kaur, M., & Mohta, A. (2019) · International Conference on Smart Systems and Inventive Technology (ICSSIT) · doi:10.1109/icssit46314.2019.8987837 · Cited by 130
Automatic Generation of Medical Imaging Diagnostic Report with Hierarchical Recurrent Neural Network
Yin, C., Qian, B., Wei, J., Li, X., Zhang, X., Li, Y., & Zheng, Q. (2019) · Proceedings · doi:10.1109/icdm.2019.00083 · Cited by 116
Progressive Transformer-Based Generation of Radiology Reports
Nooralahzadeh, F., Gonzalez, N. A. P., Frauenfelder, T., Fujimoto, K., & Krauthammer, M. O. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.241 · Cited by 121
Gan, C., Li, Y., Li, H., Sun, C., & Gong, B. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.201 · Cited by 116
Word–Sentence Framework for Remote Sensing Image Captioning
Wang, Q., Huang, W. F., Zhang, X., & Li, X. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3044054 · Cited by 126
Enhancing Video Summarization via Vision-Language Embedding
Plummer, B. A., Brown, M., & Lazebnik, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.118 · Cited by 112
STAIR Captions: Constructing a Large-Scale Japanese Image Caption Dataset
Yoshikawa, Y., Shigeto, Y., & Takeuchi, A. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2066 · Cited by 115
Multi-Level Policy and Reward-Based Deep Reinforcement Learning Framework for Image Captioning
Xu, N., Zhang, H., Liu, A.-A., Nie, W., Su, Y., Nie, J., & Zhang, Y. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2941820 · Cited by 121
In-Depth Review of YOLOv1 to YOLOv10 Variants for Enhanced Photovoltaic Defect Detection
Hussain, M., & Khanam, R. (2024) · Solar · doi:10.3390/solar4030016 · Cited by 131
“…that have proven exceptional performance across a broad spectrum of CV tasks like object detection [61], image classification [62], image captioning [63], image segmentation [64], image retrieval [65], speech processing [66], facial recognition [67], pose estimation [68], traffic sign recognition…”
Neural Programmer: Inducing Latent Programs with Gradient DescentPreprint
Neelakantan, A., Le, Q. V., & Sutskever, I. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04834 · Cited by 171
Reflective Decoding Network for Image Captioning
Ke, L., Pei, W., Li, R., Shen, X., & Tai, Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00898 · Cited by 115
Learning from Collective Intelligence
Zhang, H., Shang, X., Luan, H., Wang, M., & Chua, T. (2016) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/2978656 · Cited by 114
Context-Aware Captions from Context-Agnostic Supervision
Vedantam, R., Samy Bengio, A., Murphy, K. P., Parikh, D., & Chechik, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.120 · Cited by 120
Skeleton Key: Image Captioning by Skeleton-Attribute Decomposition
Wang, Y., Lin, Z., Shen, X., Cohen, S. D., & Cottrell, G. W. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.780 · Cited by 108
Deep Bidirectional LSTM Network Learning-Based Sentiment Analysis for Arabic Text
Elfaik, H., & Nfaoui, E. H. (2020) · Journal of Intelligent Systems · doi:10.1515/jisys-2020-0021 · Cited by 121
Houssein, E. H., Gamal, A. M., Younis, E. M. G., & Mohamed, E. (2025) · Cluster Computing · doi:10.1007/s10586-025-05281-5 · Cited by 148
Women also Snowboard: Overcoming Bias in Captioning ModelsPreprint
Hendricks, L. A., Burns, K., Saenko, K., Darrell, T., & Rohrbach, A. (2018) · European Conference on Computer Vision · doi:10.48550/arxiv.1803.09797 · Cited by 536
Monthly energy consumption forecast: A deep learning approach
Berriel, R. F., Lopes, A. T., Rodrigues, A. L., Varejão, F. M., & Oliveira-Santos, T. (2017) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966398 · Cited by 120
Sequence-to-sequence neural net models for grapheme-to-phoneme conversion
Yao, K., & Zweig, G. (2015) · Interspeech (USB) · doi:10.21437/interspeech.2015-134 · Cited by 138
Where to put the image in an image caption generator
Tanti, M., Albert Gatt, A., & Camilleri, K. P. (2017) · OAR@UM (University of Malta) · Cited by 112
Siamese Natural Language Tracker: Tracking by Natural Language Descriptions with Siamese Trackers
Feng, Q., Ablavsky, V., Bai, Q., & Sclaroff, S. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00579 · Cited by 127
Entity-Oriented Multi-Modal Alignment and Fusion Network for Fake News Detection
Li, P., Sun, X., Yu, H., Tian, Y., Yao, F., & Xu, G. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3098988 · Cited by 112
A Perceptual Measure for Deep Single Image Camera Calibration
Hold-Geoffroy, Y., Sunkavalli, K., Eisenmann, J., Fisher, M., Gambaretto, E., Hadap, S., & Lalonde, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00250 · Cited by 111
Deep Understanding of Cooking Procedure for Cross-modal Recipe Retrieval
Chen, J., Ngo, C., Feng, F., & Chua, T. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240627 · Cited by 111
Deep Adversarial Graph Attention Convolution Network for Text-Based Person Search
Liu, J., Zha, Z.-J., Hong, R., Wang, M., & Zhang, Y. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350991 · Cited by 112
Automatic Detection of Underwater Small Targets Using Forward-Looking Sonar Images
Zhou, T., Si, J., Wang, L., Xu, C., & Yu, X. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3181417 · Cited by 113
PI-LSTM: Physics-informed long short-term memory network for structural response modeling
Liu, F., Li, J., & Wang, L. (2023) · Engineering Structures · doi:10.1016/j.engstruct.2023.116500 · Cited by 111
Rich Image Captioning in the Wild
Tran, K., He, X., Zhang, L., & Sun, J. Z. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.61 · Cited by 110
Feature Space Transfer for Data Augmentation
Liu, B., Wang, X., Dixit, M., Kwitt, R., & Vasconcelos, N. M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00947 · Cited by 113
SD-RSIC: Summarization-Driven Deep Remote Sensing Image Captioning
Sümbül, G., Nayak, S., & Demir, B. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3031111 · Cited by 119
Describing and Localizing Multiple Changes with Transformers
Qiu, Y., Yamamoto, S., Nakashima, K., Suzuki, R., Iwata, K., Kataoka, H., & Satoh, Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00198 · Cited by 123
Automated Radiographic Report Generation Purely on Transformer: A Multicriteria Supervised Approach
Wang, Z., Han, H., Wang, L., Li, X., & Zhou, L. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3171661 · Cited by 118
Medical image captioning via generative pretrained transformers
Selivanov, A., Rogov, O. Y., Chesakov, D., Shelmanov, A., Fedulova, I., & Dylov, D. V. (2023) · Scientific Reports · doi:10.1038/s41598-023-31223-5 · Cited by 124
“It looks both on the LSTM state ht (5), on context vector ât (8) and the one previous word zt−1 (2): P(zt |ât ,zt−1) = so f tmax(Lo(Lhht +Laât +Ezt−1)) (11)”
BackgroundMulti-cue Zero-Shot Learning with Strong Supervision
Akata, Z., Malinowski, M., Fritz, M., & Schiele, B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.14 · Cited by 109
Analyzing multimodal public sentiment based on hierarchical semantic attentional network
Xu, N. (2017) · IEEE International Conference on Intelligence and Security Informatics (ISI) · doi:10.1109/isi.2017.8004895 · Cited by 109
Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog
Gan, Z., Cheng, Y., Kholy, A. E., Li, L., Liu, J., & Gao, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1648 · Cited by 112
Multimodal research in vision and language: A review of current and emerging trends
Uppal, S., Bhagat, S., Hazarika, D., Majumder, N., Poria, S., Zimmermann, R., & Zadeh, A. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.07.009 · Cited by 118
High-Resolution Remote Sensing Image Captioning Based on Structured Attention
Zhao, R., Shi, Z., & Zou, Z. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3070383 · Cited by 117
Dual Global Enhanced Transformer for image captioning
Xian, T., Li, Z., Zhang, C., & Ma, H. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.01.011 · Cited by 113
Chen, J., Chen, C., Xing, Z., Xu, X., Zhu, L., Li, G., & Wang, J. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380327 · Cited by 132
Trellis Networks for Sequence ModelingPreprint
Bai, S., Kolter, J. Z., & Koltun, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.06682 · Cited by 137
Semantic Regularisation for Recurrent Image Annotation
Liu, F., Xiang, T., Hospedales, T. M., Yang, W., & Sun, C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.443 · Cited by 110
Visual to Text: Survey of Image and Video Captioning
Li, S., Tao, Z., Li, K., & Fu, Y. (2019) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2019.2892755 · Cited by 112
Neural Approaches to Conversational AI
Gao, J., Galley, M., & Li, L. (2019) · now publishers, Inc. eBooks · doi:10.1561/9781680835533 · Cited by 125
Automatic Image Captioning Based on ResNet50 and LSTM with Soft Attention
Chu, Y., Xiao, Y., Yu, L., Sergei, M., & Wang, Z. (2020) · Wireless Communications and Mobile Computing · doi:10.1155/2020/8909458 · Cited by 108
Wang, J., Liu, Q., Xie, H., Yang, Z., & Zhou, H. (2021) · Cancers · doi:10.3390/cancers13040661 · Cited by 112
Gómez-Guillén, D., & Rojas‐Domínguez, A. (2015) · Neural Computation · doi:10.1162/neco_a_00793 · Cited by 107
Multi-stream Deep Networks for Person to Person Violence Detection in Videos
Dong, Z., Qin, J., & Wang, Y. (2016) · Communications in computer and information science · doi:10.1007/978-981-10-3002-4_43 · Cited by 107
Interactive neural machine translation
Peris, Á., Domingo, M., & Casacuberta, F. (2016) · Computer Speech & Language · doi:10.1016/j.csl.2016.12.003 · Cited by 108
Aligning Linguistic Words and Visual Semantic Units for Image Captioning
Guo, L., Liu, J., Tang, J., Li, J., Luo, W., & Lu, H. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350943 · Cited by 107
Zhao, B., Cheng, C., Peng, Z., Dong, X., & Meng, G. (2020) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2020.3005113 · Cited by 107
Imputer: Sequence Modelling via Imputation and Dynamic ProgrammingPreprint
Chan, W., Saharia, C., Hinton, G. E., Norouzi, M., & Jaitly, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.08926 · Cited by 102
Robust Visual-Textual Sentiment Analysis
You, Q., Cao, L., Jin, H., & Luo, J. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2964288 · Cited by 109
Le, Q. V. (2015) · Cited by 110
Connectionist Temporal Fusion for Sign Language Translation
Wang, S., Guo, D., Zhou, W., Zha, Z.-J., & Wang, M. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240671 · Cited by 109
Cross on Cross Attention: Deep Fusion Transformer for Image Captioning
Zhang, J., Xie, Y., Ding, W., & Wang, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3243725 · Cited by 109
Towards Unsupervised Image Captioning With Shared Multimodal Embeddings
Laina, I., Rupprecht, C., & Navab, N. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00751 · Cited by 111
Hardware accelerators for recurrent neural networks on FPGA
Chang, A. X. M., & Culurciello, E. (2017) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2017.8050816 · Cited by 109
Image Understanding using vision and reasoning through Scene Description Graph
Aditya, S., Yang, Y., Baral, C. R., Aloimonos, Y., & Fermuller, C. M. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.12.004 · Cited by 103
Context-Aware Visual Policy Network for Sequence-Level Image Captioning
Liu, D., Zha, Z.-J., Zhang, H., Zhang, Y., & Wu, F. H. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240632 · Cited by 110
Multi-Vehicle Collaborative Learning for Trajectory Prediction With Spatio-Temporal Tensor Fusion
Wang, Y., Zhao, S., Zhang, R., Cheng, X., & Yang, L. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.3009762 · Cited by 108
Malik, H., Farooq, M. S., Khelifi, A., Abid, A., Nasir Qureshi, J., & Hussain, M. (2020) · IEEE Access · doi:10.1109/access.2020.3004766 · Cited by 104
Describing Videos using Multi-modal Fusion
Jin, Q., Chen, J., Chen, S., Xiong, Y., & Hauptmann, A. G. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2984065 · Cited by 104
Cross-media analysis and reasoning: advances and directions
Peng, Y., Zhu, W., Zhao, Y., Xu, C., Huang, Q., Lu, H., Zheng, Q., Huang, T., & Gao, W. (2017) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1601787 · Cited by 104
A region-based image caption generator with refined descriptions
Kinghorn, P., Zhang, L., & Shao, L. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.07.014 · Cited by 104
Adelani, D. I., Mai, H., Fang, F., Nguyen, H. H., Yamagishi, J., & Echizen, I. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-44041-1_114 · Cited by 121
Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning
Wang, B. K., Li, G., Zhou, X., Chen, Z., Grossman, T., & Li, Y. (2021) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3472749.3474765 · Cited by 104
Lee, B., Baek, J., Park, S., & Yoon, S. (2016) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/2975167.2975212 · Cited by 104
Automated audio captioning with recurrent neural networks
Drossos, K., Adavanne, S., & Virtanen, T. I. (2017) · IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) · doi:10.1109/waspaa.2017.8170058 · Cited by 102
Personalized Key Frame Recommendation
Chen, X., Zhang, Y., Ai, Q., Xu, H., Yan, J., & Qin, Z. (2017) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3077136.3080776 · Cited by 105
Improving Image Captioning with Conditional Generative Adversarial Nets
Chen, C., Mu, S., Xiao, W., Ye, Z., Wu, L., & Ju, Q. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018142 · Cited by 103
Multi-modal Sentence Summarization with Modality Attention and Image Filtering
Li, H., Zhu, J., Liu, T., Zhang, J., & Zong, C. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/577 · Cited by 103
Compositional Learning of Image-Text Query for Image Retrieval
Anwaar, M. U., Labintcev, E., & Kleinsteuber, M. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00118 · Cited by 107
Predictive and generative machine learning models for photonic crystals
Christensen, T., Loh, C., Picek, S., Jakobović, D., Jing, L., Fisher, S., Čeperić, V., Joannopoulos, J. D., & Marin Soljacic, A. (2018) · Nanophotonics · doi:10.1515/nanoph-2020-0197 · Cited by 108
Deep learning in electron microscopy
Ede, J. M. (2020) · Machine Learning Science and Technology · doi:10.1088/2632-2153/abd614 · Cited by 118
Learning Word-Like Units from Joint Audio-Visual Analysis
Harwath, D. F., & Glass, J. R. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1047 · Cited by 102
Source Code Authorship Attribution Using Long Short-Term Memory Based Networks
Alsulami, B., Dauber, E., Harang, R., Mancoridis, S., & Greenstadt, R. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66402-6_6 · Cited by 106
Adding Attentiveness to the Neurons in Recurrent Neural Networks
Zhang, P., Xue, J., Lan, C., Zeng, W., Gao, Z., & Zheng, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_9 · Cited by 105
Image Captioning With End-to-End Attribute Detection and Subsequent Attributes Prediction
Huang, Y., Chen, J., Ouyang, W., Wan, W., & Xue, Y. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2969330 · Cited by 102
Truncation Cross Entropy Loss for Remote Sensing Image Captioning
Li, X., Zhang, X., Huang, W. F., & Wang, Q. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3010106 · Cited by 109
The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives
Iyyer, M., Manjunatha, V., Guha, A., Vyas, Y., Boyd-Graber, J. L., Daumé, H., & Davis, L. S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.686 · Cited by 107
Topic-Oriented Image Captioning Based on Order-Embedding
Yu, N., Hu, X., Song, B., Yang, J., & Zhang, J. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2889922 · Cited by 101
Face attribute prediction using off-the-shelf CNN features
Zhong, Y., Sullivan, J., & Li, H. (2016) · International Conference on Biometrics · doi:10.1109/icb.2016.7550092 · Cited by 101
Learning to Collocate Neural Modules for Image Captioning
Yang, X., Zhang, H., & Cai, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00435 · Cited by 100
DeeSCVHunter: A Deep Learning-Based Framework for Smart Contract Vulnerability Detection
Yu, X., Zhao, H., Hou, B., Ying, Z., & Wu, B. L. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534324 · Cited by 106
Automatic captioning for medical imaging (MIC): a rapid review of literature
Beddiar, D. R., Oussalah, M., & Seppänen, T. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10270-w · Cited by 103
Describing Like Humans: On Diversity in Image Captioning
Wang, Q., & Chan, A. B. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00432 · Cited by 100
Grounding Distributional Semantics in the Visual World
Baroni, M. (2015) · Language and Linguistics Compass · doi:10.1111/lnc3.12170 · Cited by 104
Deep Learning in Object Recognition, Detection, and Segmentation
Wang, X. (2016) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000071 · Cited by 111
Geomagnetic Index Kp Forecasting With LSTM
Tan, Y., Hu, Q., Wang, Z., & Zhong, Q. (2018) · Space Weather · doi:10.1002/2017sw001764 · Cited by 99
Paying More Attention to Saliency
Cornia, M., Baraldi, L., Serra, G., & Cucchiara, R. (2018) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3177745 · Cited by 99
Learning to Act Properly: Predicting and Explaining Affordances from Images
Chuang, C.-Y., Li, J., Torralba, A. B., & Fidler, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00108 · Cited by 98
Adaptive Reconstruction Network for Weakly Supervised Referring Expression Grounding
Liu, X., Li, L., Wang, S., Zha, Z.-J., Meng, D., & Huang, Q. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00270 · Cited by 99
Image caption generation with dual attention mechanism
Liu, M., Li, L., Hu, H., Guan, W., & Tian, J. (2019) · Information Processing & Management · doi:10.1016/j.ipm.2019.102178 · Cited by 99
Grounded Situation Recognition
Pratt, S. M., Yatskar, M., Weihs, L., Farhadi, A., & Kembhavi, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58548-8_19 · Cited by 108
Metadata
- Authors
- Vinyals, O., Toshev, A., Bengio, S., & Erhan, D.
- Affiliations
- Google (United States) (US)
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 3156-3164
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1895577753
- Cited by
- 6,318
- References
- 40
- Citations by year
- 2026: 228 · 2025: 464 · 2024: 547 · 2023: 598 · 2022: 589 · 2021: 747 · 2020: 748 · 2019: 770 · 2018: 656 · 2017: 471 · 2016: 329 · 2015: 158
- FWCI
- 220.64
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- image captioning, recurrent neural networks, generative models, machine translation, image description generation, PASCAL dataset, Flickr30k, COCO dataset, BLEU evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
