The DOI matches its Crossref record; 37 of the 40 references it cites were found in a registry or an index.
Show and tell: A neural image caption generator
Abstract
Automatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing. In this paper, we present a generative model based on a deep recurrent architecture that combines recent advances in computer vision and machine translation and that can be used to generate natural sentences describing an image. The model is trained to maximize the likelihood of the target description sentence given the training image. Experiments on several datasets show the accuracy of the model and the fluency of the language it learns solely from image descriptions. Our model is often quite accurate, which we verify both qualitatively and quantitatively. For instance, while the current state-of-the-art BLEU-1 score (the higher the better) on the Pascal dataset is 25, our approach yields 59, to be compared to human performance around 69. We also show BLEU-1 score improvements on Flickr30k, from 56 to 66, and on SBU, from 19 to 28. Lastly, on the newly released COCO dataset, we achieve a BLEU-4 of 27.7, which is the current state-of-the-art.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“They coupled a convolutional neural network for encoding of the image, with a recurrent neural network - specifically a long-short term memory net (LSTM) (Hochreiter and Schmidhuber (1997)) - for textual encoding.”
From Explainable artificial intelligence (XAI) in deep learning-based medical image analysis (2022) · cites this paper as Vinyals et al. (2015)
In the citing paper
“They used human-generated sentences as ground truth for training, and used the bilingual evaluation understudy (BLEU) metric for evaluation.”
From Explainable artificial intelligence (XAI) in deep learning-based medical image analysis (2022) · cites this paper as Vinyals et al. (2015)
In the citing paper
“Vinyals et al. (2015) provided textual explanation for images using an end-to-end image captioning framework.”
From Explainable artificial intelligence (XAI) in deep learning-based medical image analysis (2022) · cites this paper as Vinyals et al. (2015)
In the citing paper
“Similarly, an image and a natural language caption can capture complimentary but converging information about a scene [32, 35].”
From Multimodal Generative Models for Scalable Weakly-Supervised Learning (2018) · cites this paper as [32]
In the citing paper
“Dependence on complex datasets including images and textual annotations”
From Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches (2023) · cites this paper as [43]
In the citing paper
“Semantic relationship between images and textual descriptions is intuitive, even for nonexperts”
From Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches (2023) · cites this paper as [43]
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Self-citationNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273
Flagged first, then by citations
3 of the 40 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- kuznetsova, 2014, Treetalk: Composition and compression of trees for image descriptions, ACL, 2
Cited by
Sorted by citations
Items 301–400 of 1,039
Bottom-Up and Top-Down Attention for Image Captioning and VQA.Preprint
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2017) · arXiv (Cornell University) · Cited by 252
A Joint Speaker-Listener-Reinforcer Model for Referring Expressions
Yu, L., Tan, H. X., Bansal, M., & Berg, T. L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.375 · Cited by 250
Deep learning convolutional neural network in rainfall–runoff modelling
Van, S. P., Le, H. M., Thanh, D. V., Dang, T. D., Loc, H. H., & Anh, D. T. (2020) · Journal of Hydroinformatics · doi:10.2166/hydro.2020.095 · Cited by 258
Hierarchical LSTMs with Adaptive Attention for Visual Captioning
Gao, L., Li, X., Song, J., & Shen, H. T. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2894139 · Cited by 249
Song, C., Ning, N., Zhang, Y., & Wu, B. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102437 · Cited by 261
Segmental Spatiotemporal CNNs for Fine-Grained Action Segmentation
Lea, C., Reiter, A., Vidal, R., & Hager, G. D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_3 · Cited by 246
Situation Recognition: Visual Semantic Role Labeling for Image Understanding
Yatskar, M., Zettlemoyer, L., & Farhadi, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.597 · Cited by 253
An, Q., Tao, Z., Xu, X., Mansori, M. E., & Chen, M. (2019) · Measurement · doi:10.1016/j.measurement.2019.107461 · Cited by 254
Weakly-supervised Disentangling with Recurrent Transformations for 3D View SynthesisPreprint
Yang, J., Reed, S., Yang, M., & Lee, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.00706 · Cited by 247
Visual question answering: Datasets, algorithms, and future challenges
Kafle, K., & Kanan, C. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.06.005 · Cited by 259
Machine health monitoring with LSTM networks
Zhao, R., Wang, J., Yan, R., & Mao, K. (2016) · International Conference on Sensing Technology · doi:10.1109/icsenst.2016.7796266 · Cited by 248
What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment
Mei, H., Bansal, M., & Walter, M. R. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1086 · Cited by 261
pix2code: Generating Code from a Graphical User Interface Screenshot
Beltramelli, T. (2018) · Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems · doi:10.1145/3220134.3220135 · Cited by 237
A survey on automatic image caption generation
Bai, S., & An, S. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.05.080 · Cited by 253
Multimodal Recurrent Model with Attention for Automated Radiology Report Generation
Xue, Y., Xu, T., Long, L. R., Xue, Z., Antani, S., Thoma, G. R., & Huang, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_52 · Cited by 245
Sentence Compression by Deletion with LSTMs
Filippova, K., Alfonseca, E., Colmenares, C. A., Kaiser, Ł., & Vinyals, O. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1042 · Cited by 246
Self-citationImproving Description-Based Person Re-Identification by Multi-Granularity Image-Text Alignments
Niu, K., Huang, Y., Ouyang, W., & Wang, L. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2984883 · Cited by 241
Instance-Aware Image and Sentence Matching with Selective Multimodal LSTM
Huang, Y., Wang, W., & Wang, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.767 · Cited by 237
Sixty-five years of the long march in protein secondary structure prediction: the final stretch?
Yang, Y., Gao, J., Wang, J., Heffernan, R., Hanson, J., Paliwal, K. K., & Zhou, Y. (2016) · Briefings in Bioinformatics · doi:10.1093/bib/bbw129 · Cited by 240
Feng, Y., Ma, L., Liu, W., & Luo, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00425 · Cited by 228
Multi-level Attention Networks for Visual Question Answering
Yu, D., Fu, J., Mei, T., & Rui, Y. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.446 · Cited by 236
Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
Shetty, R., Rohrbach, M., Hendricks, L. A., Fritz, M., & Schiele, B. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.445 · Cited by 230
TGIF: A New Dataset and Benchmark on Animated GIF Description
Li, Y., Song, Y., Cao, L., Tetreault, J., Goldberg, L., Jaimes, A., & Luo, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.502 · Cited by 235
Progress in Machine Translation
Wang, H., Wu, H. R., He, Z., Huang, L., & Church, K. (2021) · Engineering · doi:10.1016/j.eng.2021.03.023 · Cited by 320
Deep Reinforcement Learning for Sequence-to-Sequence Models
Keneshloo, Y., Shi, T., Ramakrishnan, N., & Reddy, C. K. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2929141 · Cited by 230
Emotion Recognition using Multimodal Residual LSTM Network
Ma, J., Tang, H., Zheng, W.-L., & Lu, B.-L. (2019) · Proceedings of the 27th ACM International Conference on Multimedia · doi:10.1145/3343031.3350871 · Cited by 230
Bidirectional Attentive Fusion with Context Gating for Dense Video Captioning
Wang, J., Jiang, W., Ma, L., Liu, W., & Xu, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00751 · Cited by 237
A Shared Task on Multimodal Machine Translation and Crosslingual Image Description
Specia, L., Frank, S., Sima’an, K., & Elliott, D. (2016) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w16-2346 · Cited by 228
Seeing Voices and Hearing Faces: Cross-Modal Biometric Matching
Nagrani, A., Albanie, S., & Zisserman, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00879 · Cited by 224
Knowledge matters: Chest radiology report generation with general and specific knowledge
Yang, S., Wu, X., Ge, S., Zhou, S. K., & Xiao, L. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102510 · Cited by 225
“S&T (Vinyals et al., 2015), SA&T (Xu et al., 2015), AdaAtt (Lu et al., 2017), Top-Down (Anderson et al., 2018), and chest radiology report generation works, e.g. CoAtt (Jing et al., 2018), CMAS (Jing et al., 2019), HRGR (Li et al., 2018), KERP (Li et al., 2019) PPKED (Liu et al., 2021b), R2Gen…”
BackgroundBorys, K., Schmitt, Y. A., Nauta, M., Seifert, C., Krämer, N., Friedrich, C. M., & Nensa, F. (2023) · European Journal of Radiology · doi:10.1016/j.ejrad.2023.110786 · Cited by 233
RADnet: Radiologist level accuracy using deep learning for hemorrhage detection in CT scans
Grewal, M., Srivastava, M. M., Kumar, P., & Varadarajan, S. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363574 · Cited by 225
Can a Machine Generate Humanlike Language Descriptions for a Remote Sensing Image?
Shi, Z., & Zou, Z. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2017.2677464 · Cited by 240
Peng, X. B., Guo, Y., Halper, L., Levine, S., & Fidler, S. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530110 · Cited by 285
“Not only can these models solve challenging tasks, but they also provide powerful priors that can be reused for a wide range of downstream applications [Brown et al. 2020; Donahue et al. 2014; Lester et al. 2021; Raffel et al. 2020; Sermanet et al. 2018; Vinyals et al. 2014].”
BackgroundUnsupervised learning of spoken language with visual context
Harwath, D. F., Torralba, A. B., & Glass, J. R. (2016) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 224
Zhang, Y., Chen, R. F., Tang, J., Stewart, W. F., & Sun, J. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098109 · Cited by 224
A new hybrid deep learning model for human action recognition
Jaouedi, N., Boujnah, N., & Bouhlel, M. S. (2019) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2019.09.004 · Cited by 237
MSNovelist: de novo structure generation from mass spectra
Stravs, M. A., Dührkop, K., Böcker, S., & Zamboni, N. (2022) · Nature Methods · doi:10.1038/s41592-022-01486-3 · Cited by 226
Sharma, V., & Mir, R. N. (2020) · Computer Science Review · doi:10.1016/j.cosrev.2020.100301 · Cited by 230
Grounding of Textual Phrases in Images by Reconstruction
Rohrbach, A., Rohrbach, M., Hu, R., Darrell, T., & Schiele, B. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_49 · Cited by 462
ReNet: A Recurrent Neural Network Based Alternative to Convolutional NetworksPreprint
Visin, F., Kastner, K., Cho, K., Matteucci, M., Courville, A. C., & Bengio, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.00393 · Cited by 285
Recurrent Spatial-Temporal Attention Network for Action Recognition in Videos
Du, W., Wang, Y., & Qiao, Y. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2778563 · Cited by 220
Diverse Beam Search for Improved Description of Complex Scenes
Vijayakumar, A., Cogswell, M., Selvaraju, R., Sun, Q., Lee, S., Crandall, D., & Batra, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12340 · Cited by 219
Hierarchy Parsing for Image Captioning
Yao, T., Pan, Y., Li, Y., & Mei, T. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00271 · Cited by 219
Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question AnsweringPreprint
Gao, H., Mao, J., Zhou, J., Huang, Z., Wang, L., & Xu, W. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05612 · Cited by 441
Diehl, P. U., Zarrella, G., Cassidy, A. S., Pedroni, B. U., & Neftci, E. (2016) · IEEE International Conference on Rebooting Computing (ICRC) · doi:10.1109/icrc.2016.7738691 · Cited by 222
Learning Human Motion Models for Long-Term Predictions
Ghosh, P. S., Song, J., Aksan, E., & Hilliges, O. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00059 · Cited by 227
Multimodal Named Entity Recognition for Short Social Media Posts
Moon, S., Neves, L., & Carvalho, V. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1078 · Cited by 248
Dibia, V. C., & Demiralp, Ç. (2019) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2019.2924636 · Cited by 233
Parametric Exponential Linear Unit for Deep Convolutional Neural Networks
Trottier, L., Giguère, P., & Chaib-draa, B. (2017) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2017.00038 · Cited by 234
Areas of Attention for Image Captioning
Pedersoli, M., Lucas, T., Schmid, C., & Verbeek, J. J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.140 · Cited by 212
A Hybrid Convolutional Variational Autoencoder for Text Generation
Semeniuta, S., Severyn, A., & Barth, E. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1066 · Cited by 225
SentiCap: Generating Image Descriptions with Sentiments
Mathews, A., Xie, L., & He, X. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10475 · Cited by 219
Yuan, J., Liao, H., Luo, R., & Luo, J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32226-7_80 · Cited by 215
Zhu, B., Bedeer, E. E., Nguyen, H. H., Barton, R., & Henry, J. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3102161 · Cited by 216
Describing Videos by Exploiting Temporal StructurePreprint
Yao, L., Torabi, A., Cho, K., Ballas, N., Pal, C., Larochelle, H., & Courville, A. C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.08029 · Cited by 303
Video Captioning by Adversarial LSTM
Yang, Y., Zhou, J., Ai, J., Bin, Y., Hanjalić, A., Shen, H. T., & Ji, Y. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2855422 · Cited by 216
Show, Control and Tell: A Framework for Generating Controllable and Grounded Captions
Cornia, M., Baraldi, L., & Cucchiara, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00850 · Cited by 213
Stimulus-driven and concept-driven analysis for image caption generation
Ding, S., Qu, S., Xi, Y., & Wan, S. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.04.095 · Cited by 217
Interactive and Explainable Region-guided Radiology Report Generation
Tanida, T., Müller, P., Kaissis, G. A., & Rueckert, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00718 · Cited by 247
“Most radiology report generation methods [1,6,7,17,24,25,48,53] are inspired by influential works [9, 46, 51, 54] from the image captioning domain in computer vision.”
MethodClinically Accurate Chest X-Ray Report GenerationPreprint
Liu, G., Hsu, T.-M. H., McDermott, M. B. A., Boag, W., Weng, W., Peter Szolovits, B., & Ghassemi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02633 · Cited by 212
Transformer-based Feature Reconstruction Network for Robust Multimodal Sentiment Analysis
Yuan, Z., Li, W., Xu, H., & Yu, W. (2021) · Proceedings of the 29th ACM International Conference on Multimedia · doi:10.1145/3474085.3475585 · Cited by 211
Adversarial text-to-image synthesis: A review
Frolov, S., Hinz, T., Raue, F., Hees, J., & Dengel, A. R. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.07.019 · Cited by 228
Controlling Output Length in Neural Encoder-Decoders
Kikuchi, Y., Neubig, G., Sasano, R., Takamura, H., & Okumura, M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1140 · Cited by 222
Language Models for Image Captioning: The Quirks and What Works
Devlin, J., Cheng, H., Fang, H., Gupta, S., Deng, L., He, X., Zweig, G., & Mitchell, M. (2015) · doi:10.3115/v1/p15-2017 · Cited by 207
GAN Lab: Understanding Complex Deep Generative Models using Interactive Visual Experimentation
Kahng, M., Thorat, N., Chau, D. H., Viégas, F. B., & WATTENBERG, M. P. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2864500 · Cited by 206
TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts
Guo, C., Zuo, X., Wang, S., & Cheng, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19833-5_34 · Cited by 206
Tianjic: A Unified and Scalable Chip Bridging Spike-Based and Continuous Neural Computation
Deng, L., Wang, G., Li, G., Li, S., Liang, L., Zhu, M., Wu, Y., Yang, Z., Zou, Z., Pei, J., et al. (2020) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2020.2970709 · Cited by 211
A Survey of Evaluation Metrics Used for NLG Systems
Sai, A. B., Mohankumar, A. K., & Khapra, M. M. (2023) · ACM Computing Surveys · doi:10.1145/3485766 · Cited by 235
Multilabel Image Classification With Regional Latent Semantic Dependencies
Zhang, J., Wu, Q., Shen, C., Zhang, J., & Lu, J. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2812605 · Cited by 206
RPAN: An End-to-End Recurrent Pose-Attention Network for Action Recognition in Videos
Du, W., Wang, Y., & Qiao, Y. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.402 · Cited by 199
Video Object Detection with an Aligned Spatial-Temporal Memory
Xiao, F., & Lee, Y. J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_30 · Cited by 203
Improving Referring Expression Grounding With Cross-Modal Attention-Guided Erasing
Liu, X., Wang, Z., Shao, J., Wang, X., & Li, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00205 · Cited by 202
Applications of Deep Learning for Dense Scenes Analysis in Agriculture: A Review
Zhang, Q., Liu, Y., Gong, C., Chen, Y., & Yu, H. (2020) · Sensors · doi:10.3390/s20051520 · Cited by 210
Stack-Captioning: Coarse-to-Fine Learning for Image Captioning
Gu, J., Cai, J., Wang, G., & Chen, T. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12266 · Cited by 198
Park, D. H., Darrell, T., & Rohrbach, A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00472 · Cited by 221
Radiology report generation with a learned knowledge base and multi-modal alignment
Yang, S., Wu, X., Ge, S., Zheng, Z., Zhou, S. K., & Xiao, L. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102798 · Cited by 199
Hierarchical Boundary-Aware Neural Encoder for Video Captioning
Baraldi, L., Grana, C., & Cucchiara, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.339 · Cited by 199
You Said That?: Synthesising Talking Faces from Audio
Jamaludin, A., Chung, J. S., & Zisserman, A. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01150-y · Cited by 189
Deep Learning Approaches on Image Captioning: A Review
Ghandi, T., Pourreza, H. R., & Mahyar, H. (2023) · ACM Computing Surveys · doi:10.1145/3617592 · Cited by 219
“Vinyals et al. [ 119 ] have been the first to incorporate deep learning-based encoder-decoder framework for image captioning.”
BackgroundMethodR2GenGPT: Radiology Report Generation with frozen LLMs
Wang, Z., Liu, L., Wang, L., & Zhou, L. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100033 · Cited by 253
Sensor-Based Gait Parameter Extraction With Deep Convolutional Neural Networks
Hannink, J., Kautz, T., Pasluosta, C. F., Gasmann, K.-G., Klucken, J., & Eskofier, B. M. (2016) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2016.2636456 · Cited by 207
Predicting Visual Features From Text for Image and Video Caption Retrieval
Dong, J., Li, X., & Snoek, C. G. M. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2832602 · Cited by 198
Tan, B., Song, Y., Zhong, E., & Yang, Q. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783295 · Cited by 200
Discriminability Objective for Training Descriptive Captions
Luo, R., Shakhnarovich, G., Cohen, S. D., & Price, B. L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00728 · Cited by 194
Chained Predictions Using Convolutional Neural Networks
Gkioxari, G., Toshev, A., & Jaitly, N. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_44 · Cited by 194
Self-citationAttention Correctness in Neural Image Captioning
Liu, C., Mao, J., Sha, F., & Yuille, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11197 · Cited by 210
Deep Learning in Image Cytometry: A Review
Gupta, A., Harrison, P. J., Wieslander, H., Pielawski, N., Kartasalo, K., Partel, G., Solorzano, L., Suveer, A., Klemm, A. H., Spjuth, O., et al. (2018) · Cytometry Part A · doi:10.1002/cyto.a.23701 · Cited by 194
Image Search With Text Feedback by Visiolinguistic Attention Learning
Chen, Y., Gong, S., & Bazzani, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00307 · Cited by 200
Learning to Generate Product Reviews from Attributes
Dong, L., Huang, S., Wei, F., Lapata, M., Zhou, M., & Xu, K. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers · doi:10.18653/v1/e17-1059 · Cited by 190
Deep Learning Techniques for Future Intelligent Cross-Media RetrievalPreprint
Rehman, S. U., Waqas, M. M., Tu, S., Koubâa, A., Rehman, O. U., Ahmad, J. E., Hanif, M. I., & Han, Z. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.01191 · Cited by 191
Constrained transformer network for ECG signal processing and arrhythmia classification
Che, C., Zhang, P., Zhu, M., Qu, Y., & Jin, B. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01546-2 · Cited by 189
Wang, X. E., Xiong, W., Wang, H., & Wang, W. Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_3 · Cited by 187
RpBERT: A Text-image Relation Propagation-based BERT Model for Multimodal NER
Sun, L. J., Wang, J., Zhang, K., Su, Y., & Weng, F. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17633 · Cited by 191
Know More Say Less: Image Captioning Based on Scene Graphs
Li, X., & Jiang, S. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2896516 · Cited by 181
Doubly-Attentive Decoder for Multi-modal Neural Machine Translation
Calixto, I., Liu, Q., & Campbell, N. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1175 · Cited by 185
Pose-Guided Multi-Granularity Attention Network for Text-Based Person Search
Jing, Y., Si, C., Wang, J., Wang, W., Wang, L., & Tan, T. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6777 · Cited by 201
Zheng, X., Burdick, D., Popa, L., Zhong, X., & Wang, N. (2020) · IEEE Workshop/Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00074 · Cited by 248
Tracking by Natural Language Specification
Li, Z., Ran, T., Gavves, E., Snoek, C. G. M., & Smeulders, A. W. M. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.777 · Cited by 194
Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences
Mei, H., Bansal, M., & Walter, M. R. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10364 · Cited by 188
Metadata
- Authors
- Vinyals, O., Toshev, A., Bengio, S., & Erhan, D.
- Affiliations
- Google (United States) (US)
- Venue
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2015-06
- Pages
- 3156-3164
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W1895577753
- Cited by
- 6,318
- References
- 40
- Citations by year
- 2026: 228 · 2025: 464 · 2024: 547 · 2023: 598 · 2022: 589 · 2021: 747 · 2020: 748 · 2019: 770 · 2018: 656 · 2017: 471 · 2016: 329 · 2015: 158
- FWCI
- 220.64
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- image captioning, recurrent neural networks, generative models, machine translation, image description generation, PASCAL dataset, Flickr30k, COCO dataset, BLEU evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
