Cited by 879Multimodal Machine Learning ApplicationsImage Retrieval and Classification TechniquesGenerative Adversarial Networks and Image Synthesis

The DOI matches its Crossref record; 167 of the 167 references it cites were found in a registry or an index.

A Comprehensive Survey of Deep Learning for Image Captioning

Hossain, MD. Zakir, Sohel, Ferdous, Shiratuddin, Mohd Fairuz, Laga, Hamid·2019·ACM Computing Surveys

DOI 10.1145/3295748

Open this paper in CoChat — free account

Abstract

Generating a description of an image is called image captioning. Image captioning requires recognizing the important objects, their attributes, and their relationships in an image. It also needs to generate syntactically and semantically correct sentences. Deep-learning-based techniques are capable of handling the complexities and challenges of image captioning. In this survey article, we aim to present a comprehensive review of existing deep-learning-based image captioning techniques. We discuss the foundation of the techniques to analyze their performances, strengths, and limitations. We also discuss the datasets and the evaluation metrics popularly used in deep-learning-based automatic image captioning.

Integrity signals

Registry

Crossref · 10.1145/3295748

Cites

167 of 167 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“The architecture of this method has ResNet [53] and two LSTMs called Skel-LSTM and Attr-LSTM.”
Method
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 2×
“Moreover, recent object detection and classification methods [73, 127] show that deep, hierarchical methods are better at learning than shallower ones.”
BackgroundMethod
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 5×
“Kiros et al. [69] proposed a method that is built on a Log-Bilinear model and used AlexNet to extract visual features.”
BackgroundMethodInfluential
Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 3×
“LSTM [54] networks are a type of RNN that have special units in addition to standard units.”
BackgroundMethod
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 4×
“The bottom up attention mechanism uses Faster R-CNN [116] for region proposals that can select salient regions of an image .”
MethodInfluential
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“For example, Convolutional Neural Networks (CNN) [79] are widely used for feature learning, and a classifier such as Softmax is used for classification.”
Method
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 1×
“d techniques and (2) Deep machine learning based techniques. In traditional machine learning, hand crafted features such as Local Binary Patterns (LBP) [107], Scale-Invariant Feature Transform (SIFT) [87], the Histogram of Oriented Gradients (HOG) [27], and a combination of such features are widely used. In these tech…”
Background
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · cited 1×
“In traditional machine learning, hand crafted features such as Local Binary Patterns (LBP) [107], Scale-Invariant Feature Transform (SIFT) [87], the Histogram of Oriented Gradients (HOG) [27], and a combination of such features are widely used.”
Background
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Latent dirichlet allocation
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Fast R-CNN
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · cited 1×
“between a pair of networks: the Generator and the Discriminator. GANs have already been used successfully in a variety of applications, including image captioning[26, 126], image to image translation [56], text to image synthesis [15, 115], and text generation [36,145]. There are two issues with GAN. First, GAN can wo…”
Background
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 2×
“and test data consists of 2,000 and 1,000 images respectively. Each image contains romantic, humorous, and factual captions. 4.2 Evaluation Metrics 4.2.1 BLEU. BLEU (Bilingual evaluation understudy) [110] is a metric that is used to measure the quality of machine generated text. Individual text segments are compared w…”
MethodInfluential
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,989
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1×
“In addition to this, in this method, each filter of a convolutional layer acts as semantic detectors [159] while other methods use external sources for obtaining semantic information.”
Method
Latent Dirichlet Allocation
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,683
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992 · cited 1×
“They are then passed to a classifier such as Support Vector Machines (SVM) [17] in order to classify an object.”
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008 · cited 1×
“Another network, Gated Recurrent Unit (GRU) [25] has a similar structure to LSTM but it does not use separate memory cells and uses fewer gates to control the flow of information.”
Background
Untitled work
(2004) · Cited by 9,718
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · cited 6×
“The top-down paradigms [23, 31, 65, 93, 94, 142, 152] are used for extracting visual features first and then convert them into words.”
BackgroundMethodInfluential
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014) · SSST@EMNLP · doi:10.3115/v1/w14-4012 · Cited by 7,251 · cited 1×
“Therefore, LSTM is still facing challenges in generating long length sentences [7, 24].”
Background
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318 · cited 5×
“[23] VGGNet RNN VS,SL,WS,EDA Fang et al. 2015 [33] AlexNet, VGGNet MELM VS,SL,WS,CA Jia et al. 2015 [59] VGGNet LSTM VS,SL,WS,EDA Karpathy et al. 2015 [65] VGGNet RNN MS,SL,WS,EDA Vinyals et al. 2015 [142] GoogLeNet LSTM VS,SL,WS,EDA Xu et al. 2015 [152] AlexNet LSTM VS,SL,WS,EDA,AB Jin et al. 2015 [61] VGGNet LSTM VS…”
BackgroundMethodInfluential
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00636 · Cited by 5,416
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390 · cited 1×
“sequence to sequence learning. Several neural language models such as neural probabilistic language model [11], log-bilinear models [105], skip-gram models [98], and recurrent neural networks (RNNs) [99] have been proposed for learning sequence to sequence tasks. RNNs have widely been used in various sequence learning…”
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228 · cited 1×
“n evaluation metrics. You et al. [156] proposed a semantic attention-based image captioning method. The method providesadetailed,coherentdescriptionofsemanticallyimportantobjects.Thetop-downparadigms [23, 31, 65, 93, 94, 142, 152] are used for extracting visual features first and then convert them into words. In botto…”
Method
Untitled work
(1999) · Cited by 5,147
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107 · cited 2×
“Visual Genome dataset [72] is another dataset for image captioning.”
Method
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Freebase: a collaboratively created graph database for structuring human knowledge
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., & Taylor, J. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376746 · Cited by 4,852 · cited 1×
“generating image captions even for open domain images. It can detect a diverse set of visual concepts and generate captions for celebrities and landmarks. It uses an external knowledge base Freebase [16] in recognizingabroadrangeofentitiessuchascelebritiesandlandmarks. Aseriesofhumanjudgments are applied for evaluatin…”
Method
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · cited 2×
“ertain key entities such as celebrities and landmarks that are out of their scope. The generated captions of these methods are evaluated on automatic metrics such as BLEU [110], METEOR [1], and CIDEr [139]. These evaluation metrics have already shown good results on these methods. However, in terms of performance ther…”
BackgroundMethodInfluential
Untitled work
(2005) · Cited by 4,152
Categorical Reparameterization with Gumbel-Softmax
Jang, E., Gu, S., & Poole, B. (2016) · arXiv · doi:10.48550/arxiv.1611.01144 · Cited by 3,616 · cited 1×
“method can generate multiple captions for a single image and showed impressive improvements in generating diverse captions. GANs have limitations in backpropagating the discrete data. Gumbel sampler [58, 91] is used to overcome the discrete data problem. The two main parts of this adversarial network are the generator…”
Method
A Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,451
A maximum entropy approach to natural language processing
Berger, A. L., Pietra, V. J. D., & Pietra, S. A. D. (1996) · doi:10.5555/234285.234289 · Cited by 3,083
Convolutional Sequence to Sequence Learning
Gehring, J., Auli, M., Grangier, D., Yarats, D., & Dauphin, Y. N. (2017) · arXiv · doi:10.48550/arxiv.1705.03122 · Cited by 2,850 · cited 1×
“Therefore, recently, convolutional architectures are used in other sequence to sequence tasks, e.g., conditional image generation [137] and machine translation [42, 43, 138].”
Method
Minimum error rate training in statistical machine translation
Och, F. J. (2003) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1075096.1075117 · Cited by 2,728 · cited 1×
“Minimum Error rate training (MERT) [106] is used to learn these weights.”
Method
A neural probabilistic language model
BengioYoshua, DucharmeRéjean, VincentPascal, & JanvinChristian (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944966 · Cited by 2,646
Generative Adversarial Text to Image Synthesis
Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., & Lee, H. (2016) · arXiv · doi:10.48550/arxiv.1605.05396 · Cited by 2,494 · cited 1×
“e Generator and the Discriminator. GANs have already been used successfully in a variety of applications, including image captioning[26, 126], image to image translation [56], text to image synthesis [15, 115], and text generation [36,145]. There are two issues with GAN. First, GAN can work well in generating natural…”
Background
SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
Yu, L., Zhang, W., Wang, Y.-X., & Yu, Y. H. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10804 · Cited by 2,422
Generating Typed Dependency Parses from Phrase Structure Parses
de Marneffe, M.-C., MacCartney, B., & Manning, C. D. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3wtkaidk6ea8 · Cited by 2,289 · cited 1×
“thod works at a finer level and embeds fragments of images and fragments of sentences. This method breaks down the images into a number of objects and sentences into a dependency tree relations (DTR) [28] and reasons about their latent, inter-modal alignment. It shows that the method achieves significant improvements…”
Method
Self-Critical Sequence Training for Image Captioning
Rennie, S. J., Marcheret, E., Mroueh, Y., Ross, J., & Goel, V. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.131 · Cited by 2,231 · cited 1×
“0.580 0.439 0.332 0.266 - 1.085 - Gan et al. 2017 [41] VS,SL,WS,CA,SCB 0.741 0.578 0.444 0.341 0.261 - 1.041 - Zhang et al. 2017 [161] VS,ODL,WS,EDA - - - 0.344 0.267 0.558 1.162 - Rennie et al. 2017 [120] VS,ODL,WS,EDA - - - .319 0.255 0.543 1.06 - Yao et al. 2017 [155] VS,SL,WS,EDA,SCB 0.734 0.567 0.430 0.326 0.254…”
Influential
Actor-critic algorithms
Konda, V. R., & Tsitsiklis, J. N. (2002) · Cited by 2,202
The Concrete Distribution: A Continuous Relaxation of Discrete Random VariablesPreprint
Maddison, C. J., Mnih, A., & Teh, Y. W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.00712 · Cited by 2,170 · cited 1×
“Gumbel sampler [58, 91] is used to overcome the discrete data problem.”
Method
SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning
Chen, L., Zhang, H., Xiao, J., Nie, L., Shao, J., Liu, W., & Chua, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.667 · Cited by 2,089 · cited 2×
“[21, 59, 61, 144, 150, 152] have performed experiments using the dataset.”
Method
Conditional Image Generation with PixelCNN DecodersPreprint
van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05328 · Cited by 2,016 · cited 1×
“ucture of the sentences and they are faster in processing than LSTMs. Therefore, recently, convolutional architectures are used in other sequence to sequence tasks, e.g., conditional image generation [137] and machine translation [42,43,138]. Inspired by the above success of CNNs in sequence learning tasks, Gu et al.…”
Method
SPICE: Semantic Propositional Image Caption Evaluation
Anderson, P., Fernando, B., Johnson, M., & Gould, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_24 · Cited by 1,986 · cited 1×
“[3] is a new caption evaluation metric based on semantic concept.”
Background
Image Captioning with Semantic Attention
You, Q., Jin, H., Wang, Z., Chen, F., & Luo, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.503 · Cited by 1,869 · cited 4×
“A general approach of this category is to analyze the visual content of the image first and then generate image captions from the visual content using a language model [70, 152, 155, 156].”
BackgroundMethodInfluential
Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning
Lu, J., Xiong, C., Parikh, D., & Socher, R. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.345 · Cited by 1,724 · cited 1×
“Lu et al. [88] proposed another attention-based image captioning method.”
Method
International Conference on Learning Representations (ICLR)
Abbas, M. Z. A., Kivinen, J., & Raiko, T. (2016) · Cited by 1,694
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
Bengio, S., Vinyals, O., Jaitly, N., & Shazeer, N. (2015) · arXiv · doi:10.48550/arxiv.1506.03099 · Cited by 1,621 · cited 1×
“next word is predicted given the image and all the previously generated ground-truth words. Therefore, the generated captions look-like ground-truth captions. This phenomenon is called exposure bias [10] problem. (ii) Evaluation metrics at test time are non-differentiable. Ideally sequence models for image captioning…”
Background
Understanding inverse document frequency: on theoretical arguments for IDF
Robertson, S. E. (2004) · Journal of Documentation · doi:10.1108/00220410410560582 · Cited by 1,550 · cited 1×
“However, CIDEr achieves human consensus using term frequency-inverse document frequency (TF-IDF) [121].”
Background
DRAW: A Recurrent Neural Network For Image GenerationPreprint
Gregor, K., Danihelka, I., Graves, A., Rezende, D. J., & Wierstra, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.04623 · Cited by 1,498 · cited 1×
“However, Johnson et al. [62] use a differential, spatial soft attention mechanism [49, 57] and bilinear interpolation [57] instead of ROI pooling mechanism [44].”
Background
Unifying Visual-Semantic Embeddings with Multimodal Neural Language ModelsPreprint
Kiros, R., Salakhutdinov, R., & Zemel, R. S. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.2539 · Cited by 1,496 · cited 2×
“ceptance Date: October 2018. A Comprehensive Survey of Deep Learning for Image Captioning 0:23 Reference Datasets Evaluation Metrics Kiros et al. 2014 [69] IAPR TC-12,SBU BLEU, PPLX Kiros et al. 2014 [70] Flickr 8K, Flickr 30K R@K, mrank Mao et al. 2014 [95] IAPR TC-12, Flickr 8K/30K BLEU, R@K, mrank Karpathy et al. 2…”
BackgroundMethod
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models
Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J., & Lazebnik, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.303 · Cited by 1,444
Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics
Hodosh, M., Young, P., & Hockenmaier, J. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3994 · Cited by 1,375 · cited 4×
“, No. 0, Article 0. Acceptance Date: October 2018. 0:24 Hossain et al. Fig. 13. Captions generated by Jia et al. [59] on some sample images from the Flickr8k dataset. 4.1.3 Flickr8K Dataset. Flickr8k [55] is a popular dataset and has 8000 images collected from Flickr. The training data consists of 6000 images, the tes…”
BackgroundMethodInfluential
From captions to visual concepts and back
Fang, H., Gupta, S., Iandola, F., Srivastava, R. K., Deng, L., Dollár, P., Gao, J., He, X., Mitchell, M., Platt, J. C., et al. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298754 · Cited by 1,321
Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing
(2013) · doi:10.18653/v1/d13-1 · Cited by 1,296
Generation and Comprehension of Unambiguous Object Descriptions
Mao, J., Huang, J., Toshev, A., Camburu, O., Yuille, A., & Murphy, K. P. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.9 · Cited by 1,281 · cited 1×
“Mao et al. [92] proposed a special type of text generation method for images.”
Method
Every Picture Tells a Story: Generating Sentences from Images
Farhadi, A., Hejrati, M., Sadeghi, M. A., Young, P., Rashtchian, C., Hockenmaier, J., & Forsyth, D. A. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15561-1_2 · Cited by 1,240 · cited 2×
“of blank slots to generate captions. In these approaches, different objects, attributes, actions are detected first and then the blank spaces in the templates are filled. For example, Farhadi et al. [34] use a triplet of scene elements to fill the template slots for generating image captions. Li et al. [80] extract th…”
BackgroundMethod
Exploring the Limits of Language ModelingPreprint
Józefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., & Wu, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02410 · Cited by 1,238 · cited 1×
“The neural language models have limitations to handle a large amount of data and are inefficient to work with long term memory [64].”
Background
A Framework for Multiple-Instance Learning
Maron, O., & Lozano‐Pérez, T. (1997) · Cited by 1,232
DenseCap: Fully Convolutional Localization Networks for Dense Captioning
Johnson, J., Karpathy, A., & Fei-Fei, L. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.494 · Cited by 1,199 · cited 3×
“However, Johnson et al. [62] use a differential, spatial soft attention mechanism [49, 57] and bilinear interpolation [57] instead of ROI pooling mechanism [44].”
BackgroundMethod
Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN)Preprint
Mao, J., Xu, W., Yang, Y., Wang, J., Huang, Z., & Yuille, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6632 · Cited by 1,186 · cited 3×
“Mao et al. [94] proposed a multimodal Recurrent Neural Network (m-RNN) method for generating novel image captions.”
BackgroundMethod
Image retrieval using scene graphs
Johnson, J., Krishna, R., Stark, M., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298990 · Cited by 1,101 · cited 1×
“PICE. SPICE (Semantic Propositional Image Caption Evaluation) [3] is a new caption evaluation metric based on semantic concept. It is based on a graph-based semantic representation called scene-graph [63, 123]. This graph can extract the information of different objects, attributes and their relationships from the ima…”
Method
ReferItGame: Referring to Objects in Photographs of Natural Scenes
Kazemzadeh, S., Ordóñez, V., Matten, M., & Berg, T. L. (2014) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1086 · Cited by 1,082 · cited 2×
“This method can generate a description for an specific object or region that is called referring expression [37, 46, 68, 102, 103, 136, 141].”
Method
Im2Text: Describing Images Using 1 Million Captioned Photographs
Ordóñez, V., Kulkarni, G. A., & Berg, T. L. (2011) · Cited by 879
Show and Tell: Lessons Learned from the 2015 MSCOCO Image Captioning Challenge
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2587640 · Cited by 875 · cited 1×
“Generating well-formed sentences requires both syntactic and semantic understanding of the language [143].”
Background
Deep Fragment Embeddings for Bidirectional Image Sentence MappingPreprint
Karpathy, A., Joulin, A., & Fei-Fei, L. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.5679 · Cited by 825 · cited 1×
“n Metrics Kiros et al. 2014 [69] IAPR TC-12,SBU BLEU, PPLX Kiros et al. 2014 [70] Flickr 8K, Flickr 30K R@K, mrank Mao et al. 2014 [95] IAPR TC-12, Flickr 8K/30K BLEU, R@K, mrank Karpathy et al. 2014 [66] PASCAL1K, Flickr 8K/30K R@K, mrank Mao et al. 2015 [94] IAPR TC-12, Flickr 8K/30K, MS COCO BLEU, R@K, mrank Chen e…”
Background
Grounded Compositional Semantics for Finding and Describing Images with Sentences
Socher, R., Karpathy, A., Le, Q. V., Manning, C. D., & Ng, A. Y. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00177 · Cited by 794 · cited 2×
“Most sentence-image multimodal methods [38, 66, 70, 128] use pre-computed word embedding vectors to initialize their model.”
Method
Boosting Image Captioning with Attributes
Yao, T., Pan, Y., Li, Y., Qiu, Z., & Mei, T. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.524 · Cited by 752 · cited 2×
“A general approach of this category is to analyze the visual content of the image first and then generate image captions from the visual content using a language model [70, 152, 155, 156].”
BackgroundMethod
Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics
Lin, C.-Y., & Och, F. J. (2004) · Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics - ACL '04 · doi:10.3115/1218955.1219032 · Cited by 737 · cited 1×
“There are some cases where an increase in BLEU score does not mean that the quality of the generated text is good [82].”
Background
SALICON: Saliency in Context
Jiang, M., Huang, S., Duan, J., & Zhao, Q. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298710 · Cited by 729 · cited 1×
“For experiments, it uses SALICON dataset [60] and achieves good results.”
Method
Eye Movement Analysis for Activity Recognition Using Electrooculography
Bulling, A., Ward, J. A., Gellersen, H. W., & Tröster, G. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.86 · Cited by 720
Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization
Murray, G., Renals, S. J., Carletta, J., & Moore, J. D. (2005) · Meeting of the Association for Computational Linguistics · Cited by 716
Learning photographic global tonal adjustment with a database of input/output image pairs
Bychkovsky, V., Paris, S., Chan, E., & Durand, F. (2011) · doi:10.1109/cvpr.2011.5995413 · Cited by 666
Re-evaluating the Role of Bleu in Machine Translation Research
Callison-Burch, C., Osborne, M., & Koehn, P. (2006) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 632
Multi-task Sequence to Sequence LearningPreprint
Luong, M.-T., Le, Q. V., Sutskever, I., Vinyals, O., & Kaiser, Ł. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06114 · Cited by 606 · cited 1×
“It uses multitask sequence to sequence training [89] for identifying the style factors and then add these factors at run time for generating attractive captions.”
Method
Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns
Ojala, T., Pietikäinen, M., & Mäenpää, T. (2000) · Lecture notes in computer science · doi:10.1007/3-540-45054-8_27 · Cited by 600 · cited 1×
“ategories: (1) Traditional machine learning based techniques and (2) Deep machine learning based techniques. In traditional machine learning, hand crafted features such as Local Binary Patterns (LBP) [107], Scale-Invariant Feature Transform (SIFT) [87], the Histogram of Oriented Gradients (HOG) [27], and a combination…”
Method
Multimodal Neural Language Models
Kiros, R., Salakhutdinov, R., & Zemel, R. (2014) · Cited by 593
Sequence Level Training with Recurrent Neural Networks
Ranzato, M., Chopra, S., Auli, M., & Zaremba, W. (2016) · International Conference on Learning Representations · Cited by 572
Three new graphical models for statistical language modelling
Mnih, A., & Hinton, G. E. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273577 · Cited by 569 · cited 1×
“Several neural language models such as neural probabilistic language model [11], log-bilinear models [105], skip-gram models [98], and recurrent neural networks (RNNs) [99] have been proposed for learning sequence to sequence tasks.”
Learning photographic global tonal adjustment with a database of input / output image pairs
Bychkovsky, V., Paris, S., Chan, E., & Durand, F. (2011) · doi:10.1109/cvpr.2011.5995332 · Cited by 531
Baby talk: Understanding and generating simple image descriptions
Kulkarni, G. A., Premraj, V., Dhar, S., Li, S., Choi, Y., Berg, A. C., & Berg, T. L. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995466 · Cited by 530
Mind's eye: A recurrent visual representation for image caption generation
Chen, X., & Zitnick, C. L. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298856 · Cited by 498 · cited 3×
“The top-down paradigms [23, 31, 65, 93, 94, 142, 152] are used for extracting visual features first and then convert them into words.”
BackgroundMethod
Towards Diverse and Natural Image Descriptions via a Conditional GAN
Dai, B., Fidler, S., Urtasun, R., & Lin, D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.323 · Cited by 489 · cited 3×
“Dai et al. [26] also proposed a GAN based image captioning method.”
Method
Guiding the Long-Short Term Memory Model for Image Caption Generation
Jia, X., Gavves, E., Fernando, B., & Tuytelaars, T. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.277 · Cited by 480 · cited 3×
“[21, 59, 61, 144, 150, 152] have performed experiments using the dataset.”
Background
Semantic Compositional Networks for Visual Captioning
Gan, Z., Gan, C., He, X., Pu, Y., Tran, K., Gao, J., Carin, L., & Deng, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.127 · Cited by 465
Explain Images with Multimodal Recurrent Neural NetworksPreprint
Mao, J., Xu, W., Yang, Y., Wang, J., & Yuille, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.1090 · Cited by 458 · cited 2×
“Many image captioning methods [66, 69, 95] are built on recurrent neural networks at the contemporary times.”
Background
Convolutional Image Captioning
Aneja, J., Deshpande, A., & Schwing, A. G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00583 · Cited by 430
Improved Image Captioning via Policy Gradient optimization of SPIDEr
Liu, S., Zhu, Z., Ye, N., Guadarrama, S., & Murphy, K. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.100 · Cited by 430 · cited 1×
“Liu et al. [85] introduced a new captions evaluation metric that is a good choice by human raters.”
Background
A Convolutional Encoder Model for Neural Machine Translation
Gehring, J., Auli, M., Grangier, D., & Dauphin, Y. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1012 · Cited by 422 · cited 1×
“Therefore, recently, convolutional architectures are used in other sequence to sequence tasks, e.g., conditional image generation [137] and machine translation [42, 43, 138].”
Method
Image Captioning and Visual Question Answering Based on Attributes and External Knowledge
Wu, Q., Shen, C., Wang, P., Dick, A., & van den Hengel, A. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2708709 · Cited by 422 · cited 5×
“Image captioning methods such as [22, 65, 142, 144, 150] use this dataset for their experiments.”
BackgroundMethodInfluential
Midge: Generating Image Descriptions From Computer Vision Detections
Mitchell, M., Dodge, J., Goyal, A., Yamaguchi, K., Stratos, K., Han, X., Mensch, A., Berg, A., Berg, T. L., & Daumé, H. (2012) · Cited by 418
MaskGAN: Better Text Generation via Filling in the______Preprint
Fedus, W., Goodfellow, I., & Dai, A. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.07736 · Cited by 412 · cited 1×
“GANs have already been used successfully in a variety of applications, including image captioning[26, 126], image to image translation [56], text to image synthesis [15, 115], and text generation [36, 145].”
Method
Learning to Recognize Daily Actions Using Gaze
Fathi, A., Li, Y., & Rehg, J. M. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33718-5_23 · Cited by 396 · cited 1×
“Thus, gaze mechanisms have already shown their potential performances in eye-based user modeling [18, 35, 109, 122, 124], object localization [100] or recognition [67] and holistic scene understanding [158, 160].”
Background
Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures
Bernardi, R., Çakıcı, R., Elliott, D., Erdem, A., Erdem, E., İkizler-Cinbiş, N., Keller, F., Muscat, A., & Plank, B. (2016) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4900 · Cited by 378 · cited 1×
“So far, only three survey papers [8, 13, 75] have been published on this research topic.”
Background
Composing Simple Image Descriptions using Web-scale N-grams
Li, S., Kulkarni, G. A., Berg, T. L., Berg, A. C., & Choi, Y. (2011) · Cited by 356
Generating Semantically Precise Scene Graphs from Textual Descriptions for Improved Image Retrieval
Schuster, S., Krishna, R., Chang, A., Fei-Fei, L., & Manning, C. D. (2015) · Workshop on Vision and Language (VL) · doi:10.18653/v1/w15-2812 · Cited by 346 · cited 1×
“PICE. SPICE (Semantic Propositional Image Caption Evaluation) [3] is a new caption evaluation metric based on semantic concept. It is based on a graph-based semantic representation called scene-graph [63, 123]. This graph can extract the information of different objects, attributes and their relationships from the ima…”
Method
Deep Reinforcement Learning-Based Image Captioning with Embedding Reward
Ren, Z., Wang, X., Zhang, N., Lv, X., & Li, L.-J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.128 · Cited by 345 · cited 2×
“Many image captioning methods [26, 39, 61, 112, 119, 126, 135, 144, 149, 151, 156] use the dataset in their experiments.”
Method
Collective Generation of Natural Image Descriptions
Кузнецова, П., Ordóñez, V., Berg, A., Berg, T. L., & Choi, Y. (2012) · Cited by 320
Image Captioning with Deep Bidirectional LSTMs
Wang, C., Yang, H., Bartz, C., & Meinel, C. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2964299 · Cited by 303 · cited 4×
“[21, 59, 61, 144, 150, 152] have performed experiments using the dataset.”
MethodInfluential
StyleNet: Generating Attractive Visual Captions with Styles
Gan, C., Gan, Z., He, X., Gao, J., & Deng, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.108 · Cited by 291 · cited 2×
“Gan et al. [39] proposed a novel image captioning system called StyleNet.”
BackgroundMethod
Improving Image-Sentence Embeddings Using Large Weakly Annotated Photo Collections
Gong, Y., Wang, L., Hodosh, M., Hockenmaier, J., & Lazebnik, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10593-2_35 · Cited by 288 · cited 1×
“The captions for the query image are selected from this captions pool [47, 55, 108, 130].”
Background
The IAPR Benchmark : A New Evaluation Resource for Visual Information Systems
Grubinger, M. (2006) · RWTH Publications (RWTH Aachen) · Cited by 285
Deep Compositional Captioning: Describing Novel Object Categories without Paired Training Data
Hendricks, L. A., Venugopalan, S., Rohrbach, M., Mooney, R. J., Saenko, K., & Darrell, T. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.8 · Cited by 283 · cited 1×
“Anne et al. [6] proposed a Deep Compositional Captioner (DCC) that can represent the unseen objects in generated captions.”
Background
Image Description using Visual Dependency Representations
Elliott, D., & Keller, F. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1128 · Cited by 282 · cited 2×
“mmatically correct captions. However, templates are predefined and cannot generate variable-length captions. Moreover, later on, parsing based language models have been introduced in image captioning [2, 32, 76, 77, 101] which are more powerful than fixed template-based methods. Therefore, in this paper, we do not foc…”
Method
T ree T alk : Composition and Compression of Trees for Image Descriptions
Kuznetsova, P., Ordóñez, V., Berg, T. L., & Choi, Y. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00188 · Cited by 268 · cited 1×
“Moreover, later on, parsing-based language models have been introduced in image captioning [2, 32, 76, 77, 101], which are more powerful than fixed template-based methods.”
Background
A survey on automatic image caption generation
Bai, S., & An, S. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.05.080 · Cited by 253 · cited 1×
“So far, only three survey papers [8, 13, 75] have been published on this research topic.”
Background
Bottom-Up and Top-Down Attention for Image Captioning and VQA.Preprint
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2017) · arXiv (Cornell University) · Cited by 252
Review Networks for Caption GenerationPreprint
Yang, Z., Yuan, Y., Wu, Y., Salakhutdinov, R., & Cohen, W. W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07912 · Cited by 230 · cited 4×
“[151] proposed a semanticconcept-based image captioning method.”
MethodInfluential
Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
Shetty, R., Rohrbach, M., Hendricks, L. A., Fritz, M., & Schiele, B. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.445 · Cited by 230
SentiCap: Generating Image Descriptions with Sentiments
Mathews, A., Xie, L., & He, X. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10475 · Cited by 219 · cited 3×
“Jin et al. 2015 [61] Flickr 8K/30K, MS COCO BLEU, METEOR, ROUGE, CIDEr Wu et al. 2016 [151] MS COCO BLEU, METEOR, CIDEr Sugano et at. 2016 [129] MS COCO BLEU, METEOR, ROUGE, CIDEr Mathews et al. 2016 [97] MS COCO, SentiCap BLEU, METEOR, ROUGE, CIDEr Wang et al. 2016 [144] Flickr 8K/30K, MS COCO BLEU, R@K Johnson et al…”
BackgroundMethodInfluential
Areas of Attention for Image Captioning
Pedersoli, M., Lucas, T., Schmid, C., & Verbeek, J. J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.140 · Cited by 212 · cited 5×
“[112] proposed an area-based attention mechanism for image captioning.”
BackgroundMethodResultInfluential
Attention Correctness in Neural Image Captioning
Liu, C., Mao, J., Sha, F., & Yuille, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11197 · Cited by 210 · cited 1×
“Liu et al. [84] proposed a method for neural image captioning.”
Method
Language Models for Image Captioning: The Quirks and What Works
Devlin, J., Cheng, H., Fang, H., Gupta, S., Deng, L., He, X., Zweig, G., & Mitchell, M. (2015) · doi:10.3115/v1/p15-2017 · Cited by 207 · cited 1×
“However, in terms of performance there exists a large gap between the evaluation of the metrics and human judgement of evaluation [20, 30, 74].”
Background
Learning Attributes Equals Multi-Source Domain Generalization
Gan, C., Yang, T., & Gong, B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.17 · Cited by 190 · cited 2×
“However, Wu et al. Recent semantic concept based image captioning methods [150, 156] applied semantic-concept-detection process [40]to obtain explicit semantic concepts.”
Method
Captioning Images with Diverse Objects
Venugopalan, S., Hendricks, L. A., Rohrbach, M., Mooney, R. J., Darrell, T., & Saenko, K. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.130 · Cited by 178 · cited 3×
“recognize the unseen objects from unpaired images and incorporate them with LSTM to generate captions. Generating captions for the unseen images is a challenging research problem. Venugopalan et al. [140] introduced a Novel Object Captioner (NOC) for generating captions for unseen objects in the image. They used exter…”
BackgroundInfluential
Dense Captioning with Joint Inference and Visual Context
Yang, L., Tang, K., Yang, J., & Li, L.-J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.214 · Cited by 174 · cited 1×
“Yang et al. [153] proposed another dense captioning method.”
Method
Attend to You: Personalized Image Captioning with Context Sequence Memory Networks
Park, C. C., Kim, B., & Kim, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.681 · Cited by 170 · cited 1×
“Park et al. [111] introduced a different type of attention-based image captioning method.”
Method
Incorporating Copying Mechanism in Image Captioning for Learning Novel Objects
Yao, T., Pan, Y., Li, Y., & Mei, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.559 · Cited by 157 · cited 1×
“Yao et al. [154] proposed a copying mechanism to generate description for novel objects.”
Background
An Empirical Study of Language CNN for Image Captioning
Gu, J., Wang, G., Cai, J., & Chen, T. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.138 · Cited by 156 · cited 1×
“Inspired by the above success of CNNs in sequence learning tasks, Gu et al. [51] proposed a CNN language model-based image captioning method.”
Method
Show, Adapt and Tell: Adversarial Training of Cross-Domain Image Captioner
Chen, T.-H., Liao, Y.-H., Chuang, C.-Y., Hsu, W.-T., Fu, J., & Sun, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.64 · Cited by 153
Learning Like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images
Mao, J., Xu, W., Yang, Y., Wang, J., Huang, Z., & Yuille, A. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.291 · Cited by 149 · cited 1×
“The top-down paradigms [23, 31, 65, 93, 94, 142, 152] are used for extracting visual features first and then convert them into words.”
Method
A Game-Theoretic Approach to Generating Spatial Descriptions
Golland, D., Liang, P., & Klein, D. (2010) · Cited by 132
Training Object Class Detectors from Eye Tracking Data
Papadopoulos, D. P., Clarke, A. D. F., Keller, F., & Ferrari, V. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10602-1_24 · Cited by 131 · cited 1×
“Thus, gaze mechanisms have already shown their potential performances in eye-based user modeling [18, 35, 109, 122, 124], object localization [100] or recognition [67] and holistic scene understanding [158, 160].”
Background
Automatic Concept Discovery from Parallel Text and Visual Corpora
Sun, C., Gan, C., & Nevatia, R. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.298 · Cited by 111 · cited 1×
“The captions for the query image are selected from these captions pool [47, 55, 108, 130].”
Background
Rich Image Captioning in the Wild
Tran, K., He, X., Zhang, L., & Sun, J. Z. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.61 · Cited by 110
METEOR, M-BLEU and M-TER
Agarwal, A., & Lavie, A. (2008) · Conference on Machine Translation (WMT) · doi:10.3115/1626394.1626406 · Cited by 109 · cited 1×
“nt. There are certain key entities such as celebrities and landmarks that are out of their scope. The generated captions of these methods are evaluated on automatic metrics such as BLEU [110], METEOR [1], and CIDEr [139]. These evaluation metrics have already shown good results on these methods. However, in terms of p…”
Method
Skeleton Key: Image Captioning by Skeleton-Attribute Decomposition
Wang, Y., Lin, Z., Shen, X., Cohen, S. D., & Cottrell, G. W. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.780 · Cited by 108 · cited 1×
“Wang et al. [148] proposed a method that locates the objects and their interactions first and then identifies and extracts the relevant attributes to generate image captions.”
Method
The use of spatial relations in referring expression generation
Viethen, J., & Dale, R. (2008) · doi:10.3115/1708322.1708334 · Cited by 105 · cited 1×
“edding space. Mao et al. [92] proposed a special type of text generation method for images. This method can generate a description for an specific object or region that is called referring expression [37, 46, 68, 102, 103, 136, 141]. Using this expression it can then infer the object or region which is being described…”
Method
Framing image description as a ranking task
HodoshMicah, YoungPeter, & HockenmaierJulia (2013) · doi:10.5555/2566972.2566993 · Cited by 98
Aligning where to see and what to tell: image caption with region-based attention and scene factorizationPreprint
Jin, J., Fu, K., Cui, R., Sha, F., & Zhang, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06272 · Cited by 97 · cited 3×
“[21, 59, 61, 144, 150, 152] have performed experiments using the dataset.”
Method
Active Visual Segmentation
Mishra, A. K., Aloimonos, Y., Cheong, L. F., & Kassim, A. A. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.171 · Cited by 91 · cited 1×
“Thus, gaze mechanisms have already shown their potential performances in eye-based user modeling [18, 35, 109, 122, 124], object localization [100] or recognition [67] and holistic scene understanding [158, 160].”
Background
Generating Image Descriptions Using Dependency Relational Patterns
Aker, A., & Gaizauskas, R. (2010) · Cited by 85
Paying Attention to Descriptions Generated by Image Captioning Models
Tavakoli, H. R., Shetty, R., Borji, A., & Laaksonen, J. T. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.272 · Cited by 82
Learning Distributions over Logical Forms for Referring Expression Generation
FitzGerald, N., Artzi, Y., & Zettlemoyer, L. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1197 · Cited by 81 · cited 1×
“edding space. Mao et al. [92] proposed a special type of text generation method for images. This method can generate a description for an specific object or region that is called referring expression [37, 46, 68, 102, 103, 136, 141]. Using this expression it can then infer the object or region which is being described…”
Method
Building a semantically transparent corpus for the generation of referring expressions
van Deemter, K., van der Sluis, I., & Albert Gatt, A. (2006) · International Conference on Natural Language Generation · doi:10.3115/1706269.1706296 · Cited by 75
Attention Correctness in Neural Image CaptioningPreprint
Liu, C., Mao, J., Sha, F., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09553 · Cited by 73 · cited 1×
“Liu et al. [84] proposed a method for neural image captioning.”
Method
Generating Expressions that Refer to Visible Objects
Mitchell, M., Hltcoe, J. H., van Deemter, K., & Reiter, E. (2013) · Aberdeen University Research Archive (Aberdeen University) · Cited by 64
A parallel-fusion RNN-LSTM architecture for image caption generation
Wang, M., Song, L., Yang, X., & Luo, C. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7533201 · Cited by 58 · cited 1×
“Wang et al. [146] proposed a parallel-fusion RNN-LSTM architecture for image caption generation.”
Background
Eye tracking assisted extraction of attentionally important objects from videos
Karthikeyan, S., Ngo, T., Eckstein, M. P., & Manjunath, B. S. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298944 · Cited by 54 · cited 1×
“Thus, gaze mechanisms have already shown their potential performances in eye-based user modeling [18, 35, 109, 122, 124], object localization [100] or recognition [67], and holistic scene understanding [158, 160].”
Background
Exploring the role of gaze behavior and object detection in scene understanding
Yun, K., Peng, Y., Samaras, D., Zelinsky, G. J., & Berg, T. L. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00917 · Cited by 53 · cited 1×
“hus, gaze mechanisms have already shown their potential performances in eye-based user modeling [18, 35, 109, 122, 124], object localization [100] or recognition [67] and holistic scene understanding [158, 160]. However, Sugano et al. [129] claimed that gaze information has not yet been integrated in image captioning…”
Method
Prediction of search targets from fixations in open-world settings
Sattar, H., Muller, S., Fritz, M., & Bulling, A. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298700 · Cited by 50 · cited 1×
“ption of humans about a scene. Human gaze can identify the important locations of objects in an image. Thus, gaze mechanisms have already shown their potential performances in eye-based user modeling [18, 35, 109, 122, 124], object localization [100] or recognition [67] and holistic scene understanding [158, 160]. How…”
Background
Joint Image-Text Representation by Gaussian Visual-Semantic Embedding
Ren, Z., Jin, H., Lin, Z., Chen, F., & Yuille, A. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2967212 · Cited by 48 · cited 1×
“Visual semantic embedding [117, 118] is used to compute the actual reward value in predicting the correct word.”
Method
Natural reference to objects in a visual domain
Mitchell, M., van Deemter, K., & Reiter, E. (2010) · International Conference on Natural Language Generation · Cited by 44
Text Generation Based on Generative Adversarial Nets with Latent Variables
Wang, H., Qin, Z., & Wan, T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93037-4_8 · Cited by 41
Prediction of Search Targets From Fixations in Open-World Settings
Sattar, H., Müller, S., Fritz, M., & Bulling, A. (2015) · Cited by 39
Describing images by feeding LSTM with structural words
Ma, S., & Han, Y. (2016) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2016.7552883 · Cited by 34 · cited 1×
“Maet al. [90] proposed another compositional network-based image captioning method.”
Rich Image Captioning in the WildPreprint
Tran, K., He, X., Zhang, L., Sun, J., Carapcea, C., Thrasher, C., Buehler, C., & Sienkiewicz, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.09016 · Cited by 34 · cited 2×
“However, Tran et al. [135] introduced a different image captioning method.”
Method
From Where and How to What We See
Karthikeyan, S., Jagadeesh, V., Shenoy, R., Ecksteinz, M., & Manjunath, B. S. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.83 · Cited by 33 · cited 1×
“nt locations of objects in an image. Thus, gaze mechanisms have already shown their potential performances in eye-based user modeling [18, 35, 109, 122, 124], object localization [100] or recognition [67] and holistic scene understanding [158, 160]. However, Sugano et al. [129] claimed that gaze information has not ye…”
Method
Simple Image Description Generator via a Linear Phrase-Based ApproachPreprint
Lebret, R., Pinheiro, P. O., & Collobert, R. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.8419 · Cited by 28 · cited 1×
“In bottom-up approaches, [32, 34, 74, 76, 78, 80], visual concepts (e.”
Understanding scene understanding
Zelinsky, G. J. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00954 · Cited by 27 · cited 1×
“Thus, gaze mechanisms have already shown their potential performances in eye-based user modeling [18, 35, 109, 122, 124], object localization [100] or recognition [67] and holistic scene understanding [158, 160].”
Background
BLEU in characters: towards automatic MT evaluation in languages without word delimiters
Denoual, E., & Lepage, Y. (2004) · Cited by 25
Learning-based Composite Metrics for Improved Caption Evaluation
Sharif, N., White, L., Bennamoun, M., & Shah, S. A. A. (2018) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/p18-3003 · Cited by 23 · cited 1×
“e 0. Acceptance Date: October 2018. 0:26 Hossain et al. aspects. However, commonly used evaluation metrics consider only some specific features (e.g., lexical or semantic) of languages. Sharif et al. [125] proposed learning-based composite metrics for evaluation of image captions. The composite metric incorporates a s…”
Background
Actor-citic agorithms
Konda, V. R., & Tsitsiklis, J. N. (1999) · neural information processing systems · Cited by 20
A survey of evolution of image captioning techniques
Kumar, A., & Goel, S. (2017) · International Journal of Hybrid Intelligent Systems · doi:10.3233/his-170246 · Cited by 20 · cited 1×
“d on image captioning with deep machine learning being popularly used. Deep learning algorithms can handle complexities and challenges of image captioning quite well. So far, only three survey papers [8, 13, 75] have been published on this research topic. Although the papers have presented a good literature survey of…”
Background
Image Captioning with an Intermediate Attributes Layer.
Wu, Q., Shen, C., van den Hengel, A., Liu, L., & Dick, A. (2015) · Cited by 17
Proceedings of the 5th Workshop on Vision and Language
Pezzelle, S., Ravi Shekhar, R. S., & Bernardi, R. (2016) · doi:10.18653/v1/w16-32 · Cited by 17
Skeleton Key: Image Captioning by Skeleton-Attribute DecompositionPreprint
Wang, Y., Lin, Z., Shen, X., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.06972 · Cited by 16 · cited 1×
“Wang et al. [148] proposed a method that locates the objects and their interactions first and then identifies and extracts the relevant attributes to generate image captions.”
Method
Describing Images with Hierarchical Concepts and Object Class Localization
Han, Y., & Li, G. (2015) · ACM International Conference on Multimedia Retrieval (ICMR) · doi:10.1145/2671188.2749290 · Cited by 8 · cited 1×
“It also uses a multi-task method similar to multiple instance learning method [33], and multi-layer optimization method [52] to generate structural words.”
Method
Dense Captioning with Joint Inference and Visual ContextPreprint
Yang, L., Tang, K., Yang, J., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06949 · Cited by 8 · cited 1×
“Yang et al. [153] proposed another dense captioning method.”
Method

Flagged first, then by citations

Research works printed but not linked: 58

10.5555/1626394.1626406 · DOI
Aker Ahmet, 2010, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1250--1258
Bahdanau Dzmitry, 2015, International Conference on Learning Representations (ICLR’15)
Cristian Bodnar. 2018. Text to image synthesis using generative adversarial networks. arXiv preprint arXiv:1805.00676. Cristian Bodnar. 2018. Text to image synthesis using generative adversarial networks. arXiv preprint arXiv:1805.00676. · arXiv
Chen Xinlei, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2422--2431
Cho Kyunghyun, 2014, Dzmitry Bahdanau, and Yoshua Bengio.
Denoual Etienne, Companion
Fang Hao, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1473--1482
10.5555/1888089.1888092 · DOI
Fedus William, 2018, Dai
FitzGerald Nicholas, 2013, Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 1914--1925
Frome Andrea, 2013, Devise: A deep visual-semantic embedding model. In Advances in Neural Information Processing Systems. 2121--2129.
Gehring Jonas, 2016, Dauphin
Gehring Jonas, 2017, Dauphin
10.5555/1870658.1870698 · DOI
Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680. Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680.
Gregor Karol, 2015, Proceedings of Machine Learning Research. 1462--1471
Grubinger Michael, 2006, International Workshop Ontoimage, 5
Isola Phillip, Proceedings of the IEEE International Conference on Computer Vision (CVPR’17)
Max Jaderberg Karen Simonyan Andrew Zisserman and Koray Kavukcuoglu. 2015. Spatial transformer networks. In Advances in Neural Information Processing Systems. 2017--2025. Max Jaderberg Karen Simonyan Andrew Zisserman and Koray Kavukcuoglu. 2015. Spatial transformer networks. In Advances in Neural Information Processing Systems. 2017--2025.
Jang Eric, 2017, International Conference on Learning Representations (ICLR’17)
Kazemzadeh Sahar, 2014, Berg
Kiros Ryan, 2014, Proceedings of the 31st International Conference on Machine Learning (ICML’14)
Kiros Ryan, Workshop on Neural Information Processing Systems (NIPS’14)
Konda Vijay R., 2000, Tsitsiklis
Krizhevsky Alex, 2012, Hinton
10.5555/2390524.2390575 · DOI
Kuznetsova Polina, 2014, TREETALK: Composition and compression of trees for image descriptions.TACL 2, 10
Lebret RÃl’mi, 2015, Workshop on International Conference on Learning Representations (ICLR’15)
Li Siming, 2011, Proceedings of the 15th Conference on Computational Natural Language Learning. Association for Computational Linguistics, 220--228
Lin Chin-Yew, 2004, Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, 8
Liu Chenxi, 2017, Yuille
Maddison Chris J., 2017, International Conference on Learning Representations (ICLR’17)
Mao Junhua, 2015, International Conference on Learning Representations (ICLR’15)
Mao Junhua, 2014, Yuille
Mitchell Margaret, 2012, Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 747--756
10.5555/1873738.1873752 · DOI
Mitchell Margaret, 2013, Kees Van Deemter, and Ehud Reiter
10.1145/1273496.1273577 · DOI
10.5555/645313.649410 · DOI
Ordonez Vicente, 2011, Berg
Ranzato Marc’Aurelio, 2016, International Conference on Learning Representations (ICLR’16)
Zhou Ren Hailin Jin Zhe Lin Chen Fang and Alan Yuille. 2015. Multi-instance visual-semantic embedding. arXiv preprint arXiv:1512.06963. Zhou Ren Hailin Jin Zhe Lin Chen Fang and Alan Yuille. 2015. Multi-instance visual-semantic embedding. arXiv preprint arXiv:1512.06963. · arXiv
Vadivel Karthikeyan Shanmuga, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3241--3250
Simonyan Karen, 2015, International Conference on Learning Representations (ICLR’15)
Yusuke Sugano and Andreas Bulling. 2016. Seeing with humans: Gaze-assisted neural image captioning. arXiv preprint arXiv:1608.05203. Yusuke Sugano and Andreas Bulling. 2016. Seeing with humans: Gaze-assisted neural image captioning. arXiv preprint arXiv:1608.05203. · arXiv
Sutskever Ilya, 2014, Le
van Deemter Kees, Proceedings of the 4th International Natural Language Generation Conference. Association for Computational Linguistics, 130--132
Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998--6008. Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998--6008.
10.5555/1708322.1708334 · DOI
Wang Qingzhong, 2018, Chan
Wang Yufei, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17)
Yang Zhilin, 30th Conference on Neural Image Processing System (NIPS’16)
Xu Kelvin, 2015, International Conference on Machine Learning. 2048--2057
Yang Linjie, 2016, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16)
Yu Lantao, 2017, Proceedings of the 31st AAAI Conference on Artificial Intelligence.
Matthew, 2014, Zeiler and Rob Fergus
Zhang Li, 2017, Hospedales

Sorted by citations

A Survey of the Usages of Deep Learning for Natural Language Processing
Otter, D. W., Medina, J. R., & Kalita, J. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2979670 · Cited by 1,836
“The use of deep learning for image captioning has been surveyed very recently [230], [231], and tasks that generate text given textual inputs are discussed in Sections IV-E–IV-G.”
Background
Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data
Bender, E. M., & Koller, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.463 · Cited by 1,082
“One approach to providing grounding is to train distributional models on corpora augmented with perceptual data, such as photos (Hossain et al., 2019) or other modalities (Kiela and Clark, 2015; Kiela et al., 2015).”
Background
Convolutional neural networks in medical image understanding: a survey
Sarvamangala, D. R., & Kulkarni, R. V. (2021) · Evolutionary Intelligence · doi:10.1007/s12065-020-00540-3 · Cited by 1,043
“Researchers are also working on image captioning (textual representations of the image) [29].”
Background
Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things
Zhang, J., & Tao, D. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3039359 · Cited by 774
“video [179] while text-to-image generation aims to generate a realistic image that matches the given text description [180].”
Background
Learning Dynamics and Heterogeneity of Spatial-Temporal Graph Data for Traffic Forecasting
Guo, S., Lin, Y., Wan, H., Li, X., & Cong, G. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3056502 · Cited by 800
“It has been widely used and achieved great successes in various tasks such as natural language processing (NLP) [39], [40], image caption [41] and speech recognition [42].”
Background

See all 879

Authors: Hossain, M. Z., Sohel, F., Shiratuddin, M. F., & Laga, H.
Affiliations: Murdoch University (AU)
ORCID: Hossain, MD. Zakir
Sohel, Ferdous
Shiratuddin, Mohd Fairuz
Laga, Hamid
Venue: ACM Computing Surveys
Publisher: Association for Computing Machinery (ACM)
Published: 2019-02-04
Volume: 51
Issue: 6
Pages: 1-36
ISSN: 0360-0300 (ISSN-L)
1557-7341
DOI: 10.1145/3295748
OpenAlex: W2896348597
Cited by: 879
References: 166
Citations by year: 2026: 55 · 2025: 113 · 2024: 124 · 2023: 167 · 2022: 153 · 2021: 127 · 2020: 104 · 2019: 36
FWCI: 47.02
Type: journal article
Language: en
Open access: closed
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: ACM Computing Surveys (publishedVersion)
Murdoch University Research Portal (submittedVersion)
Funders: Australian Research Council (DE120102960)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords: image captioning, object recognition, sentence generation, evaluation metrics, deep learning architectures

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A Comprehensive Survey of Deep Learning for Image Captioning

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks