The DOI matches its Crossref record; 167 of the 167 references it cites were found in a registry or an index.
A Comprehensive Survey of Deep Learning for Image Captioning
Abstract
Generating a description of an image is called image captioning. Image captioning requires recognizing the important objects, their attributes, and their relationships in an image. It also needs to generate syntactically and semantically correct sentences. Deep-learning-based techniques are capable of handling the complexities and challenges of image captioning. In this survey article, we aim to present a comprehensive review of existing deep-learning-based image captioning techniques. We discuss the foundation of the techniques to analyze their performances, strengths, and limitations. We also discuss the datasets and the evaluation metrics popularly used in deep-learning-based automatic image captioning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Most prior works have relied on a set of metrics inspired by similar setups in natural language generation, where radiology report text is treated as generic text.22”
From Evaluating progress in automatic chest X-ray radiology report generation (2023) · cites this paper as [22]
In the citing paper
“If we take a broader view, many works are using cross-modal deep generative models (such as GAN) in data synthesis, such as from video to audio [267], from text to image and vice versa [268,269].”
From Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances (2022) · cites this paper as [268]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“The architecture of this method has ResNet [53] and two LSTMs called Skel-LSTM and Attr-LSTM.”
MethodVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 2×
“Moreover, recent object detection and classification methods [73, 127] show that deep, hierarchical methods are better at learning than shallower ones.”
BackgroundMethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 5×
“Kiros et al. [69] proposed a method that is built on a Log-Bilinear model and used AlexNet to extract visual features.”
BackgroundMethodInfluentialHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 3×
“LSTM [54] networks are a type of RNN that have special units in addition to standard units.”
BackgroundMethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 4×
“The bottom up attention mechanism uses Faster R-CNN [116] for region proposals that can select salient regions of an image .”
MethodInfluentialFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“For example, Convolutional Neural Networks (CNN) [79] are widely used for feature learning, and a classifier such as Softmax is used for classification.”
MethodDistinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 1×
“d techniques and (2) Deep machine learning based techniques. In traditional machine learning, hand crafted features such as Local Binary Patterns (LBP) [107], Scale-Invariant Feature Transform (SIFT) [87], the Histogram of Oriented Gradients (HOG) [27], and a combination of such features are widely used. In these tech…”
BackgroundGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Flagged first, then by citations
Research works printed but not linked: 58
- 10.5555/1626394.1626406 · DOI
- Aker Ahmet, 2010, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1250--1258
- Bahdanau Dzmitry, 2015, International Conference on Learning Representations (ICLR’15)
- Cristian Bodnar. 2018. Text to image synthesis using generative adversarial networks. arXiv preprint arXiv:1805.00676. Cristian Bodnar. 2018. Text to image synthesis using generative adversarial networks. arXiv preprint arXiv:1805.00676. · arXiv
- Chen Xinlei, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2422--2431
- Cho Kyunghyun, 2014, Dzmitry Bahdanau, and Yoshua Bengio.
- Denoual Etienne, Companion
- Fang Hao, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1473--1482
- 10.5555/1888089.1888092 · DOI
- Fedus William, 2018, Dai
- FitzGerald Nicholas, 2013, Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 1914--1925
- Frome Andrea, 2013, Devise: A deep visual-semantic embedding model. In Advances in Neural Information Processing Systems. 2121--2129.
- Gehring Jonas, 2016, Dauphin
- Gehring Jonas, 2017, Dauphin
- 10.5555/1870658.1870698 · DOI
- Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680. Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680.
- Gregor Karol, 2015, Proceedings of Machine Learning Research. 1462--1471
- Grubinger Michael, 2006, International Workshop Ontoimage, 5
- Isola Phillip, Proceedings of the IEEE International Conference on Computer Vision (CVPR’17)
- Max Jaderberg Karen Simonyan Andrew Zisserman and Koray Kavukcuoglu. 2015. Spatial transformer networks. In Advances in Neural Information Processing Systems. 2017--2025. Max Jaderberg Karen Simonyan Andrew Zisserman and Koray Kavukcuoglu. 2015. Spatial transformer networks. In Advances in Neural Information Processing Systems. 2017--2025.
- Jang Eric, 2017, International Conference on Learning Representations (ICLR’17)
- Kazemzadeh Sahar, 2014, Berg
- Kiros Ryan, 2014, Proceedings of the 31st International Conference on Machine Learning (ICML’14)
- Kiros Ryan, Workshop on Neural Information Processing Systems (NIPS’14)
- Konda Vijay R., 2000, Tsitsiklis
- Krizhevsky Alex, 2012, Hinton
- 10.5555/2390524.2390575 · DOI
- Kuznetsova Polina, 2014, TREETALK: Composition and compression of trees for image descriptions.TACL 2, 10
- Lebret RÃl’mi, 2015, Workshop on International Conference on Learning Representations (ICLR’15)
- Li Siming, 2011, Proceedings of the 15th Conference on Computational Natural Language Learning. Association for Computational Linguistics, 220--228
- Lin Chin-Yew, 2004, Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, 8
- Liu Chenxi, 2017, Yuille
- Maddison Chris J., 2017, International Conference on Learning Representations (ICLR’17)
- Mao Junhua, 2015, International Conference on Learning Representations (ICLR’15)
- Mao Junhua, 2014, Yuille
- Mitchell Margaret, 2012, Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 747--756
- 10.5555/1873738.1873752 · DOI
- Mitchell Margaret, 2013, Kees Van Deemter, and Ehud Reiter
- 10.1145/1273496.1273577 · DOI
- 10.5555/645313.649410 · DOI
- Ordonez Vicente, 2011, Berg
- Ranzato Marc’Aurelio, 2016, International Conference on Learning Representations (ICLR’16)
- Zhou Ren Hailin Jin Zhe Lin Chen Fang and Alan Yuille. 2015. Multi-instance visual-semantic embedding. arXiv preprint arXiv:1512.06963. Zhou Ren Hailin Jin Zhe Lin Chen Fang and Alan Yuille. 2015. Multi-instance visual-semantic embedding. arXiv preprint arXiv:1512.06963. · arXiv
- Vadivel Karthikeyan Shanmuga, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3241--3250
- Simonyan Karen, 2015, International Conference on Learning Representations (ICLR’15)
- Yusuke Sugano and Andreas Bulling. 2016. Seeing with humans: Gaze-assisted neural image captioning. arXiv preprint arXiv:1608.05203. Yusuke Sugano and Andreas Bulling. 2016. Seeing with humans: Gaze-assisted neural image captioning. arXiv preprint arXiv:1608.05203. · arXiv
- Sutskever Ilya, 2014, Le
- van Deemter Kees, Proceedings of the 4th International Natural Language Generation Conference. Association for Computational Linguistics, 130--132
- Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998--6008. Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998--6008.
- 10.5555/1708322.1708334 · DOI
- Wang Qingzhong, 2018, Chan
- Wang Yufei, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17)
- Yang Zhilin, 30th Conference on Neural Image Processing System (NIPS’16)
- Xu Kelvin, 2015, International Conference on Machine Learning. 2048--2057
- Yang Linjie, 2016, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16)
- Yu Lantao, 2017, Proceedings of the 31st AAAI Conference on Artificial Intelligence.
- Matthew, 2014, Zeiler and Rob Fergus
- Zhang Li, 2017, Hospedales
Cited by
Sorted by citations
Items 401–500 of 878
MicroNN: An On-device Disk-resident Updatable Vector Database
Pound, J., Chabert, F., Bhushan, A., Goswami, A., Pacaci, A., & Chowdhury, S. R. (2025) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3722212.3724444 · Cited by 5
“Advances in modern machine learning models are enabling rich representation of text, images, and other assets as dense vectors that capture semantically meaningful information [16, 25].”
Berger, U., Stanovsky, G., Abend, O., & Frermann, L. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.52 · Cited by 13
Facklam, D., Sweeney, S. J., Majumdar, S., & Dmello Kamath, R. N. (2026) · The Electronic Library · doi:10.1108/el-06-2025-0270 · Cited by 5
In Search of a Suitable Epoch Value for the Optimum Result in Image Captioning
Dinda, S., Pal, A. R., & Pal, A. (2023) · International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT) · doi:10.1109/icaect57570.2023.10117839 · Cited by 5
“[1] carried out a vivid survey in this domain.”
BackgroundA Comprehensive Analysis of Object Detectors in Adverse Weather Conditions
Patel, V. S., Agrawal, K., & Nguyen, T. (2024) · Conference on Information Sciences and Systems · doi:10.1109/ciss59072.2024.10480197 · Cited by 5
Anzum, F., Asha, A. Z., Dey, L., Gavrilov, A., Iffath, F., Ohi, A. Q., Pond, L., Shopon, M., & Gavrilova, M. L. (2024) · Advances in information security, privacy, and ethics book series · doi:10.4018/978-1-6684-8127-1.ch001 · Cited by 5
Jha, P., Dias, G. H., Lechervy, A., Moreno, J. G., Jangra, A., Pais, S., & Saha, S. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548299 · Cited by 6
“Interestingly, some recent studies have emerged that tackle vision-grounded natural language representations [9, 28, 33, 34, 45] and applications [2, 21, 31, 35, 51, 56].”
BackgroundGuiding Attention using Partial-Order Relationships for Image Captioning
Popattia, M., Rafi, M., Qureshi, R., & Nawaz, S. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00513 · Cited by 5
“Generating descriptive and meaningful captions for images, and to capture its semantic meaning, is one such multimodal inference problem [3,10].”
BackgroundComparison of various CNN encoders for image captioning
Veena, S., Ashwin, K. S., & Gupta, P. (2022) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2335/1/012029 · Cited by 5
Image Caption Method Based on Graph Attention Network with Global Context
Sui, J., Yu, H., Liang, X., & Ping, P. (2022) · International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc55077.2022.9886239 · Cited by 5
“In the first stage, a series of convolution neural network [4-7] were proposed to represent visual information with grid features, and these grid features have achieved excellent performance on visual tasks like image classification and multi-modal tasks like image captioning, as shown in Fig.”
A Framework for Image Captioning Based on Relation Network and Multilevel Attention Mechanism
Sharma, H., & Srivastava, S. (2022) · Neural Processing Letters · doi:10.1007/s11063-022-11106-y · Cited by 5
“To give a fair comparison, we have conducted the experiments on similar hyperparameter settings as done by most of the state-of-the-art methods [66].”
MethodRIDA: Cross-App Record and Replay for Android
Liang, J., Wang, S., Deng, X., & Liu, Y. (2023) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst57152.2023.00031 · Cited by 5
“Different image captioning techniques vary in the CNN and RNN structures they applied [29].”
MethodA data science and machine learning approach to continuous analysis of Shakespeare's plays
Swisher, C., & Shamir, L. (2023) · Journal of Data Mining & Digital Humanities · doi:10.46298/jdmdh.10829 · Cited by 5
“…text generation (Gatt and Krahmer, 2018; de Rosa and Papa, 2021), text summarization (Gambhir and Gupta, 2017; El-Kassas et al., 2021), sentiment analysis (Zhang et al., 2018; Yadav and Vishwakarma, 2020), automatic caption generation (Bai and An, 2018; Hossain et al., 2019; Predić et al., 2022).”
BackgroundHybrid Graph Models for Traffic Prediction
Chen, R., & Yao, H. (2023) · Applied Sciences · doi:10.3390/app13158673 · Cited by 5
Artificial intelligence-based spatio-temporal vision sensors: applications and prospects
Li, Z., & Sun, H. (2023) · Frontiers in Materials · doi:10.3389/fmats.2023.1269992 · Cited by 6
Normal-Abnormal Decoupling Memory for Medical Report Generation
Zhao, G., Yan, Y., & Zhao, Z. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.131 · Cited by 5
“Early approaches utilized templates or retrieval-based methods for caption generation (Hossain et al., 2019).”
Rigoll, P., Petersen, P., Stage, H., Ries, L., & Sax, E. (2023) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc57777.2023.10422271 · Cited by 5
“There are methods to caption images automatically [15].”
MethodMeaning Guided Video Captioning
Babaria, R. J., & Tamaki, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-41299-9_37 · Cited by 5
English to Hindi Multi Modal Image Caption Translation
Kaur, J., & Josan, G. S. (2020) · Journal of scientific research · doi:10.37398/jsr.2020.640238 · Cited by 5
“Hossain et al. (2019) describes nicely the various techniques for generation of image captions.”
MethodExplorations into Deep Learning Text Architectures for Dense Image Captioning
Toshevska, M., Stojanovska, F., Zdravevski, E., Lameski, P., & Gievska, S. (2020) · Annals of Computer Science and Information Systems · doi:10.15439/2020f57 · Cited by 5
“For most applications, the image captioning system must give an accurate description of the scene [4].”
BackgroundGenerating Attribution Maps with Disentangled Masked Backpropagation
Ruiz, A., Agudo, A., & Moreno-Noguer, F. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00094 · Cited by 5
“Despite their outstanding performance in multiple tasks [10, 14, 40], they are still characterized as black-boxes whose internal inference rules are difficult to interpret.”
BackgroundAutomatic Caption Generation via Attention Based Deep Neural Network Model
Tiwari, V., & Bhatnagar, C. (2021) · International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO) · doi:10.1109/icrito51393.2021.9596255 · Cited by 5
Smartphone-based Multi-parametric Glucose Prediction using Recurrent Neural Networks
PALAZ, Z., Doğan, V., & Kılıç, V. (2022) · European Journal of Science and Technology · doi:10.31590/ejosat.1041547 · Cited by 5
Bangla Image Captioning With Bidirectional GRU & Attention Mechanism
Zohora, F. T., & Abedin, Z. (2022) · International Conference on Innovations in Science, Engineering and Technology (ICISET) · doi:10.1109/iciset54810.2022.9775869 · Cited by 5
Clinical Comparable Corpus Describing the Same Subjects with Different Expressions
Nakamura, Y., Hanaoka, S., Nomura, Y., Hayashi, N., Abe, O., Yada, S., Wakamiya, S., & Aramaki, E. (2022) · Studies in health technology and informatics · doi:10.3233/shti220073 · Cited by 5
CAPTION: Caption Analysis with Proposed Terms, Image of Objects, and Natural Language Processing
Ferreira, L. A., Meneghetti, D. D. R., Lopes, M., & Santos, P. E. (2022) · SN Computer Science · doi:10.1007/s42979-022-01322-7 · Cited by 5
“Automatic Image Captioning has been the subject of a number of recently published survey papers [11–14].”
BackgroundImage Caption Generator in Hindi Using Attention
Sethi, A. R., Jain, A., & Dhiman, C. (2022) · Advances in transdisciplinary engineering · doi:10.3233/atde220727 · Cited by 5
The Role of Transformer-based Image Captioning for Indoor Environment Visual Understanding
Fudholi, D. H., & Nayoan, R. A. N. (2022) · International Journal of Computing and Digital Systems · doi:10.12785/ijcds/120138 · Cited by 7
“Deep learning is known for its ability to handle a large and diverse set of images or videos [21].”
BackgroundHuang, P.-C., Shakya, E., Song, M., & Subramaniam, M. (2022) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm55620.2022.9994867 · Cited by 5
“However, none of the current methods use image captions and contents together for biofilm document classification [16].”
MethodTowards Captioning an Image Collection from a Combined Scene Graph Representation Approach
Phueaksri, I., Kastner, M. A., Kawanishi, Y., Komamizu, T., & Ide, I. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-27077-2_14 · Cited by 6
“Image captioning [12] is an image-to-text translation task that aims to describe the scene, location, objects, and interactions in an image in the form of a sentence.”
BackgroundAsynchronous evolution of deep neural network architectures
Liang, J., Shahrzad, H., & Miikkulainen, R. P. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.111209 · Cited by 5
DeepCap: A Deep Learning Model to Caption Black and White Images
Pandit, V. R., Gulati, R., Singla, C., & Singh, S. K. (2020) · International Conference on Cloud Computing, Data Science & Engineering (Confluence) · doi:10.1109/confluence47617.2020.9058164 · Cited by 5
Reading Between the Lines: Exploring Infilling in Visual Narratives
Chandu, K. R., Dong, R.-P., & Black, A. W. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.93 · Cited by 5
Towards an AI assistant for human grid operators
Marot, A., Rozier, A., Dussartre, M., Crochepierre, L., & Donnot, B. (2022) · HAL · Cited by 5
Facial Expression Recognition and Image Description Generation in Vietnamese
Lâm, K. N., Nguyen, K.-N. T., Nguy, L. H., & Kalita, J. K. (2021) · Frontiers in artificial intelligence and applications · doi:10.3233/faia210176 · Cited by 5
“Publishing efforts on IDG may be grouped into 3 categories [16,17]: (i) Models rely on computer vision techniques to identify objects in the input image and extract their features [18,19].”
BackgroundShow and Speak: Directly Synthesize Spoken Description of Images
Wang, X., Feng, S., Zhu, J., Hasegawa‐Johnson, M., & Scharenborg, O. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414021 · Cited by 6
“Recent research in this direction is called image captioning [1], which aims to automatically generate textual descriptions of images.”
BackgroundAn Image Captioning Model Based on Bidirectional Depth Residuals and its Application
Zhou, Z., Xu, L., Wang, C., Xie, W., Wang, S., Ge, S., & Zhang, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3057091 · Cited by 6
“by a convolutional neural network (CNN) and recurrent neural network (RNN) [1]–[3].”
BackgroundHùng, B. T., & Huy, V. Q. (2025) · The Visual Computer · doi:10.1007/s00371-025-04072-8 · Cited by 4
“Numerous studies have explored this task, including [6–9].”
Joshi, A., Alkhayyat, A. H., Gunwant, H., Tripathi, A., & Sharma, M. (2024) · AIP conference proceedings · doi:10.1063/5.0184395 · Cited by 4
Deep Learning as a New Framework for Passive Vehicle Safety Design Using Finite Elements Models Data
Navarro, M. L., Jehle, J. S., Apellániz, P. A., Parras, J., Zazo, S., & Gerdts, M. (2024) · Applied Sciences · doi:10.3390/app14209296 · Cited by 5
Yuan, B., Zhao, D., Liu, Z., Li, W., & Li, T. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680654 · Cited by 4
“Image captioning [20] describes an image with words or sentences to meet human-like semantic understanding.”
BackgroundProgress-Aware Video Frame Captioning
Xue, Z., An, J., Yang, X., & Grauman, K. L. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01273 · Cited by 4
“Image captioning [23] generates a single, isolated description for each image, with no contextual linkage among different images.”
Leveraging Linguistically-aware Object Relations and NASNet for Image Captioning
Sharif, N., Jalwana, M. A. A. K., Bennamoun, M., Liu, W., & Shah, S. A. A. (2020) · International Conference on Image and Vision Computing New Zealand (IVCNZ) · doi:10.1109/ivcnz51579.2020.9290719 · Cited by 4
“Existing works in literature either extract visual features from the penultimate layer of a pretrained CNN or a lower convolutional layer [28].”
BackgroundMethodFactors Involved in Artificial Intelligence-based Automated HTML Code Generation Tool
Agrawal, S., Suryawanshi, S., Arsude, V., Maid, N., & Kawarkhe, M. (2020) · International Conference on Smart Innovations in Design, Environment, Management, Planning and Computing (ICSIDEMPC) · doi:10.1109/icsidempc49020.2020.9299609 · Cited by 4
Theft Detection and Monitoring System Using Machine Learning
Arora, J., Bangroo, A., & Garg, S. (2021) · Lecture notes in electrical engineering · doi:10.1007/978-981-16-1342-5_76 · Cited by 4
Santosh, K., Das, N., & Ghosh, S. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-823504-1.00011-8 · Cited by 7
Interdisciplinary perspectives on privacy awareness in lifelogging technology development
Wilkowska, W., Offermann, J., Colonna, L., Flórez‐Revuelta, F., Climent-Pérez, P., Mihailidis, A., Poli, A., Spinsante, S., & Ziefle, M. (2022) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-022-04486-5 · Cited by 6
Artificial Intelligences-Based Approaches for Generating Image Caption
Ingale, S. P., & Bamnote, G. R. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-4687-5_42 · Cited by 5
Accident Prevention System using Machine Learning
Thokala, S., Jakkani, R., Alli, M., & Hariharan, S. (2023) · International Conference on Computer Communication and Informatics (ICCCI) · doi:10.1109/iccci56745.2023.10128202 · Cited by 4
Foundation Models for Speech, Images, Videos, and Control
Paaß, G., & Giesselbach, S. (2023) · Artificial intelligence: foundations, theory, and algorithms/Artificial intelligence: Foundations, theory, and algorithms · doi:10.1007/978-3-031-23190-2_7 · Cited by 6
Smart Assistant for Visually Impaired People using Deep Learning Algorithms
Bipin, P. B., & Abirami, S. (2023) · International Conference on Sustainable Computing and Smart Systems (ICSCSS) · doi:10.1109/icscss57650.2023.10169851 · Cited by 4
HDFL: A Heterogeneity and Client Dropout-Aware Federated Learning Framework
Zawad, S., Anwar, A., Zhou, Y., Baracaldo, N., & Yan, F. (2023) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid57682.2023.00037 · Cited by 4
“Such data can be potentially used to train state-of-the-art machine learning models to address a wide variety of ML tasks such as keyword spotting [1], image captioning [2], translation [3] and medical applications [4].”
BackgroundInvestigating CLIP Performance for Meta-data Generation in AD Datasets
Gannamaneni, S. S., Sadaghiani, A., Rao, R. P., Möck, M., & Akila, M. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00398 · Cited by 4
“It is an active field of research at the convergence of language description and image understanding, and several survey papers [28, 36, 53] have attempted to provide some structure to the large quantity of work.”
BackgroundUser-Aware Prefix-Tuning Is a Good Learner for Personalized Image Captioning
Wang, X., Wang, G., Chai, W., Zhou, J., & Wang, G. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8540-1_31 · Cited by 4
“Owing to the personalities of the users, similar images may exhibit varying descriptions. applications, such as e-commerce product description, assisting individuals with visual impairments, and automating captioning for social media posts [4,24].”
A Large Model Assisted Remote Sensing Image Scene Understanding Algorithm Based on Object Detection
Wang, Z., Xu, Z., Yang, W., Chen, W., & Yang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5597-4_5 · Cited by 4
Omni-Q: Omni-Directional Scene Understanding for Unsupervised Visual Grounding
Wang, S., Lin, Y., & Wu, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01352 · Cited by 5
“Its purpose is to locate relevant objects in an image based on a language query and serves as a fundamental component for various multi-modal tasks, such as image captioning [10, 41] and visual question answering [2, 27, 47].”
BackgroundPrimer on Generative Artificial Intelligence and Large Language Models in Medical Imaging
Kim, K., Hong, G.-S., & Kim, N. (2024) · Journal of the Korean Society of Radiology · doi:10.3348/jksr.2024.0066 · Cited by 4
“Generation tasks include visual question answering (15), visual reasoning (16), visual captioning (17), and visual generation (18), including image recognition (19, 20), visual grounding (21), and image retrieval (22).”
BackgroundReferring flexible image restoration
Guan, R., Hu, R., Zhou, Z., Xue, T., Man, K. L., Smith, J. S., Lim, E. G., Ding, W., & Yue, Y. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126857 · Cited by 6
Robust image classification with multi-modal large language models
Villani, F., Maljkovic, I., Lazzaro, D., Sotgiu, A., Ciná, A. E., & Roli, F. (2025) · Pattern Recognition Letters · doi:10.1016/j.patrec.2025.04.022 · Cited by 6
The Phantom Menace: Unmasking Privacy Leakages in Vision-Language Models
Caldarella, S., Mancini, M., Ricci, E., & Aljundi, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-92648-8_26 · Cited by 18
AI4AR: An AI-Based Mobile Application for the Automatic Generation of AR Contents
Pierdicca, R., Paolanti, M., Frontoni, E., & Baraldi, L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58465-8_21 · Cited by 4
WEmbSim: A Simple yet Effective Metric for Image Captioning
Sharif, N., White, L., Bennamoun, M., Liu, W., & Shah, S. A. A. (2020) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta51227.2020.9363392 · Cited by 4
“captioning models [1] are based on Recurrent Neural Network (RNN) [36] language models for text generation.”
BackgroundMethodA Scientometric Visualization Analysis of Image Captioning Research From 2010 to 2020
Liu, W., Wu, H., Hu, K., Luo, Q., & Cheng, X. (2021) · IEEE Access · doi:10.1109/access.2021.3129782 · Cited by 4
“To date, researchers have presented a number of review papers [17-21] to summarize the development of image description techniques.”
BackgroundTechnology, Science and Culture - A Global Vision, Volume II
Picazo-Vela, S., & Hernández, L. R. (2020) · IntechOpen eBooks · doi:10.5772/intechopen.90099 · Cited by 4
A core region captioning framework for automatic video understanding in story video contents
Suh, H., Kim, J., So, J., & Jung, J.-J. (2022) · International Journal of Engineering Business Management · doi:10.1177/18479790221078130 · Cited by 4
“To 12 – 18 This dense captioning is based on the Visual Genome 19 region caption data set based on MS COCO and the YFCC100M image data set.”
MethodCaption Generation Based on Emotions Using CSPDenseNet and BiLSTM with Self-Attention
Priya, S. K., Pon Karthika, K., Kaliappan, J., Selvaraj, S. K., Nagalakshmi, R., & Molla, B. (2022) · Applied Computational Intelligence and Soft Computing · doi:10.1155/2022/2756396 · Cited by 4
“Recently developed deep learning-based caption generation techniques [7–9] considers only the factual and semantic content of the image.”
Backgroundvani, M. S., & Priya, S. (2022) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2022.47348 · Cited by 4
Afriyie, Y., Weyori, B. A., & Opoku, A. A. (2023) · Journal of Experimental & Theoretical Artificial Intelligence · doi:10.1080/0952813x.2023.2165721 · Cited by 4
Singh, A., Shah, A., Kumar, P., Chaudhary, H., Sharma, A., Chaudhary, A. S., & Dixit, A. (2023) · International Conference on Power, Instrumentation, Energy and Control (PIECON) · doi:10.1109/piecon56912.2023.10085724 · Cited by 4
CapEnrich: Enriching Caption Semantics for Web Images via Cross-modal Pre-trained Knowledge
Yao, L., Chen, W., & Jin, Q. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583232 · Cited by 9
“Indexing and labeling them with textual descriptions can greatly benefit realistic applications, such as Content-based Image Retrieval and Recommendation [11].”
BackgroundAsymmetrical Attention Networks Fused Autoencoder for Debiased Recommendation
Zhang, Y., Zhao, C., Liao, W., Zhou, W. G., & Yuan, M. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3596498 · Cited by 4
“The past few years have witnessed the success of deep learning technologies in fields such as computer vision and natural language processing [14, 16, 33, 51, 56].”
BackgroundMulyawan, R., Sunyoto, A., & Muhammad, A. H. (2023) · JOIV International Journal on Informatics Visualization · doi:10.30630/joiv.7.2.1387 · Cited by 4
“The task of photo caption generation has already had a significant impact in several fields, such as image search also various disciplines, such as software development for people with disabilities, video surveillance and security, and the interface between humans and computers [2].”
BackgroundShanthi, N., Gowthami, J., Aravindhraj, N., Jagadeesh, S., Karthick, R. V., & Bharanidharan, M. (2023) · International Conference on Sustainable Communication Networks and Application (ICSCNA) · doi:10.1109/icscna58489.2023.10370414 · Cited by 4
“The author talks about the idea of image captioning, what it entails, and how deep learning fits into this topic[12].”
BackgroundBharne, S., & Bhaladhare, P. (2024) · International Conference on Recent Advances in Science and Engineering (RAiSE) · doi:10.3390/engproc2023059182 · Cited by 4
Imagery contents descriptions for People with visual impairments
Jandrey, A. H., Ruiz, D. D., & Silveira, M. S. (2023) · Brazilian Symposium on Human Factors in Computing Systems (IHC) · doi:10.1145/3638067.3638105 · Cited by 4
“Its applications span diverse fields [24, 32] including assistive technologies [35, 49].”
BackgroundTowards Generating Diverse Audio Captions via Adversarial Training
Mei, X., Liu, X., Sun, J., Plumbley, M. D., & Wang, W. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3416686 · Cited by 12
“Similar to other types of multi-media captioning systems [17]–[19], we argue that an audio captioning system is expected to possess three properties: (1) fidelity-generated captions should be semantically faithful to the content of the described audio clip; (2) naturalness-the style of the generated captions should be…”
BackgroundParallel Image Captioning Using 2D Masked Convolution
Poleak, C., & Kwon, J.-W. (2019) · Applied Sciences · doi:10.3390/app9091871 · Cited by 4
“As challenging as it seems, image captioning also holds great interest in the research community because it has many benefits, ranging from assisting the visually impaired in understanding image content to improving the image indexing and annotation tasks [1].”
BackgroundComponent based comparative analysis of each module in image captioning
Choi, S.-H., Jo, S. Y., & Jung, S. H. (2020) · ICT Express · doi:10.1016/j.icte.2020.08.004 · Cited by 5
Evaluating Gender-Neutral Training Data for Automated Image Captioning
Amend, J. J., Wazzan, A., & Souvenir, R. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671774 · Cited by 4
Learning Spatial Relations with a Standard Convolutional Neural Network
Swingler, K. M., & Bath, M. (2020) · International Joint Conference on Computational Intelligence · doi:10.5220/0010170204640470 · Cited by 4
“(Hossain et al., 2019) provide a useful review of image captioning techniques.”
BackgroundCapGen: A Neural Image Caption Generator with Speech Synthesis
Kumar, A., & Verma, S. (2021) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-15-8335-3_46 · Cited by 4
Adversarial Image Caption Generator Network
Dehaghi, A. M., Seydi, V., & Madadi, Y. (2021) · SN Computer Science · doi:10.1007/s42979-021-00486-y · Cited by 4
A Comprehensive Review on Recent Methods and Challenges of Video DescriptionPreprint
Singh, A., Singh, T. D., & Bandyopadhyay, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.14752 · Cited by 4
“The descriptions generated using template-based approaches are grammatically correct, but they lack in generating variable-length description [29].”
MethodEnsemble Models for Neural Source Code Summarization of Subroutines
LeClair, A., Aakash, B., & McMillan, C. (2021) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.26226/morressier.613b5418842293c031b5b62e · Cited by 27
Pixels to Prose: Understanding the Art of Image CaptioningPreprint
Sharma, A., Singh, H., & Pant, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5351410 · Cited by 4
‘We can see a savage’: a case study of the colonial gaze in generative AI algorithms
Alenichev, A., Shaffer, J. D., Kingori, P., Grietens, K. P., Muldoon, J., & Rocher, L. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02685-0 · Cited by 3
“It generates a string of words that adequately describes objects, locations, contexts, and relationships in an image (Herdade et al. 2019; Hossain et al. 2019) as well as the broader context.”
Pang, T., Wong, J. H. D., Ng, W. L., Chan, C. S., Wang, C., Zhou, X., & Yu, Y. (2024) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ad2a95 · Cited by 3
Sharma, A., Singh, H., & Pant, M. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.132385 · Cited by 11
“A language encoder, a vision module, a multi-modal space component, and a language decoder are the main architectural elements of a typical multi-modal space based system[35].”
Frequency-adaptive prototype learning for long-tail scene graph generation
Xie, F., Wu, Y., & Liu, F. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-00546-4 · Cited by 3
Performance Evaluation of CNN-based Encoders for Image Captioning
Hoang, A. C., Nguyen, D. C., & Nguyen, H. L. (2023) · International Conference on Control, Automation and Information Sciences (ICCAIS) · doi:10.1109/iccais59597.2023.10382370 · Cited by 3
“Research directions and accomplishments in this domain can be explored within research documents [5], [9] with numerous real-world applications today, such as smart transportation, assistance for the visually impaired, and large-scale data retrieval.”
BackgroundSingh, P. K., Sachan, K., Khandelwal, V., Singh, S., & Singh, S. (2024) · Recent Patents on Biotechnology · doi:10.2174/0118722083297406240313090140 · Cited by 3
Hakimov, S., Cheema, G. S., & Ewerth, R. (2024) · Springer eBooks · doi:10.1007/978-3-031-64451-1_4 · Cited by 3
Prakash, J., V, Vijay, S. A. A., Murugan, M., Gopikrishnan, S., Rivera, M. E., & Wheeler, P. W. (2025) · Applied Sciences · doi:10.3390/app15094909 · Cited by 3
Voice-based Image Captioning using Florence-2
Mittal, D., Ahlawat, P., Nag, T., & Shrimaroj, O. B. (2024) · International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES) · doi:10.1109/scopes64467.2024.10990818 · Cited by 3
Balachandran, A., & Masum, M. (2025) · International Journal of Information Management Data Insights · doi:10.1016/j.jjimei.2025.100355 · Cited by 3
Ahmed, B. H. A., & Saad, M. (2021) · Palestinian International Conference on Information and Communication Technology (PICICT) · doi:10.1109/picict53635.2021.00017 · Cited by 4
The New Dataset MITWPU-1K for Object Recognition and Image Captioning Tasks
Bhalekar, M. A., & Bedekar, M. V. (2022) · Engineering Technology & Applied Science Research · doi:10.48084/etasr.5039 · Cited by 3
“In [8], authors provide the categorization of existing image captioning systems and commonly used datasets for image captioning.”
BackgroundNatural Language Processing with Optimal Deep Learning-Enabled Intelligent Image Captioning System
Marzouk, R., Alabdulkreem, E. A., Nour, M. K., Duhayyim, M. A., Othman, M., Zamani, A. S., Yaseen, I., & Motwakel, A. (2022) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2023.033091 · Cited by 3
Semantic-embedding Guided Graph Network for cross-modal retrieval
Yuan, M., Zhang, H., Liu, D., Wang, L., & Liu, L. (2023) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2023.103807 · Cited by 3
Triple-level relationship enhanced transformer for image captioning
Zheng, A., Zheng, S., Bai, C., & Chen, D. (2023) · Multimedia Systems · doi:10.1007/s00530-023-01073-2 · Cited by 4
“With reference to machine translation tasks, these methods follow the “Encoder-Decoder” architecture [1] basically, in which CNN [2] and RNN [3] are widely used.”
Method
Metadata
- Authors
- Hossain, M. Z., Sohel, F., Shiratuddin, M. F., & Laga, H.
- Affiliations
- Murdoch University (AU)
- Venue
- ACM Computing Surveys
- Published
- 2019-02-04
- Volume
- 51
- Issue
- 6
- Pages
- 1-36
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W2896348597
- Cited by
- 879
- References
- 166
- Citations by year
- 2026: 55 · 2025: 113 · 2024: 124 · 2023: 167 · 2022: 153 · 2021: 127 · 2020: 104 · 2019: 36
- FWCI
- 47.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Computing Surveys (publishedVersion)
- Murdoch University Research Portal (submittedVersion)
- Funders
- Australian Research Council (DE120102960)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- image captioning, object recognition, sentence generation, evaluation metrics, deep learning architectures
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
