Cited by 879Multimodal Machine Learning ApplicationsImage Retrieval and Classification TechniquesGenerative Adversarial Networks and Image Synthesis

The DOI matches its Crossref record; 167 of the 167 references it cites were found in a registry or an index.

A Comprehensive Survey of Deep Learning for Image Captioning

Hossain, MD. Zakir, Sohel, Ferdous, Shiratuddin, Mohd Fairuz, Laga, Hamid·2019·ACM Computing Surveys

DOI 10.1145/3295748

Open this paper in CoChat — free account

Abstract

Generating a description of an image is called image captioning. Image captioning requires recognizing the important objects, their attributes, and their relationships in an image. It also needs to generate syntactically and semantically correct sentences. Deep-learning-based techniques are capable of handling the complexities and challenges of image captioning. In this survey article, we aim to present a comprehensive review of existing deep-learning-based image captioning techniques. We discuss the foundation of the techniques to analyze their performances, strengths, and limitations. We also discuss the datasets and the evaluation metrics popularly used in deep-learning-based automatic image captioning.

Integrity signals

Registry

Crossref · 10.1145/3295748

Cites

167 of 167 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“The architecture of this method has ResNet [53] and two LSTMs called Skel-LSTM and Attr-LSTM.”
Method
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 2×
“Moreover, recent object detection and classification methods [73, 127] show that deep, hierarchical methods are better at learning than shallower ones.”
BackgroundMethod
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 5×
“Kiros et al. [69] proposed a method that is built on a Log-Bilinear model and used AlexNet to extract visual features.”
BackgroundMethodInfluential
Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 3×
“LSTM [54] networks are a type of RNN that have special units in addition to standard units.”
BackgroundMethod
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 4×
“The bottom up attention mechanism uses Faster R-CNN [116] for region proposals that can select salient regions of an image .”
MethodInfluential
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 1×
“For example, Convolutional Neural Networks (CNN) [79] are widely used for feature learning, and a classifier such as Softmax is used for classification.”
Method
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589 · cited 1×
“d techniques and (2) Deep machine learning based techniques. In traditional machine learning, hand crafted features such as Local Binary Patterns (LBP) [107], Scale-Invariant Feature Transform (SIFT) [87], the Histogram of Oriented Gradients (HOG) [27], and a combination of such features are widely used. In these tech…”
Background
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460

Flagged first, then by citations

Show all 166 references

Research works printed but not linked: 58

10.5555/1626394.1626406 · DOI
Aker Ahmet, 2010, Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1250--1258
Bahdanau Dzmitry, 2015, International Conference on Learning Representations (ICLR’15)
Cristian Bodnar. 2018. Text to image synthesis using generative adversarial networks. arXiv preprint arXiv:1805.00676. Cristian Bodnar. 2018. Text to image synthesis using generative adversarial networks. arXiv preprint arXiv:1805.00676. · arXiv
Chen Xinlei, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2422--2431
Cho Kyunghyun, 2014, Dzmitry Bahdanau, and Yoshua Bengio.
Denoual Etienne, Companion
Fang Hao, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1473--1482
10.5555/1888089.1888092 · DOI
Fedus William, 2018, Dai
FitzGerald Nicholas, 2013, Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. 1914--1925
Frome Andrea, 2013, Devise: A deep visual-semantic embedding model. In Advances in Neural Information Processing Systems. 2121--2129.
Gehring Jonas, 2016, Dauphin
Gehring Jonas, 2017, Dauphin
10.5555/1870658.1870698 · DOI
Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680. Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680.
Gregor Karol, 2015, Proceedings of Machine Learning Research. 1462--1471
Grubinger Michael, 2006, International Workshop Ontoimage, 5
Isola Phillip, Proceedings of the IEEE International Conference on Computer Vision (CVPR’17)
Max Jaderberg Karen Simonyan Andrew Zisserman and Koray Kavukcuoglu. 2015. Spatial transformer networks. In Advances in Neural Information Processing Systems. 2017--2025. Max Jaderberg Karen Simonyan Andrew Zisserman and Koray Kavukcuoglu. 2015. Spatial transformer networks. In Advances in Neural Information Processing Systems. 2017--2025.
Jang Eric, 2017, International Conference on Learning Representations (ICLR’17)
Kazemzadeh Sahar, 2014, Berg
Kiros Ryan, 2014, Proceedings of the 31st International Conference on Machine Learning (ICML’14)
Kiros Ryan, Workshop on Neural Information Processing Systems (NIPS’14)
Konda Vijay R., 2000, Tsitsiklis
Krizhevsky Alex, 2012, Hinton
10.5555/2390524.2390575 · DOI
Kuznetsova Polina, 2014, TREETALK: Composition and compression of trees for image descriptions.TACL 2, 10
Lebret RÃl’mi, 2015, Workshop on International Conference on Learning Representations (ICLR’15)
Li Siming, 2011, Proceedings of the 15th Conference on Computational Natural Language Learning. Association for Computational Linguistics, 220--228
Lin Chin-Yew, 2004, Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, 8
Liu Chenxi, 2017, Yuille
Maddison Chris J., 2017, International Conference on Learning Representations (ICLR’17)
Mao Junhua, 2015, International Conference on Learning Representations (ICLR’15)
Mao Junhua, 2014, Yuille
Mitchell Margaret, 2012, Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 747--756
10.5555/1873738.1873752 · DOI
Mitchell Margaret, 2013, Kees Van Deemter, and Ehud Reiter
10.1145/1273496.1273577 · DOI
10.5555/645313.649410 · DOI
Ordonez Vicente, 2011, Berg
Ranzato Marc’Aurelio, 2016, International Conference on Learning Representations (ICLR’16)
Zhou Ren Hailin Jin Zhe Lin Chen Fang and Alan Yuille. 2015. Multi-instance visual-semantic embedding. arXiv preprint arXiv:1512.06963. Zhou Ren Hailin Jin Zhe Lin Chen Fang and Alan Yuille. 2015. Multi-instance visual-semantic embedding. arXiv preprint arXiv:1512.06963. · arXiv
Vadivel Karthikeyan Shanmuga, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3241--3250
Simonyan Karen, 2015, International Conference on Learning Representations (ICLR’15)
Yusuke Sugano and Andreas Bulling. 2016. Seeing with humans: Gaze-assisted neural image captioning. arXiv preprint arXiv:1608.05203. Yusuke Sugano and Andreas Bulling. 2016. Seeing with humans: Gaze-assisted neural image captioning. arXiv preprint arXiv:1608.05203. · arXiv
Sutskever Ilya, 2014, Le
van Deemter Kees, Proceedings of the 4th International Natural Language Generation Conference. Association for Computational Linguistics, 130--132
Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998--6008. Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998--6008.
10.5555/1708322.1708334 · DOI
Wang Qingzhong, 2018, Chan
Wang Yufei, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17)
Yang Zhilin, 30th Conference on Neural Image Processing System (NIPS’16)
Xu Kelvin, 2015, International Conference on Machine Learning. 2048--2057
Yang Linjie, 2016, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16)
Yu Lantao, 2017, Proceedings of the 31st AAAI Conference on Artificial Intelligence.
Matthew, 2014, Zeiler and Rob Fergus
Zhang Li, 2017, Hospedales

Sorted by citations

Items 601–700 of 878

Best RNN+CNN combination in image captioning
Tang, J. (2024) · International Conference on Computer Science and Management Technology (ICCSMT) · doi:10.1145/3708036.3708085 · Cited by 1
“This is consistent with the papers of other scholars, which increases the credibility of this paper [10].”
Whether Current Large Language Models is Suitable for Multimodal Aspect-based Sentiment Analysis?
Song, S., & Zhao, S. (2024) · International Conference on Advances in Artificial Intelligence and Applications (AAIA) · doi:10.1145/3712623.3712644 · Cited by 1
“Recently, some researchers have developed models with strong multimodal capabilities, such as Flamingo and Vicuna [4], which can complete tasks such as complex machine translation [7], problems related to images and videos [3].”
Exploring Semantic Attributes for Image Caption Synthesis in Low-Resource Assamese Language
Choudhury, P., Guha, P., & Nandi, S. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3717612 · Cited by 2
“The semantic attributes are defined as the most important words for conveying semantic information of an image [17].”
An automatic image description generation technology and application for visually impaired individuals
Qu, X., Dong, H., Li, Z., & Tan, X. (2025) · International Journal of Modern Physics C · doi:10.1142/s0129183125420033 · Cited by 1
Image captioning deep learning model using ResNet50 encoder and hybrid LSTM–GRU decoder optimized with beam search
Kavitha, P. V., & Karpagam, V. (2025) · Automatika · doi:10.1080/00051144.2025.2485695 · Cited by 1
“Not only does it comprehend the image’s contents, but it also annotates them using a caption format [1].”
Modeling Omnichannel Retail Marketing Driven by Artificial Intelligence
Samanta, I., & Arkoudis, N. (2025) · Springer proceedings in business and economics · doi:10.1007/978-3-031-76654-1_15 · Cited by 1
Text-Guided Local Control for Regional Editing
Li, J., Liu, L., Zhang, H., & Lu, X. C. (2024) · Asia Symposium on Image and Graphics (ASIG) · doi:10.1145/3718441.3718470 · Cited by 1
“The techniques of cross-modal generation can also be used for various tasks such as image description generation [9], image transformation [18] and image editing [16].”
Controllable and Diverse Image Captioning via Keyword-Driven Diffusion Networks
Pan, M., Li, Y., Li, S., & Wang, S. (2024) · International Conference on Communication and Information Processing (ICCIP) · doi:10.1145/3708657.3708748 · Cited by 1
“In previous research, the most prevalent approach involves using a visual encoder (VE) to extract features from the image, followed by a language decoder to generate sentences [14, 15, 25].”
Synthetic Aperture Radar Image Captioning: Building a Dataset and Explore Models
Li, Y., Liu, W., Lu, W., Li, R., Li, F., Huan, T., Wang, S., & Wu, Y. (2025) · International Conference on Neural Networks, Information and Communication Engineering (NNICE) · doi:10.1109/nnice64954.2025.11063765 · Cited by 1
CorGPT: Coronary Angiography Imaging Analysis Using Large Medical Vision-Language Models
Huang, B., Zhang, H., Liu, Z., Wang, J., & Zhou, S. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0027-7_24 · Cited by 1
Brain Tumor Classification Using Convolutional Neural Networks
Eldin, N. M. B., Khayat, G. A. E., & Amer, A. A. (2025) · International Conference on Machine Intelligence and Smart Innovation (ICMISI) · doi:10.1109/icmisi65108.2025.11115797 · Cited by 1
“[4] This enables CNNs to operate proficiently across diverse tasks, such as picture segmentation, feature extraction, and classification.”
Integrating Frequency-Domain Representations with Low-Rank Adaptation in Vision-Language Models
Khan, M. A., Gangopadhyay, A., & Wang, J. (2025) · International Conference on Advanced Machine Learning and Data Science (AMLDS) · doi:10.1109/amlds63918.2025.11159352 · Cited by 1
“Similarly, the author’s approaches in [10], focus on aligning vision and language modalities with minimal parameter changes, achieving robust performance in tasks such as image captioning [11] and Visual Question Answering (VQA) [12].”
Enhancing Visual Reliance in Text Generation: A Bayesian Perspective on Mitigating Hallucination in Large Vision-Language Models
Hu, N., Duan, X., Zhang, J., & Kang, G. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755606 · Cited by 2
“…connector, Large Vision-Language Models (LVLMs) [1, 26, 31, 32, 44, 49] can simultaneously deal with visual and textural information and have demonstrated remarkable capabilities across various of multimodal tasks, including image captioning [16, 55] and visual question answering [12, 43].”
Definition of AI
Steckelmacher, D., & Bogaerts, B. (2025) · Law, governance and technology series · doi:10.1007/978-3-031-98406-8_4 · Cited by 2
Efficient image captioning with CLIP and GPT-2 via prefix mapping
Yen, N. D. T., Huu, V. L. N., & Tran, A. C. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.09.093 · Cited by 1
Towards Robust Text-Based Person Retrieval: A Framework for Correspondence Rectification and Description Synthesis
Yu, L., Xiong, L. Y., Li, W., & Feng, Y. (2025) · Electronics · doi:10.3390/electronics14234619 · Cited by 1
ChitraVivran: Real-Time Attention-Based Hindi Image Captioning with Boosted Contextual Descriptors
Sharma, H., Padha, D., & Singh, Y. P. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3779214 · Cited by 1
“Therefore, IC represents the integration of two cutting-edge domains in artificial intelligence: computer vision and natural language processing [13].”
Influential
A Novel Hybrid Deep Learning Framework for Image Captioning Using Combined Semantic and Object-Level Features
Chauhan, H. N., & Thacker, C. B. (2025) · Cybernetics and Information Technologies · doi:10.2478/cait-2025-0036 · Cited by 1
Adaptive in Adapter: Boosting Open-Vocabulary Semantic Segmentation With Adaptive Dropout Adapter
Wang, C., Xu, W., Xu, R., Zhang, Z., Xu, S., Zhang, J., Teng, X., Meng, W., & Zhang, X. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3654453 · Cited by 1
VL-GRiP3: A hierarchical pipeline leveraging vision-language models for autonomous robotic 3D grasping
Polonara, M., Yang, X., Carbonari, L., & Zhang, X. (2026) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2026.103244 · Cited by 2
Boosting adversarial transferability of vision-language pre-trained models via optimal transport
Qin, S., Li, G., Gao, S., Han, D., Jia, X., Bai, Y., Gu, J., & Cao, X. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.113729 · Cited by 1
CLIP-Guided Dynamic Feature Fusion With Relation-Enhanced Attention for Remote Sensing Image Captioning
Sima, H., Jiang, W., Wang, J., Li, C., & Xu, M. (2026) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2026.3686075 · Cited by 1
“…[1], [2] or scene classiﬁcation [3], [4], image captioning facilitates a better understanding of the rich information contained in remote sensing imagery,thus providing valuable guidance for application domains, such as environmental monitoring, urban planning, and disaster assessment [5], [6].”
MLFF-DA: Multi-level feature fusion with dual attention for enhanced image captioning
Hossain, M. A., Hossen, M. B., Rahman, M. M., Islam, M. S., Abdullah, M. I., & Ye, Z. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133888 · Cited by 1
Self-citation
Multiple Visual-Semantic Embedding for Video Retrieval from Query Sentence
Nguyen, H. M., Miyazaki, T., Sugaya, Y., & Omachi, S. (2021) · Applied Sciences · doi:10.3390/app11073214 · Cited by 3
“Image captioning is a task for generating description of images [19].”
Intelligent Calibration of Static FEA Computations Based on Terrestrial Laser Scanning Reference
Xu, W., Bao, X., Chen, G., & Neumann, I. (2020) · Sensors · doi:10.3390/s20226439 · Cited by 1
VisDG: Towards Spontaneous Visual Dialogue Generation
Yang, Q., Zhang, L., Zhang, Y., & Gao, J. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534326 · Cited by 1
“For a comprehensive survey, readers can refer to [20] and [21].”
Background
Overview on Image Captioning Techniques
(2021) · International Journal of Emerging Trends in Engineering Research · doi:10.30534/ijeter/2021/15982021 · Cited by 1
Automatic Threat Detection Using Deep Neural Networks
Tripathy, S. S., Barik, L., Sahu, P. P., & Naik, H. (2021) · OITS International Conference on Information Technology (OCIT) · doi:10.1109/ocit53463.2021.00024 · Cited by 1
Images2Poem in different contexts with Dual‐CharRNN
Yan, J., Xie, Y., & Luan, X. (2022) · CAAI Transactions on Intelligence Technology · doi:10.1049/cit2.12089 · Cited by 1
Oppositional Harris Hawks Optimization with Deep Learning-Based Image Captioning
Kavitha, V. R., Nimala, K., Beno, A., Ramya, K. C., Kadry, S., Kang, B.-G., & Nam, Y. (2022) · Computer Systems Science and Engineering · doi:10.32604/csse.2023.024553 · Cited by 1
A Reconstruction-Based Visual-Acoustic-Semantic Embedding Method for Speech-Image Retrieval
Cheng, W., Tang, W. D., Huang, Y., Luo, Y., & Wang, L. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3171090 · Cited by 1
Research on image content description in Chinese based on fusion of image global and local features
Kong, D., Zhao, H. P., & Zeng, X. (2022) · PLoS ONE · doi:10.1371/journal.pone.0271322 · Cited by 1
“Introduction Image content description, also known as image semantic understanding, uses computer vision and deep learning technology to extract the semantics contained in the image, and use natural language processing technology to generate a reasonable text description [1, 2].”
Background
Molecular-InChI: Automated Recognition of Optical Chemical Structure
Kumar, N., Rashmi, M., Ramu, S., & Guddeti, R. M. R. (2022) · IEEE Region 10 Symposium (TENSYMP) · doi:10.1109/tensymp54529.2022.9864516 · Cited by 1
“1: Skeletal formula and corresponding InChI label [2] by [3].”
Background
A Survey on Text Description to Image Generation Using GAN
Yeshasvi, M., Kayal, P., & Subetha, T. (2022) · Advances in intelligent systems and computing · doi:10.1007/978-981-19-3590-9_52 · Cited by 1
Gaussian Distributed Graph Constrained Multi-Modal Gaussian Process Latent Variable Model for Ordinal Labeled Data
Maeda, K., Ogawa, T., & Haseyama, M. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9898070 · Cited by 1
“Multi-modal learning is an effective way to handle these data and has been applied to cross-modal retrieval [1], image captioning [2], and audio-visual analysis [3].”
Comparative Study on Image Captioning
Patel, H. K., & Rathod, J. M. (2022) · INTERNATIONAL JOURNAL OF NEXT-GENERATION COMPUTING · doi:10.47164/ijngc.v13i4.769 · Cited by 1
A Survey on Attention-Based Image Captioning: Taxonomy, Challenges, and Future Perspectives
Sharma, H., Padha, D., & Selwal, A. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-981-99-0047-3_58 · Cited by 1
Novel Object Captioning with Semantic Match from External Knowledge
Du, S., Zhu, H., Lin, G., Wang, D. Y., & Shi, J. (2023) · Applied Sciences · doi:10.3390/app13137868 · Cited by 1
A Survey on Efficient Neural Network Compression Techniques
Jain, N., Wyawahare, M. V., Mankar, V., & Paratkar, T. (2023) · Algorithms for intelligent systems · doi:10.1007/978-981-99-2854-5_24 · Cited by 2
Human Trespass Detection Based on Lightweight YOLO-v5 and RNN in Restricted Area
Tian, L., & Wang, Y. (2023) · Highlights in Science Engineering and Technology · doi:10.54097/hset.v57i.10003 · Cited by 1
Design exploration of a tactile path defect detection vehicle
Huang, C., Wang, J., & Ma, S. (2023) · International Conference on Computer Vision and Pattern Analysis (ICCPA) · doi:10.1117/12.2684219 · Cited by 1
Glove-Ing Attention: A Multi-Modal Neural Learning Approach to Image Captioning
Anundskaas, L. H., Afridi, H., Tarekegn, A. N., Yamin, M. M., Ullah, M., Yamin, S., & Cheikh, F. A. (2023) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw59220.2023.10193011 · Cited by 1
“The state-of-the-art image captioning models are mostly based on Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), which can effectively capture the visual features of an image and generate a coherent sentence that describes its content [5].”
Method
Automated Clinical Impression Generation for Medical Signal Data Searches
Lee, W., Yang, J.-W., 박도영, & Kim, Y. (2023) · Applied Sciences · doi:10.3390/app13158931 · Cited by 1
Efficient scheme to perform semantic segmentation on 3-D brain tumor using 3-D u-net architecture
Shaukat, Z., Farooq, Q. A., Xiao, C., Ali, S., Akhtar, F., Azeem, M., & Zulfiqar, A. A. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16458-8 · Cited by 1
“In last few years, significant breakthroughs have been made by methods based on deep learning in various fields including image segmentation and classification, object detection and tracking and image captioning [3, 9, 15, 24, 31].”
Scene recognition and image caption generation
Bhatlawande, S. S., Khire, N. R., Kinge, M., Kshirsagar, C., Kudal, P., Shilaskar, S., & Madake, J. (2023) · AIP conference proceedings · doi:10.1063/5.0166868 · Cited by 1
Text-driven Face Image Generation and Manipulation via Multi-level Residual Mapper
Li, Z., Zhang, S., Liu, Y., Zhang, Z., Zhang, W., & Huang, Q. (2023) · International Journal of Software and Informatics · doi:10.21655/ijsi.1673-7288.00313 · Cited by 1
Integrating Scene and Text Recognition for Improved Scene Caption to Assist Visually Impaired
Madake, J., Jha, M., Meshram, N., Muhal, N., Naik, U., & Bhatlawande, S. S. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-45124-9_4 · Cited by 1
Labelling of Annotated Condition Monitoring Data Through Technical Language Processing
Löwenmark, K., Taal, C., Vurgaft, A. V., Nivre, J., Liwicki, M., & Sandin, F. (2023) · Annual Conference of the PHM Society · doi:10.36001/phmconf.2023.v15i1.3507 · Cited by 2
“Early work in mapping language and signals focused mainly on a uni-directional pipeline, going for instance from image to caption through image captioning (Hossain et al., 2019), or from image + question to image and question-based answer through visual question answering (VQA) (Wu et al., 2017).”
Background
Research On the Innovation of Host Ability Integrating Natural Language Processing Technology
Sun, H., & You, X. (2024) · Journal of Machine and Computing · doi:10.53759/7669/jmc202404017 · Cited by 1
Automated Image Captioning and Speech Synthesis
Mankawade, A., Pavitha, N., & VPatil, A. (2023) · International Conference on Futuristic Technologies (INCOFT) · doi:10.1109/incoft60753.2023.10425625 · Cited by 1
“The introduction of the article[4] discusses the value of picture captioning in different domain spaces like e-commerce and entertainment..”
Background
Automated Image Caption Generation using Deep Learning
Hasan, M., Prithila, S. J., Ahasan, T. A., Hassan, M., Hossain, A., Bhuiyan, S. A., & Rasel, A. A. (2023) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit60459.2023.10441058 · Cited by 1
“Recognizing speech, and translation by machine, and image labeling are a few examples of sequence learning applications that utilize RNN [10].”
CLIP-M-Cap: CLIP Mean Teacher for Image Captioning
Chen, S., & Zhang, J. (2023) · International Symposium on Computing and Artificial Intelligence (ISCAI) · doi:10.1145/3640771.3640783 · Cited by 1
“While humans can effortlessly describe the content of an image, teaching machines to do the same has proven to be a challenging task in the field of computer vision and natural language processing [1] [2] [3] [4].”
Background
Image-to-Text Translation for Interactive Image Recognition: A Comparative User Study with Non-expert Users
Kawabe, W., & Sugano, Y. (2024) · Journal of Information Processing · doi:10.2197/ipsjjip.32.358 · Cited by 1
DICKT—Deep Learning-Based Image Captioning using Keras and TensorFlow
Vi, P. T., Mishra, S. K., Ngoc, L. A., Mishra, S., & Phuc, V. M. (2022) · Annals of Computer Science and Information Systems · doi:10.15439/2023r55 · Cited by 1
Detection and Recognition Approaches for Medical Labels in Algeria
Meghatria, R., & Abderrahim, I. A. (2024) · International Conference on Image and Signal Processing and their Applications (ISPA) · doi:10.1109/ispa59904.2024.10536801 · Cited by 1
“…text, classification methods like template matching or neural networks are used. on the other hand, in recent years, deep learning approaches [9], particularly Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) [10], have shown remarkable performance in text…”
Background
A Comparative Evaluation of Image Caption Synthesis Using Deep Neural Network
Tisha, S. N., Rahaman, M. S., & Rivas, P. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-61816-1_13 · Cited by 1
Preliminary Study on Image Captioning for Construction Hazards
Hsiao, W.-T., Yu, W., Cheng, T.-M., & Bulgakov, A. (2024) · IEEE International Conference on Electronic Communications, Internet of Things and Big Data (ICEIB) · doi:10.3390/engproc2024074020 · Cited by 1
Bi-directional Contextual Attention for 3D Dense Captioning
Kim, M., Lim, H. S., Lee, S., Kim, B., & Kim, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72649-1_22 · Cited by 1
Reflecting on machine-generated image descriptions of personal photos
Jung, J. Y., & Kim, J. (2024) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3677045.3685494 · Cited by 1
“A quest in computer vision research is to build machines that understand what they see, or that at least can replicate humans’ ability to describe what they visually perceive [10, 14].”
A Survey on Image Captioning Using Encoder-Decoder Based Deep Learning Models
Narendrabhai, C. H., & Thacker, C. B. (2024) · Parul International Conference on Engineering and Technology (PICET) · doi:10.1109/picet60765.2024.10716063 · Cited by 1
Image Captioning Based on Semantic Scenes
Zhao, F., Yu, Z., Wang, T., & Lv, Y. (2024) · Entropy · doi:10.3390/e26100876 · Cited by 1
“Image captioning is a task in the field of computer vision which involves the generation of descriptive natural language for images by comprehending the visual information contained within them [1].”
Background
Toward a Multi-dimensional Humor Dataset for Social Robots
Zhang, H., Hei, X., Cárdenas, J. J. G., Miao, X., & Tapus, A. (2024) · IEEE RO-MAN · doi:10.1109/ro-man60168.2024.10731268 · Cited by 1
“Additionally, it can utilize image captioning algorithms [42], [43] to extract relevant keywords from the current context.”
Recent Advances on Multi-modal Dialogue Systems: A Survey
Cheng, F., Li, X., Wu, H., Sang, J., & Zhao, W. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0847-8_3 · Cited by 1
Automated Medical Labels Detection and Text Extraction Using Tesseract
Abderrahim, I. A., & Benrached, Y. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-82112-7_16 · Cited by 1
Multimodal Shannon Game with Images
Zouhar, V., Bhattacharya, S., & Bojar, O. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-82487-6_3 · Cited by 2
Generating Descriptions for Visual Content Using Deep Learning Approach
Rajkumar, B., Jaya Lakshmi, A., Deekshitha, P., s, V. A. R. T. A., & Dharavath, K. (2024) · International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES) · doi:10.1109/scopes64467.2024.10990400 · Cited by 1
“This review paper examines various deep learning methods for image captioning, covering their underlying principles, performance, advantages, limitations, datasets, and common evaluation metrics used in automatic image captioning [3].”
ReFrame: Rectification Framework for Image Explaining Architectures
Adhikary, D. D., Hazra, A., & Chakrabarti, P. (2024) · COMAD/CODS · doi:10.1145/3703323.3703332 · Cited by 1
“In particular, Image Captioning , the first of these, involves creating descriptive text for an image, encapsulating its contents in a concise sentence [11].”
Machine-learning Applications to Membrane Active Peptides
Bjerrum, E. J., & Maria, L. D. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-801238-3.11544-2 · Cited by 1
The Design Patent Images Classification Based on Image Caption Model
Liu, H., Dai, Q., Li, Y., Zhang, C., Yi, S., & Yuan, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-39431-8_34 · Cited by 1
Auto Image Comment via Deep Attention
Shi, J., Li, H., Wu, S., & Wang, M. (2019) · International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc47709.2019.8980999 · Cited by 1
“It is an interdisciplinary cross-modal and cross-cutting application, which is similar to translating an image into a piece of descriptive text [2], [3].”
Method
Improving E-Learning Material Quality With the Aid of Deep Learning and Workflow Management.
Pap, M., Nagy, L. Z., & Fekete, D. (2020) · Cited by 1
Intelligent Assistance for End-Users in creating Point and Click Games Storylines
Cau, F. M., Mereu, A., & Spano, L. D. (2020) · doi:10.1109/hccai49649.2020.00018 · Cited by 1
“We can ﬁnd different approaches for labelling the objects inside an image [2] or for providing a good descriptive caption [3].”
Method
Iraqi Plate Number Detection Based on Deep Learning and Recognize by SURF
Alhamza, D. A. A., & Alaythawy, A. D. (2020) · Information Technology To Enhance e-learning and Other Application (IT-ELA) · doi:10.1109/it-ela50150.2020.9253085 · Cited by 1
“The accuracy of various pattern recognition tasks was considerably improved with the introduction of Deep Learning (DL) techniques [15][16].”
Method
Identifying User Interests and Habits Using Object Detection and Semantic Segmentation Models
Volokha, V., & Gladilin, P. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72610-2_16 · Cited by 1
On Randomized Classification Layers and Their Implications in Natural Language Generation
Shalev, G., Shalev, G., & Keshet, J. (2021) · doi:10.18653/v1/2021.maiworkshop-1.2 · Cited by 1
“Deep neural networks enabled breakthroughs in natural language generation tasks such as machine-translation (Zhang and Zong, 2015), image captioning (Hossain et al., 2019), and more.”
Background
Generating Classified Ad Product Image Titles with Image Captioning
Atıcı, B., & Omurca, S. İ. (2021) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-030-79357-9_21 · Cited by 1
Comprehensive Comparative Study on Several Image Captioning Techniques Based on Deep Learning Algorithm
Ningthoujam, C., & Chingtham, T. S. (2021) · Lecture notes in networks and systems · doi:10.1007/978-981-16-4244-9_18 · Cited by 1
Attention-Based Comparison of Automatic Image Caption Generation Encoders
NagaDurga, C. S., & Anuradha, T. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-16-8550-7_16 · Cited by 1
Paint to Better Describe: Learning Image Caption by Using Text-to-Image Synthesis
Wang, R., & Liu, L. (2021) · IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech) · doi:10.1109/dasc-picom-cbdcom-cyberscitech52372.2021.00160 · Cited by 1
A Study on Different Deep Learning Architectures on Image Captioning
Rage, K. (2022) · International Conference on Smart Structures and Systems (ICSSS) · doi:10.1109/icsss54381.2022.9782260 · Cited by 1
Advancement of Deep Learning and Its Substantial Impact on the Diagnosis of COVID-19 Cases
Shahabade, S., & Londhe, R. R. (2022) · Smart innovation, systems and technologies · doi:10.1007/978-981-19-2719-5_25 · Cited by 1
SLT at SemEval-2023 Task 1: Enhancing Visual Word Sense Disambiguation through Image Text Retrieval using BLIP
Molavi, M., & Zeinali, H. (2023) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2023.semeval-1.264 · Cited by 1
“For example, in image captioning(Hossain et al., 2019), the system needs to identify the correct sense of a word in a given context to generate an accurate and meaningful caption.”
Background
CLSE: Corpus of Linguistically Significant Entities
Chuklin, A., Zhao, J., & Kale, M. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.6 · Cited by 2
Deep Unified Multimodal Embeddings for Understanding both Content and Users in Social Media NetworksPreprint
Sikka, K., Bramer, L. V., & Divakaran, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.07075 · Cited by 3
“Over the last three years we have seen significant progress in image captioning with the advances in vision and language based deep learning [31].”
Creative Captioning: An AI Grand Challenge Based on the Dixit Board GamePreprint
Kunda, M., & Rabkina, I. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00048 · Cited by 5
“Creative captioning is very related to image captioning (Hossain et al. 2019), though it goes beyond to incorporate more sophisticated aspects of vision (e.g., multiple and often abstract interpretations of an image), natural language processing (including idioms, cultural references, etc.), story…”
Background
Denoising Large-Scale Image Captioning from Alt-text Data using Content Selection ModelsPreprint
Chandu, K. R., Sharma, P. K., Changpinyo, S., Thapliyal, A., & Soricut, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.05175 · Cited by 3
“In contrast to most IC approaches (Hossain et al., 2018; Sharma et al., 2020), which hallucinate incorrect content from noisy training data (i.”
Result
Concadia: Tackling image accessibility with context.Preprint
Kreiss, E., Goodman, N. D., & Potts, C. (2021) · arXiv (Cornell University) · Cited by 1
Goal-driven text descriptions for imagesPreprint
Luo, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.12575 · Cited by 4
DeepLSF: Fusing Knowledge and Data for Time Series ForecastingPreprint
Chattha, M. A. (2021) · doi:10.36227/techrxiv.17003944 · Cited by 1
“ing [3], speech recognition [4], visual question answering (VQA) [5], image captioning [6], etc.”
Background
Improving image captioning methods using machine learning approaches
Atliha, V. (2023) · doi:10.20334/2023-021-m · Cited by 1
Vision-and-Language Pretraining: Methods, Applications, and Future ChallengesPreprint
Nguyen, T., Nguyen, C.-D. T., Wu, X., Ng, S.-K., & Luu, A.-T. (2023) · Research Square · doi:10.21203/rs.3.rs-3120051/v1 · Cited by 1
A Comprehensive Survey on Image Captioning for Indian Languages: Techniques, Datasets, and ChallengesPreprint
Jayaswal, V., Rani, R., & Kaur, J. (2023) · Research Square · doi:10.21203/rs.3.rs-3601177/v1 · Cited by 1
Construction Safety Inspection Workflow with Clip-Based Image Captioning and Attention GenerationPreprint
Tsai, W.-L., Lin, J. J., Ho, W.-F., Tang, S., & Hsieh, S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4819831 · Cited by 1
Integrating Multiple Deep Learning Models to Classify Disaster Scene Videos
Li, Y. (2021) · doi:10.12794/metadc1873789 · Cited by 1
Representation, information extraction, and summarization for automatic multimedia understanding
Harrando, I. (2022) · theses.fr (ABES) · doi:10.70675/702b8ba3z3acaz4ce3za9b9zf2cb1609e328 · Cited by 2
A Framework for Decoding Event-Related Potentials from Text
Yan, S., & White, A. S. (2019) · Workshop on Cognitive Modeling and Computational Linguistics (CMCL) · doi:10.18653/v1/w19-2910
“Inspired by multimodal tasks like image captioning (see Hossain et al., 2019, for a review) and visual question answering (Antol et al., 2015), we propose to model EEG using standard convolutional neural networks (CNNs) pre-trained ar X iv :1 90 2.”
Method
Inteligência Social de uma cidade: escutar e aprender os contextos narrados pelo cidadão em meios as redes sociais
Kwecko, V., de Tolêdo, F. P., Moraes, G., & Botelho, S. S. D. C. (2019) · RENOTE · doi:10.22456/1679-1916.96171
Text-to-Image Synthesis Based on Machine Generated Captions
Menardi, M., Falcon, A., Mohamed, S. S., Seidenari, L., Serra, G., Bimbo, A. D., & Tasso, C. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-39905-4_7
Creating Information-maximizing Natural Language Messages Through Image Captioning-Retrieval.
Karl, F., Lauri, M., & Biemann, C. (2019) · KONVENS
Background Hardly Matters: Understanding Personality Attribution in Deep Residual NetworksPreprint
Ras, G., Dotsch, R., Ambrogioni, L., Güçlü, U., & van Gerven, M. A. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.09831
“The recent advances in contextual feature extraction indicate that it is possible to create networks with a human-like capability of image understanding for certain tasks, especially image captioning [17].”
Background

Authors: Hossain, M. Z., Sohel, F., Shiratuddin, M. F., & Laga, H.
Affiliations: Murdoch University (AU)
ORCID: Hossain, MD. Zakir
Sohel, Ferdous
Shiratuddin, Mohd Fairuz
Laga, Hamid
Venue: ACM Computing Surveys
Publisher: Association for Computing Machinery (ACM)
Published: 2019-02-04
Volume: 51
Issue: 6
Pages: 1-36
ISSN: 0360-0300 (ISSN-L)
1557-7341
DOI: 10.1145/3295748
OpenAlex: W2896348597
Cited by: 879
References: 166
Citations by year: 2026: 55 · 2025: 113 · 2024: 124 · 2023: 167 · 2022: 153 · 2021: 127 · 2020: 104 · 2019: 36
FWCI: 47.02
Type: journal article
Language: en
Open access: closed
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: ACM Computing Surveys (publishedVersion)
Murdoch University Research Portal (submittedVersion)
Funders: Australian Research Council (DE120102960)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Keywords: image captioning, object recognition, sentence generation, evaluation metrics, deep learning architectures

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

A Comprehensive Survey of Deep Learning for Image Captioning

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks