The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Attention on Attention for Image Captioning
Open accessAbstract
Attention mechanisms are widely used in current encoder/decoder frameworks of image captioning, where a weighted average on encoded vectors is generated at each time step to guide the caption decoding process. However, the decoder has little idea of whether or how well the attended vector and the given attention query are related, which could make the decoder give misled results. In this paper, we propose an Attention on Attention (AoA) module, which extends the conventional attention mechanisms to determine the relevance between attention results and queries. AoA first generates an information vector and an attention gate using the attention result and the current context, then adds another attention by applying element-wise multiplication to them and finally obtains the attended information, the expected useful knowledge. We apply AoA to both the encoder and the decoder of our image captioning model, which we name as AoA Network (AoANet). Experiments show that AoANet outperforms all previously published methods and achieves a new state-of-the-art performance of 129.8 CIDEr-D score on MS COCO Karpathy offline test split and 129.6 CIDEr-D (C40) score on the official online testing server. Code is available at https://github.com/husthuaan/AoANet.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“DropToken [174] reduces the training complexity via random dropping a portion of the video and audio tokens from input sequence during training.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [174]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Flagged first, then by citations
Cited by
Sorted by citations
Items 801–900 of 1,002
Macroscopic Control of Text Generation for Image CaptioningPreprint
Zhu, Z., Wang, T., & Qu, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.08000 · Cited by 2
“In [16, 42, 25, 40, 24, 34], attention mechanism is used to enable the model to selectively get the target area of interest that needs special attention at each time step.”
MethodConnecting What to Say With Where to Look by Modeling Human Attention TracesPreprint
Meng, Z., Yu, L., Zhang, N., Berg, T. L., Damavandi, B., Singh, V., & Bearman, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.05964 · Cited by 2
“Architectures for both tasks often rely on attention [38, 15, 17, 25], a mechanism inspired by the human visual system [31, 11].”
Self-Annotated Training for Controllable Image CaptioningPreprint
Zhu, Z., Wang, T., & Qu, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08446 · Cited by 2
“To examine the impact of our proposed SAT method and finetuning method, we choose AoANet (Huang et al. 2019) as our baseline model and conduct ablation studies in the task of controlling the sentence length and tense simultaneously.”
BackgroundMethodInfluentialSemi-Supervised Image Captioning by Adversarially Propagating Labeled DataPreprint
Kim, D.-J., Oh, T.-H., Choi, J., & Kweon, I. S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4583222 · Cited by 2
Enhancing Lightweight Image Captioning with Localized Features and Keyword ExtractionPreprint
Jeon, M., Kim, S., Lee, S., & Kim, D.-J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4866145 · Cited by 2
Ohata, K., Kitada, S., & Iyatomi, H. (2022) · IEEE International Conference on Smart Communities: Improving Quality of Life using AI, Robotics and IoT (HONET) · doi:10.1109/honet56683.2022.10019010 · Cited by 1
“These frameworks have improved the captioning performance under poor conditions by using models with improved long short-term memory (LSTM) [22] or selfattention-based AoANet [16] models.”
MethodThink as People: Context-Driven Multi-Image News Captioning with Adaptive Dual Attention
Yang, Q., Wu, X., Chen, X., Gao, X., & Zhang, X. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446024 · Cited by 1
“Additionally, Transformer-based models have shown promise in image captioning, utilizing Transformer as the image encoder and text decoder [9, 10, 11, 12].”
MethodA Unified Visual and Linguistic Semantics Method for Enhanced Image Captioning
Peng, J., & Tang, T. (2024) · Applied Sciences · doi:10.3390/app14062657 · Cited by 1
“Huang et al. [25] introduce an “Attention on Attention” (AoA) module to refine traditional attention mechanisms in image captioning, emphasizing the calculation of relevance between attention results and queries.”
BackgroundSample-Adaptive Classification Inference Network
Yang, J., Zhou, G., Wang, R., & Xue, L. (2024) · Neural Processing Letters · doi:10.1007/s11063-024-11629-6 · Cited by 1
“Lun Huang et al. [16] applied AOA to the image cap-tioning model and achieved excellent results.”
Image captioning using attention network and machine learning approaches
Pandey, M. (2024) · AIP conference proceedings · doi:10.1063/5.0232649 · Cited by 1
Reinforced visual interaction fusion radiology report generation
Wang, L., Chen, H., Liu, Y., Lyu, Y., & Qiu, F. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01504-8 · Cited by 3
“Beginly, some researchers directly applied image captioning methods in the field of image captioning for radiology report generation, such as transformer [11] and AOANet [13], but achieved limited results.”
InfluentialBromonschenkel, G., Oliveira, H., & Paixão, T. M. (2024) · SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi62404.2024.10716325 · Cited by 1
“In the context of the Brazilian Portuguese language, the earliest architecture explored in a native dataset for IC was an Attention-on-Attention Network (AoANet) [13] as reported by Santos et al. [3].”
Exploring Semantic Attributes for Image Caption Synthesis in Low-Resource Assamese Language
Choudhury, P., Guha, P., & Nandi, S. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3717612 · Cited by 2
“Furthermore, the introduction of attention mechanisms enhances the image captioning performance [1, 18, 42].”
FFusion: Feature Fusion Transformer for Zero-Shot Learning
Tao, W., Xie, J., An, Z., & Meng, X. (2025) · Electronics · doi:10.3390/electronics14050926 · Cited by 1
EFCNet enhances the efficiency of segmenting clinically significant small medical objects
Kong, L., Wei, Q., Xu, C., Ye, X., Liu, W., Wang, M., Fu, Y., & Chen, H. (2025) · Scientific Reports · doi:10.1038/s41598-025-93171-6 · Cited by 1
“Numerous attention-based methods have emerged in recent years 36–44 , and have been applied to diverse tasks in computer vision and natural language processing.”
Gao, L., & Pang, X. (2025) · IEEE Access · doi:10.1109/access.2025.3565682 · Cited by 1
“Meanwhile, Huang et al. [18] introduced an extension of the attention operator, namely the Attention-on-Attention (AoA) module.”
Lu, J. J., Wu, C., Wang, L., Li, R., & Shen, X. (2025) · Tsinghua Science & Technology · doi:10.26599/tst.2024.9010093 · Cited by 1
Positional-enhanced and normalized transformer for image captioning
Gong, F., Shi, Z., & Zhao, X. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-04066-y · Cited by 1
Integrating visual memory for image captioning
Wei, J., & Wu, T. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-07045-7 · Cited by 1
“Following a general practice in image captioning [1, 12, 28, 46, 47], our training process is divided into two stages, starting with word-level cross-entropy loss (XE), followed by sentence-level reinforcement learning (RL).”
InfluentialControllable and Diverse Image Captioning via Keyword-Driven Diffusion Networks
Pan, M., Li, Y., Li, S., & Wang, S. (2024) · International Conference on Communication and Information Processing (ICCIP) · doi:10.1145/3708657.3708748 · Cited by 1
$$T^2$$ Transformer for Image Captioning
Liu, Q., Liu, G., Li, Y., Zheng, X., & Guo, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8180-8_11 · Cited by 1
Training strategies for semi-supervised remote sensing image captioning
Cheng, Q., Cheng, H., Yuan, L., Du, Y., Tu, S., & Xu, Y. (2025) · Scientific Reports · doi:10.1038/s41598-025-09853-8 · Cited by 1
“Specifically, for the first two datasets, we compared our approach against two supervised methods, SAT 46 and AoANet 47 , as well as two semi-supervised methods, SFRC 6 and FRIC 7 .”
InfluentialSynthetic Aperture Radar Image Captioning: Building a Dataset and Explore Models
Li, Y., Liu, W., Lu, W., Li, R., Li, F., Huan, T., Wang, S., & Wu, Y. (2025) · International Conference on Neural Networks, Information and Communication Engineering (NNICE) · doi:10.1109/nnice64954.2025.11063765 · Cited by 1
Location-Guided Automated Lesion Captioning in Whole-Body PET/CT Images
Yu, M., Gao, Y., Shu, Y., Chen, Y., Liu, J. Y., Jiang, C., Sun, K., Cui, Z., Zhang, W., Zhan, Y., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04971-1_33 · Cited by 1
U 2 DP: Unlocking Unlabeled Data Potential for Semi-Supervised Remote Sensing Image Captioning
Zhong, S.-P., Guan, Z., Zhang, Y., Lian, X., & Yang, Y. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3620299 · Cited by 2
“One adopts a conventional CNN-based encoder and LSTM-based decoder framework (i.e., AoANet [30]), while the other utilizes a transformer-based model [21].”
Ju, S., Wang, M., Xiao, Y., Shen, Y., Cheng, J., Su, Y., & Liao, X. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.109322 · Cited by 2
Image captioning for life ecological experiment of China’s space station
Liu, Y., Wang, Y., Du, C., Deng, Y., Liu, A., Li, S., & Li, S. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-02163-6 · Cited by 1
“Note that AoANet is solely trained with the image captioning generation loss, which does not fully leverage category labels.”
InfluentialZhong, P., Hu, W., Zhao, Y., & Zhang, F. (2026) · npj Heritage Science · doi:10.1038/s40494-026-02343-8 · Cited by 1
“AoANet 8 adds “ Attention on Attention ” improving focus on critical visual features and textual context; among compared models, it performsbest.”
Engage for All: Making Ordinary Image Descriptions Appealing Again!
Chen, Y., Jiang, Y., Zhou, L., Cao, J., Guan, Y., Yang, M., & Guo, Q. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01798 · Cited by 1
Show, Conceive and Tell: Image Captioning with Prospective Linguistic Information
Huang, Y., & Chen, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69544-6_29 · Cited by 1
Self-citationTopic Guided Image Captioning with Scene and Spatial Features
Zia, U., Riaz, M. M., & Ghafoor, A. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-030-99587-4_16 · Cited by 1
Dynamic Transformer for Image Captioning
Xian, T., Li, Z., Chen, T., & Ma, H. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859885 · Cited by 1
“Most image captioning models are based on the encoder-decoder framework[1, 2, 3, 4, 5].”
MethodDM-CATN: Deep Modular Co-Attention Transformer Networks for image captioning
Wang, X., Fang, X., & Yang, Y. (2022) · International Conference on Artificial Intelligence and Intelligent Information Processing (AIIIP) · doi:10.1117/12.2659580 · Cited by 1
Dual attention-guided state controller mechanism for Text-to-Image generation
Lai, B., Zhang, Z., Tian, J., & Ma, L. (2022) · IEEE Region 10 Conference (TENCON) · doi:10.1109/tencon55691.2022.9978127 · Cited by 1
“Attention (AoA) module [12] to construct W2V attentional features as follows, which can reduce the above interference.”
BackgroundvieCap4H Challenge 2021: A transformer-based method for Healthcare Image Captioning in Vietnamese
Cao, D. B., Trinh, T. T. T., Nguyen, T.-T., Nguyen, D.-V., & Vo, N. D. (2022) · VNU Journal of Science Computer Science and Communication Engineering · doi:10.25073/2588-1086/vnucsce.371 · Cited by 1
Post-Attention Modulator for Dense Video Captioning
Guo, Z., Wang, T.-J. J., & Laaksonen, J. T. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956260 · Cited by 2
“Motivated by [12], we propose a novel Post-Attention Modulator (PAM) to be applied to Transformer-based models for dense VC problem.”
BackgroundMethodResultInfluentialICDT: Incremental Context Guided Deliberation Transformer for Image Captioning
Lai, X., Lyu, Y., Zhong, J., Wang, C., Dai, Q., & Li, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20865-2_33 · Cited by 1
Semantic-Guided Selective Representation for Image Captioning
Li, Y., Ma, Y., Zhou, Y., & Yu, X. (2023) · IEEE Access · doi:10.1109/access.2023.3243952 · Cited by 1
“Following [4], [27], we also present the performances of our proposed RAS-FSG by building an ensemble of trained models, i.e. , averaging the output probability distributions of multiple trained models.”
BackgroundMethodInfluentialEvaluating Captioning Models using Markov Logic Networks
Shah, M., Sarkhel, S., & Venugopal, D. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020793 · Cited by 1
“We perform experiments using the well-known MSCOCO dataset [11] and compare the performance of state-of-theart captioning systems such as SGAE [21] and attentionon attention transformer models (AoANet) [7].”
MethodInfluentialAttention-aware Memory Aggregation Network for Hyperspectral Image Classification
Rime, M. H. K., Hossain, M. A., & Qazi, A. (2022) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit57492.2022.10055318 · Cited by 1
“Attention Mechanism Attention mechanism has been widely used in image classification [15], image synthesis [16], image super-resolution [17] and many more [18]–[20].”
BackgroundEvaluación de atención selectiva en niños de 7 a 8 años
Hernández, D. C. V., Hernández, R. E. V., Albán, J. R. A., & Hernández, R. S. V. (2023) · ConcienciaDigital · doi:10.33262/concienciadigital.v6i1.4.2005 · Cited by 1
Controllable Image Captioning with Feature Refinement and Multilayer Fusion
Du, S., Zhu, H., Zhang, Y., Wang, D. Y., Shi, J., Xing, N., Lin, G., & Zhou, H. (2023) · Applied Sciences · doi:10.3390/app13085020 · Cited by 1
A Survey on Attention-Based Image Captioning: Taxonomy, Challenges, and Future Perspectives
Sharma, H., Padha, D., & Selwal, A. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-981-99-0047-3_58 · Cited by 1
Complementary Shifted Transformer for Image Captioning
Liu, Y., Yang, Y., Xiang, R., & Ma, J. (2023) · Neural Processing Letters · doi:10.1007/s11063-023-11314-0 · Cited by 1
“We compare with the top-performing models include SCST [45], Up-Down [3], GCN-LSTM [46], ORT [7], ETA [61], AoANet [47], NGSAN [8], M 2 [14], GET [62], X-Transformer [21], TCIC [48], RSTNet [9], DLCT [27], GAT [49], VPNet [50], A 2 [22], MDSAN [40].”
MethodNovel Object Captioning with Semantic Match from External Knowledge
Du, S., Zhu, H., Lin, G., Wang, D. Y., & Shi, J. (2023) · Applied Sciences · doi:10.3390/app13137868 · Cited by 1
Folk Games Image Captioning using Object Attention
Akbar, S., Sitohang, B., Pardede, J., Amal, I. I., Yunastrian, K., Ahmada, M. T., & Prameswari, A. (2023) · Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) · doi:10.29207/resti.v7i4.4708 · Cited by 1
“Image captioning is a process of automatically giving text descriptions to images using computer vision and natural language processing[1], [2].”
BackgroundEMPIRICAL STUDY OF FEATURE EXTRACTION APPROACHES FOR IMAGE CAPTIONING IN VIETNAMESE
Nguyen, K. T. T. M. (2022) · Journal of Computer Science and Cybernetics · doi:10.15625/1813-9663/38/4/17548 · Cited by 1
“After Transformer was born with a powerful Self-Attention mechanism, almost all new methods are based on this architecture, which can be mentioned as AoANet [11], Object Relation Transformer [10], Meshed-Memory Transformer [6], X-Transformer [19], RSTNet [35].”
MethodVideo Captioning Based on Cascaded Attention-Guided Visual Feature Fusion
Chen, S., Li, Y., & Hu, Y. (2023) · Neural Processing Letters · doi:10.1007/s11063-023-11386-y · Cited by 1
“In recent years, Huang et al. [10] and Pan et al. [29] have improved the performance of the attention mechanism by further strengthening the interaction between language and visual features.”
BackgroundImproving Image Captioning with Feature Filtering and Injection
Guo, M.-H., Chen, Q., Fang, X., Bao, J., & Xiang, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44210-0_30 · Cited by 1
End-to-end Image Captioning via Visual Region Aggregation and Dual-level Collaboration
Song, J., Zeng, P., Gu, J., Zhu, J., & Gao, L. (2023) · International Journal of Software and Informatics · doi:10.21655/ijsi.1673-7288.00316 · Cited by 1
RelNet-MAM: Relation Network with Multilevel Attention Mechanism for Image Captioning
Srivastava, S., & Sharma, H. (2023) · Microprocessors and Microsystems · doi:10.1016/j.micpro.2023.104931 · Cited by 1
Towards Arabic Image Captioning: A Transformer-Based Approach
Abdou, M., Adel, M., Nagaty, R., Hatem, A., & Khoriba, G. A. (2023) · Novel Intelligent and Leading Emerging Sciences Conference (NILES) · doi:10.1109/niles59815.2023.10296781 · Cited by 1
“(2019) [5] used MS COCO dataset for image captioning in English.”
MethodMulti-attention TransUNet—A Transformer Approach for Image Description Generation
Mishra, P., Shrivastava, M., Jain, U., Prasad, A. O., & Satapathy, S. C. (2023) · Smart innovation, systems and technologies · doi:10.1007/978-981-99-6702-5_2 · Cited by 1
A study on attention-based deep learning architecture model for image captioning
Fudholi, D. H., Al-Faruq, U. A. A., Nayoan, R. A. N., & Zahra, A. (2023) · IAES International Journal of Artificial Intelligence · doi:10.11591/ijai.v13.i1.pp23-34 · Cited by 1
Choudhury, P., Guha, P., & Nandi, S. (2023) · International Conference on Asian Language Processing · doi:10.1109/ialp61005.2023.10337310 · Cited by 1
“Huang et al. [10] applied attention on attention with multi-head self-attention to enhance the visual features.”
BackgroundMethodTransformer with a Parallel Decoder for Image Captioning
Wei, P., Liu, X., Luo, J., Pu, H., Huang, X., Wang, S., Cao, H., Yang, S., Zhuang, X., Wang, J., et al. (2023) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001423540290 · Cited by 1
Enhancing Medical Report Generation in Multi-Slice Fusion Scenarios
Bu, S., Li, T., & Dai, Z. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385537 · Cited by 1
Lightweight Image Captioning Model Based on Knowledge Distillation
Cui, Z., Tang, Z., Li, J., & Chen, K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-53308-2_23 · Cited by 1
Jaiswal, S., Pallthadka, H., Chinchewadi, R. P., & Jaiswal, T. (2023) · International Journal of Computer Engineering in Research Trends · doi:10.22362/ijcert/2023/v10/i12/v10i123 · Cited by 1
Multi-Source Dynamic Interactive Network Collaborative Reasoning Image Captioning
Su, Q., & Li, Z. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446104 · Cited by 1
“B-1 B-4 M R-L C AoANet [23] 81.7 40.2 29.1 59.3 132.0 SGAE [27] 81.0 39.0 28.4 58.9 129.0 GCN-LSTM [22] 80.9 38.3 28.7 58.5 128.9 X-Transformer [13] Referring to a few models [23, 11], the ensemble model is used to show the performance of the model.”
InfluentialImage Captioning with Reinforcement Learning
Verma, A., Agarwal, S., Arya, K. V., Petrlik, I., Esparza, R., & Rodriguez, C. R. (2023) · IEEE International Conference on Computer Vision and Machine Intelligence (CVMI) · doi:10.1109/cvmi59935.2023.10464514 · Cited by 1
Tag‐inferring and tag‐guided Transformer for image captioning
Yi, Y., Liang, Y., Kong, D., Tang, Z., & Peng, J. (2024) · IET Computer Vision · doi:10.1049/cvi2.12280 · Cited by 1
Zhang, S., Zhang, J., Zhang, H., & Li, Z. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-09633-x · Cited by 1
“Several scholars have introduced region-aware attention to generate region-level features by modeling the correlation between object regions [6, 7].”
Geometrically-Aware Dual Transformer Encoding Visual and Textual Features for Image Captioning
Chang, Y., Ma, H.-S., Li, S.-C., & Huang, J.-W. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-2262-4_2 · Cited by 1
Sun, J.-T., & Min, X. (2024) · Signal Image and Video Processing · doi:10.1007/s11760-024-03268-0 · Cited by 1
“By assessing the correlation between attention outcomes, Transformers expand upon the conventional attention mechanisms, thereby elevating their impact [15].”
Multi-Granularity Cross-Attention Network for Visual Question Answering
Wang, Y., Gao, W., Cheng, X., Wang, X., Zhao, H., Xie, Z., & Xu, L. (2023) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom60117.2023.00291 · Cited by 1
“Attention Mechanism Based on the research progress of the attention mechanism in NLP and CV, several studies have introduced the attention mechanism into the field of VQA, which has proved to to be effective for better performance [16], [31].”
BackgroundImage Captioning with Masked Diffusion Model
Tian, W., Xu, W. Z., Zhao, J., & Zhao, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5603-2_18 · Cited by 1
SG-DLCT: Saliency-Guided Dual-Level Collaborative Transformer for Image Captioning
Zhu, X., Deng, J., Zhang, K., & Zhang, J. (2024) · IEEE Data Driven Control and Learning Systems Conference (DDCLS) · doi:10.1109/ddcls61622.2024.10606918 · Cited by 1
“We compare our method with various state-of-the-art approaches, including MT[24], AOA[23], ORT[10], M2-Transformer[11], X-transformer[22], RSTNet[6] and DLCT[7].”
MethodHIST: Hierarchical and sequential transformer for image captioning
Lv, F., Wang, R., Jing, L., & Dai, P. (2024) · IET Computer Vision · doi:10.1049/cvi2.12305 · Cited by 2
Zero-Shot Image Caption Inference System Based on Pretrained Models
Zhang, X., Shen, J., Wang, Y., Xiao, J., & Li, J. (2024) · Electronics · doi:10.3390/electronics13193854 · Cited by 1
Multi-view Attention Networks for Visual Question Answering
Li, M., Bai, Z., & Deng, J. (2024) · doi:10.1109/icnlp60986.2024.10692598 · Cited by 1
“Inspired by the tasks including image captioning [5], visual question answering [3], and object detection [6], we design a Multi-view attention model to gradually pay attention to the most important regions.”
BackgroundMethodImage Captioning Based on Semantic Scenes
Zhao, F., Yu, Z., Wang, T., & Lv, Y. (2024) · Entropy · doi:10.3390/e26100876 · Cited by 1
Engaging Live Video Comments Generation
Luo, G., Ma, Y., Zhang, M., Huang, J., Li, S., Qian, Z., & Zhang, X. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681195 · Cited by 1
“Relevant studies include image and video captioning[1, 4, 5, 17], multimodal dialogue generation[2, 15], image and video question-answering[7, 9, 23], and so forth.”
M-RAT: a Multi-grained Retrieval Augmentation Transformer for Image Captioning
Song, J., Pan, R., Zhou, J., & Yang, H. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0908-6_11 · Cited by 1
“(1) Encoder-decoder Transformer-based methods [4, 7, 18, 37, 51] consist of a two-stage learning process.”
Enriching visual feature representations for vision–language tasks using spectral transforms
Ondeng, O., Absaloms, H. O., & Akuon, P. O. (2024) · Image and Vision Computing · doi:10.1016/j.imavis.2024.105390 · Cited by 1
Disentangling Fine-Tuning from Pre-Training in Visual Captioning with Hybrid Markov Logic
Shah, M., Sarkhel, S., & Venugopal, D. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825003 · Cited by 1
“We evaluate both inference methods on the MSCOCO dataset where we evaluate varied types of captioning models such as SGAE [43], transformer-based models [9, 21, 30] and BLIP2 (a Visual Large Language Model (VLLM) based model) [25].”
Supporting vision-language model few-shot inference with confounder-pruned knowledge prompt
Li, J., Mo, W., Song, F., Sun, C., Qiang, W., Su, B., & Zheng, C. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107173 · Cited by 9
CAVI: A Contrastive Image Captioning Framework with the Contributions of All Visual Information
Shang, Z., An, G., & Cao, S. (2024) · International Conference on Signal Processing · doi:10.1109/icsp62129.2024.10846256 · Cited by 1
“Examples include AoANet[13] which establishes self-attention or cross attention mechanisms between vision and language,Meshed-memory Transformer[9] which enables interaction between multi-level visual and language features,and COSNet[14] which infers the missing language semantics from the image.”
Incorporating Transformer-based Scene Graphs Progressively for Image Captioning
Zhang, L., An, G., & Cao, S. (2024) · International Conference on Signal Processing · doi:10.1109/icsp62129.2024.10846478 · Cited by 1
Yang, X., & Chen, Z. (2024) · China Automation Congress (CAC) · doi:10.1109/cac63892.2024.10865664 · Cited by 1
Li, J., Yang, B., Liu, J., Amevor, F. K., Guo, Y., Zhou, Y., Deng, Q., & Zhao, X. (2025) · Animals · doi:10.3390/ani15111546 · Cited by 1
Wu, Y., Wu, J., Yang, H., Zhou, Y., Xie, Y., & Chen, Z. (2025) · International Conference on Software Engineering and Computer Science (CSECS) · doi:10.1109/csecs64665.2025.11009656 · Cited by 1
Shifted Window Fourier Transform and Retention for Image Captioning
Hu, J. C., Cavicchioli, R., & Capotondi, A. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-6596-9_12 · Cited by 2
Enhanced group relation learning via aligned attention masking for fashion product captioning
Tang, Y., Ye, D., Tao, F., & Du, G. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00195-z · Cited by 3
“Second, architectural refinements for multi-level feature processing have gained traction (Huang et al. 2019; Cornia et al. 2020; Kuo and Kira 2022; Hu et al. 2023).”
OPCap:Object-aware Prompting Captioning
Huang, F., Cao, Y., & Zhong, J. (2025) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3743093.3770956 · Cited by 1
“• AoA [13] : Introduces a dual-attention mechanism, adding a second attention module on top of the first to refine the attention weights and capture the relationship between image and text more precisely.”
Retrieval-based objects and relations prompt for image captioning
Gu, J., Qin, T., Pu, Y., & Zhao, Z. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114518 · Cited by 1
Exploiting Knowledge Embedding to Improve the Description for Image Captioning
Song, D., Peng, C., Yang, H., & Liao, L. (2021) · Communications in computer and information science · doi:10.1007/978-981-16-1964-9_25 · Cited by 1
Descriptive Image Captioning with Salient Retrieval Priors
Shi, Z., Liu, H., & Zhu, X. (2021) · doi:10.21428/594757db.898c7976 · Cited by 1
“AoANet [28] adopts attention on attention module, which achieves superior performance over automatic evaluation metrics.”
MethodRelational Attention with Textual Enhanced Transformer for Image Captioning
Song, L., Shi, Y., Xiao, X., Zhang, C., & Xiang, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88010-1_13 · Cited by 1
HMER-Image To LaTeX : A Variational Dropout Approach
Garkal, A., Pal, A., & Singh, K. P. (2021) · IEEE Conference on Information and Communication Technology (CICT) · doi:10.1109/cict53865.2020.9672359 · Cited by 1
Feature Enhancement Method for Text Information Generation of Remote Sensing Images
Lin, Q., Li, S., Xia, L., & Shan, Y. (2022) · International Conference on Intelligent Control, Measurement and Signal Processing (ICMSP) · doi:10.1109/icmsp55950.2022.9858985 · Cited by 1
“[7] point out that in the self-attention mechanism commonly used at present, the model would return a weight matrix for each query, no matter whether there is a relevant region or not, which might lead to errors.”
BackgroundTri-RAT: optimizing the attention scores for image captioning
Yang, Y., An, Y., Hu, J., & Pan, L. (2022) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-022-00260-7 · Cited by 1
Asymmetry-aware bilinear pooling in multi-modal data for head pose estimation
Chen, J., Li, Q., Ren, D., Cao, H., & Ling, H. (2022) · Signal Processing Image Communication · doi:10.1016/j.image.2022.116895 · Cited by 1
Self-citationImage captioning based on an improved attention mechanism
Liang, L., & Xiang, N. (2023) · International Conference on Artificial Intelligence, Automation and High Performance Computing (AIAHPC) · doi:10.1117/12.2685386 · Cited by 1
“IDA concatenates query and attention result, which is similar to AoANet [11] .”
MethodResultDeep Gaussian Denoiser Epistemic Uncertainty and Decoupled Dual-Attention FusionPreprint
Ma, X., Lin, X., Helou, M. E., & Susstrunk, S. E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.04631 · Cited by 1
Denoising Large-Scale Image Captioning from Alt-text Data using Content Selection ModelsPreprint
Chandu, K. R., Sharma, P. K., Changpinyo, S., Thapliyal, A., & Soricut, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.05175 · Cited by 3
“, 2017) following recent state-of-the-art approaches (Sharma et al., 2018; Yu et al., 2019; Changpinyo et al., 2019; Huang et al., 2019; Cornia et al., 2020).”
Goal-driven text descriptions for imagesPreprint
Luo, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.12575 · Cited by 4
Generating Radiology Report Via Auxiliary Signal Guidance and Memory-Driven NetworkPreprint
Xue, Y., Tan, Y., Tan, L., Qin, J., & Xiang, X. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4149520 · Cited by 1
CATANIC: Automatic generation model of image captions based on multiple attention mechanismPreprint
Zhang, T., Zhang, T., Zhuo, Y., & Ma, F. (2023) · Research Square · doi:10.21203/rs.3.rs-2718040/v1 · Cited by 1
Improving image captioning methods using machine learning approaches
Atliha, V. (2023) · doi:10.20334/2023-021-m · Cited by 1
Metadata
- Authors
- Huang, L., Wang, W., Chen, J., & Wei, X.-Y.
- Affiliations
- Peking University (CN)
- Macau University of Science and Technology (MO)
- Peng Cheng Laboratory (CN)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 4633-4642
- OpenAlex
- W2986670728
- Semantic Scholar
- 4c163d4942117179d3e97182e1b280027d7d60a9
- Cited by
- 1,092
- References
- 54
- Citations by year
- 2026: 52 · 2025: 142 · 2024: 197 · 2023: 228 · 2022: 212 · 2021: 188 · 2020: 70 · 2019: 2
- FWCI
- 38.47
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (U-1613209, 61972217, 61872256)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- image captioning, attention mechanism, attention gate, encoder-decoder architecture, MS COCO
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
