The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Attention on Attention for Image Captioning
Open accessAbstract
Attention mechanisms are widely used in current encoder/decoder frameworks of image captioning, where a weighted average on encoded vectors is generated at each time step to guide the caption decoding process. However, the decoder has little idea of whether or how well the attended vector and the given attention query are related, which could make the decoder give misled results. In this paper, we propose an Attention on Attention (AoA) module, which extends the conventional attention mechanisms to determine the relevance between attention results and queries. AoA first generates an information vector and an attention gate using the attention result and the current context, then adds another attention by applying element-wise multiplication to them and finally obtains the attended information, the expected useful knowledge. We apply AoA to both the encoder and the decoder of our image captioning model, which we name as AoA Network (AoANet). Experiments show that AoANet outperforms all previously published methods and achieves a new state-of-the-art performance of 129.8 CIDEr-D score on MS COCO Karpathy offline test split and 129.6 CIDEr-D (C40) score on the official online testing server. Code is available at https://github.com/husthuaan/AoANet.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“DropToken [174] reduces the training complexity via random dropping a portion of the video and audio tokens from input sequence during training.”
From Multimodal learning with transformers: A survey (2023) · cites this paper as [174]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Cited by
Sorted by citations
Items 901–1,000 of 1,002
Explainable Artificial Intelligence approaches for Image Captioning
Elguendouze, S. (2024) · theses.fr (ABES) · doi:10.70675/c9dab409zaeaaz436fz95cdz57fca46aa107 · Cited by 1
Learning to Caption Images with Two-Stream Attention and Sentence Auto-EncoderPreprint
Goel, A., Fernando, B., Nguyen, T.-S., & Bilen, H. (2019) · arXiv (Cornell University)
Recurrent Nested Model for Sequence Generation
Jiang, W., Ma, L., & Lu, W. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6768
T-EMDE: Sketching-based global similarity for cross-modal retrievalPreprint
Rychalska, B., Wieczorek, M., & Dabrowski, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.04242
“in database retrieval tasks of photos via their textual descriptions [14, 18, 33], image captioning [22, 24, 57], and multimodal neural machine translation [25, 36].”
MethodContrastive Semantic Similarity Learning for Image Captioning Evaluation with Intrinsic Auto-encoderPreprint
Zeng, C., Zhao, T., & Kwong, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.15312
“Huang et al. [15] build attention on attention framework and also achieve great performances.”
BackgroundExplain and Improve: LRP-Inference Fine-Tuning for Image Captioning ModelsPreprint
Sun, J., Lapuschkin, S., Samek, W., & Binder, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01037
“We will analyze the adaptive attention mechanism [7], [8], [9], [10] and the multi-head attention mechanism [11], [36], [12], [13], [14], [15].”
BackgroundMethodInfluentialIncorporating additional knowledge into image captioners
Yang, X. (2021) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/151726
Policy and Value Deep RL for Temporal Language-Agnostic Street Image Captioning
Chan, C. C. K., Chen, C.-C., Delaney, S., & Yang, C. (2021) · IEEE Eurasia Conference on Biomedical Engineering, Healthcare and Sustainability (ECBIOS) · doi:10.1109/ecbios51820.2021.9510291
“Popular image captioning works include [1–7] to mention a few, which primarily focus on encoder-decoder convolutional and recurrent frameworks coupled with training large corpora.”
BackgroundAnalysis of the Fuzziness of Image Caption Generation Models due to Data Augmentation Techniques
Reddy, K. A., Satish, C., Polsani, J., Chintapalli, T. N., & Ananya, G. S. (2021) · International Journal of Recent Technology and Engineering (IJRTE) · doi:10.35940/ijrte.c6439.0910321
“Many image captioning models [5,6] tend to have a two stage approach at solving this problem.”
BackgroundLooking Back and Forward: Enhancing Image Captioning with Global Semantic Guidance
Feng, C., Tang, J., Li, S., Li, X., & Wang, T. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533418
“However, when analyzing outputs of existing models [2], [12], we observe that: the pre-generated captions are not always correct, and some captions suffer from non-fluency and non-consistency problems.”
BackgroundMethodInfluentialImage and Video Captioning Using Deep Architectures
Francis, D., & Huet, B. (2012) · Springer eBooks · doi:10.1007/978-3-030-74478-6_7 · Cited by 1
Adversarial Training for Image Captioning Incorporating Relation Attention
Chen, T., Li, Z., Zhang, C., & Ma, H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-89188-6_15
Velastegui, R., Flores, F., Velastegui, D., & Fenellós, C. B. (2021) · Innovaciones docentes en tiempos de pandemia. · doi:10.26754/cinaic.2021.0083
CIDEr-R: Robust Consensus-based Image Description EvaluationPreprint
dos Santos, G. O., Colombini, E. L., & Avila, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.13701 · Cited by 3
Attention on Attention for Text to Image Synthesis using Mode-Seeking Loss Function
Bhise, N., Krzyżak, A., & Bui, T. D. (2021) · Spectrum Research Repository (Concordia University)
Graph Structural Attention and Increased Global Attention for Image Captioning
Zheng, T., Qian, W., Nie, R., Cao, J., & Xu, D. (2021) · International Conference on Intelligent Control and Information Processing · doi:10.1109/icicip53388.2021.9642211
DSAMT: Dual-Source Aligned Multimodal Transformers for TextCaps
Liao, C., Liu, R., & Gao, S. (2021) · IEEE International Conference on Network Infrastructure and Digital Content (IC-NIDC) · doi:10.1109/ic-nidc54101.2021.9660575
Recorrect Net: Visual Guidance for Image Captioning
Guo, Q., Xu, Y., & Gao, S. (2021) · IEEE International Conference on Network Infrastructure and Digital Content (IC-NIDC) · doi:10.1109/ic-nidc54101.2021.9660494
Evaluación de atención selectiva en niños de 7 a 8 años
Hernández, D. C. V., Hernández, R. E. V., Albán, J. R. A., & Hernández, R. S. V. (2022) · ConcienciaDigital · doi:10.33262/concienciadigital.v5i1.1.2005 · Cited by 2
Image Retrieval with Text Feedback based on Transformer Deep Model
Huynh, T. L.-P., & Ly, N. Q. (2021) · NAFOSTED Conference on Information and Computer Science (NICS) · doi:10.1109/nics54270.2021.9701539
An Augmented Embedding Spaces approach for Text-based Image Captioning
Bui, D. C., Trinh, T., Vo, N. D., & Nguyen, K. T. T. M. (2021) · NAFOSTED Conference on Information and Computer Science (NICS) · doi:10.1109/nics54270.2021.9701576
Critical Analysis of Deconfounded Pretraining to Improve Visio-Linguistic Models
Cornille, N., Laenen, K., & Moens, M.-F. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.736791 · Cited by 1
Diversity Enhanced Image Captioning with Adversarial Learning
Ding, Q., Wang, Q., Jin, L., & Shen, R. (2021) · 2021 2nd International Conference on Electronics, Communications and Information Technology (CECIT) · doi:10.1109/cecit53797.2021.00098
Qin, C., Gong, W., & Li, X. (2022) · Mathematical Problems in Engineering · doi:10.1155/2022/2315341
“[15] proposed the AoA (Attention on Attention) module, which expands the conventional attention mechanism by adding an additional attention to achieve better results in the image text description.”
BackgroundText Augmentation for Compressed Image Captioning Models
Atliha, V., & Šešok, D. (2022) · IEEE Open Conference of Electrical, Electronic and Information Sciences (eStream) · doi:10.1109/estream56157.2022.9781641 · Cited by 1
ZoDIAC: Zoneout Dropout Injection Attention CalculationPreprint
Zohourianshahzadi, Z., & Kalita, J. K. (2022) · Research Square · doi:10.21203/rs.3.rs-1798795/v1
Transformer-based image captioning by leveraging sentence information
Chahkandi, V., Fadaeieslam, M. J., & Yaghmaee, F. (2022) · Journal of Electronic Imaging · doi:10.1117/1.jei.31.4.043005
“30,33,35 3 Approach In this section, we have presented our encoder – decoder architecture to overcome the main draw-back of nonautoregressive models, which is a big gap in generated sentence quality compared to its autoregressive counterpart.”
BackgroundImage Captioning with X-Gelu Activated Xgl TransformerPreprint
Sharma, D., Dhiman, C., & Kumar, D. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4127610
A Sub-captions Semantic-Guided Network for Image Captioning
Tian, W., Zhu, J., Wu, S., Zhao, Z., Zhang, Y.-Z., & Zhang, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-13832-4_31
Relation-Aware Global-Augmented Transformer for TextCaps
Li, Q., Li, B., & Ma, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-15919-0_62
CSTNET: Enhancing Global-To-Local Interactions for Image Captioning
Yang, X. Y., Wang, Y., Chen, H., & Li, J. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897810 · Cited by 1
“We compare our model with previous works that include Up-down [13], RFNet [14], GCN-LSTM [15], ORT [16], AOANet [17] and M 2 [4].”
ResultHybrid-Spatial Transformer for Image Captioning
Zheng, J., & Pun, C. (2022) · International Conference on Control and Computer Vision (ICCCV) · doi:10.1145/3561613.3561617
BiTMulV: Bidirectional-Decoding Based Transformer with Multi-view Visual Representation
Yu, Q., Wang, X., Wang, D., Chu, X. Y., Liu, B., & Liu, P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-18907-4_57
Refining AttnGAN Using Attention on Attention Network
Bhise, N., Krzyżak, A., & Bui, T. D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-23028-8_29
From Less to More: Common-Sense Semantic Perception Benefits Image Captioning
Chen, F., Li, X., Tang, J., Li, S., & Wang, T. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25198-6_27
Research on Image Description Generation Method Based on Visual Sentinel
Qiao, P., Shen, R., & Li, Y. (2023) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-20738-9_70
Sequential Memory Modelling for Video Captioning
Puttaraja, P., Nayaka, C., Manikesh, M., Sharma, N., & Kumar, M. A. (2022) · IEEE India Conference · doi:10.1109/indicon56171.2022.10039829
A Survey on Causal Inference in Image Captioning
Kim, J., & Kim, J. (2023) · International Conference on Electronics, Information, and Communication (ICEIC) · doi:10.1109/iceic57457.2023.10049869
“There have been countless research in IC, having a long history [10]–[13].”
BackgroundResearch on Data Augmentation Strategy Methods for Image Caption
Lin, N., Li, S., Han, Y., & Liu, M. (2022) · International Conference on Electronic Information Technology and Computer Engineering (EITCE) · doi:10.1145/3573428.3573673
“Due to the large number of images in the dataset processed with the data augmentation strategy, this paper uses the ResNext-101 model to extract image features directly and train on the AoA [14] model.”
MethodInfluentialGraph Alignment Transformer for More Grounded Image Captioning
Tian, C., Hu, H., & Li, Z. (2022) · International Conference on Industrial IoT, Big Data and Supply Chain (IIoTBDSC) · doi:10.1109/iiotbdsc57192.2022.00028
SATNet: Captioning with Semantic Alignment and Feature Enhancement
Bai, W., Zhang, C., Li, Z., Wei, P., & Wang, Z. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-30111-7_38
Effectively Utilizing the Category Labels for Image Captioning
Feng, J., & Zhao, J. (2023) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2022dlp0013
Fine-Grained Features for Image Captioning
Shao, M., Feng, J. M., Wu, J., Zhang, H., & Zheng, Y. (2023) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2023.036564 · Cited by 1
Improving Image Captioning with Control Signal of Sentence Quality
Zhu, Z., Wang, S., & Qu, H. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094586
“2019), significant improvements are made in the field of image captioning (Anderson et al. 2018; Huang et al. 2019; Pan et al. 2020; Lu et al. 2017).”
BackgroundResearch on Image Description Generation Method Based on G-AoANet
Qiao, P., Shen, R., & Li, Y. (2022) · doi:10.1145/3573942.3574072
“[3] proposed the AoANet model in order to generate better descriptions, extending the traditional attention mechanism to measure and determine the correlation between attention results and queries.”
MethodLiu, J., Lin, X. A., & He, L. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191656
Probing Cross-modal Semantics Alignment Capability from the Textual Perspective
Ma, Z., Zong, S., Pan, M., Zhang, J., Huang, S., Dai, X., & Chen, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.421
Self-citationIncorporating Unlikely Negative Cues for Distinctive Image Captioning
Fei, Z., & Huang, J. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/83
Balanced Overall and Local: Improving Image Captioning with Enhanced Transformer Model
Xian, H., Guo, B., & Zhou, Y. (2023) · International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT) · doi:10.1109/ainit59027.2023.10212804
“In 2019, Huang et al.[10] proposed the Attention on Attention (AoA) Module, which addresses the problem that during the computation of the attention mechanism, normalized weights are generated regardless of whether the query vector is correlated with the value vector or not, leading to the emergence of a problem where…”
BackgroundLearning from the Past: Fast NAS for Tasks and Datasets
Cheung, M. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3618000
“For example, ImageNet, 1 an open dataset collected for image classification, has been used for image segmentation [30], caption generation [18], and image generation [11], and many of these models are available for reuse [25].”
Towards Explainable Navigation and Recounting
Poppi, S., Bigazzi, R., Rawal, N., Cornia, M., Cascianelli, S., Baraldi, L., & Cucchiara, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43148-7_15
“The language model can be implemented as a recurrent neural network [19, 7, 18] or Transformer-based fully-attentive models [15, 14, 32].”
MethodGenerating Radiology Reports via Multi-feature Optimization Transformer
Wang, R., & Hua, R. (2023) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2023.10.010 · Cited by 1
Multilayer Vision and Language Augmented Transformer for Image Captioning
Su, Q., & Li, Z. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-7022-3_19
Fine-Grained Attribute-Object Feature Representation in Compositional Zero-Shot Learning
Shabbir, N., Rout, R. K., Umer, S., & Mohanta, P. P. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-45170-6_17
Image Caption Generator for Sinhala Using Deep Learning
Ranasinghe, D., Pushpananda, R., & Weerasinghe, R. (2023) · International Journal on Advances in ICT for Emerging Regions (ICTer) · doi:10.4038/icter.v16i2.7266
“Huang et al. [24] proposed the ‘Attention on Attention (AoA)’ module to extend the conventional attention mechanism to identify the relevance of the query and the attention result.”
BackgroundResultAdaptive Multi-granularity Aggregation Transformer for Image Captioning
Li, D., Wang, Y., & Liu, Q. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-50959-9_24
Feature extraction semilearning and augmented representation for image captioning in crowd scenes
Nguyen, K. T. T. M. (2023) · Science and Technology Development Journal · doi:10.32508/stdj.v26i4.4028
Research on Image Semantic Description Method Based on RVC Network
Zhou, H. (2023) · IEEE International Conference on Automation, Electronics and Electrical Engineering (AUTEEE) · doi:10.1109/auteee60196.2023.10407719
“Huang et al. [5] combined the Transformer model with a dual-attention mechanism to greatly improve the accuracy of descriptive sentences.”
MethodImage Captioning with Global Information Enhanced Image Representation
Wang, X., Liu, Y., Liu, H., Guo, D., Zhai, J., & Li, J. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-0068-4_42
Enhancing Image Descriptions with Image Transformers: A Journey into Advanced Image Captioning
Jaiswal, S., Pallthadka, H., Chinchewadi, R. P., & Jaiswal, T. (2023) · International Journal of Computer Engineering in Research Trends · doi:10.22362/ijcert/2023/v10/i12/v10i122
(2024) · International Research Journal of Modernization in Engineering Technology and Science · doi:10.56726/irjmets49310
Zhao, H., Huang, Y., Zhao, K., & Wang, S. (2024) · Applied and Computational Engineering · doi:10.54254/2755-2721/44/20230093
“Most of the attention mechanisms are focused attention which has been applied to various fields, such as image-based analysis [8, 9], text classification [10, 11], video classification [12], image captioning [13], and recommendation [14, 15, 16].”
BackgroundSemi-Supervised Image Captioning by Adversarially Propagating Labeled DataPreprint
Kim, D.-J., Oh, T.-H., Choi, J., & Kweon, I. S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4757425
GRU-Enhanced Decoding by Lightweight Transformer for Image Captioning
Chaudhary, D., Sharma, K., Sharma, V., & Jain, P. (2024) · International Conference on Cloud Computing, Data Science & Engineering (Confluence) · doi:10.1109/confluence60223.2024.10463460
Performance and Cost Balancing in Vision Transformer-Based Image Captioning
Lyu, Y., Liu, Y., & Zhao, Q. (2023) · Asia Service Sciences and Software Engineering Conference (ASSE) · doi:10.1145/3634814.3634842
Wu, B., & Wo, Y. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01309-9
“( 2) Attention-based image captioning model: Up-Down [2], MAD-SAP [48], AoANet [3], X-LAN [18], DCA [51].”
InfluentialPositional Feature Generator-Based Transformer for Image Captioning
He, S., Yang, X., Ma, S., Song, B., He, Z., & Luo, W. (2023) · International Conference on Intelligent Systems and Knowledge Engineering (ISKE) · doi:10.1109/iske60036.2023.10481500
“…to make a comparison with some advanced image captioning algorithms like LSTM-A [34], RFNet [35], Up-Down [36], GCN-L [37], LBPF [38], SGAE [39], AoANet [40], X-T [42], GAT [41], CITE [43], TD-V [44], MPRE [45] and RA [46], we also evaluate our model on the ’Karpathy’ test split of MS COCO2014.”
MethodDSCJA-Captioner: Dual-Branch Spatial and Channel Joint Attention for Image Captioning
Tian, X., Yang, X., Ma, S., Song, B., & He, Z. (2023) · International Conference on Intelligent Systems and Knowledge Engineering (ISKE) · doi:10.1109/iske60036.2023.10481319
“In the last few years, attention has been a focus of research in multi-modal reasoning [3]–[6], which serves as a central step in capturing the correlation between two modalities.”
BackgroundBenefit from AMR: Image Captioning with Explicit Relations and Endogenous Knowledge
Chen, F., Li, X., Tang, J., Li, S., & Wang, T. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-2390-4_25
Analysis and reconstruction of noise probability density functions in images
Khatamov, O., & Тавбоев, С. А. (2024) · AIP conference proceedings · doi:10.1063/5.0210509
Kilimci, Z. H., & Küçükmanіşa, A. (2024) · Gazi Üniversitesi Mühendislik-Mimarlık Fakültesi Dergisi · doi:10.17341/gazimmfd.1245400
The Current Landscape of Audio Description Research: A Survey
Rajiv, L. G. A. (2023) · International Journal of Science and Research (IJSR) · doi:10.21275/sr23209131618
Learn to Prompt: Soft-Prompting Adaptive Attention for Image Captioning
Cui, M., & Cui, M. (2024) · International Conference on Information and Computer Technologies (ICICT) · doi:10.1109/icict62343.2024.00070
“Huang et al. [7] presented Attention on Attention, a study that added another layer of attention on top of the attention mechanism.”
BackgroundMethodAudio Description of Videos Using Machine Learning
Roy, R., Manepalli, S., Rajan, S., Mishra, S., & Sampatrao, G. S. (2024) · IEEE International Conference for Convergence in Technology (I2CT) · doi:10.1109/i2ct61223.2024.10544216
“This strategy improves decoding accuracy, exceeding previous methods with a cutting-edge CIDEr-D score of 129.8 on the MS COCO Karpathy offline test split [10].”
MethodInfluentialA Systematic Survey of Automatic Image Description Generation Systems
Sreela, S. R., & Idicula, S. M. (2024) · International Journal of Image and Graphics · doi:10.1142/s0219467826500026
Dual Transformer with Gated-Attention Fusion for News Disaster Image Captioning
Li, Y., Zhu, Y., Zhang, Y., & Yang, C. (2024) · Communications in computer and information science · doi:10.1007/978-981-97-3623-2_15
Self-correcting text-aware strategy for text-caption
Lian, J., Cao, X., & Dai, P. (2024) · International Conference on Remote Sensing, Mapping, and Image Processing (RSMIP) · doi:10.1117/12.3029640
Image Captioning with Visual Positional Embedding and Bi-linear Pooling
Nair, S., & Guha, P. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-58181-6_35
Liu, Z., Huo, J., & Hu, W. (2024) · Youth Academic Annual Conference of Chinese Association of Automation (YAC) · doi:10.1109/yac63405.2024.10598716
Improving Image Captioning with Image Concepts of Words
Wang, Y., Xiang, X., Jing, K., Xu, J., & Sun, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5495-3_27
Image Captioning untuk Gambar Rambu Lalu Lintas Indonesia Menggunakan Pretrained CNN dan Transformer
Aprilia, N. P., & Rochadiani, T. H. (2024) · Indonesian Journal of Computer Science · doi:10.33022/ijcs.v13i3.4012
Fusion of Computer Vision and Natural Language Processing: Automated Image Caption Generation
Akku, C. P., Cheemalwar, S., Chamakura, V., Gogu, S. S., Swaraja, K., & Vignesh, N. A. (2024) · International Conference on Intelligent Technologies (CONIT) · doi:10.1109/conit61985.2024.10627348
Cui, Z. (2024) · Smart innovation, systems and technologies · doi:10.1007/978-981-97-2144-3_22
Bypass network for semantics driven image paragraph captioning
Zheng, Q., Wang, C., & Wang, D. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104154
Image Caption Method from Coarse to Fine Based On Dual Encoder-Decoder Framework
Li, Z., Liu, Y., & Song, Y. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650584
Enhance Training Objectives for Image Captioning with Decomposed Sequence-level Metric
Wang, Y., Long, J., Hu, J. X., & Shang, L. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650884
“Transformers [19] have also been introduced into image caption tasks and achieves new state-of-the-arts [5]–[8], [10], [11].”
BackgroundMethodScale-aware Multi-head Information Aggregation for Image Captioning
Jia, A., & Zhang, X. (2024) · Chinese Control Conference (CCC) · doi:10.23919/ccc63176.2024.10661680
“Comparison meth-ods for offline evaluation include: RNN based methods SCST [38], Up-Down [4], STMA [27], GCN-LSTM [39], SGAE [41], AoANet [26], and Transformer based methods ORT [8], M2 [7], NG-SAN [9], X-Transformer [40], RST-Net [12], DIFNet [13].”
MethodUTStyleCap4K: Generating Image Captions with Sentimental Styles
Zhang, C., Li, T., & Yamasaki, T. (2024) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2024edp7036
“While they achieve good results, their performance is significantly lower than traditional image captioning models [2], [11] on conventional datasets like MSCOCO [12] in terms of evaluation metric scores.”
Image Captioning With Sentiment for Indonesian
Khumaeni, & Khodra, M. L. (2024) · International Conference on Artificial Intelligence, Blockchain, Cloud Computing, and Data Analytics (ICoABCD) · doi:10.1109/icoabcd63526.2024.10704319
“…Cornia et al. [24] and Huang et al. [25] have shown that using Multihead Attention in captioning models significantly enhances the ability to generate captions that are not only accurate and descriptive but also sentimentally appropriate, which is crucial for applications like e-commerce [24] [25].”
MethodDMR$$^2$$G: diffusion model for radiology report generation
Ouyang, H., Chang, Z., Tang, B., & Li, S. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20206-x · Cited by 1
“Lateron,attention-basedmethodshavebeenproposedtocapturemultimodalalignment[29– 31]andperformobject-relationalreasoning[32,33].”
A Comparative Analysis of Attention Mechanism in RNN-LSTMs for Improved Image Captioning Performance
Mirza, M., & Siddiqui, M. T. (2024) · International Journal of Innovative Science and Research Technology (IJISRT) · doi:10.38124/ijisrt/ijisrt24oct678
Multi-Branch Information Network for Advanced Image Captioning
Song, C. (2024) · International Conference on Artificial Intelligence, Internet of Things and Cloud Computing Technology (AIoTC) · doi:10.1109/aiotc63215.2024.10748270
“These methods include GCN-LSTM [11], AoANet [12], M 2 transformer [13], DLCT [14], RSTNet [15], CBTIC [16], EARL [7].”
BackgroundMethodDual Branch Non-Autoregressive Image Captioning
Liu, Y., Yu, H., Li, H., Han, X., & Liu, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78456-9_21
Bengali Image Caption Generation using Attention Mechanism
De, S., Das, R., & Patel, A. S. (2024) · IEEE International Conference on Computer Vision and Machine Intelligence (CVMI) · doi:10.1109/cvmi61877.2024.10781602
Cominelli, L., Galatolo, F. A., Cimino, M. G. C. A., Sirigu, G., Jiang, Y., & Scilingo, E. P. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5062470
Enhancing Cost-Efficient Image Captioning Through ExpansionNet v2 Optimization
Vo, A.-K., Nguyen, M.-T., & Huynh‐The, T. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-76197-3_9
Bimodality Approach to Mental Disorder Diagnosis via Dual-pathway Fusion Perceptual Network
Xu, R., Gan, S., Zhao, F., Zuo, Z., & Zhuo, T. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10821891
“Attention modules can improve interpretability by focusing on important brain areas while suppressing unnecessary ones [18], [19], [20].”
Raju, S. M. T. U., Mohammad, A. R., Islam, M. M., & Karray, F. O. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.10831147
Metadata
- Authors
- Huang, L., Wang, W., Chen, J., & Wei, X.-Y.
- Affiliations
- Peking University (CN)
- Macau University of Science and Technology (MO)
- Peng Cheng Laboratory (CN)
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 4633-4642
- OpenAlex
- W2986670728
- Semantic Scholar
- 4c163d4942117179d3e97182e1b280027d7d60a9
- Cited by
- 1,092
- References
- 54
- Citations by year
- 2026: 52 · 2025: 142 · 2024: 197 · 2023: 228 · 2022: 212 · 2021: 188 · 2020: 70 · 2019: 2
- FWCI
- 38.47
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (U-1613209, 61972217, 61872256)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- image captioning, attention mechanism, attention gate, encoder-decoder architecture, MS COCO
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
