The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Task-Adaptive Attention for Image Captioning
Abstract
Attention mechanisms are now widely used in image captioning models. However, most attention models only focus on visual features. When generating syntax related words, little visual information is needed. In this case, these attention models could mislead the word generation. In this paper, we propose Task-Adaptive Attention module for image captioning, which can alleviate this misleading problem and learn implicit non-visual clues which can be helpful for the generation of non-visual words. We further introduce a diversity regularization to enhance the expression ability of the Task-Adaptive Attention module. Extensive experiments on the MSCOCO captioning dataset demonstrate that by plugging our Task-Adaptive Attention module into a vanilla Transformer-based image captioning model, performance improvement can be achieved.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107
Self-citationBottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S., & Zhang, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00636 · Cited by 4,948
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Self-Critical Sequence Training for Image Captioning
Rennie, S. J., Marcheret, E., Mroueh, Y., Ross, J., & Goel, V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.131 · Cited by 2,112
Sequence Level Training with Recurrent Neural Networks
Ranzato, M., Chopra, S., Auli, M., & Zaremba, W. (2015) · arXiv · doi:10.48550/arxiv.1511.06732 · Cited by 1,426
Boosting Image Captioning with Attributes
Yao, T., Pan, Y., Li, Y., Qiu, Z., & Mei, T. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.524 · Cited by 733
Video Captioning With Attention-Based LSTM and Semantic Consistency
Gao, L., Guo, Z., Zhang, H., Xu, X., & Shen, H. T. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2729019 · Cited by 637
Deep Multi-View Enhancement Hashing for Image Retrieval
Yan, C., Gong, B., Wei, Y., & Gao, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2975798 · Cited by 504
Self-citationMultimodal Transformer With Multi-View Visual Representation for Image Captioning
Yu, J., Li, J., Yu, Z., & Huang, Q. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2947482 · Cited by 468
Guiding the Long-Short Term Memory Model for Image Caption Generation
Jia, X., Gavves, E., Fernando, B., & Tuytelaars, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.277 · Cited by 426
Deep Learning and Convolutional Neural Networks for Medical Image Computing
(2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-42999-1 · Cited by 289
Recurrent Fusion Network for Image Captioning
Jiang, W., Ma, L., Jiang, Y.-G., Liu, W., & Zhang, T. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01216-8_31 · Cited by 286
From Deterministic to Generative: Multimodal Stochastic RNNs for Video Captioning
Song, J., Guo, Y., Gao, L., Li, X., Hanjalic, A., & Shen, H. T. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2851077 · Cited by 259
Hierarchical LSTMs with Adaptive Attention for Visual Captioning
Gao, L., Li, X., Song, J., & Shen, H. T. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2894139 · Cited by 245
Multi-Scale 3D Convolution Network for Video Based Person Re-Identification
Li, J., Zhang, S., & Huang, T. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018618 · Cited by 183
An Empirical Study of Language CNN for Image Captioning
Gu, J., Wang, G., Cai, J., & Chen, T. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.138 · Cited by 175
Learning visual relationship and context-aware attention for image captioning
Wang, J., Wang, W., Wang, L., Wang, Z., Feng, D. D., & Tan, T. N. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.107075 · Cited by 165
Pose-Guided Representation Learning for Person Re-Identification
Li, J., Zhang, S. S., Tian, Q., Wang, M., & Gao, W. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929036 · Cited by 111
Learning to Collocate Neural Modules for Image Captioning
Yang, X., Zhang, H., & Cai, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00435 · Cited by 102
Multi-Objective Matrix Normalization for Fine-Grained Visual Recognition
Min, S., Yao, H., Xie, H., Zha, Z.-J., & Zhang, Y. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2977457 · Cited by 95
Zhou, L., Xu, C., Koch, P. A., & Corso, J. J. (2017) · Thematic Workshops of ACM Multimedia · doi:10.1145/3126686.3126717 · Cited by 73
Viewpoint and Scale Consistency Reinforcement for UAV Vehicle Re-Identification
Teng, S., Zhang, S. S., Huang, Q., & Sebe, N. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01402-2 · Cited by 69
Bidirectional Attention-Recognition Model for Fine-Grained Object Classification
Liu, C., Xie, H., Zha, Z.-J., Yu, L., Chen, Z., & Zhang, Y. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2954747 · Cited by 65
Self-citationDomain Adaptive Person Re-Identification via Camera Style Generation and Label Propagation
Ren, C.-X., Liang, B.-H., Ge, P., Zhai, Y., & Lei, Z. (2019) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2019.2939750 · Cited by 55
Structure-Aware Deep Learning for Product Image Classification
Chen, Z., Ai, S., & Jia, C. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3231742 · Cited by 46
Self-citationGated Hierarchical Attention for Image Captioning
Wang, Q., & Chan, A. B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20870-7_2 · Cited by 19
Cited by
Gait Recognition in the Wild with Dense 3D Representations and A Benchmark
Zheng, J., Liu, X., Liu, W., He, L., Yan, C., & Mei, T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01959 · Cited by 260
Self-citationKılıç, B., & Aydın, S. (2022) · Neuroinformatics · doi:10.1007/s12021-022-09579-2 · Cited by 112
Metadata
- Authors
- Yan, C., Hao, Y., Li, L., Yin, J., Liu, A., Mao, Z., Chen, Z., & Gao, X.
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2022-01
- Volume
- 32
- Issue
- 1
- Pages
- 43-51
- ISSN
- 1051-8215 (ISSN-L)
- 1558-2205
- OpenAlex
- W3136792391
- Cited by
- 367
- References
- 28
- FWCI
- 20.72
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.