Region-Aware Image Captioning via Interaction Learning
The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Abstract
Image captioning is one of the primary goals in computer vision which aims to automatically generate natural descriptions for images. Intuitively, human visual system can notice some stimulating regions at first glance, and then volitionally focus on interesting objects within the region. For example, to generate a free-form sentence about “boy-catch-baseball”, the visual region involving “boy” and “baseball” could be first attended and then guide the salient object discovery for the word-by-word generation. Till now, previous captioning works mainly rely on the object-wise modeling and ignore the rich regional patterns. To mitigate the drawback, this paper proposes the region-aware interaction learning method, which aims to explicitly capture the semantic correlations in the region and object dimensions for the word inference. First, given an image, we extract a set of regions which contain diverse objects and their relations. Second, we present the spatial-GCN interaction refining structure which can establish the connection between regions and objects to effectively capture contextual information. Third, we design the dual-attention interaction inference procedure, which enables attention to be calculated in region and object dimensions jointly for the word generation. Specifically, the guidance mechanism is proposed to selectively emphasize semantic inter-dependencies from region to object attentions. Extensive experiments on the MSCOCO dataset demonstrate the superiority…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It is not recommended to use the RBN's hidden layers alone to extract features from the data [105-109].”
From Vehicle Detection and Classification via YOLOv8 and Deep Belief Network over Aerial Image Sequences (2023) · cites this paper as [106]
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Cited by
Medical image analysis using deep learning algorithms
Li, M., Jiang, Y., Zhang, Y., & Zhu, H. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1273253 · Cited by 505
Self-citationZhang, B., Shi, H., & Wang, H. (2023) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s410301 · Cited by 347
Robust human locomotion and localization activity recognition over multisensory
Khan, D., Alonazi, M., Abdelhaq, M., Mudawi, N. A., Algarni, A. M., Jalal, A., & Liu, H. (2024) · Frontiers in Physiology · doi:10.3389/fphys.2024.1344887 · Cited by 220
Bashir, A. K., Victor, N., Bhattacharya, S., Huynh-The, T., Chengoden, R., Yenduri, G., Maddikunta, P. K. R., Pham, Q.-V., Gadekallu, T. R., & Liyanage, M. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3304790 · Cited by 183
“D lung object to zoom in and rotate for diagnosis) without touching the screen or button-aided control [51].”
BackgroundYenduri, G., Kaluri, R., Rajput, D. S., Lakshmanna, K., Gadekallu, T. R., Mahmud, M., & Brown, D. J. (2023) · IEEE Access · doi:10.1109/access.2023.3289496 · Cited by 95
Metadata
- Authors
- Liu, A.-A., Zhai, Y., Xu, N., Nie, W., Li, W., & Zhang, Y.
- Affiliations
- Tianjin University (CN)
- University of Science and Technology of China (CN)
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2022-06
- Volume
- 32
- Issue
- 6
- Pages
- 3685-3696
- ISSN
- 1051-8215 (ISSN-L)
- 1558-2205
- OpenAlex
- W3195680250
- Cited by
- 128
- References
- 70
- Citations by year
- 2026: 7 · 2025: 12 · 2024: 23 · 2023: 72 · 2022: 14
- FWCI
- 9.81
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Circuits and Systems for Video Technology (publishedVersion)
- Funders
- National Natural Science Foundation of China (61772359, 62002257)
- Grant of Tianjin New Generation Artificial Intelligence Major Program (19ZXZNGX00110, 18ZXZNGX00150)
- China Postdoctoral Science Foundation (2021M692395)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- image captioning, interaction learning, dual attention mechanism, visual attention, context modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Region-Aware Image Captioning via Interaction Learning. https://app.cochat.ai/papers/10.1109/tcsvt.2021.3107035
- The paper
- Liu, A.-A., Zhai, Y., Xu, N., Nie, W., Li, W., & Zhang, Y. (2022). Region-Aware Image Captioning via Interaction Learning. IEEE Transactions on Circuits and Systems for Video Technology. https://doi.org/10.1109/tcsvt.2021.3107035
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.