The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Retrieval-Augmented Transformer for Image Captioning
Abstract
Image captioning models aim at connecting Vision and Language by providing natural language descriptions of input images. In the past few years, the task has been tackled by learning parametric models and proposing visual feature extraction advancements or by modeling better multi-modal connections. In this paper, we investigate the development of an image captioning approach with a kNN memory, with which knowledge can be retrieved from an external corpus to aid the generation process. Our architecture combines a knowledge retriever based on visual similarities, a differentiable encoder, and a kNN-augmented attention layer to predict tokens based on the past context and on text retrieved from the external memory. Experimental results, conducted on the COCO dataset, demonstrate that employing an explicit external memory can aid the generation process and increase caption quality. Our work opens up new avenues for improving image captioning models at larger scale.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The RA-Transformer [46] and EXTRA [308], both retrieval-augmented transformerbased captioning models, utilize cross-attention over encoded retrieved captions.”
From Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024) · cites this paper as [46]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“Large-scale language models can perfectly memorize parts of their training data [12] and increasing model size predictably improves performance on a wide range of downstream tasks [10, 30, 46].”
BackgroundDeep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jegou, H. (2021) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2019.2921572 · Cited by 4,228
Billion-scale similarity search with GPUs
Johnson, J., Douze, M., & Jégou, H. (2017) · arXiv · doi:10.48550/arxiv.1702.08734 · Cited by 4,140
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555
Meshed-Memory Transformer for Image Captioning
Cornia, M., Stefanini, M., Baraldi, L., & Cucchiara, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01059 · Cited by 1,259
Self-citationAttention on Attention for Image Captioning
Huang, L., Wang, W., Chen, J., & Wei, X.-Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00473 · Cited by 1,092 · cited 3×
“Following standard image captioning approaches [3, 16, 27], we train and evaluate our model on the COCO dataset [38], thus not relying on large-scale image-text datasets [15].”
BackgroundMethodNeural Machine Translation of Rare Words with Subword Units
Sennrich, R., Haddow, B., & Birch, A. (2015) · arXiv · doi:10.48550/arxiv.1508.07909 · Cited by 950
(2020) · doi:10.1109/sc41405.2020.00024 · Cited by 928
Unified Vision-Language Pre-Training for Image Captioning and VQA
Zhou, L., Palangi, H., Zhang, L., Hu, H., Corso, J., & Gao, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.7005 · Cited by 864
Research works printed but not linked: 53
- Peter Anderson , Basura Fernando , Mark Johnson , and Stephen Gould . 2016 . SPICE: Semantic Propositional Image Caption Evaluation. In ECCV. Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2016. SPICE: Semantic Propositional Image Caption Evaluation. In ECCV.
- Peter Anderson Xiaodong He Chris Buehler Damien Teney Mark Johnson Stephen Gould and Lei Zhang. 2018. Bottom-up and top-down attention for image captioning and visual question answering. In CVPR. Peter Anderson Xiaodong He Chris Buehler Damien Teney Mark Johnson Stephen Gould and Lei Zhang. 2018. Bottom-up and top-down attention for image captioning and visual question answering. In CVPR. · DOI · Paper page
- Jyoti Aneja Aditya Deshpande and Alexander G Schwing. 2018. Convolutional image captioning. In CVPR. Jyoti Aneja Aditya Deshpande and Alexander G Schwing. 2018. Convolutional image captioning. In CVPR. · DOI · Paper page
- Satanjeev Banerjee and Alon Lavie . 2005 . METEOR: An automatic metric for MT evaluation with improved correlation with human judgments . In ACL Workshops. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In ACL Workshops.
- Manuele Barraco , Marcella Cornia , Silvia Cascianelli , Lorenzo Baraldi , and Rita Cucchiara . 2022 . The Unreasonable Effectiveness of CLIP Features for Image Captioning: An Experimental Analysis. In CVPR Workshops. Manuele Barraco, Marcella Cornia, Silvia Cascianelli, Lorenzo Baraldi, and Rita Cucchiara. 2022. The Unreasonable Effectiveness of CLIP Features for Image Captioning: An Experimental Analysis. In CVPR Workshops.
- Manuele Barraco Matteo Stefanini Marcella Cornia Silvia Cascianelli Lorenzo Baraldi and Rita Cucchiara. 2022. CaMEL: Mean Teacher Learning for Image Captioning. In ICPR. Manuele Barraco Matteo Stefanini Marcella Cornia Silvia Cascianelli Lorenzo Baraldi and Rita Cucchiara. 2022. CaMEL: Mean Teacher Learning for Image Captioning. In ICPR. · DOI
- Roberto Bigazzi Federico Landi Marcella Cornia Silvia Cascianelli Lorenzo Baraldi and Rita Cucchiara. 2020. Explore and Explain: Self-supervised Navigation and Recounting. In ICPR. Roberto Bigazzi Federico Landi Marcella Cornia Silvia Cascianelli Lorenzo Baraldi and Rita Cucchiara. 2020. Explore and Explain: Self-supervised Navigation and Recounting. In ICPR. · DOI
- Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell 2020. Language models are few-shot learners. In NeurIPS. Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell 2020. Language models are few-shot learners. In NeurIPS.
- Marco Cagrandi Marcella Cornia Matteo Stefanini Lorenzo Baraldi and Rita Cucchiara. 2021. Learning to Select: A Fully Attentive Approach for Novel Object Captioning. In ICMR. Marco Cagrandi Marcella Cornia Matteo Stefanini Lorenzo Baraldi and Rita Cucchiara. 2021. Learning to Select: A Fully Attentive Approach for Novel Object Captioning. In ICMR. · DOI
- Marcella Cornia Lorenzo Baraldi and Rita Cucchiara. 2020. SMArT: Training Shallow Memory-aware Transformers for Robotic Explainability. In ICRA. Marcella Cornia Lorenzo Baraldi and Rita Cucchiara. 2020. SMArT: Training Shallow Memory-aware Transformers for Robotic Explainability. In ICRA. · DOI
- Marcella Cornia , Lorenzo Baraldi , and Rita Cucchiara . 2021. Explaining transformer-based image captioning models: An empirical analysis. AI Communications ( 2021 ), 1–19. Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara. 2021. Explaining transformer-based image captioning models: An empirical analysis. AI Communications (2021), 1–19.
- Marcella Cornia , Lorenzo Baraldi , Giuseppe Fiameni , and Rita Cucchiara . 2022 . Universal Captioner: Inducing Content-Style Separation in Vision-and-Language Model Training. arXiv preprint arXiv:2111.12727(2022). Marcella Cornia, Lorenzo Baraldi, Giuseppe Fiameni, and Rita Cucchiara. 2022. Universal Captioner: Inducing Content-Style Separation in Vision-and-Language Model Training. arXiv preprint arXiv:2111.12727(2022). · arXiv
- Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2018 . BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL.
- Jeffrey Donahue Lisa Anne Hendricks Sergio Guadarrama Marcus Rohrbach Subhashini Venugopalan Kate Saenko and Trevor Darrell. 2015. Long-term recurrent convolutional networks for visual recognition and description. In CVPR. Jeffrey Donahue Lisa Anne Hendricks Sergio Guadarrama Marcus Rohrbach Subhashini Venugopalan Kate Saenko and Trevor Darrell. 2015. Long-term recurrent convolutional networks for visual recognition and description. In CVPR. · DOI · Paper page
- Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly Jakob Uszkoreit and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR. Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly Jakob Uszkoreit and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In ICLR.
- Shiv Ram Dubey Satish Kumar Singh and Wei-Ta Chu. 2021. Vision Transformer Hashing for Image Retrieval. In ICME. Shiv Ram Dubey Satish Kumar Singh and Wei-Ta Chu. 2021. Vision Transformer Hashing for Image Retrieval. In ICME. · DOI
- Alaaeldin El-Nouby Natalia Neverova Ivan Laptev and Hervé Jégou. 2021. Training Vision Transformers for Image Retrieval. arXiv preprint arXiv:2102.05644(2021). Alaaeldin El-Nouby Natalia Neverova Ivan Laptev and Hervé Jégou. 2021. Training Vision Transformers for Image Retrieval. arXiv preprint arXiv:2102.05644(2021). · arXiv
- Longteng Guo Jing Liu Xinxin Zhu Peng Yao Shichen Lu and Hanqing Lu. 2020. Normalized and Geometry-Aware Self-Attention Network for Image Captioning. In CVPR. Longteng Guo Jing Liu Xinxin Zhu Peng Yao Shichen Lu and Hanqing Lu. 2020. Normalized and Geometry-Aware Self-Attention Network for Image Captioning. In CVPR.
- Kelvin Guu , Kenton Lee , Zora Tung , Panupong Pasupat , and Ming-Wei Chang . 2020 . REALM: Retrieval-Augmented Language Model Pre-Training. In ICML. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. REALM: Retrieval-Augmented Language Model Pre-Training. In ICML.
- Kaiming He Xiangyu Zhang Shaoqing Ren and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. Kaiming He Xiangyu Zhang Shaoqing Ren and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR.
- Simao Herdade , Armin Kappeler , Kofi Boakye , and Joao Soares . 2019 . Image Captioning: Transforming Objects into Words. In NeurIPS. Simao Herdade, Armin Kappeler, Kofi Boakye, and Joao Soares. 2019. Image Captioning: Transforming Objects into Words. In NeurIPS.
- Xiaowei Hu Zhe Gan Jianfeng Wang Zhengyuan Yang Zicheng Liu Yumao Lu and Lijuan Wang. 2022. Scaling Up Vision-Language Pre-Training for Image Captioning. In CVPR. Xiaowei Hu Zhe Gan Jianfeng Wang Zhengyuan Yang Zicheng Liu Yumao Lu and Lijuan Wang. 2022. Scaling Up Vision-Language Pre-Training for Image Captioning. In CVPR.
- Gautier Izacard and Edouard Grave. 2021. Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering. In EACL. Gautier Izacard and Edouard Grave. 2021. Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering. In EACL. · DOI · Paper page
- Urvashi Khandelwal Omer Levy Dan Jurafsky Luke Zettlemoyer and Mike Lewis. 2020. Generalization through Memorization: Nearest Neighbor Language Models. In ICLR. Urvashi Khandelwal Omer Levy Dan Jurafsky Luke Zettlemoyer and Mike Lewis. 2020. Generalization through Memorization: Nearest Neighbor Language Models. In ICLR.
- Diederik P Kingma and Jimmy Ba . 2015 . Adam: A Method for Stochastic Optimization. In ICLR. Diederik P Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.
- 10.1016/j.neunet.2021.08.030 · DOI · Paper page
- Patrick Lewis Ethan Perez Aleksandra Piktus Fabio Petroni Vladimir Karpukhin Naman Goyal Heinrich Küttler Mike Lewis Wen-tau Yih Tim Rocktäschel Sebastian Riedel and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In NeurIPS. Patrick Lewis Ethan Perez Aleksandra Piktus Fabio Petroni Vladimir Karpukhin Naman Goyal Heinrich Küttler Mike Lewis Wen-tau Yih Tim Rocktäschel Sebastian Riedel and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In NeurIPS.
- Xiujun Li , Xi Yin , Chunyuan Li , Pengchuan Zhang , Xiaowei Hu , Lei Zhang , Lijuan Wang , Houdong Hu , Li Dong , Furu Wei , 2020 . Oscar: Object-semantics aligned pre-training for vision-language tasks. In ECCV. Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, 2020. Oscar: Object-semantics aligned pre-training for vision-language tasks. In ECCV.
- Chin-Yew Lin . 2004 . Rouge: A package for automatic evaluation of summaries . In ACL Workshops. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In ACL Workshops.
- Tsung-Yi Lin Michael Maire Serge Belongie James Hays Pietro Perona Deva Ramanan Piotr Dollár and C Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. In ECCV. Tsung-Yi Lin Michael Maire Serge Belongie James Hays Pietro Perona Deva Ramanan Piotr Dollár and C Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. In ECCV.
- Fenglin Liu , Xuancheng Ren , Xian Wu , Shen Ge , Wei Fan , Yuexian Zou , and Xu Sun . 2020 . Prophet Attention: Predicting Attention with Future Attention. In NeurIPS. Fenglin Liu, Xuancheng Ren, Xian Wu, Shen Ge, Wei Fan, Yuexian Zou, and Xu Sun. 2020. Prophet Attention: Predicting Attention with Future Attention. In NeurIPS.
- Wei Liu , Sihan Chen , Longteng Guo , Xinxin Zhu , and Jing Liu . 2021 . CPTR: Full Transformer Network for Image Captioning. arXiv preprint arXiv:2101.10804(2021). Wei Liu, Sihan Chen, Longteng Guo, Xinxin Zhu, and Jing Liu. 2021. CPTR: Full Transformer Network for Image Captioning. arXiv preprint arXiv:2101.10804(2021). · arXiv
- Yunpeng Luo Jiayi Ji Xiaoshuai Sun Liujuan Cao Yongjian Wu Feiyue Huang Chia-Wen Lin and Rongrong Ji. 2021. Dual-Level Collaborative Transformer for Image Captioning. In AAAI. Yunpeng Luo Jiayi Ji Xiaoshuai Sun Liujuan Cao Yongjian Wu Feiyue Huang Chia-Wen Lin and Rongrong Ji. 2021. Dual-Level Collaborative Transformer for Image Captioning. In AAAI.
- Paulius Micikevicius Sharan Narang Jonah Alben Gregory Diamos Erich Elsen David Garcia Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh and Hao Wu. 2018. Mixed Precision Training. In ICLR. Paulius Micikevicius Sharan Narang Jonah Alben Gregory Diamos Erich Elsen David Garcia Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh and Hao Wu. 2018. Mixed Precision Training. In ICLR.
- Yingwei Pan Ting Yao Yehao Li and Tao Mei. 2020. X-Linear Attention Networks for Image Captioning. In CVPR. Yingwei Pan Ting Yao Yehao Li and Tao Mei. 2020. X-Linear Attention Networks for Image Captioning. In CVPR.
- Kishore Papineni Salim Roukos Todd Ward and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In ACL. Kishore Papineni Salim Roukos Todd Ward and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In ACL. · DOI · Paper page
- Alec Radford Jong Wook Kim Chris Hallacy Aditya Ramesh Gabriel Goh Sandhini Agarwal Girish Sastry Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In ICML. Alec Radford Jong Wook Kim Chris Hallacy Aditya Ramesh Gabriel Goh Sandhini Agarwal Girish Sastry Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In ICML.
- Alec Radford , Jeffrey Wu , Rewon Child , David Luan , Dario Amodei , and Ilya Sutskever . 2019 . Language Models are Unsupervised Multitask Learners . OpenAI Blog 1 , 8 (2019), 9 . Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. OpenAI Blog 1, 8 (2019), 9.
- Steven J Rennie Etienne Marcheret Youssef Mroueh Jarret Ross and Vaibhava Goel. 2017. Self-critical sequence training for image captioning. In CVPR. Steven J Rennie Etienne Marcheret Youssef Mroueh Jarret Ross and Vaibhava Goel. 2017. Self-critical sequence training for image captioning. In CVPR. · DOI · Paper page
- Sheng Shen Liunian Harold Li Hao Tan Mohit Bansal Anna Rohrbach Kai-Wei Chang Zhewei Yao and Kurt Keutzer. 2021. How Much Can CLIP Benefit Vision-and-Language Tasks?arXiv preprint arXiv:2107.06383(2021). Sheng Shen Liunian Harold Li Hao Tan Mohit Bansal Anna Rohrbach Kai-Wei Chang Zhewei Yao and Kurt Keutzer. 2021. How Much Can CLIP Benefit Vision-and-Language Tasks?arXiv preprint arXiv:2107.06383(2021). · arXiv
- Zhan Shi Xu Zhou Xipeng Qiu and Xiaodan Zhu. 2020. Improving Image Captioning with Better Use of Captions. In ACL. Zhan Shi Xu Zhou Xipeng Qiu and Xiaodan Zhu. 2020. Improving Image Captioning with Better Use of Captions. In ACL. · DOI
- Giorgos Tolias Ronan Sicre and Hervé Jégou. 2016. Particular object retrieval with integral max-pooling of CNN activations. In ICLR. Giorgos Tolias Ronan Sicre and Hervé Jégou. 2016. Particular object retrieval with integral max-pooling of CNN activations. In ICLR.
- Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles and Hervé Jégou. 2021. Training data-efficient image transformers & distillation through attention. In ICML. Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles and Hervé Jégou. 2021. Training data-efficient image transformers & distillation through attention. In ICML.
- Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS. Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS.
- Ramakrishna Vedantam C Lawrence Zitnick and Devi Parikh. 2015. CIDEr: Consensus-based Image Description Evaluation. In CVPR. Ramakrishna Vedantam C Lawrence Zitnick and Devi Parikh. 2015. CIDEr: Consensus-based Image Description Evaluation. In CVPR. · DOI · Paper page
- Oriol Vinyals Alexander Toshev Samy Bengio and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In CVPR. Oriol Vinyals Alexander Toshev Samy Bengio and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In CVPR. · DOI · Paper page
- Yuhuai Wu Markus N Rabe DeLesley Hutchins and Christian Szegedy. 2022. Memorizing Transformers. In ICLR. Yuhuai Wu Markus N Rabe DeLesley Hutchins and Christian Szegedy. 2022. Memorizing Transformers. In ICLR.
- Xu Yang Kaihua Tang Hanwang Zhang and Jianfei Cai. 2019. Auto-Encoding Scene Graphs for Image Captioning. In CVPR. Xu Yang Kaihua Tang Hanwang Zhang and Jianfei Cai. 2019. Auto-Encoding Scene Graphs for Image Captioning. In CVPR. · DOI · Paper page
- Xu Yang Hanwang Zhang and Jianfei Cai. 2019. Learning to Collocate Neural Modules for Image Captioning. In ICCV. Xu Yang Hanwang Zhang and Jianfei Cai. 2019. Learning to Collocate Neural Modules for Image Captioning. In ICCV. · DOI
- Ting Yao Yingwei Pan Yehao Li and Tao Mei. 2018. Exploring Visual Relationship for Image Captioning. In ECCV. Ting Yao Yingwei Pan Yehao Li and Tao Mei. 2018. Exploring Visual Relationship for Image Captioning. In ECCV. · DOI · Paper page
- Dani Yogatama , Cyprien de Masson d’Autume , and Lingpeng Kong . 2021 . Adaptive Semiparametric Language Models . TACL 9 (2021), 362 – 373 . Dani Yogatama, Cyprien de Masson d’Autume, and Lingpeng Kong. 2021. Adaptive Semiparametric Language Models. TACL 9(2021), 362–373. · DOI
- Yang You Jing Li Sashank Reddi Jonathan Hseu Sanjiv Kumar Srinadh Bhojanapalli Xiaodan Song James Demmel Kurt Keutzer and Cho-Jui Hsieh. 2020. Large Batch Optimization for Deep Learning: Training BERT in 76 minutes. In ICLR. Yang You Jing Li Sashank Reddi Jonathan Hseu Sanjiv Kumar Srinadh Bhojanapalli Xiaodan Song James Demmel Kurt Keutzer and Cho-Jui Hsieh. 2020. Large Batch Optimization for Deep Learning: Training BERT in 76 minutes. In ICLR.
- Xuying Zhang Xiaoshuai Sun Yunpeng Luo Jiayi Ji Yiyi Zhou Yongjian Wu Feiyue Huang and Rongrong Ji. 2021. RSTNet: Captioning with Adaptive Attention on Visual and Non-Visual Words. In CVPR. Xuying Zhang Xiaoshuai Sun Yunpeng Luo Jiayi Ji Yiyi Zhou Yongjian Wu Feiyue Huang and Rongrong Ji. 2021. RSTNet: Captioning with Adaptive Attention on Visual and Non-Visual Words. In CVPR. · DOI · Paper page
Cited by
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., & Li, Q. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671470 · Cited by 827
Retrieval-Augmented Generation for AI-Generated Content: A Survey
Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2024) · arXiv · doi:10.48550/arxiv.2402.19473 · Cited by 240
“The RA-Transformer [46] and EXTRA [335], both retrieval-augmented transformer-based captioning models, utilize cross-attention over encoded retrieved captions.”
(2024) · doi:10.1109/cvprw63382.2024.00188 · Cited by 51
Accelerating Retrieval-Augmented Generation
Quinn, D., Nouri, M., Patel, N., Salihu, J., Salemi, A., Lee, S., Zamani, H., & Alian, M. (2025) · Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1 · doi:10.1145/3669940.3707264 · Cited by 48
Retrieval-Augmented Egocentric Video Captioning
Xu, J., Huang, Y., Hou, J., Chen, G., Zhang, Y., Feng, R., & Xie, W. (2024) · arXiv · doi:10.48550/arxiv.2401.00789 · Cited by 24
Metadata
- Authors
- Sarto, S., Cornia, M., Baraldi, L., & Cucchiara, R.
- Affiliations
- Department of Engineering, University of Modena and Reggio Emilia, Italy
- Department of Education and Humanities, University of Modena and Reggio Emilia, Italy
- Venue
- International Conference on Content-based Multimedia Indexing
- Publisher
- ACM
- Published
- 2022-09-14
- Pages
- 1-7
- Cited by
- 49
- References
- 13
- Funders
- Fondazione di Modena
- Italian Ministry of University and Research (B87G22000460001)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Retrieval-Augmented Transformer for Image Captioning. https://app.cochat.ai/papers/10.1145/3549555.3549585
- The paper
- Sarto, S., Cornia, M., Baraldi, L., & Cucchiara, R. (2022). Retrieval-Augmented Transformer for Image Captioning. International Conference on Content-based Multimedia Indexing. https://doi.org/10.1145/3549555.3549585
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.