Visual Dialog
Abstract
We introduce the task of Visual Dialog, which requires an AI agent to hold a meaningful dialog with humans in natural, conversational language about visual content. Specifically, given an image, a dialog history, and a question about the image, the agent has to ground the question in image, infer context from history, and answer the question accurately. Visual Dialog is disentangled enough from a specific downstream task so as to serve as a general test of machine intelligence, while being grounded in vision enough to allow objective evaluation of individual responses and benchmark progress. We develop a novel two-person chat data-collection protocol to curate a large-scale Visual Dialog dataset (VisDial). VisDial contains 1 dialog (10 question-answer pairs) on ~140k images from the COCO dataset, with a total of ~1.4M dialog question-answer pairs. We introduce a family of neural encoder-decoder models for Visual Dialog with 3 encoders (Late Fusion, Hierarchical Recurrent Encoder and Memory Network) and 2 decoders (generative and discriminative), which outperform a number of sophisticated baselines. We propose a retrieval-based evaluation protocol for Visual Dialog where the AI agent is asked to sort a set of candidate answers and evaluated on metrics such as mean-reciprocal-rank of human response. We quantify gap between machine and human performance on the Visual Dialog task via human studies. Our dataset, code, and trained models will be released publicly at https://visualdialog.org. Putting it all together, we demonstrate the first visual chatbot!.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Visual Dialog is interactive and context-related.
“Interactive and context-related dialog”
From Explainable AI in medical imaging: An overview for clinical practitioners – Beyond saliency-based XAI approaches (2023) · cites this paper as [40]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
(2016) · Cited by 31,590 · see citing works
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612 · see citing works
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · see citing works
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228 · see citing works
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902 · see citing works
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487 · see citing works
Self-citationTeaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,779 · see citing works
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
Goyal, Y., Khot, T., Summers-Stay, D., Batra, D., & Parikh, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.670 · Cited by 2,287 · see citing works
Self-citationStacked Attention Networks for Image Question Answering
Yang, Z., He, X., Gao, J., Deng, L., & Smola, A. J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.10 · Cited by 2,036 · see citing works
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Serban, I. V., Sordoni, A., Bengio, Y., Courville, A. C., & Pineau, J. (2016) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v30i1.9883 · Cited by 1,639 · see citing works
Sequence to Sequence -- Video to Text
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R., Darrell, T., & Saenko, K. (2015) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2015.515 · Cited by 1,507 · see citing works
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models
Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J., & Lazebnik, S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.303 · Cited by 1,444 · see citing works
Vinyals, O., & Le, Q. (2015) · arXiv · doi:10.48550/arxiv.1506.05869 · Cited by 1,400 · see citing works
From captions to visual concepts and back
Fang, H., Gupta, S., Iandola, F., Srivastava, R. K., Deng, L., Dollár, P., Gao, J., He, X., Mitchell, M., Platt, J. C., et al. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298754 · Cited by 1,305 · see citing works
Hierarchical Question-Image Co-Attention for Visual Question AnsweringPreprint
Lu, J., Yang, J., Batra, D., & Parikh, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00061 · Cited by 1,303 · see citing works
Self-citationLiu, C., Lowe, R., Serban, I. V., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1230 · Cited by 1,262 · see citing works
Deep Reinforcement Learning for Dialogue Generation
Li, J., Monroe, W., Ritter, A., Jurafsky, D., Galley, M., & Gao, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1127 · Cited by 1,221 · see citing works
Translating Videos to Natural Language Using Deep Recurrent Neural Networks
Venugopalan, S., Xu, H., Donahue, J., Rohrbach, M., Mooney, R. J., & Saenko, K. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1173 · Cited by 996 · see citing works
A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues
Serban, I. V., Sordoni, A., Lowe, R., Charlin, L., Pineau, J., Courville, A. C., & Bengio, Y. (2017) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v31i1.10983 · Cited by 734 · see citing works
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merriënboer, B., Joulin, A., & Mikolov, T. (2015) · arXiv · doi:10.48550/arxiv.1502.05698 · Cited by 670 · see citing works
MovieQA: Understanding Stories in Movies through Question-Answering
Tapaswi, M., Zhu, Y., Stiefelhagen, R., Torralba, A. B., Urtasun, R., & Fidler, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.501 · Cited by 646 · see citing works
Ask Your Neurons: A Neural-Based Approach to Answering Questions about Images
Malinowski, M., Rohrbach, M., & Fritz, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.9 · Cited by 633 · see citing works
Exploring Models and Data for Image Question AnsweringPreprint
Ren, M., Kiros, R., & Zemel, R. S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.02074 · Cited by 618 · see citing works
Large-scale Simple Question Answering with Memory Networks
Bordes, A., Usunier, N., Chopra, S., & Weston, J. (2015) · arXiv · doi:10.48550/arxiv.1506.02075 · Cited by 528 · see citing works
Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering
Goyal, Y., Khot, T., Agrawal, A., Summers-Stay, D., Batra, D., & Parikh, D. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-018-1116-0 · Cited by 487 · see citing works
Self-citationLearning End-to-End Goal-Oriented DialogPreprint
Bordes, A., Boureau, Y.-L., & Weston, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07683 · Cited by 481 · see citing works
Segmentation from Natural Language Expressions
Hu, R., Rohrbach, M., & Darrell, T. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_7 · Cited by 467 · see citing works
A dataset for Movie Description
Rohrbach, A., Rohrbach, M., Tandon, N., & Schiele, B. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298940 · Cited by 450 · see citing works
Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question AnsweringPreprint
Gao, H., Mao, J., Zhou, J., Huang, Z., Wang, L., & Xu, W. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05612 · Cited by 437 · see citing works
A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain InputPreprint
Malinowski, M., & Fritz, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0210 · Cited by 432 · see citing works
Human Attention in Visual Question Answering: Do Humans and Deep Networks look at the same regions?
Das, A., Agrawal, H., Zitnick, C. L., Parikh, D., & Batra, D. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1092 · Cited by 431 · see citing works
Self-citationGrounding of Textual Phrases in Images by Reconstruction
Rohrbach, A., Rohrbach, M., Hu, R., Darrell, T., & Schiele, B. (2016) · Max Planck Digital Library · Cited by 422 · see citing works
GuessWhat?! Visual Object Discovery through Multi-modal Dialogue
de Vries, H., Strub, F., Chandar, S., Pietquin, O., Larochelle, H., & Courville, A. C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.475 · Cited by 378 · see citing works
Visual Turing test for computer vision systems
Geman, D., Geman, S., Hallonquist, N., & Laurent Younes, B. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1422953112 · Cited by 323 · see citing works
Yin and Yang: Balancing and Answering Binary Visual Questions
Zhang, P., Goyal, Y., Summers-Stay, D. A., Batra, D., & Parikh, D. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.542 · Cited by 312 · see citing works
Self-citationThe Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems
Lowe, R., Pow, N., Serban, I. V., & Pineau, J. (2015) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w15-4640 · Cited by 282 · see citing works
Generating Factoid Questions With Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus
Serban, I. V., García-Durán, A., Gulcehre, C., Ahn, S., Chandar, S., Courville, A., & Bengio, Y. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1056 · Cited by 280 · see citing works
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merriënboer, B., Joulin, A., & Mikolov, T. (2016) · International Conference on Learning Representations · Cited by 261 · see citing works
Huang, T.-H., Ferraro, F., Mostafazadeh, N., Misra, I., Agrawal, A., Devlin, J., Girshick, R., He, X., Kohli, P., Batra, D., et al. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1147 · Cited by 254 · see citing works
Self-citationRevisiting Visual Question Answering Baselines
Jabri, A., Joulin, A., & van der Maaten, L. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_44 · Cited by 235 · see citing works
What Are You Talking About? Text-to-Image Coreference
Kong, C., Lin, D., Bansal, M., Urtasun, R., & Fidler, S. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.455 · Cited by 191 · see citing works
Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences
Mei, H., Bansal, M., & Walter, M. R. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10364 · Cited by 184 · see citing works
Kannan, A., Kurach, K., Ravi, S., Kaufmann, T., Tomkins, A., Miklos, B., Corrado, G. S., Lukacs, L., Ganea, M., Young, P., et al. (2016) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2939672.2939801 · Cited by 182 · see citing works
Knowledge Guided Disambiguation for Large-Scale Scene Classification With Multi-Resolution CNNs
Wang, L., Guo, S., Huang, W., Xiong, Y., & Qiao, Y. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2675339 · Cited by 154 · see citing works
Evaluating Prerequisite Qualities for Learning End-to-End Dialog SystemsPreprint
Dodge, J., Gane, A., Zhang, X., Bordes, A., Chopra, S., Miller, A., Szlam, A. D., & Weston, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06931 · Cited by 150 · see citing works
Image-Grounded Conversations: Multimodal Context for Natural Question and Response GenerationPreprint
Mostafazadeh, N., Brockett, C., Dolan, B., Galley, M., Gao, J., Spithourakis, G., & Vanderwende, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.08251 · Cited by 125 · see citing works
Joint Video and Text Parsing for Understanding Events and Answering Queries
Tu, K., Meng, M., Lee, M. W., Choe, T. E., & Zhu, S. C. (2014) · IEEE Multimedia · doi:10.1109/mmul.2014.29 · Cited by 113 · see citing works
An ISU dialogue system exhibiting reinforcement learning of dialogue policies
Lemon, O., Georgila, K., Henderson, J., & Stuttle, M. N. (2006) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/1608974.1608986 · Cited by 95 · see citing works
VizWiz: Nearly Real-time Answers to Visual Questions
Bigham, J. P., Jayant, R., Ji, H., Little, G., Millerγ, A., Miller, R. C., Miller, R., Tatarowicz, A., White, Y., White, S., et al. (2010) · Cited by 80 · see citing works
Sort Story: Sorting Jumbled Images and Captions into Stories
Agrawal, H., Chandrasekaran, A., Batra, D., Parikh, D., & Bansal, M. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1091 · Cited by 61 · see citing works
Self-citationQuestion Relevance in VQA: Identifying Non-Visual And False-Premise Questions
Ray, A., Christie, G. A., Bansal, M., Batra, D., & Parikh, D. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1090 · Cited by 48 · see citing works
Self-citationEmpirical methods for evaluating dialog systems
Paek, T. (2001) · doi:10.3115/1118053.1118054 · Cited by 47 · see citing works
Measuring Machine Intelligence Through Visual Question Answering
Zitnick, C. L., Agrawal, A., Antol, S., Mitchell, M., Batra, D., & Parikh, D. (2016) · AI Magazine · doi:10.1609/aimag.v37i1.2647 · Cited by 29 · see citing works
Self-citationChristie, G. A., Laddha, A., Agrawal, A., Antol, S., Goyal, Y., Kochersberger, K. B., & Batra, D. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1156 · Cited by 17 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909 · see citing works
Self-citationA Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524 · see citing works
Anderson, P., Wu, Q., Teney, D., Bruce, J., Johnson, M., Sunderhauf, N., Reid, I., Gould, S., & van den Hengel, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00387 · Cited by 1,187 · see citing works
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072 · see citing works
Graph R-CNN for Scene Graph Generation
Yang, J., Lu, J., Lee, S., Batra, D., & Parikh, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_41 · Cited by 889 · see citing works
Self-citationScaling Egocentric Vision: The "Equation missing" Dataset
Damen, D., Doughty, H., Farinella, G. M., Fidler, S., Furnari, A., Kazakos, E., Moltisanti, D., Munro, J., Perrett, T., Price, W., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01225-0_44 · Cited by 875 · see citing works
Flamingo: a visual language model for few-shot learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1723 · Cited by 966 · see citing works
Intern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02283 · Cited by 734 · see citing works
VizWiz Grand Challenge: Answering Visual Questions from Blind People
Gurari, D., Li, Q., Stangl, A. J., Guo, A., Lin, C., Grauman, K., Luo, J., & Bigham, J. P. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00380 · Cited by 621 · see citing works
“Our findings show the statistics found in the existing artificially constructed VQA datasets, nicely summarized in [13] and [20], match statistics observed in practice.”
ResultWang, X., Huang, Q., Celikyilmaz, A., Gao, J., Shen, D., Wang, Y.-F., Wang, W. Y., & Zhang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00679 · Cited by 522 · see citing works
TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering
Jang, Y., Song, Y., Yu, Y., Kim, Y., & Kim, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.149 · Cited by 496 · see citing works
Inferring and Executing Programs for Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L., Hoffman, J., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.325 · Cited by 487 · see citing works
Counterfactual VQA: A Cause-Effect Look at Language Bias
Niu, Y., Tang, K., Zhang, H., Lu, Z., Hua, X., & Wen, J.-R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01251 · Cited by 445 · see citing works
Convolutional Image Captioning
Aneja, J., Deshpande, A., & Schwing, A. G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00583 · Cited by 426 · see citing works
Lu, J., Yang, J., Batra, D., & Parikh, D. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00754 · Cited by 412 · see citing works
Self-citationCross-Modal Causal Relational Reasoning for Event-Level Visual Question Answering
Liu, Y., Li, G., & Lin, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3284038 · Cited by 354 · see citing works
MUREL: Multimodal Relational Reasoning for Visual Question Answering
Cadene, R., Ben-younes, H., Cord, M., & Thome, N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00209 · Cited by 330 · see citing works
VLP: A Survey on Vision-language Pre-training
Chen, F., Zhang, D., Han, M., Chen, X.-Y., Shi, J. C., Xu, S., & Xu, B. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1369-5 · Cited by 295 · see citing works
Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback
Wu, H., Gao, Y., Guo, X., Al-Halah, Z., Rennie, S. J., Grauman, K. L., & Feris, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01115 · Cited by 297 · see citing works
VLN↻BERT: A Recurrent Vision-and-Language BERT for Navigation
Hong, Y., Wu, Q., Qi, Y., Rodriguez-Opazo, C., & Gould, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00169 · Cited by 264 · see citing works
Large-scale Multi-modal Pre-trained Models: A Comprehensive Survey
Wang, X., Chen, G., Qian, G., Gao, P., Wei, X.-Y., Wang, Y., Tian, Y., & Gao, W. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1410-8 · Cited by 245 · see citing works
Borys, K., Schmitt, Y. A., Nauta, M., Seifert, C., Krämer, N., Friedrich, C. M., & Nensa, F. (2023) · European Journal of Radiology · doi:10.1016/j.ejrad.2023.110786 · Cited by 233 · see citing works
Gao, J., Galley, M., & Li, L. C. (2019) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000074 · Cited by 44 · see citing works
CPT: Colorful Prompt Tuning for pre-trained vision-language models
Yao, Y., Zhang, A., Zhang, Z., Liu, Z., Chua, T., & Sun, M. (2024) · AI Open · doi:10.1016/j.aiopen.2024.01.004 · Cited by 206 · see citing works
Audio Visual Scene-Aware Dialog
Alamri, H., Cartillier, V., Das, A., Wang, J., Cherian, A., Essa, I. A., Batra, D., Marks, T. K., Hori, C., Anderson, P., et al. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00774 · Cited by 172 · see citing works
Self-citationEngaging Image Captioning via Personality
Shuster, K., Humeau, S., Hu, H., Bordes, A., & Weston, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01280 · Cited by 160 · see citing works
Visual Coreference Resolution in Visual Dialog Using Neural Module Networks
Kottur, S., Moura, J. M. F., Parikh, D., Batra, D., & Rohrbach, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01267-0_10 · Cited by 161 · see citing works
Self-citationVision-Language Pre-Training: Basics, Recent Advances, and Future Trends
Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., & Gao, J. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000105 · Cited by 155 · see citing works
Two Causal Principles for Improving Visual Dialog
Qi, J., Niu, Y., Huang, J., & Zhang, H. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01087 · Cited by 150 · see citing works
Parallel Attention: A Unified Framework for Visual Object Discovery Through Dialogs and Queries
Zhuang, B., Wu, Q., Shen, C., Reid, I. D., & van den Hengel, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00447 · Cited by 142 · see citing works
Reasoning Visual Dialogs With Structural and Partial Observations
Zheng, Z., Wang, W., Qi, S., & Zhu, S. C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00683 · Cited by 131 · see citing works
Are You Talking to Me? Reasoned Visual Dialog Generation Through Adversarial Learning
Wu, Q., Wang, P., Shen, C., Reid, I. D., & van den Hengel, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00639 · Cited by 131 · see citing works
Deconfounded Image Captioning: A Causal Retrospect
Yang, X., Zhang, H., & Cai, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3121705 · Cited by 130 · see citing works
Friend, Collaborator, Student, Manager
Guzdial, M., Liao, N., Chen, J., Chen, S.-Y., Shah, S., Shah, V., Reno, J. O., Smith, G., & Riedl, M. O. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300854 · Cited by 134 · see citing works
Region-Aware Image Captioning via Interaction Learning
Liu, A.-A., Zhai, Y., Xu, N., Nie, W., Li, W., & Zhang, Y. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3107035 · Cited by 128 · see citing works
Kaleido-BERT: Vision-Language Pre-training on Fashion Domain
Zhuge, M., Gao, D., Fan, D.-P., Jin, L., Chen, B., Zhou, H., Qiu, M., & Shao, L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01246 · Cited by 128 · see citing works
Deep Learning for Image-to-Text Generation: A Technical Overview
He, X., & Deng, L. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2741510 · Cited by 121 · see citing works
Multi-Level Policy and Reward-Based Deep Reinforcement Learning Framework for Image Captioning
Xu, N., Zhang, H., Liu, A.-A., Nie, W., Su, Y., Nie, J., & Zhang, Y. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2941820 · Cited by 115 · see citing works
Explainable AI for Medical Data: Current Methods, Limitations, and Future Directions
Hossain, M. I., Zamzmi, G., Mouton, P. R., Salekin, M. S., Sun, Y., & Goldgof, D. B. (2023) · ACM Computing Surveys · doi:10.1145/3637487 · Cited by 120 · see citing works
Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision
Teney, D., Abbasnedjad, E., & van den Hengel, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58607-2_34 · Cited by 106 · see citing works
Schwartz, I., Yu, S., Hazan, T., & Schwing, A. G. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00214 · Cited by 108 · see citing works
Large-Scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline
Murahari, V., Batra, D., Parikh, D., & Das, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58523-5_20 · Cited by 107 · see citing works
Self-citationTowards Unsupervised Image Captioning With Shared Multimodal Embeddings
Laina, I., Rupprecht, C., & Navab, N. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00751 · Cited by 111 · see citing works
Learning to Collocate Neural Modules for Image Captioning
Yang, X., Zhang, H., & Cai, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00435 · Cited by 101 · see citing works
Ishmam, M. F., Shovon, M. S. H., Mridha, M. F., & Dey, N. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102270 · Cited by 100 · see citing works
High-Order Interaction Learning for Image Captioning
Wang, Y., Xu, N., Liu, A.-A., Li, W., & Zhang, Y. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3121062 · Cited by 95 · see citing works
Tell, Draw, and Repeat: Generating and Modifying Images Based on Continual Linguistic Instruction
El-Nouby, A., Sharma, S., Schulz, H., Hjelm, D., Asri, L. E., Kahou, S. E., Bengio, Y., & Taylor, G. W. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01040 · Cited by 93 · see citing works
VisualCOMET: Reasoning About the Dynamic Context of a Still Image
Park, J. S., Bhagavatula, C., Mottaghi, R., Farhadi, A., & Choi, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58558-7_30 · Cited by 91 · see citing works
Transferable Representation Learning in Vision-and-Language Navigation
Huang, H., Jain, V., Mehta, H., Ku, A., Magalhaes, G., Baldridge, J., & Ie, E. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00750 · Cited by 82 · see citing works
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action
Lu, J., Clark, C., Lee, S.-H., Zhang, Z., Khosla, S., Marten, R., Hoiem, D., & Kembhavi, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02497 · Cited by 98 · see citing works
Izadi, S., & Forouzanfar, M. (2024) · AI · doi:10.3390/ai5020041 · Cited by 102 · see citing works
VD-BERT: A Unified Vision and Dialog Transformer with BERT
Wang, Y., Joty, S., Lyu, M. R.-T., King, I., Xiong, C., & Hoi, S. C. H. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.269 · Cited by 91 · see citing works
Answer Them All! Toward Universal Visual Question Answering Models
Shrestha, R., Kafle, K., & Kanan, C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01072 · Cited by 85 · see citing works
Image-Question-Answer Synergistic Network for Visual Dialog
Guo, D., Xu, C., & Tao, D. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01068 · Cited by 78 · see citing works
Sequential Attention GAN for Interactive Image Editing
Cheng, Y., Gan, Z., Li, Y., Liu, J., & Gao, J. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413551 · Cited by 78 · see citing works
VideoLLM-online: Online Video Large Language Model for Streaming Video
Chen, J., Lv, Z., Wu, S., Lin, K. Q., Song, C., Gao, D., Liu, J.-W., Gao, Z., Mao, D., & Shou, M. Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01742 · Cited by 83 · see citing works
Active visual information gathering for vision-language navigation
Wang, H., Wang, W., Shu, T., Liang, W., & Shen, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58542-6_19 · Cited by 74 · see citing works
Bilinear Graph Networks for Visual Question Answering
Guo, D., Xu, C., & Tao, D. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3104937 · Cited by 76 · see citing works
Radial Graph Convolutional Network for Visual Question Generation
Xu, X., Wang, T., Yang, Y., Hanjalić, A., & Shen, H. T. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2986029 · Cited by 72 · see citing works
Video Captioning With Object-Aware Spatio-Temporal Correlation and Aggregation
Zhang, J., & Peng, Y. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2988435 · Cited by 73 · see citing works
A Simple Baseline for Audio-Visual Scene-Aware Dialog
Schwartz, I., Schwing, A. G., & Hazan, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01283 · Cited by 69 · see citing works
Shuffle-Then-Assemble: Learning Object-Agnostic Visual Relationship Features
Yang, X., Zhang, H., & Cai, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01258-8_3 · Cited by 69 · see citing works
Bridging Text and Video: A Universal Multimodal Transformer for Audio-Visual Scene-Aware Dialog
Li, Z., Li, Z., Zhang, J., Feng, Y., & Zhou, J. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3065823 · Cited by 86 · see citing works
Making History Matter: History-Advantage Sequence Training for Visual Dialog
Yang, T., Zha, Z.-J., & Zhang, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00265 · Cited by 66 · see citing works
Unshuffling Data for Improved Generalization in Visual Question Answering
Teney, D., Abbasnejad, E., & van den Hengel, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00145 · Cited by 66 · see citing works
Violin: A Large-Scale Dataset for Video-and-Language Inference
Liu, J., Chen, W., Cheng, Y., Gan, Z., Yu, L., Yang, Y., & Liu, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01091 · Cited by 62 · see citing works
EmoSen: Generating Sentiment and Emotion Controlled Responses in a Multimodal Dialogue System
Firdaus, M., Chauhan, H., Ekbal, A., & Bhattacharyya, P. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3015491 · Cited by 59 · see citing works
VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs
Lin, X., Bertasius, G., Wang, J., Chang, S., Parikh, D., & Torresani, L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00693 · Cited by 57 · see citing works
Self-citationCollecting and Characterizing Natural Language Utterances for Specifying Data Visualizations
Srinivasan, A., Nyapathy, N., Lee, B., Drucker, S. M., & Stasko, J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445400 · Cited by 58 · see citing works
Multi-Task Learning of Hierarchical Vision-Language Representation
Nguyen, D.-K., & Okatani, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01074 · Cited by 55 · see citing works
Environment-Agnostic Multitask Learning for Natural Language Grounded Navigation
Wang, X. E., Jain, V., Ie, E., Wang, W. Y., Kozareva, Z., & Ravi, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58586-0_25 · Cited by 54 · see citing works
Look Before you Speak: Visually Contextualized Utterances
Seo, P. H., Nagrani, A., & Schmid, C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01660 · Cited by 55 · see citing works
Context-Aware Graph Inference With Knowledge Distillation for Visual Dialog
Guo, D., Wang, H., & Wang, M. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3085755 · Cited by 53 · see citing works
Comparison of VGG and ResNet used as Encoders for Image Captioning
Atliha, V., & Šešok, D. (2020) · IEEE Open Conference of Electrical, Electronic and Information Sciences (eStream) · doi:10.1109/estream50540.2020.9108880 · Cited by 52 · see citing works
Remote sensing image caption generation via transformer and reinforcement learning
Shen, X., Liu, B., Zhou, Y. Z., & Zhao, J. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09294-7 · Cited by 52 · see citing works
Prompt-Based Learning for Unpaired Image Captioning
Zhu, P., Wang, X., Zhu, L., Sun, Z., Zheng, W., Wang, Y., & Chen, C. W. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3265842 · Cited by 52 · see citing works
Nguyen, V.-Q., Suganuma, M., & Okatani, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58586-0_14 · Cited by 53 · see citing works
Automatic Reminiscence Therapy for Dementia
Carós, M., Garolera, M., Radeva, P. I., & Giró-i-Nieto, X. (2020) · International Conference on Multimedia Retrieval · doi:10.1145/3372278.3391927 · Cited by 56 · see citing works
Recurrent Attention Network with Reinforced Generator for Visual Dialog
Fan, H., Zhu, L., Yang, Y., & Wu, F. Q. (2020) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3390891 · Cited by 49 · see citing works
VLC-BERT: Visual Question Answering with Contextualized Commonsense Knowledge
Ravi, S., Chinchure, A., Sigal, L., Liao, R., & Shwartz, V. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00121 · Cited by 49 · see citing works
Image synthesis: a review of methods, datasets, evaluation metrics, and future outlook
Baraheem, S. S., Le, T.-N., & Nguyen, T. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10434-2 · Cited by 48 · see citing works
Efficient GPT-4V level multimodal large language model for deployment on edge devices
Yao, Y., Yu, T., Zhang, A., Wang, C., Cui, J., Zhu, H., Cai, T., Chen, C., Li, H., Zhao, W., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-61040-5 · Cited by 50 · see citing works
Chart Question Answering: State of the Art and Future Directions
Hoque, E., Kavehzadeh, P., & Masry, A. A. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14573 · Cited by 50 · see citing works
Video Question Answering with Spatio-Temporal Reasoning
Jang, Y., Song, Y., Kim, C. D., Yu, Y., Kim, Y., & Kim, G. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01189-x · Cited by 46 · see citing works
LION : Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge
Chen, G., Shen, L., Shao, R., Deng, X., & Nie, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02506 · Cited by 53 · see citing works
Vision + X: A Survey on Multimodal Learning in the Light of Data
Zhu, Y., Wu, Y., Sebe, N., & Yan, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3420239 · Cited by 46 · see citing works
UTC: A Unified Transformer with Inter-Task Contrastive Learning for Visual Dialog
Chen, C., Tan, Z., Cheng, Q., Jiang, X., Liu, Q., Zhu, Y., & Gu, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01757 · Cited by 44 · see citing works
iVQA: Inverse Visual Question Answering
Liu, F., Xiang, T., Hospedales, T. M., Yang, W., & Sun, C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00898 · Cited by 43 · see citing works
Adaptive Spatio-Temporal Graph Enhanced Vision-Language Representation for Video QA
Jin, W., Zhao, Z., Cao, X., Zhu, J., He, X., & Zhuang, Y. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3076556 · Cited by 41 · see citing works
Challenges and Prospects in Vision and Language Research
Kafle, K., Shrestha, R., & Kanan, C. (2019) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2019.00028 · Cited by 39 · see citing works
Toward Region-Aware Attention Learning for Scene Graph Generation
Liu, A.-A., Tian, H., Xu, N., Nie, W., Zhang, Y., & Kankanhalli, M. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3086066 · Cited by 40 · see citing works
Multitask Learning for Visual Question Answering
Ma, J., Liu, J., Lin, Q., Wu, B., Wang, Y., & You, Y. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3105284 · Cited by 39 · see citing works
Long-Form Video Question Answering via Dynamic Hierarchical Reinforced Networks
Zhao, Z., Zhang, Z., Xiao, S., Xiao, Z., Yan, X., Yu, J., Cai, D., & Wu, F. Q. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2922062 · Cited by 39 · see citing works
Knowledge-Based Embodied Question Answering
Tan, S., Ge, M., Guo, D., Liu, H., & Sun, F. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3277206 · Cited by 37 · see citing works
Continual Learning for Smart City: A Survey
Yang, L., Luo, Z., Zhang, S., Teng, F., & Li, T. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3447123 · Cited by 41 · see citing works
Neural Multimodal Belief Tracker with Adaptive Attention for Dialogue Systems
Zhang, Z., Liao, L., Huang, M., Zhu, X., & Chua, T. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313598 · Cited by 36 · see citing works
Learning Dual Encoding Model for Adaptive Visual Understanding in Visual Dialogue
Yu, J., Jiang, X., Qin, Z., Zhang, W., Hu, Y., & Wu, Q. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3034494 · Cited by 36 · see citing works
Exploring Video Captioning Techniques: A Comprehensive Survey on Deep Learning Methods
Islam, S., Dash, A., Seum, A., Raj, A. H., Hossain, T., & Shah, F. M. (2021) · SN Computer Science · doi:10.1007/s42979-021-00487-x · Cited by 36 · see citing works
Towards General Purpose Vision Systems: An End-to-End Task-Agnostic Vision-Language Architecture
Gupta, T., Kamath, A., Kembhavi, A., & Hoiem, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01591 · Cited by 54 · see citing works
FLIPDIAL: A Generative Model for Two-Way Visual Dialogue
Dokania, P. K., Torr, P. H. S., Siddharth, N., & Massiceti, D. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00638 · Cited by 33 · see citing works
- Authors
- Das, A., Kottur, S., Gupta, K., Singh, A., Yadav, D., Moura, J. M. F., Parikh, D., & Batra, D.
- Affiliations
- Georgia Institute of Technology (US)
- Carnegie Mellon University (US)
- University of California, Berkeley (US)
- Virginia Tech (US)
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 1080-1089
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4249013746
- Cited by
- 445
- References
- 59
- Citations by year
- 2026: 19 · 2025: 45 · 2024: 51 · 2023: 57 · 2022: 67 · 2021: 78 · 2020: 52 · 2019: 49 · 2018: 24 · 2017: 3
- FWCI
- 10.23
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- visual dialogue, visual question answering, human-AI dialogue, chatbots, encoder-decoder architecture, memory networks, decision fusion, mean reciprocal rank
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review