The DOI matches its Crossref record; 105 of the 105 references it cites were found in a registry or an index.
Visual question answering: A survey of methods and datasets
Open accessAbstract
Visual Question Answering (VQA) is a challenging task that has received increasing attention from both the computer vision and the natural language processing communities. Given an image and a question in natural language, it requires reasoning over visual elements of the image and general knowledge to infer the correct answer. In the first part of this survey, we examine the state of the art by comparing modern approaches to the problem. We classify methods by their mechanism to connect the visual and textual modalities. In particular, we examine the common approach of combining convolutional and recurrent neural networks to map images and questions to a common feature space. We also discuss memory-augmented and modular architectures that interface with structured knowledge bases. In the second part of this survey, we review the datasets available for training and evaluating VQA systems. The various datatsets contain questions at different levels of complexity, which require different capabilities and types of reasoning. We examine in depth the question/answer pairs from the Visual Genome project, and evaluate the relevance of the structured annotations of images with scene graphs for VQA. Finally, we discuss promising future directions for the field, in particular the connection to structured knowledge bases and the use of natural language processing models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“This allows retrieving information that is not present in the common visual datasets such as ImageNet [14] or COCO [45], which are only labeled with classes, bounding boxes, and/or captions.”
MethodGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107
Freebase: a collaboratively created graph database for structuring human knowledge
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., & Taylor, J. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376746 · Cited by 4,852
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Jurafsky, D. S., & Martin, J. (2000) · Cited by 4,132
Prud’hommeaux, E. (2008) · Medical Entomology and Zoology · Cited by 3,097
Understanding natural language
Winograd, T. (1972) · Cognitive Psychology · doi:10.1016/0010-0285(72)90002-3 · Cited by 2,799
DBpedia: A Nucleus for a Web of Open Data
Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., & Ives, Z. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-76298-0_52 · Cited by 2,454
Young, P., Lai, A., Hodosh, M., & Hockenmaier, J. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00166 · Cited by 2,426
Toward an architecture for never-ending language learning
Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka, E., & Mitchell, T. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7519 · Cited by 2,000
Stacked Attention Networks for Image Question Answering
Yang, Z., He, X., Gao, J., Deng, L., & Smola, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.10 · Cited by 1,998
Microsoft COCO Captions: Data Collection and Evaluation Server
Chen, X., Fang, H., Lin, T.-Y., Vedantam, R., Gupta, S., Dollar, P., & Zitnick, C. L. (2015) · arXiv · doi:10.48550/arxiv.1504.00325 · Cited by 1,614
Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
Fukui, A., Park, D. H., Yang, D., Rohrbach, A., Darrell, T., & Rohrbach, M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1044 · Cited by 1,582
ConceptNet — A Practical Commonsense Reasoning Tool-Kit
Liu, H., & Singh, P. (2004) · BT Technology Journal · doi:10.1023/b:bttj.0000047600.45421.6d · Cited by 1,558
Open information extraction from the web
Banko, M., Cafarella, M., Soderland, S. G., Broadhead, M., & Etzioni, O. (2007) · International Joint Conference on Artificial Intelligence · Cited by 1,398
Sukhbaatar, S., Szlam, A., Weston, J., & Fergus, R. (2015) · arXiv · doi:10.48550/arxiv.1503.08895 · Cited by 1,345
Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics
Hodosh, M., Young, P., & Hockenmaier, J. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3994 · Cited by 1,314
YAGO2: A spatially and temporally enhanced knowledge base from Wikipedia
Hoffart, J., Suchanek, F. M., Berberich, K., & Weikum, G. (2013) · Artificial Intelligence · doi:10.1016/j.artint.2012.06.001 · Cited by 1,232
Generation and Comprehension of Unambiguous Object Descriptions
Mao, J., Huang, J., Toshev, A., Camburu, O., Yuille, A., & Murphy, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.9 · Cited by 1,174
Identifying Relations for Open Information Extraction
Fader, A., Soderland, S. G., & Etzioni, O. (2011) · Cited by 1,144
Hierarchical Question-Image Co-Attention for Visual Question Answering
Lu, J., Yang, J., Batra, D., & Parikh, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00061 · Cited by 1,129
Robotics: Science and Systems (RSS)
Trinkle, J., & Matsuoka, Y. (2010) · AI Magazine · doi:10.1609/aimag.v31i1.2237 · Cited by 1,081
Resource description framework: metadata and its applications
Candan, K. S., Liu, H., & Suvarna, R. (2001) · ACM SIGKDD Explorations Newsletter · doi:10.1145/507533.507536 · Cited by 1,067
ReferItGame: Referring to Objects in Photographs of Natural Scenes
Kazemzadeh, S., Ordonez, V., Matten, M., & Berg, T. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1086 · Cited by 1,048
The Stanford typed dependencies representation
de Marneffe, M.-C., & Manning, C. D. (2008) · Coling 2008: Proceedings of the workshop on Cross-Framework and Cross-Domain Parser Evaluation - CrossParser '08 · doi:10.3115/1608858.1608859 · Cited by 918
Andreas, J., Rohrbach, M., Darrell, T., & Klein, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.12 · Cited by 908
VQA: Visual Question Answering
Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Batra, D., & Parikh, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.00468 · Cited by 900
Visual7W: Grounded Question Answering in Images
Zhu, Y., Groth, O., Bernstein, M., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.540 · Cited by 794 · cited 8×
“The Visual7w [100] dataset is a subset of the Visual Genome that contains additional annotations.”
BackgroundMethodResultInfluentialSemantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base
Yih, W.-T., Chang, M.-W., He, X., & Gao, J. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1128 · Cited by 756
Deep Fragment Embeddings for Bidirectional Image Sentence Mapping
Karpathy, A., Joulin, A., & Fei-Fei, L. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.5679 · Cited by 739
Language to Logical Form with Neural Attention
Dong, L., & Lapata, M. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1004 · Cited by 700
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merriënboer, B., Joulin, A., & Mikolov, T. (2015) · arXiv · doi:10.48550/arxiv.1502.05698 · Cited by 670
Ask Your Neurons: A Neural-Based Approach to Answering Questions about Images
Malinowski, M., Rohrbach, M., & Fritz, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.9 · Cited by 631
Dynamic Memory Networks for Visual and Textual Question Answering
Xiong, C., Merity, S., & Socher, R. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.01417 · Cited by 603
VQA: Visual Question Answering: www.visualqa.org
Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Parikh, D., & Batra, D. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0966-6 · Cited by 570
Show and Tell: A Neural Image Caption GeneratorPreprint
Vinyals, O., Toshev, A., Samy Bengio, A., & Erhan, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.4555 · Cited by 553
Learning Dependency-Based Compositional Semantics
Liang, P., Jordan, M. I., & Klein, D. (2013) · Computational Linguistics · doi:10.1162/coli_a_00127 · Cited by 546
Natural Language Object Retrieval
Hu, R., Xu, H., Rohrbach, M., Feng, J., Saenko, K., & Darrell, T. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.493 · Cited by 532
Large-scale Simple Question Answering with Memory Networks
Bordes, A., Usunier, N., Chopra, S., & Weston, J. (2015) · arXiv · doi:10.48550/arxiv.1506.02075 · Cited by 528 · cited 1×
“They were described in [42], with a number of variants proposed concurrently [83, 70, 8, 59].”
BackgroundWhere to Look: Focus Regions for Visual Question Answering
Shih, K. J., Singh, S., & Hoiem, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.499 · Cited by 493
1985 ACM-SIGMOD international conference on management of data
(1985) · Information & Management · doi:10.1016/0378-7206(85)90026-6 · Cited by 485
FVQA: Fact-Based Visual Question Answering
Wang, P., Wu, Q., Shen, C., Dick, A., & van den Hengel, A. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2754246 · Cited by 479
Self-citationWhat Value Do Explicit High Level Concepts Have in Vision to Language Problems?
Wu, Q., Shen, C., Liu, L., Dick, A., & Van Den Hengel, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.29 · Cited by 452
Self-citationVerb Semantics and Lexical SelectionPreprint
Wu, Z., & Palmer, M. S. (1994) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9406033 · Cited by 433
Open information extraction: the second generation
Etzioni, O., Fader, A., Christensen, J., Soderland, S. G., & Mausam, M. (2011) · doi:10.5591/978-1-57735-516-8/ijcai11-012 · Cited by 430
Online Learning of Relaxed CCG Grammars for Parsing to Logical Form
Zettlemoyer, L., & Collins, M. (2007) · Cited by 387
Ask Me Anything: Free-Form Visual Question Answering Based on Knowledge from External Sources
Wu, Q., Wang, P., Shen, C., Dick, A., & Van Den Hengel, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.500 · Cited by 355
Self-citationComposing Simple Image Descriptions using Web-scale N-grams
Li, S., Kulkarni, G. A., Berg, T. L., Berg, A. C., & Choi, Y. (2011) · Cited by 353
Image Question Answering Using Convolutional Neural Network with Dynamic Parameter Prediction
Noh, H., Seo, P. H., & Han, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.11 · Cited by 345
Describing Videos by Exploiting Temporal StructurePreprint
Yao, L., Torabi, A., Cho, K., Ballas, N., Pal, C., Larochelle, H., & Courville, A. C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.08029 · Cited by 330
Visual Turing test for computer vision systems
Geman, D., Geman, S., Hallonquist, N., & Younes, L. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1422953112 · Cited by 321
Yin and Yang: Balancing and Answering Binary Visual Questions
Zhang, P., Goyal, Y., Summers-Stay, D., Batra, D., & Parikh, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.542 · Cited by 312
A Neural Network for Factoid Question Answering over Paragraphs
Iyyer, M., Boyd-Graber, J., Claudino, L., Socher, R., & Daumé III, H. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1070 · Cited by 301
ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answering
Chen, K., Wang, J., Chen, L.-C., Gao, H., Xu, W., & Nevatia, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05960 · Cited by 254
Simple Baseline for Visual Question Answering
Zhou, B., Tian, Y., Sukhbaatar, S., Szlam, A. D., & Fergus, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.02167 · Cited by 254
Learning to Answer Questions from Image Using Convolutional Neural Network
Ma, L., Lu, Z., & Li, H. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10442 · Cited by 253
A survey on question answering technology from an information retrieval perspective
Kolomiyets, O., & Moens, M.-F. (2011) · Information Sciences · doi:10.1016/j.ins.2011.07.047 · Cited by 252
A Joint Model of Language and Perception for Grounded Attribute Learning
Matuszek, C., FitzGerald, N., Zettlemoyer, L., Bo, L., & Fox, D. (2012) · Maryland Shared Open Access Repository · doi:10.13016/m2ehu6-1m90 · Cited by 248
Explicit Knowledge-based Reasoning for Visual Question Answering
Wang, P., Wu, Q., Shen, C., Dick, A., & van den Hengel, A. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/179 · Cited by 233
Self-citationAre You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering
Gao, H., Mao, J., Zhou, J., Huang, Z., Wang, L., & Xu, W. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05612 · Cited by 216
Multimodal Residual Learning for Visual QA
Kim, J.-H., Lee, S., Kwak, D., Heo, M.-O., Kim, J., Ha, J.-W., & Zhang, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.01455 · Cited by 209
Bringing Semantics into Focus Using Visual Abstraction
Zitnick, C. L., & Parikh, D. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.387 · Cited by 193
Empirical Methods in Natural Language Processing
Rodríguez, K. J., Bryant, M., Blanke, T., & Luszczynska, M. (2012) · Cited by 178
Learning the Visual Interpretation of Sentences
Zitnick, C. L., Parikh, D., & Vanderwende, L. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.211 · Cited by 171
Open information extraction for the web
Etzioni, O., & Banko, M. (2009) · Cited by 152
A Focused Dynamic Attention Model for Visual Question AnsweringPreprint
Ilievski, I., Yan, S., & Feng, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.01485 · Cited by 150
Answer-Type Prediction for Visual Question Answering
Kafle, K., & Kanan, C. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.538 · Cited by 148
Image Question Answering: A Visual Semantic Embedding Model and a New Dataset.Preprint
Ren, M., Kiros, R., & Zemel, R. S. (2015) · arXiv (Cornell University) · Cited by 134
Joint Video and Text Parsing for Understanding Events and Answering Queries
Tu, K., Meng, M., Lee, M. W., Choe, T. E., & Zhu, S. C. (2014) · IEEE Multimedia · doi:10.1109/mmul.2014.29 · Cited by 108
Learning to Compose Neural Networks for Question AnsweringPreprint
Andreas, J., Rohrbach, M., Darrell, T., & Klein, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.01705 · Cited by 100
Adopting Abstract Images for Semantic Scene Understanding
Zitnick, C. L., Vedantam, R., & Parikh, D. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2366143 · Cited by 89
Zero-Shot Learning via Visual Abstraction
Antol, S., Zitnick, C. L., & Parikh, D. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10593-2_27 · Cited by 87
Learning Common Sense through Visual Abstraction
Vedantam, R., Lin, X., Batra, T., Zitnick, C. L., & Parikh, D. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.292 · Cited by 86
Training Recurrent Answering Units with Joint Loss Minimization for VQAPreprint
Noh, H., & Han, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.03647 · Cited by 83
A Joint Model of Language and Perception for Grounded Attribute LearningPreprint
Matuszek, C., FitzGerald, N., Zettlemoyer, L., Bo, L., & Fox, D. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.6423 · Cited by 82
Building a Large-scale Multimodal Knowledge Base System for Answering Visual QueriesPreprint
Zhu, Y., Zhang, C., Ré, C., & Fei-Fei, L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.05670 · Cited by 76
Weakly Supervised Memory Networks.Preprint
Sukhbaatar, S., Szlam, A. D., Weston, J., & Fergus, R. (2015) · arXiv (Cornell University) · Cited by 76
Visual Madlibs: Fill in the blank Image Generation and Question AnsweringPreprint
Yu, L., Park, E., Berg, A. C., & Berg, T. L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00278 · Cited by 75
Towards Neural Network-based ReasoningPreprint
Peng, B., Lu, Z., Li, H., & Wong, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.05508 · Cited by 74
Predicting Object Dynamics in Scenes
Fouhey, D. F., & Zitnick, C. L. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.260 · Cited by 73
A Survey of Current Datasets for Vision and Language Research
Ferraro, F., Mostafazadeh, N., Huang, T.-H., Vanderwende, L., Devlin, J., Galley, M., & Mitchell, M. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1021 · Cited by 68 · cited 1×
“Those datasets go beyond the scope of this article (see [19] for a review), but are a potential source of additional training data for VQA since they combine images with textual annotations.”
BackgroundRecurrent Neural Networks with External Memory for Spoken Language Understanding
Peng, B., Yao, K., Li, J., & Wong, K. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-25207-0_3 · Cited by 68
Acquiring Comparative Commonsense Knowledge from the Web
Tandon, N., de Melo, G., & Weikum, G. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v28i1.8735 · Cited by 65
DualNet: Domain-invariant network for visual question answering
Saito, K., Shin, A., Ushiku, Y., & Harada, T. (2017) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2017.8019436 · Cited by 64
Don't just listen, use your imagination: Leveraging visual common sense for non-visual tasks
Lin, X., & Parikh, D. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298917 · Cited by 61
Robust spoken instruction understanding for HRI
Cantrell, R., Scheutz, M. J., Schermerhorn, P., & Wu, X. (2010) · Cited by 59
Compositional Memory for Visual Question AnsweringPreprint
Jiang, A., Wang, F., Porikli, F., & Li, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05676 · Cited by 44
Recurrent Neural Networks with External Memory for Language UnderstandingPreprint
Peng, B., & Yao, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00195 · Cited by 37
Conversational robots: building blocks for grounding word meaning
Roy, D. K., Hsiao, K.-Y., & Mavridis, N. (2003) · doi:10.3115/1119212.1119222 · Cited by 34
Toward Interactive Grounded Language Acqusition
Kollar, T., Krishnamurthy, J., & Strimel, G. P. (2013) · Robotics: Science and Systems · doi:10.15607/rss.2013.ix.005 · Cited by 33
Wu, Q., Shen, C., van den Hengel, A., Wang, P., & Dick, A. (2016) · arXiv (Cornell University) · Cited by 32
Self-citationDualNet: Domain-Invariant Network for Visual Question AnsweringPreprint
Saito, K., Shin, A., Ushiku, Y., & Harada, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.06108 · Cited by 20
Singh, V., & Dwivedi, S. K. (2014) · International Journal of Information Retrieval Research · doi:10.4018/ijirr.2014070102 · Cited by 5
Cited by
Detecting Visual Relationships with Deep Relational Networks
Dai, B., Zhang, Y., & Lin, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.352 · Cited by 502
Unleashing the potential of prompt engineering for large language models
Chen, B., Zhang, Z., Langrené, N., & Zhu, S. (2025) · Patterns · doi:10.1016/j.patter.2025.101260 · Cited by 490
The multi-modal fusion in visual question answering: a review of attention mechanisms
Lu, S., Liu, M., Yin, L., Yin, Z., Liu, X., & Zheng, W. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1400 · Cited by 477
Inferring and Executing Programs for Visual Reasoning
Johnson, J., Hariharan, B., Van Der Maaten, L., Hoffman, J., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.325 · Cited by 467
Graph-Structured Representations for Visual Question Answering
Teney, D., Liu, L., & Van Den Hengel, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.344 · Cited by 447
Self-citation
Metadata
- Authors
- Wu, Q., Teney, D., Wang, P., Shen, C., Dick, A., & van den Hengel, A.
- Venue
- Computer Vision and Image Understanding
- Publisher
- Elsevier BV
- Published
- 2017-10
- Volume
- 163
- Pages
- 21-40
- ISSN
- 1077-3142 (ISSN-L)
- OpenAlex
- W2964138343
- Cited by
- 438
- References
- 105
- FWCI
- 18.53
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.