VizWiz Grand Challenge: Answering Visual Questions from Blind People
Open accessAbstract
The study of algorithms to automatically answer visual questions currently is motivated by visual question answering (VQA) datasets constructed in artificial VQA settings. We propose VizWiz, the first goal-oriented VQA dataset arising from a natural VQA setting. VizWiz consists of over 31,000 visual questions originating from blind people who each took a picture using a mobile phone and recorded a spoken question about it, together with 10 crowdsourced answers per visual question. VizWiz differs from the many existing VQA datasets because (1) images are captured by blind photographers and so are often poor quality, (2) questions are spoken and so are more conversational, and (3) often visual questions cannot be answered. Evaluation of modern algorithms for answering visual questions and deciding if a visual question is answerable reveals that VizWiz is a challenging dataset. We introduce this dataset to encourage a larger community to develop more generalized algorithms that can assist blind people.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
VizWiz had over 40% disagreement between crowdworkers and in-house experts on whether a photo contained privacy or quality issues.
“Prior work reported over 40% disagreement between crowdworkers and in-house experts on whether a photo contained privacy or quality issues [36]”
From Disability-First AI Dataset Annotation: Co-designing Stuttered Speech Annotation Guidelines with People Who Stutter (2026) · cites this paper as [36]
The supplementary section “I - Filtering Visual Questions” reports disagreement among three domain experts: only one expert marked 44 of the 64 visual questions flagged for removal. Section 3.2 describes a separate, sequential review by crowd workers and in-house experts, but does not report over 40% disagreement between those groups. The claim appears to conflate the expert-only disagreement with the two-stage review. (LLM, conf=0.88)
Experimental — accuracy not yet measuredAcademic scholars and industry corporations have emphasized the pressing need for more disability-centered datasets.
“Although academic scholars and industry corporations have emphasized the pressing need for more disability-centered datasets [20, 38, 55, 79, 97, 111, 124]”
From "It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models (2026) · cites this paper as [55]
The Introduction identifies a gap in existing VQA datasets: none addressed images and questions from blind people. It introduces VizWiz to address that gap. This supports the academic call for disability-centered datasets, but the paper does not establish that industry corporations also emphasized the need. (LLM, conf=0.84)
Experimental — accuracy not yet measuredPrior efforts of disability-first datasets often focus on the data collection process.
“While prior efforts of disabilityfirst datasets often focus on the data collection process [36, 47, 63, 68, 88, 98], our study represents an important step in centering disabled people and their expertise throughout the AI development pipeline.”
From Disability-First AI Dataset Annotation: Co-designing Stuttered Speech Annotation Guidelines with People Who Stutter (2026) · cites this paper as [36]
Blind users often struggle to take high-quality photographs, and many visual questions go unanswered because images fail to capture the relevant objects.
“From analyzing VizWiz images, Gurari et al. [55] found that blind users often struggle to take high-quality photographs, and many visual questions go unanswered because images fail to capture the relevant objects [55].”
From "It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models (2026) · cites this paper as [55]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 2× · see citing works
“Following prior work [14], we first computed the average image from all images in VizWiz.”
BackgroundMethodMicrosoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · cited 4× · see citing works
“0: Real [7] MSCOCO [25] Crowd workers (AMT) Typed Visual Madlibs [43] MSCOCO [25] Automatically generated (templates) ——– FM-IQA [16] MSCOCO [25] Crowd workers (Baidu) Typed KB-VQA [40] MSCOCO [25] In-house participants Typed COCO-QA [32] MSCOCO [25] Automatically generated (captions) ——– VQA v2.”
BackgroundMethodInfluentialImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 2× · see citing works
“We employ five metrics commonly used for evaluating VQA and image description algorithms: Accuracy [7], CIDEr [38], BLEU4 [29], and ME-”
MethodIndoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760 · cited 1× · see citing works
“DAQUAR [28] NYU Depth V2 [34] Automatically generated (templates) ——– VQA v1.”
MethodVisual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107 · see citing works
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902 · see citing works
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · cited 2× · see citing works
“We employ five metrics commonly used for evaluating VQA and image description algorithms: Accuracy [7], CIDEr [38], BLEU4 [29], and ME-”
MethodVQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487 · see citing works
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,682 · see citing works
SUN database: Large-scale scene recognition from abbey to zoo
Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., & Torralba, A. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539970 · Cited by 3,283 · cited 1× · see citing works
“Historically, progress in the research community on a given computer vision problem is typically preceded by a largescale, publicly-shared dataset [12, 25, 30, 33, 42].”
BackgroundCLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.215 · Cited by 1,781 · see citing works
Thomée, B., Shamma, D. A., Friedland, G. H., Elizalde, B., Ni, K. S., Poland, D., Borth, D., & Li, L.-J. (2016) · Communications of the ACM · doi:10.1145/2812802 · Cited by 1,632 · see citing works
Microsoft COCO Captions: Data Collection and Evaluation Server
Chen, X., Fang, H., Lin, T.-Y., Vedantam, R., Gupta, S., Dollar, P., & Zitnick, C. L. (2015) · arXiv · doi:10.48550/arxiv.1504.00325 · Cited by 1,614 · cited 1× · see citing works
“Historically, progress in the research community on a given computer vision problem is typically preceded by a largescale, publicly-shared dataset [12, 25, 30, 33, 42].”
BackgroundVQA: Visual Question AnsweringPreprint
Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Batra, D., & Parikh, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.00468 · Cited by 968 · cited 10× · see citing works
“For example, all images were either taken from an existing vision dataset (e.g., MSCOCO [28]) or artificially constructed (e.g., Abstract Scenes [8], computer graphics [7, 21]).”
BackgroundMethodInfluentialAndreas, J., Rohrbach, M., Darrell, T., & Klein, D. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.12 · Cited by 933 · cited 9× · see citing works
“0: Real [17] MSCOCO [25] Crowd workers (AMT) Typed Visual7W [45] MSCOCO [25] Crowd workers (AMT) Typed CLEVR [19] Synthetic Shapes Automatically generated (templates) ——– SHAPES [6] Synthetic Shapes Automatically generated (templates) ——– Visual Genome [23] MSCOCO [25] & YFCC100M [35] Crowd workers (AMT) Typed FVQA [3…”
BackgroundMethodInfluentialSUN attribute database: Discovering, annotating, and recognizing scene attributes
Patterson, G., & Hays, J. H. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247998 · Cited by 878 · see citing works
Visual7W: Grounded Question Answering in Images
Zhu, Y., Groth, O., Bernstein, M., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.540 · Cited by 794 · cited 6× · see citing works
“0: Real [17] MSCOCO [25] Crowd workers (AMT) Typed Visual7W [45] MSCOCO [25] Crowd workers (AMT) Typed CLEVR [19] Synthetic Shapes Automatically generated (templates) ——– SHAPES [6] Synthetic Shapes Automatically generated (templates) ——– Visual Genome [23] MSCOCO [25] & YFCC100M [35] Crowd workers (AMT) Typed FVQA [3…”
BackgroundMethodInfluentialExploring Models and Data for Image Question AnsweringPreprint
Ren, M., Kiros, R., & Zemel, R. S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.02074 · Cited by 618 · cited 6× · see citing works
“Over the past three years, a plethora of VQA datasets have been publicly shared to encourage a larger community to collaborate on developing algorithms that answer visual questions [6, 7, 16, 17, 19, 20, 23, 28, 32, 39, 40, 43, 45].”
BackgroundMethodInfluentialBigham, J. P., Jayant, C., Ji, H., Little, G., Miller, A. E., Miller, R. C., Tatarowicz, A., White, B., White, S., & YEH, T. P. (2010) · International Cross-Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1806020 · Cited by 536 · see citing works
Self-citationFVQA: Fact-Based Visual Question Answering
Wang, P., Wu, Q., Shen, C., Dick, A., & van den Hengel, A. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2754246 · Cited by 490 · cited 6× · see citing works
“0: Real [17] MSCOCO [25] Crowd workers (AMT) Typed Visual7W [45] MSCOCO [25] Crowd workers (AMT) Typed CLEVR [19] Synthetic Shapes Automatically generated (templates) ——– SHAPES [6] Synthetic Shapes Automatically generated (templates) ——– Visual Genome [23] MSCOCO [25] & YFCC100M [35] Crowd workers (AMT) Typed FVQA [3…”
BackgroundMethodInfluentialDas, A., Kottur, S., Gupta, K., Singh, A., Yadav, D., Moura, J. M. F., Parikh, D., & Batra, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.121 · Cited by 445 · cited 2× · see citing works
“Our findings show the statistics found in the existing artificially constructed VQA datasets, nicely summarized in [13] and [20], match statistics observed in practice.”
ResultAre You Talking to a Machine? Dataset and Methods for Multilingual Image Question AnsweringPreprint
Gao, H., Mao, J., Zhou, J., Huang, Z., Wang, L., & Xu, W. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05612 · Cited by 437 · see citing works
A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain InputPreprint
Malinowski, M., & Fritz, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0210 · Cited by 432 · cited 7× · see citing works
“Over the past three years, a plethora of VQA datasets have been publicly shared to encourage a larger community to collaborate on developing algorithms that answer visual questions [6, 7, 16, 17, 19, 20, 23, 28, 32, 39, 40, 43, 45].”
BackgroundMethodInfluentialDas, A., Kottur, S., Gupta, K., Singh, A., Jain, D., Moura, J. M. F., Parikh, D., & Batra, D. (2017) · Cited by 432 · see citing works
Image Description using Visual Dependency Representations
Elliott, D., & Keller, F. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1128 · Cited by 282 · cited 1× · see citing works
“We evaluate with respect to four metrics: Accuracy [8], CIDEr [42], BLEU4 [32], and METEOR [16].”
MethodAn Analysis of Visual Question Answering Algorithms
Kafle, K., & Kanan, C. (2017) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2017.217 · Cited by 272 · cited 7× · see citing works
“0: Real [17] MSCOCO [25] Crowd workers (AMT) Typed Visual7W [45] MSCOCO [25] Crowd workers (AMT) Typed CLEVR [19] Synthetic Shapes Automatically generated (templates) ——– SHAPES [6] Synthetic Shapes Automatically generated (templates) ——– Visual Genome [23] MSCOCO [25] & YFCC100M [35] Crowd workers (AMT) Typed FVQA [3…”
BackgroundMethodInfluential(2017) · doi:10.1145/2998181.2998364 · Cited by 258 · see citing works
Explicit Knowledge-based Reasoning for Visual Question Answering
Wang, P., Wu, Q., Shen, C., Dick, A., & van den Hengel, A. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/179 · Cited by 240 · cited 7× · see citing works
“Over the past three years, a plethora of VQA datasets have been publicly shared to encourage a larger community to collaborate on developing algorithms that answer visual questions [6, 7, 16, 17, 19, 20, 23, 28, 32, 39, 40, 43, 45].”
BackgroundMethodInfluentialVisual challenges in the everyday lives of blind people
Brady, E. L., Morris, M. R., Zhong, Y., White, S., & Bigham, J. P. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2481291 · Cited by 236 · cited 2× · see citing works
“Our work builds off previous work [8] which established a mobile phone application that supported blind people to ask over 70,000 visual questions [10] by taking a photo and asking a question about it.”
BackgroundMethodSelf-citationUnderstanding Blind People's Experiences with Computer-Generated Captions of Social Media Images
MacLeod, H., Bennett, C. L., Morris, M. R., & Cutrell, E. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025814 · Cited by 200 · cited 1× · see citing works
“In addition, social media platforms, such as Facebook and Twitter, provide people a way to maintain connections with friends by enabling them to identify and tag friends in posted images as well as respond to images automatically described to them [26, 41].”
BackgroundJayant, C., Ji, H., White, S., & Bigham, J. P. (2011) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2049536.2049573 · Cited by 168 · cited 1× · see citing works
“In VizWiz, ∼28% of visual questions are deemed unanswerable by crowd workers, despite the availability of several automated systems designed to guide blind photographers to improve the image focus [3], lighting [8], or composition [18, 37, 44].”
BackgroundSelf-citationShow, Ask, Attend, and Answer: A Strong Baseline For Visual Question AnsweringPreprint
Kazemi, V., & Elqursh, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.03162 · Cited by 139 · see citing works
VizWiz::LocateIt - enabling blind people to locate objects in their environment
Bigham, J. P., Jayant, C., Miller, A. E., White, B., & YEH, T. P. (2010) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2010.5543821 · Cited by 137 · cited 1× · see citing works
“Such VQA systems have been shown to be valuable for many daily tasks including grocery shopping [8], locating a specific object in a complex scene [9], and choosing clothes to wear [11].”
BackgroundSelf-citationPrivacy Concerns and Behaviors of People with Visual Impairments
Ahmed, T., Hoyle, R., Connelly, K., Crandall, D., & Kapadia, A. C. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702334 · Cited by 130 · cited 2× · see citing works
“We begin our work by implementing a rigorous filtering process to remove all visual questions that could risk compromising the safety and/or privacy of any individuals associated with them, since blind people often willingly share personal information with strangers to overcome personal obstacles [5].”
BackgroundMethodCrowdsourcing subjective fashion advice using VizWiz
Burton, M. A., Brady, E. L., Brewer, R. N., Neylan, C., Bigham, J. P., & Hurst, A. K. (2012) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2384916.2384941 · Cited by 95 · cited 3× · see citing works
“Success in developing automated methods would mitigate concerns about the many undesired consequences from today’s status quo for blind people of relying on humans to answer visual questions [8, 11, 24]; e.”
BackgroundSelf-citationVizWiz: Nearly Real-time Answers to Visual Questions
Bigham, J. P., Jayant, R., Ji, H., Little, G., Millerγ, A., Miller, R. C., Miller, R., Tatarowicz, A., White, Y., White, S., et al. (2010) · Cited by 80 · see citing works
Self-citationVisual Madlibs: Fill in the blank Image Generation and Question AnsweringPreprint
Yu, L., Park, E., Berg, A. C., & Berg, T. L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00278 · Cited by 75 · cited 6× · see citing works
“Over the past three years, a plethora of VQA datasets have been publicly shared to encourage a larger community to collaborate on developing algorithms that answer visual questions [6, 7, 16, 17, 19, 20, 23, 28, 32, 39, 40, 43, 45].”
BackgroundMethodInfluentialAnswering visual questions with conversational crowd assistants
Lasecki, W. S., Thiha, P., Zhong, Y., Brady, E. L., & Bigham, J. P. (2013) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2513383.2517033 · Cited by 70 · cited 3× · see citing works
“Success in developing automated methods would mitigate concerns about the many undesired consequences from today’s status quo for blind people of relying on humans to answer visual questions [8, 11, 24]; e.”
BackgroundSelf-citationA qualitative study to support a blind photography mobile application
Adams, D., Morales, L. M., & Kurniawan, S. (2013) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/2504335.2504360 · Cited by 50 · cited 1× · see citing works
“For nearly a decade, blind people have been both taking pictures [4, 8] and asking questions about the pictures they take [8, 11, 24].”
BackgroundQuestion Relevance in VQA: Identifying Non-Visual And False-Premise Questions
Ray, A., Christie, G. A., Bansal, M., Batra, D., & Parikh, D. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1090 · Cited by 48 · see citing works
An Assisted Photography Framework to Help Visually Impaired Users Properly Aim a Camera
Vázquez, M., & Steinfeld, A. M. (2014) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2651380 · Cited by 46 · cited 1× · see citing works
“In VizWiz, ∼28% of visual questions are deemed unanswerable by crowd workers, despite the availability of several automated systems designed to guide blind photographers to improve the image focus [3], lighting [8], or composition [18, 37, 44].”
BackgroundReal time object scanning using a mobile phone and cloud-based visual search engine
Zhong, Y., Garrigues, P. J., & Bigham, J. P. (2013) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2513383.2513443 · Cited by 42 · see citing works
Self-citationQuestion Part Relevance and Editing for Cooperative and Context-Aware VQA (C2VQA)
Toor, A. S., Wechsler, H., & Nappi, M. (2017) · International Workshop on Content-Based Multimedia Indexing (CBMI) · doi:10.1145/3095713.3095718 · Cited by 11 · cited 2× · see citing works
“Our work relates to recent “relevance” datasets which were artificially constructed to include irrelevant visual questions by injecting questions that are unrelated to the contents of high quality images [20, 27, 31, 36].”
BackgroundResult
Flagged first, then by citations
1 of the 45 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- P. Anderson, X. He, C. Buehler, D. Teney, M. Johnson, S. Gould, and L. Zhang. Bottom-up and top-down attention for image captioning and visual question answering. arXiv preprint arXiv:1707.07998, 2017. 7, 8, 12, 14 · arXiv
- S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, pages 448- 456, 2015. 12
- D. Kingma and J. Ba. Adam: A method for stochastic opti- mization. arXiv preprint arXiv:1412.6980, 2014. 12 · arXiv
- A. Mahendru, V. Prabhu, A. Mohapatra, D. Batra, and S. Lee. The promise of premise: Harnessing question premises in visual question answering. arXiv preprint arXiv:1705.00601, 2017. 1, 3, 6, 8, 14 · arXiv
- N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neu- ral networks from overfitting. Journal of machine learning research, 15(1):1929-1958, 2014. 12
- B. Thomee, D. A. Shamma, G. Friedland, B. Elizalde, K. Ni, D. Poland, D. Borth, and L. Li. YFCC100M: The new data in multimedia research. Communications of the ACM, 59(2):64-73, 2016. 4
Cited by
Sorted by citations
Improved Baselines with Visual Instruction Tuning
Liu, H., Li, C., Li, Y., & Lee, Y. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02484 · Cited by 1,748 · see citing works
“VizWiz [19] contains 8,000 images to evaluate model’s zero-shot generalization on visual questions asked by visually impaired people.”
BackgroundMethodInfluentialFlamingo: a visual language model for few-shot learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1723 · Cited by 966 · see citing works
Towards VQA Models That Can Read
Singh, A., Natarajan, V., Shah, M., Jiang, Y., Chen, X., Batra, D., Parikh, D., & Rohrbach, M. (2019) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2019.00851 · Cited by 2,473 · see citing works
“Despite collecting open-ended answers from annotators, we find that human accuracy is 85.01%, consistent with that on VQA 2.0 [10] and VizWiz [13].”
MethodResultInfluentialIntern VL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Chen, Z., Wu, J., Wang, W., Su, W., Chen, G., Xing, S., Zhong, M., Zhang, Q., Zhu, X., Lu, L., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02283 · Cited by 734 · see citing works
Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
Lei, J., Li, L., Zhou, L., Gan, Z., Berg, T. L., Bansal, M., & Liu, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00725 · Cited by 606 · see citing works
“It is trained on Visual Genome [29] for object detection and attribute classification, and produces effective grid features for image VQA tasks [3, 20].”
MethodMultimodal intelligence: Representation learning, information fusion, and applications
Zhang, C., Yang, Z., He, X., & Deng, L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2987728 · Cited by 511 · see citing works
“This dataset includes many text-related questions [253].”
A dataset of clinically generated visual questions and answers about radiology images
Lau, J. J., Gayen, S., Abacha, A. B., & Demner‐Fushman, D. (2018) · Scientific Data · doi:10.1038/sdata.2018.251 · Cited by 496 · see citing works
MMBench: Is Your Multi-modal Model an All-Around Player?
Liu, Y., Duan, H., Zhang, Y., Li, B., Zhang, S., Zhao, W., Yuan, Y., Wang, J., He, C., Liu, Z., et al. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72658-3_13 · Cited by 668 · see citing works
The multi-modal fusion in visual question answering: a review of attention mechanisms
Lu, S., Liu, M., Yin, L., Yin, Z., Liu, X., & Zheng, W. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1400 · Cited by 478 · see citing works
“…et al., 2022 ; Li et al., 2022 ; Gong et al., 2021 ; Zhan et al., 2020 ; Wang et al., 2022a ; Wang et al., 2022b ), remote sensing ( Bazi et al., 2022 ; Zheng et al., 2022b ; Al Rahhal et al., 2022 ), and blindness help ( Gurari et al., 2018 ; Tung et al., 2021 ; Tung, Huy Tien & Minh Le, 2021 ).”
Dynamic Fusion With Intra- and Inter-Modality Attention Flow for Visual Question Answering
Gao, P., Jiang, Z., You, H., Lu, P., Hoi, S. C. H., Wang, X., & Li, H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00680 · Cited by 415 · see citing works
“It has extensive applications in practice, such as assisting blind people [12] and education of young children, and therefore become a hot research topic recently.”
BackgroundIn Defense of Grid Features for Visual Question Answering
Jiang, H., Misra, I., Rohrbach, M., Miller, E. G., & Chen, X. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01028 · Cited by 379 · see citing works
“Through a comprehensive set of experiments, we verified that our observations generalize across different network backbones, different VQA models [16, 49], different VQA benchmarks [3, 12], and even to other relevant tasks (e.”
BackgroundMethodThe hateful memes challenge: Detecting hate speech in multimodal memes
Kiela, D., Firooz, H., Mohan, A., Goswami, V., Singh, A., Ringshia, P., & Testuggine, D. (2020) · arXiv · doi:10.48550/arxiv.2005.04790 · Cited by 352 · see citing works
“In the past few years there has been a surge of interest in multimodal problems, from image captioning [8, 43, 88, 28, 68] to visual question answering (VQA) [3, 26, 27, 33, 71] and beyond.”
BackgroundInfluentialScene Text Visual Question Answering
Biten, A. F., Tito, R., Mafla, A., Gómez, L., Rusiñol, M., Jawahar, C. V., Valveny, E., & Karatzas, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00439 · Cited by 311 · see citing works
“In total, we used six different datasets, namely: ICDAR 2013[23] and ICDAR2015[22], ImageNet [6], VizWiz[13], IIIT Scene Text Retrieval[37], Visual Genome [29] and COCO-Text [50].”
MethodRSVQA: Visual Question Answering for Remote Sensing Data
Lobry, S., Marcos, D., Murray, J., & Tuia, D. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.2988782 · Cited by 305 · see citing works
“[30] confirm that networks trained on generic data sets do not generalize to their specific one.”
BackgroundVILA: On Pre-training for Visual Language Models
Lin, J., Yin, H., Ping, W., Molchanov, P. A., Shoeybi, M., & Han, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02520 · Cited by 284 · see citing works
“VQA-v2 [24]; GQA [28]; VisWiz [25]; SQA I : ScienceQA-IMG [39]; VQA T : TextVQA [50]; POPE [34]; MME [23]; MMB: MMBench [38]; MMB CN : MMBench-Chinese [38]; SEED: SEED-Bench [31]; LLaVA W : LLaVA-Bench (In-the-Wild) [37]; MM-Vet [63].”
ShareGPT4V: Improving Large Multi-modal Models with Better Captions
Chen, L., Li, J., Dong, X., Zhang, P., He, C., Wang, J., Zhao, F., & Lin, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72643-9_22 · Cited by 250 · see citing works
RUBi: Reducing Unimodal Biases in Visual Question AnsweringPreprint
Cadène, R., Dancette, C., Ben-younes, H., Cord, M., & Parikh, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.10169 · Cited by 255 · see citing works
“Several large real image VQA datasets have recently emerged [8, 9, 10, 11, 12, 13, 14].”
BackgroundIterative Answer Prediction With Pointer-Augmented Multimodal Transformers for TextVQA
Hu, R., Singh, A., Darrell, T., & Rohrbach, M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01001 · Cited by 229 · see citing works
“The ST-VQA dataset [8] contains natural images from multiple sources including ICDAR 2013 [24], ICDAR 2015 [23], ImageNet [12], VizWiz [18], IIIT STR [36], Visual Genome [26], and COCO-Text [49].”
MethodFoundations & Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions
Liang, P. P., Zadeh, A., & Morency, L. (2024) · ACM Computing Surveys · doi:10.1145/3656580 · Cited by 306 · see citing works
“These approaches essentially result in a single joint model across all modalities.”
BackgroundMathew, M., Bagal, V., Tito, R., Karatzas, D., Valveny, E., & Jawahar, C. V. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00264 · Cited by 214 · see citing works
“VQA received significant attention over the past few years [15, 5, 16, 20, 23, 3].”
BackgroundCaptioning Images Taken by People Who Are Blind
Gurari, D., Zhao, Y., Zhang, M., & Bhattacharya, N. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58520-4_25 · Cited by 182 · see citing works
Self-citationHuman-Centered Artificial Intelligence for Designing Accessible Cultural Heritage
Pisoni, G., Díaz-Rodríguez, N., Gijlers, H., & Tonolli, L. (2021) · Applied Sciences · doi:10.3390/app11020870 · Cited by 183 · see citing works
“One of the advances in the NLP models brings also the possibility to produce explanations about an image though visual question answering [125], which in our view can be especially useful for the blind.”
Engaging Image Captioning via Personality
Shuster, K., Humeau, S., Hu, H., Bordes, A., & Weston, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01280 · Cited by 160 · see citing works
“Hence, they are useful for developing models that understand content, but are not useful as an end application unless the human cannot see the image, e.g. due to visual impairment [16].”
BackgroundLLaMA-VID: An Image is Worth 2 Tokens in Large Language Models
Li, Y., Wang, C., & Jia, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72952-2_19 · Cited by 197 · see citing works
Language Is Not All You Need: Aligning Perception with Language Models
Huang, S., Dong, L., Wang, W., Hao, Y., Singhal, S., Ma, S., Lv, T., Cui, L., Mohammed, O. K., Patra, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.14045 · Cited by 235 · see citing works
“Visual Question Answering Table 2b reports the visual question answering results on VQAv2 [41] and VizWiz [42].”
BackgroundMethodInfluentialVision-Language Pre-Training: Basics, Recent Advances, and Future Trends
Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., & Gao, J. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000105 · Cited by 155 · see citing works
Multimodal Foundation Models: From Specialists to General-Purpose Assistants
Li, C., Gan, Z., Yang, Z., Yang, J., Li, L., Wang, L., & Gao, J. (2024) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000110 · Cited by 225 · see citing works
Monkey: Image Resolution and Text Label are Important Things for Large Multi-Modal Models
Li, Z., Yang, B., Liu, Q., Ma, Z., Zhang, S., Yang, J., Sun, Y., Liu, Y., & Bai, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02527 · Cited by 174 · see citing works
“Detailed Caption 213k COCO Caption [22] 82k TextCaps [44] 109k General VQA VQAV2 [16] 100k OKVQA [34] 18k GQA [20] 150k ScienceQA [33] 18k VizWiz [17] 20k Scene Text-centric VQA TextVQA [45] 34k OCRVQA [38] 250k AI2D [23] 24k Doc-oriented VQA DocVQA [36] 118k ChartQA [35] 84k InfoVQA [37] 47k…”
Overcoming Language Priors in Visual Question Answering with Adversarial RegularizationPreprint
Ramakrishnan, S., Agrawal, A., & Lee, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.03649 · Cited by 141 · see citing works
“In response to these challenges, there has been extensive work on VQA in recent years both in terms of dataset curation [6, 12, 2, 17, 13, 32, 3] and modeling [2, 5, 28, 14, 16, 4, 22, 20].”
BackgroundMulla, N., & Gharpure, P. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00295-9 · Cited by 132 · see citing works
Generative Multimodal Models are In-Context Learners
Sun, Q., Cui, Y., Zhang, X., Zhang, F., Yu, Q., Wang, Y., Rao, Y., Liu, J., Huang, T., & Wang, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01365 · Cited by 147 · see citing works
“Our Emu2-Chat is evaluated on academic-task-oriented benchmarks including image question-answering datasets (VQAv2 [28], OKVQA [53], GQA [32], VizWiz [29], TextVQA [68]) and video question-answering datasets (MSVD [82] and MSRVTT [82]).”
MethodTAP: Text-Aware Pre-training for Text-VQA and Text-Caption
Yang, Z., Lu, Y., Wang, J., Yin, X. P., Florencio, D. A. F., Wang, L., Zhang, C., Zhang, L., & Luo, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00864 · Cited by 130 · see citing works
“The ST-VQA dataset [8] contains 21,892 images from multiple sources including ICDAR 2013 [27], ICDAR 2015 [26], ImageNet [13], VizWiz [18], IIIT STR [39], Visual Genome [29], and COCO-Text [54].”
BackgroundMethodInfluentialSelf-citationBERT Representations for Video Question Answering
Yang, Z., García, N., Chu, C., Otani, M., Nakashima, Y., & Takemura, H. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093596 · Cited by 108 · see citing works
“Besides these improvements, many other related work, such as [33, 25, 30, 12, 3, 46, 11, 1, 16, 2, 34], has contributed to the advancement of the field.”
BackgroundMorris, M. R. (2020) · Communications of the ACM · doi:10.1145/3356727 · Cited by 114 · see citing works
“Efforts to directly source data from under-represented user groups, such as the VizWiz data set [5], which contains thousands of images and related questions captured by people who are blind, are a step in the right direction.”
BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questions
Hu, W., Xu, Y., Li, Y., Li, W., Chen, Z., & Tu, Z. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i3.27999 · Cited by 117 · see citing works
Towards Unsupervised Image Captioning With Shared Multimodal Embeddings
Laina, I., Rupprecht, C., & Navab, N. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00751 · Cited by 111 · see citing works
“Assisting visually impaired people [23, 62] and human-robot interaction [12, 39] are some examples of the importance of image captioning.”
BackgroundGurari, D., Li, Q., Lin, C., Zhao, Y., Guo, A., Stangl, A. J., & Bigham, J. P. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00103 · Cited by 103 · see citing works
“When comparing the questions reported in prior work about non-private questions [25] to these private questions, we observe great similarity.”
BackgroundMethodResultInfluentialSelf-citationDoes Object Recognition Work for Everyone?Preprint
DeVries, T., Misra, I., Wang, C., & van der Maaten, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02659 · Cited by 132 · see citing works
“Recent advances in the accuracy of object-recognition systems [19, 20, 42] have spurred a large variety of realworld deployments of such systems, for instance, in aids for the visually impaired [17], in photo album organization software [45], in image search, and in popular cloud services [2, 8, 16, 21, 32].”
BackgroundRe-Attention for Visual Question Answering
Guo, W., Zhang, Y., Yang, J., & Yuan, X. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3097180 · Cited by 99 · see citing works
Ishmam, M. F., Shovon, M. S. H., Mridha, M. F., & Dey, N. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102270 · Cited by 100 · see citing works
“Both VQA and VQA v2 have been popular benchmarks for model evaluation throughout the years.”
BackgroundInfluentialLiu, H., Lai, V., & Tan, C. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479552 · Cited by 111 · see citing works
“AI assistance has been shown to improve people’s efficiency in tasks such as transcription by enhancing their computational capacity [16, 35], support creativity in producing music [15, 41, 45], and even allow the visually impaired to “see” images [22, 68].”
BackgroundLaTr: Layout-Aware Transformer for Scene-Text VQA
Biten, A. F., Litman, R., Xie, Y., Appalaraju, S., & Manmatha, R. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01605 · Cited by 95 · see citing works
“ST-VQA [8] is an amalgamation of well-known computer vision datasets, namely: ICDAR 2013 [25], IC-DAR2015 [24], ImageNet [10], VizWiz [18], IIIT Scene Text Retrieval [46], Visual Genome [28] and COCO-Text [64].”
MethodOneLLM: One Framework to Align All Modalities with Language
Han, J., Gong, K., Zhang, Y., Wang, J., Zhang, K., Lin, D., Qiao, Y., Gao, P., & Yue, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02510 · Cited by 103 · see citing works
“ScienceQA [52] MMbench [50] SEED-Bench [41] NextQA [86] How2QA [46] { Question } { Options } Answer with the option’s letter from the given choices directly VizWiz [29] { Question } When the provided information is insufficient, respond with ’Unanswerable’.”
BackgroundToward a Collective Agenda on AI for Earth Science Data Analysis
Tuia, D., Roscher, R., Wegner, J. D., Jacobs, N., Zhu, X. X., & Camps‐Valls, G. (2021) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2020.3043504 · Cited by 89 · see citing works
FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning
Chen, H., Zhang, Y., Krompaß, D., Gu, J., & Volker Tresp, C. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i10.29007 · Cited by 94 · see citing works
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action
Lu, J., Clark, C., Lee, S.-H., Zhang, Z., Khosla, S., Marten, R., Hoiem, D., & Kembhavi, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02497 · Cited by 97 · see citing works
VQA-E: Explaining, Elaborating, and Enhancing Your Answers for Visual Questions
Li, Q., Tao, Q., Joty, S., Cai, J., & Luo, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_34 · Cited by 88 · see citing works
“In recent years, visual question answering (VQA) has been widely studied by researchers in both computer vision and natural language processing communities [2,34,8,27,31,11].”
BackgroundSelf-citation“Factual” or “Emotional”: Stylized Image Captioning with Adaptive Learning and Attention
Chen, T., Zhang, Z., You, Q., Chen, F., Wang, Z., Jin, H., & Luo, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_32 · Cited by 87 · see citing works
“Automatically generating coherent captions for images has attracted remarkable attention for its strong applicability, such as picture auto-commenting [23] and helping blind people to see [11].”
BackgroundSelf-citationContinual Learning of Large Language Models: A Comprehensive Survey
Shi, H., Xu, Z., Wang, H., Qin, W., Wang, W., Wang, Y., Wang, Z., Ebrahimi, S., & Wang, H. (2025) · ACM Computing Surveys · doi:10.1145/3735633 · Cited by 136 · see citing works
“After discussing general observations on CFT in Section 4.3.1, we will delve into each sub-category in detail.”
InfluentialThe Revolution of Multimodal Large Language Models: A Survey
Caffagni, D., Cocchi, F., Barsellotti, L., Moratelli, N., Sarto, S., Baraldi, L., Baraldi, L., Cornia, M., & Cucchiara, R. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.807 · Cited by 98 · see citing works
“While retrieval-augmented generation (RAG) is a consolidated technique in LLMs (Lewis et al., 2020; Asai et al., 2023), its application in MLLMs is still under-explored.”
VisualMRC: Machine Reading Comprehension on Document Images
Tanaka, R., Nishida, K., & Yoshida, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17635 · Cited by 93 · see citing works
“VizWiz-VQA (Gurari et al. 2018) consists of questions originating from blind people who each took a picture using a mobile phone.”
BackgroundOn the General Value of Evidence, and Bilingual Scene-Text Visual Question Answering
Wang, X., Liu, Y., Shen, C., Ng, C. C., Luo, C., Jin, L., Chan, C. S., van den Hengel, A., & Wang, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01014 · Cited by 83 · see citing works
“[4] 19k 26k 3k 4k [6, 8, 13, 14, 17, 23, 32] [24] 180k 900k 20k 100k [9] [29] 25k 39k 3k 5k [16]”
BackgroundReducing Language Biases in Visual Question Answering with Visually-Grounded Question Encoder
Gouthaman, K. V., & Mittal, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58601-0_2 · Cited by 82 · see citing works
“With the success of deep-learning in CV and NLP, many datasets [3,14,21,15,19,44], and models [2,4,6,5,22] have been proposed to solve VQA.”
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models
Lin, B., Tang, Z., Ye, Y., Huang, J., Zhang, J., Pang, Y., Jin, P., Ning, M., Luo, J., & Yuan, L. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3654458 · Cited by 81 · see citing works
“E VALUATION BENCHMARKS INCLUDE VQA-V2 [79]; GQA [80], V IS W IZ [81]; SQA I (S CIENCE QA-IMG) [82]; VQA T (T EXT VQA) [83]; POPE [21]; MME [84]; MMB (MMB ENCH ) [85]; LL A VA W : LLAVA-BENCH ( IN -”
Self-citationChoi, D., Lee, S., Kim, S.-I., Lee, K., Yoo, H. J., Lee, S., & Hong, H. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641989 · Cited by 79 · see citing works
“Propelled by research examining AI bias related to gender and race [9, 34, 42, 63, 72], recent years have seen growing attention to AI’s inclusivity toward identity-based minority groups, such as people with disabilities [38–40, 51, 96, 99, 101, 109] and LGBTQ communities [27, 82, 103].”
BackgroundVisual question answering: a state-of-the-art review
Manmadhan, S., & Kovoor, B. C. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09832-7 · Cited by 81 · see citing works
“This can directly provide an extension to the Viz-Wiz challenge where the questions are asked against photographs taken by visually impaired people using their mobiles.”
BackgroundMethodInfluentialOn the Adversarial Robustness of Multi-Modal Foundation Models
Schlarmann, C., & Hein, M. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00395 · Cited by 78 · see citing works
Multi-modal Misinformation Detection: Approaches, Challenges and Opportunities
Abdali, S., Shaham, S., & Krishnamachari, B. (2024) · ACM Computing Surveys · doi:10.1145/3697349 · Cited by 91 · see citing works
“More recently, Cao et al. leverage a pre-trained vision-language models (PVLMs) in a zero-shot and fine-tuning free VQA setting to address the problem of meme detection by generating hateful content-centric image captions [16].”
BackgroundLLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model
Zhu, Y., Zhu, M., Liu, N., Xu, Z., & Peng, Y. (2024) · doi:10.1145/3688863.3689575 · Cited by 94 · see citing works
“VQA v2 [12]; GQA [16]; VizWiz [14]; SQA I : ScienceQA-IMG [28]; VQA T : TextVQA [31]; POPE [22]; MME [10]; MMB: MMBench [27]; SEED: SEED-Bench [18]; MM-Vet [36].”
MethodStructured Multimodal Attentions for TextVQA
Gao, C., Zhu, Q., Wang, P., Li, H., Liu, Y., van den Hengel, A., & Wu, Q. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3132034 · Cited by 71 · see citing works
“The VizWiz [3] first identified this problem and found that nearly a quarter of questions asked by visually-impaired people are text-reading related.”
BackgroundInfluentialVisionZip: Longer is Better but Not Necessary in Vision Language Models
Yang, S., Chen, Y., Tian, Z., Wang, C., Li, J., Yu, B., & Jia, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01843 · Cited by 93 · see citing works
Focal Visual-Text Attention for Memex Question Answering
Liang, J., Jiang, L., Cao, L., Kalantidis, Y., Li, L.-J., & Hauptmann, A. G. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2890628 · Cited by 69 · see citing works
“VizWiz [35] collected visual questions from blind people and targeted to build systems that could assist blind people.”
BackgroundImage captioning for effective use of language models in knowledge-based visual question answering
Salaberria, A., Azkune, G., de Lacalle, O. L., Soroa, A., & Agirre, E. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118669 · Cited by 69 · see citing works
Shuffle-Then-Assemble: Learning Object-Agnostic Visual Relationship Features
Yang, X., Zhang, H., & Cai, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01258-8_3 · Cited by 69 · see citing works
“Thanks to the maturity of mid-level vision solutions such as object classification and detection [19,41,15], we are more ambitious to pursue higher-level visionlanguage tasks such as image captioning [13,14,5,31], visual Q&A [22,27,18], and visual chatbot [7].”
BackgroundAMAM: An Attention-based Multimodal Alignment Model for Medical Visual Question Answering
Pan, H., He, S., Zhang, K., Qu, B., Chen, C., & Shi, K. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109763 · Cited by 66 · see citing works
Generative Bias for Robust Visual Question Answering
Cho, J. W., Kim, D.-J., Ryu, H., & Kweon, I. S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01124 · Cited by 66 · see citing works
BrowseWithMe: An Online Clothes Shopping Assistant for People with Visual Impairments
Stangl, A. J., Kothari, E., Jain, S. D., Yeh, T., Grauman, K., & Gurari, D. (2018) · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3234695.3236337 · Cited by 66 · see citing works
“, Table 2; Q4 and Q5), and more specifically responds to Lily who shared “I miss browsing through magazines to get outfit inspiration!” The importance of these chosen features is reinforced by the findings of prior work [6, 13, 14, 20, 44, 46].”
BackgroundSelf-citation"I look at it as the king of knowledge": How Blind People Use and Understand Generative AI Tools
Adnin, R., & Das, M. (2024) · The 26th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663548.3675631 · Cited by 73 · see citing works
“Related to information seeking, one unique use context of GenAI for blind individuals is visual question answering [31, 35], for which participants primarily used Be My AI but also sometimes newer GenAI models that can describe visual information e.g., ChatGPT 4.”
MethodResultNatural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey
Kuang, J., Shen, Y., Xie, J., Luo, H., Xu, Z., Li, R., Li, Y., Cheng, X., Lin, X., & Han, Y. (2025) · ACM Computing Surveys · doi:10.1145/3711680 · Cited by 69 · see citing works
Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions
Li, Q., Fu, J., Yu, D., Mei, T., & Luo, J. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1164 · Cited by 65 · see citing works
Self-citationA holistic representation guided attention network for scene text recognition
Yang, L., Wang, P., Li, H., Li, Z., & Zhang, Y.-N. (2019) · Neurocomputing · doi:10.1016/j.neucom.2020.07.010 · Cited by 63 · see citing works
“Text reading has been integrated in a variety of vision tasks, such as fine-grained image classification [1, 2, 3], image retrieval [1, 4] and visual question answering [5, 6].”
BackgroundAssessing Image Quality Issues for Real-World Problems
Chiu, T.-Y., Zhao, Y., & Gurari, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00370 · Cited by 63 · see citing works
“Our models perform comparably to the answerability baseline [17].”
BackgroundMethodInfluentialSelf-citationCascade Reasoning Network for Text-based Visual Question Answering
Liu, F., Xu, G., Wu, Q., Du, Q., Wei, J., & Tan, M. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413924 · Cited by 61 · see citing works
“Compared with common VQA [12, 30], the text-based visual question answering (T-VQA) is more practical as it has the ability to aid visually impaired users in better understanding the information of their surrounding physical world, such as identifying time, temperatures and brand names [16].”
BackgroundMM1: Methods, Analysis and Insights from Multimodal LLM Pre-training
McKinzie, B., Gan, Z., Fauconnier, J.-P., Dodge, S., Zhang, B., Dufter, P., Shah, D., Du, X., Peng, F., Belyi, A., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73397-0_18 · Cited by 120 · see citing works
Recent Advances and Trends in Multimodal Deep Learning: A ReviewPreprint
Jabeen, S., Li, X., Amin, M. S., Li, S., & Abdul, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.11087 · Cited by 88 · see citing works
“So far, VizWiz [49] and VQA-MED [4]are two publicly available goal-oriented datasets for VQA research.”
BackgroundDocument Understanding Dataset and Evaluation (DUDE)
Landeghem, J. V., Powalski, R., Tito, R., Jurkiewicz, D., Blaschko, M. B., Borchmann, Ł., Coustaty, M., Moens, S., Pietruszka, M., Ackaert, B., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01789 · Cited by 67 · see citing works
“For example, the question-answering approach is combined with modalities such as videos [44, 13, 14, 28, 17], images [98, 3, 29, 68, 7, 8], speech [99, 43], knowledge graphs [93, 84, 80, 22, 37], and maps [70, 15].”
MethodWhy Does a Visual Question Have Different Answers?
Bhattacharya, N., Li, Q., & Gurari, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00437 · Cited by 59 · see citing works
Self-citationGenAssist: Making Image Generation Accessible
Huh, M., Peng, Y.-H., & Pavel, A. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606735 · Cited by 67 · see citing works
“Previous research has explored what visual questions blind people would like to have answered [9] and provided on-demand visual question answering support using both crowdsourcing [6, 25] and automated methods [17].”
BackgroundGhosh, A., Acharya, A., Saha, S., Jain, V., & Chadha, A. R. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4783140 · Cited by 94 · see citing works
Sharma, C. B., Batra, I., Sharma, S., Malik, A., Hosen, A. S. M. S., & Ra, I.-H. (2022) · IEEE Access · doi:10.1109/access.2022.3214310 · Cited by 57 · see citing works
“Other technologies like 3D Question Answering models [104], and VizWiz Grand Challenge [105] can also be integrated in the Smarthome to help elderly people in more specified way.”
MethodCan Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?
Chen, Y., Hu, H., Luan, Y., Sun, H., Changpinyo, S., Ritter, A., & Chang, M.-W. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.925 · Cited by 78 · see citing works
Robust Visual Question Answering: Datasets, Methods, and Future Challenges
Ma, J., Wang, P., Kong, D., Wang, Z., Liu, J., Pei, H., & Zhao, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3366154 · Cited by 57 · see citing works
“ID settings typically represent that the test split’s distribution is similar to that of the corresponding training split: A variety of ID datasets [1], [12], [15], [56], [57], [58], [59] have been developed from different angles and have made significant contributions to robust VQA research.”
BackgroundICDAR 2019 Competition on Scene Text Visual Question Answering
Biten, A. F., Tito, R. P., Mafla, A., Gómez, L., Rusiñol, M., Mathew, M., Jawahar, C., Valveny, E., & Karatzas, D. (2019) · IEEE International Conference on Document Analysis and Recognition · doi:10.1109/icdar.2019.00251 · Cited by 91 · see citing works
“Many real-life visual question answering cases (see for example the VizWiz challenge 1 ), are frequently grounded on scene text present in the scene.”
MethodInfluentialQuestion Type Guided Attention in Visual Question Answering
Shi, Y., Furlanello, T., Zha, S., & Anandkumar, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01225-0_10 · Cited by 54 · see citing works
“Vizwiz [18] is another recently released dataset that uses pictures taken by blind people.”
BackgroundAccessible Cultural Heritage through Explainable Artificial Intelligence
Díaz-Rodríguez, N., & Pisoni, G. (2020) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3386392.3399276 · Cited by 55 · see citing works
“Another NLP model to produce explanations about an image is tackled by the problem of visual question answering [28], specially useful for the blind3 or image captioning projects456.”
Velioğlu, R., & Rose, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.12975 · Cited by 55 · see citing works
“There has been a surge of interest in multimodal problems since 2015 in visual question answering [5, 6], image captioning [7, 8], speech recognition [9, 10] and beyond.”
BackgroundGrounding Answers for Visual Questions Asked by Visually Impaired People
Chen, C., Anjum, S., & Gurari, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01851 · Cited by 53 · see citing works
“First, we chose the top-performing VQA algorithms with publicly-available code for the 2021 VizWiz-VQA dataset challenge [15] and the mainstream 2021 VQA dataset challenge [13], that is LXMERT [36] and OSCAR [24] with VinVL image features [40] respectively.”
BackgroundMethodInfluentialSelf-citationVisual question answering: Which investigated applications?
Barra, S., Bisogni, C., Marsico, M. D., & Ricciardi, S. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.09.008 · Cited by 53 · see citing works
“To this aim, the dataset proposed and described in Gurari et al. (2018) contains 31000 visual questions originated by blind people who took a photo with their mobile phone and recorded a spoken question about it.”
MethodInfluentialGoing Denser with Open-Vocabulary Part Segmentation
Sun, P., Chen, S., Zhu, C., Xiao, F., Luo, P., Xie, S., & Yan, Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01417 · Cited by 53 · see citing works
“A universal representation for vision and language is needed in various tasks such as visual question answering [25, 29], image/video-text retrieval [51,64,81], visual reasoning [22, 75] and so on.”
BackgroundInvestigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision People
Penuela, R. E. G., Collins, J., Bennett, C. L., & Azenkot, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642211 · Cited by 52 · see citing works
“Gurari et al. presented this study to encourage the development of CV algorithms that work with BLV users’ photos [16], due to the challenges they present to VQA.”
BackgroundMethodAn Outlook into the Future of Egocentric Vision
Plizzari, C., Goletto, G., Furnari, A., Bansal, S., Ragusa, F., Farinella, G. M., Damen, D., & Tommasi, T. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02095-7 · Cited by 52 · see citing works
“Seminal Works Ye et al (2012) developed the first system capable of recognising eye contact between the wearer and a person in front of the camera.”
BackgroundMethodInfluentialTemporal Reasoning via Audio Question Answering
Fayek, H. M., & Johnson, J. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3010650 · Cited by 51 · see citing works
Prompt-Based Learning for Unpaired Image Captioning
Zhu, P., Wang, X., Zhu, L., Sun, Z., Zheng, W., Wang, Y., & Chen, C. W. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3265842 · Cited by 52 · see citing works
“It has been widely used in many applications, such as human-robot interaction [8], [9], visual aid for the blind [10], [11], [12], and automatic driving [13], [14], [15].”
MethodImage Captioning as an Assistive Technology: Lessons Learned from VizWiz 2020 Challenge
Dognin, P., Melnyk, I., Mroueh, Y., Padhi, I., Rigotti, M., Ross, J., Schiff, Y., Young, R. A., & Belgodere, B. M. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13113 · Cited by 54 · see citing works
“…systems that constitute mature assistive technologies motivated the introduction of the VizWiz Challenges for captioning images taken by people who are blind in (Gurari et al., 2018), (Simons, Gurari, & Fleischmann, 2020) and(Gurari, Zhao, Zhang, & Bhattacharya, 2020). dataset (generic captions).”
BackgroundMethodInfluentialEfficient GPT-4V level multimodal large language model for deployment on edge devices
Yao, Y., Yu, T., Zhang, A., Wang, C., Cui, J., Zhu, H., Cai, T., Chen, C., Li, H., Zhao, W., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-61040-5 · Cited by 50 · see citing works
The Emerging Professional Practice of Remote Sighted Assistance for People with Visual Impairments
Lee, S., Reddie, M., Tsai, C.-H., Beck, J., Rosson, M. B., & Carroll, J. M. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376591 · Cited by 48 · see citing works
“They have also used their initial findings to further develop and expand their work through VizWiz Social [17], which connected the user with visual impairment to their social networks, such as friends and family members, for assistance obtaining visual information.”
BackgroundA survey of methods, datasets and evaluation metrics for visual question answering
Sharma, H., & Jalal, A. S. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2021.104327 · Cited by 47 · see citing works
An analysis of graph convolutional networks and recent datasets for visual question answering
Yusuf, A. A., Feng, C., & Xianling, M. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10151-2 · Cited by 45 · see citing works
“For instance, the VizWiz dataset (Gurari et al. 2018) is challenging and can be deployed to real-world applications.”
BackgroundMethodInfluentialDesigning an Online Infrastructure for Collecting AI Data From People With Disabilities
Park, J. S., Bragg, D., Kamar, E., & Morris, M. R. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445870 · Cited by 50 · see citing works
“The VizWiz app allows blind users to upload photos and receive descriptions from paid crowdworkers [7]; many VizWiz users have opted to share images to create public datasets for AI training [25, 26].”
BackgroundRUArt: A Novel Text-Centered Solution for Text-Based Visual Question Answering
Jin, Z., Wu, H., Yang, C. H., Zhou, F., Qin, J., Xiao, L., & Yin, X.-C. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3120194 · Cited by 44 · see citing works
- Authors
- Gurari, D., Li, Q., Stangl, A. J., Guo, A., Lin, C., Grauman, K., Luo, J., & Bigham, J. P.
- Affiliations
- The University of Texas at Austin (US)
- University of Science and Technology of China (CN)
- University of Colorado Boulder (US)
- Carnegie Mellon University (US)
- University of Rochester (US)
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 3608-3617
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2788643321
- Cited by
- 621
- References
- 45
- Citations by year
- 2026: 92 · 2025: 133 · 2024: 108 · 2023: 65 · 2022: 56 · 2021: 66 · 2020: 55 · 2019: 29 · 2018: 17
- FWCI
- 12.58
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1704309, 1514118, 1755593)
- Microsoft Research
- Baidu
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- visual question answering, blind people, assistive technology, low-quality images, answerability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
