Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension
Abstract
We introduce the task of Multi-Modal Machine Comprehension (M3C), which aims at answering multimodal questions given a context of text, diagrams and images. We present the Textbook Question Answering (TQA) dataset that includes 1,076 lessons and 26,260 multi-modal questions, taken from middle school science curricula. Our analysis shows that a significant portion of questions require complex parsing of the text and the diagrams and reasoning, indicating that our dataset is more complex compared to previous machine comprehension and visual question answering datasets. We extend state-of-the-art methods for textual machine comprehension and visual question answering to the TQA dataset. Our experiments show that these models do not perform well on TQA. The presented dataset opens new challenges for research in question answering and reasoning across multiple modalities.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Standalone question generation has been applied to education, news, and social media.
“If we consider the different domains, standalone question generation has been applied to education [123], news [124] and social media [126].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [126]
Using multiple modalities can target more real-world applications, including educational questions based on pictures and diagrams and helping visually challenged people identify objects.
“Using multiple modalities, more real-world applications can be targeted, including generating questions based on pictures and diagrams in educational domain [126] and helping the visually challenged identify objects around them [127].”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [126]
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · see citing works
SQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · see citing works
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487 · see citing works
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,779 · see citing works
Bidirectional Attention Flow for Machine ComprehensionPreprint
Seo, M., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1611.01603 · Cited by 2,322 · see citing works
Self-citationSemantic Parsing on Freebase from Question-Answer Pairs
Berant, J., Chou, A., Frostig, R., & Liang, P. (2013) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1160 · Cited by 2,318 · see citing works
Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
Fukui, A., Park, D. H., Yang, D., Rohrbach, A., Darrell, T., & Rohrbach, M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1044 · Cited by 1,582 · see citing works
Hierarchical Question-Image Co-Attention for Visual Question AnsweringPreprint
Lu, J., Yang, J., Batra, D., & Parikh, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00061 · Cited by 1,303 · see citing works
Memory NetworksPreprint
Weston, J., Chopra, S., & Bordes, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.3916 · Cited by 819 · see citing works
Visual7W: Grounded Question Answering in Images
Zhu, Y., Groth, O., Bernstein, M., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.540 · Cited by 794 · cited 3× · see citing works
“released the Visual7W dataset [30] which contained multiple choice visual answers in addition to textual answers.”
BackgroundMethodMCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
Richardson, M., Burges, C., & Renshaw, E. (2013) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1020 · Cited by 778 · see citing works
Dynamic Memory Networks for Visual and Textual Question AnsweringPreprint
Xiong, C., Merity, S., & Socher, R. (2016) · International Conference on Machine Learning · doi:10.48550/arxiv.1603.01417 · Cited by 770 · see citing works
Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering
Xu, H., & Saenko, K. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_28 · Cited by 753 · see citing works
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merriënboer, B., Joulin, A., & Mikolov, T. (2015) · arXiv · doi:10.48550/arxiv.1502.05698 · Cited by 670 · see citing works
MovieQA: Understanding Stories in Movies through Question-Answering
Tapaswi, M., Zhu, Y., Stiefelhagen, R., Torralba, A. B., Urtasun, R., & Fidler, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.501 · Cited by 646 · see citing works
Ask Your Neurons: A Neural-Based Approach to Answering Questions about Images
Malinowski, M., Rohrbach, M., & Fritz, M. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.9 · Cited by 633 · see citing works
Exploring Models and Data for Image Question AnsweringPreprint
Ren, M., Kiros, R., & Zemel, R. S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.02074 · Cited by 618 · see citing works
A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task
Chen, D., Bolton, J., & Manning, C. D. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1223 · Cited by 494 · see citing works
The Goldilocks Principle: Reading Children's Books with Explicit Memory RepresentationsPreprint
Hill, F., Bordes, A., Chopra, S., & Weston, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.02301 · Cited by 478 · see citing works
Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question AnsweringPreprint
Gao, H., Mao, J., Zhou, J., Huang, Z., Wang, L., & Xu, W. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05612 · Cited by 437 · see citing works
A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain InputPreprint
Malinowski, M., & Fritz, M. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.0210 · Cited by 432 · see citing works
What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA
Wang, M., Smith, N. A., & Mitamura, T. (2007) · Cited by 388 · see citing works
Hierarchical Question-Image Co-Attention for Visual Question Answering
Lu, J. (2024) · TIB Data Manager · doi:10.57702/uhdmfu1r · Cited by 357 · see citing works
A Diagram is Worth a Dozen Images
Kembhavi, A., Salvato, M., Kolve, E., Seo, M., Hajishirzi, H., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_15 · Cited by 290 · see citing works
Self-citationLarge-scale Semantic Parsing via Schema Matching and Lexicon Extension
Cai, Q., & Yates, A. (2013) · Cited by 277 · see citing works
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merriënboer, B., Joulin, A., & Mikolov, T. (2016) · International Conference on Learning Representations · Cited by 261 · see citing works
Visual Madlibs: Fill in the Blank Description Generation and Question Answering
Yu, L., Park, E., Berg, A. C., & Berg, T. L. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.283 · Cited by 216 · see citing works
Hierarchical Memory NetworksPreprint
Chandar, S., Ahn, S., Larochelle, H., Vincent, P., Tesauro, G. J., & Bengio, Y. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.07427 · Cited by 59 · see citing works
Flagged first, then by citations
2 of the 34 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092
“The out-of-domain datasets are TextbookQA (Kembhavi et al., 2017), RACE (Lai et al., 2017), BioASQ ( http://bioasq. org/ ), RE (Levy et al., 2017), DuoRC (Saha et al., 2018), and DROP (Dua et al., 2019). model (+ 3 .”
BackgroundInfluentialThe Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096 · see citing works
“The out-of-domain datasets are TextbookQA (Kembhavi et al., 2017), RACE (Lai et al., 2017), BioASQ ( http://bioasq. org/ ), RE (Levy et al., 2017), DuoRC (Saha et al., 2018), and DROP (Dua et al., 2019). model (+ 3 .”
BackgroundInfluentialTowards VQA Models That Can Read
Singh, A., Natarajan, V., Shah, M., Jiang, Y. L., Chen, X., Batra, D., Parikh, D., & Rohrbach, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00851 · Cited by 777 · see citing works
“Textbook QA (TQA) [24] considers the task of answering questions from middle-school textbooks, which often require understanding and reasoning about text and diagrams.”
BackgroundCan a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
Mihaylov, T., Clark, P., Khot, T., & Sabharwal, A. (2018) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1260 · Cited by 2,991 · see citing works
“ing the most useful pieces of knowledge remains an open challenge, our ‘oracle’ experiments with the fact fused while generating a question qand an interpretation (by 2Only ˘5% of the TQA questions ofKembhavi et al. (2017) require additional common knowledge. 3To avoid ambiguity in the term ‘human performance’, Sectio…”
BackgroundDocVQA: A Dataset for VQA on Document Images
Mathew, M., Karatzas, D., & Jawahar, C. V. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00225 · Cited by 570 · see citing works
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., & Tafjord, O. (2018) · arXiv · doi:10.48550/arxiv.1803.05457 · Cited by 462 · see citing works
“al., 2016), to our task. As BiDAF is a direct answer system, we adapted it to multiple-choice QA following the approach used in several previous projects (Khashabi et al., 2018b; Welbl et al., 2017a; Kembhavi et al., 2017) as follows: First, given a question, we create a single paragraph by concatenating a set of retr…”
MethodLooking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences
Khashabi, D., Chaturvedi, S., Roth, M., Upadhyay, S., & Roth, D. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1023 · Cited by 374 · see citing works
“Since BiDAF was designed for cloze style questions, we apply it to our multiple-choice setting following the procedure by Kembhavi et al. (2017): Specifically, we score each answer-option by computing the similarity value of it’s output span with each of the candidate answers, computed by phrasal…”
MethodIQA: Visual Question Answering in Interactive Environments
Gordon, D., Kembhavi, A., Rastegari, M., Redmon, J., Fox, D., & Farhadi, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00430 · Cited by 362 · see citing works
Self-citationHow far are we to GPT-4V? Closing the gap to commercial multimodal models with open-source suites
Chen, Z., Wang, W., Tian, H., Ye, S., Gao, Z., Cui, E., Tong, W., Hu, K., Luo, J., Ma, Z., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4231-5 · Cited by 437 · see citing works
“…(en) [28], GQA (en) [34], OKVQA (en) [76], General QA VSR (en) [55], VisualDialog (en) [24] Science AI2D (en) [39], ScienceQA (en) [69], TQA (en) [40] ChartQA (en) [77], MMC-Inst (en) [57], DVQA (en) [38], Chart PlotQA (en) [81], LRV-Instruction (en) [56] GeoQA+ (en) [14], TabMWP (en) [70],…”
Scene Text Visual Question Answering
Biten, A. F., Tito, R., Mafla, A., Gómez, L., Rusiñol, M., Jawahar, C. V., Valveny, E., & Karatzas, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00439 · Cited by 311 · see citing works
“The Textbook Question Answering (TQA) dataset [29] aims at answering multimodal questions given a context of text, diagrams and images, but textual information is provided in computer readable format.”
BackgroundDVQA: Understanding Data Visualizations via Question Answering
Kafle, K., Price, B. L., Cohen, S. D., & Kanan, C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00592 · Cited by 260 · see citing works
MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension
Fisch, A., Talmor, A., Jia, R., Seo, M., Choi, E., & Chen, D. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5801 · Cited by 220 · see citing works
“TextbookQA (Kembhavi et al., 2017) Text-bookQA is collected from lessons from middle school Life Science, Earth Science, and Physical Science textbooks.”
BackgroundMethodSelf-citationMathew, M., Bagal, V., Tito, R., Karatzas, D., Valveny, E., & Jawahar, C. V. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00264 · Cited by 214 · see citing works
KVQA: Knowledge-Aware Visual Question Answering
Shah, S. B., Mishra, A., Yadati, N., & Talukdar, P. P. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018876 · Cited by 171 · see citing works
“Other recent datasets such as (Kembhavi et al. 2017; Tapaswi et al. 2016; Lu et al. 2018) do not require external knowledge as ours.”
BackgroundPlotQA: Reasoning over Scientific Plots
Methani, N., Ganguly, P., Khapra, M. M., & Kumar, P. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093523 · Cited by 165 · see citing works
“There are some datasets [14, 15] which contain questions asked over diagrams found in text books but these datasets are smaller and contain multiple-choice questions.”
BackgroundMulla, N., & Gharpure, P. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00295-9 · Cited by 132 · see citing works
“If we consider the different domains, standalone question generation has been applied to education [123], news [124] and social media [126].”
BackgroundRecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipes
Yagcioglu, S., Erdem, A., Erdem, E., & İkizler-Cinbiş, N. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1166 · Cited by 122 · see citing works
A Survey on Machine Reading Comprehension—Tasks, Evaluation Metrics and Benchmark Datasets
Zeng, C., Li, S., Li, Q., Hu, J., & Hu, J. (2020) · Applied Sciences · doi:10.3390/app10217640 · Cited by 109 · see citing works
Ishmam, M. F., Shovon, M. S. H., Mridha, M. F., & Dey, N. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102270 · Cited by 100 · see citing works
“The surveys can be roughly categorized into generalized surveys that provide a bird’s eye view on the whole domain itself and specialized or critical surveys that extensively focus on a part of the domain.”
BackgroundSu, D., Xu, Y., Yu, T., Siddique, F. B., Barezi, E. J., & Fung, P. (2020) · Workshop on NLP for COVID-19 (NLP-COVID19) · doi:10.18653/v1/2020.nlpcovid19-2.14 · Cited by 91 · see citing works
VisualMRC: Machine Reading Comprehension on Document Images
Tanaka, R., Nishida, K., & Yoshida, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17635 · Cited by 93 · see citing works
FitzGerald, N., Michael, J., He, L., & Zettlemoyer, L. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1191 · Cited by 84 · see citing works
“3 Dataset We gathered annotations for 133,479 verb mentions in 64,018 sentences (1.27M tokens) across 3 domains: Wikipedia, Wikinews, and science textbook text from the Textbook Question Answering (TQA) dataset (Kembhavi et al., 2017).”
MethodOn the General Value of Evidence, and Bilingual Scene-Text Visual Question Answering
Wang, X., Liu, Y., Shen, C., Ng, C. C., Luo, C., Jin, L., Chan, C. S., van den Hengel, A., & Wang, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01014 · Cited by 82 · see citing works
PathVQA: 30000+ Questions for Medical Visual Question Answering
He, X., Zhang, Y., Mou, L., Xing, E., & Xie, P. (2020) · arXiv · doi:10.48550/arxiv.2003.10286 · Cited by 79 · see citing works
“CLEVR [4, 12] is a dataset developed on rendered images of spatially related objects (including cube, sphere, and cylinder) with different sizes, materials, and colors.”
BackgroundFew-Shot Question Answering by Pretraining Span Selection
Ram, O., Kirstain, Y., Berant, J., Globerson, A., & Levy, O. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.239 · Cited by 79 · see citing works
“…format was popularized by SQuAD (Rajpurkar et al., 2016), and has since been adopted by several datasets in various domains (Trischler et al., 2017; Kembhavi et al., 2017) and languages (Lewis et al., 2020; Clark et al., 2020), with some extensions allowing for unanswerable questions (Levy et al.,…”
BackgroundMethodInfluentialStructured Multimodal Attentions for TextVQA
Gao, C., Zhu, Q., Wang, P., Li, H., Liu, Y., van den Hengel, A., & Wu, Q. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3132034 · Cited by 75 · see citing works
“For example, Textbook QA [15] asks multimodal questions given text, diagrams and images from middle school textbooks.”
BackgroundPathological Visual Question AnsweringPreprint
He, X., Cai, Z., Wei, W., Zhang, Y., Mou, L., Xing, E. P., & Xie, P. (2020) · doi:10.36227/techrxiv.13127537 · Cited by 70 · see citing works
Question Answering as Global Reasoning Over Semantic Abstractions
Khashabi, D., Khot, T., Sabharwal, A., & Roth, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11574 · Cited by 70 · see citing works
“Since B I DAF was designed for fill-in-the-blank style questions, we follow the variation used by Kembhavi et al. (2017) to apply it to our multiple-choice setting.”
MethodGeneralizing Question Answering System with Pre-trained Language Model Fine-tuning
Su, D., Xu, Y., Winata, G. I., Xu, P., Kim, H., Liu, Z., & Fung, P. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5827 · Cited by 72 · see citing works
“Many datasets, such as SQuAD (Rajpurkar et al., 2016), NewsQA (Trischler et al., 2017), TriviaQA (Joshi et al., 2017), SearchQA (Dunn et al., 2017), HotpotQA (Yang et al., 2018), NaturalQuestions (Kwiatkowski et al., 2019), DROP (Dua et al., 2019), RACE (Lai et al., 2017), DueRC (Saha et al., 2018), BioASQ (Tsatsaroni…”
BackgroundSlideVQA: A Dataset for Document Visual Question Answering on Multiple Images
Tanaka, R., Nishida, K., Nishida, K., Hasegawa, T., Saito, I., & Saito, K. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26598 · Cited by 63 · see citing works
FriendsQA: Open-Domain Question Answering on TV Show Transcripts
Yang, Z., & Choi, J. D. (2019) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w19-5923 · Cited by 57 · see citing works
“TQA gives passages from middle school science lessons and textbooks (Kembhavi et al., 2017).”
BackgroundSelf-citationErdem, E., Kuyu, M., Yagcioglu, S., Frank, A., Parcalabescu, L., Plank, B., Babii, A., Turuta, O., Erdem, A., Calixto, I., et al. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12918 · Cited by 54 · see citing works
Towards Visual Question Answering on Pathology Images
He, X., Cai, Z., Wei, W., Zhang, Y., Mou, L., Xing, E. P., & Xie, P. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-short.90 · Cited by 55 · see citing works
“CLEVR (Johnson et al., 2017; Kembhavi et al., 2017) is a dataset developed on rendered images of spatially related objects (including cube, sphere, and cylinder) with different sizes, materials, and colors.”
BackgroundPathological Visual Question AnsweringPreprint
He, X., Cai, Z., Wei, W., Zhang, Y., Mou, L., Xing, E. P., & Xie, P. (2020) · doi:10.36227/techrxiv.13127537.v1 · Cited by 54 · see citing works
Answering Questions about Data Visualizations using Efficient Bimodal Fusion
Kafle, K., Shrestha, R., Price, B. L., Cohen, S. D., & Kanan, C. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093494 · Cited by 53 · see citing works
“As a remedy, some subsequent datasets have focused on synthetic scenes and diagrams where reasoning capacities can be better studied [5, 13, 21, 22].”
BackgroundSPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval
Zhao, T., Lu, X., & Lee, K. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.47 · Cited by 62 · see citing works
HEAD-QA: A Healthcare Dataset for Complex Reasoning
Vilares, D., & Gómez‐Rodríguez, C. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1092 · Cited by 50 · see citing works
“For this reason, our baselines are based on state-of-the-art systems used in open-domain and multi-choice QA (Chen et al., 2017; Kembhavi et al., 2017; Khot et al., 2018; Clark et al., 2018).”
MethodDeep learning based question answering system in Bengali
Mayeesha, T. T., Sarwar, A. M., & Rahman, R. M. (2020) · Journal of Information and Telecommunication · doi:10.1080/24751839.2020.1833136 · Cited by 50 · see citing works
“Many other reading comprehension datasets in English have also used school science curriculum previously (ARC, OpenBookQA,SciTail,TQA) (details in section 2.1), so our studies are also grounded in historical work.”
BackgroundInfluentialAlawwad, H. A., Alhothali, A., Naseem, U., Alkhathlan, A., & Jamal, A. T. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2024.111332 · Cited by 78 · see citing works
“Examples from the textbook question answering dataset [4].”
InfluentialWebSRC: A Dataset for Web-Based Structural Reading Comprehension
Chen, X., Zhao, Z., Chen, L., Ji, J., Zhang, D., Luo, A., Xiong, Y., & Yu, K. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.343 · Cited by 52 · see citing works
“Multi-modal MRC datasets with both images and texts are proposed, such as MovieQA (Tapaswi et al., 2016), TQA (Kembhavi et al., 2017), COMICS (Iyyer et al., 2017) and RecipeQA (Yagcioglu et al., 2018).”
MethodAn analysis of graph convolutional networks and recent datasets for visual question answering
Yusuf, A. A., Feng, C., & Xianling, M. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10151-2 · Cited by 45 · see citing works
“Textbook Question Answering (TQA) (Kembhavi et al. 2017) is also similar to DVQA but, it does not contain questions specific to bar charts and uses a multiple choice scheme to reduce the ranking problem.”
MethodMachine Reading Comprehension: The Role of Contextualized Language Models and BeyondPreprint
Zhang, Z., Zhao, H., & Wang, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.06249 · Cited by 54 · see citing works
“TextbookQA (Kembhavi et al. 2017) 26K textbook X figures involved”
RUArt: A Novel Text-Centered Solution for Text-Based Visual Question Answering
Jin, Z., Wu, H., Yang, C. H., Zhou, F., Qin, J., Xiao, L., & Yin, X.-C. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3120194 · Cited by 46 · see citing works
Mini-InternVL: a flexible-transfer pocket multi-modal model with 5% parameters and 90% performance
Gao, Z., Chen, Z., Cui, E., Ren, Y., Wang, W., Zhu, J., Tian, H., He, J., He, J., Zhu, X., et al. (2024) · Visual Intelligence · doi:10.1007/s44267-024-00067-6 · Cited by 50 · see citing works
“Ultimately, we develop a vision encoder, termed InternViT-300M, which is infused with diverse knowledge and is adaptable to various language models.”
Wang, L., Hu, Y., He, J., Xu, X., Liu, N., Liu, H., & Shen, H. T. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29884 · Cited by 50 · see citing works
A survey of deep learning techniques for machine reading comprehension
Kazi, S., Khoja, S. A., & Daud, A. Y. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10583-4 · Cited by 43 · see citing works
Deep Vision Multimodal Learning: Methodology, Benchmark, and Trend
Chai, W., & Wang, G. (2022) · Applied Sciences · doi:10.3390/app12136588 · Cited by 43 · see citing works
A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Dataset
Boratko, M., Padigela, H., Mikkilineni, D., Yuvraj, P., Das, R., McCallum, A., Chang, M., Fokoue, A., Kapanipathi, P., Mattei, N., et al. (2018) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/w18-2607 · Cited by 40 · see citing works
“While it is stated that SQuAD requires logical reasoning, the complexity of reasoning required is far less than that for the AI2 standardized tests dataset (Clark and Etzioni, 2016; Kembhavi et al., 2017).”
English Machine Reading Comprehension Datasets: A Survey
Dzendzik, D., Foster, J., & Vogel, C. M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.693 · Cited by 44 · see citing works
“Textbook Question Answering (TQA) (Kembhavi et al., 2017) is a multi-modal dataset requiring not only text understanding but also picture processing.”
BackgroundManyModalQA: Modality Disambiguation and QA over Diverse Inputs
Hannan, D., Jain, A., & Bansal, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6294 · Cited by 37 · see citing works
Roy, R. S., & Anand, A. (2021) · Synthesis lectures on information concepts, retrieval, and services · doi:10.2200/s0113ed1v01y202109icr076 · Cited by 36 · see citing works
Localize, Group, and Select: Boosting Text-VQA by Scene Text Modeling
Lu, X., Fan, Z., Wang, Y., Oh, J., & Rosé, C. P. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00297 · Cited by 36 · see citing works
“…scene texts in visual question answering tasks was originally emphasized by [2], due to the lack of available large-scale datasets, early development of question answering tasks related to embedded text understanding was limited in narrow domains such as bar-charts [14] or diagrams [17, 18].”
BackgroundWhat Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge
Richardson, K., & Sabharwal, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00331 · Cited by 35 · see citing works
Review on Neural Question Generation for Education Purposes
Faraby, S. A., Adiwijaya, A., & Romadhony, A. (2023) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00374-x · Cited by 31 · see citing works
“Several datasets and research on various subjects use texts from the world of education, among others: TQA (life science, earth science, physical science),SciQA(physics,chemistry,biology),(CaoandWang,2021)(science,history,politicalscience),(Steueretal.,2021)(anatomy,biologychemistry,physics, psychology, sociology), an…”
InfluentialKim, D., Kim, S., & Kwak, N. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1347 · Cited by 31 · see citing works
“n and corresponding context paragraph. For above 3 models, we use experimental results newly reported in (Li et al.,2018). Challenge This is the one that obtained the top results in TQA competition (Kembhavi et al., 2017). The results in the table are mixed with each of top score inthetext-question track and the diagr…”
MethodDomain-agnostic Question-Answering with Adversarial Training
Lee, S., Kim, D., & Park, J. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5826 · Cited by 34 · see citing works
“h are BioASQ (BA) (Tsatsaronis et al.,2012), DROP (DP) (Dua et al.,2019), DuoRC (DR) (Saha et al.,2018), RACE (RA) (Lai et al.,2017), RelationExtraction (RE) (Levy et al., 2017), and TextbookQA (TQ) (Kembhavi et al., 2017). Table 1 shows the statistics and description of these datasets. Each dataset has about 1k sampl…”
MethodMMCoQA: Conversational Question Answering over Text, Tables, and Images
Li, Y., Li, W., & Nie, L. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.290 · Cited by 31 · see citing works
“Besides, Textbook QA (Kembhavi et al., 2017) and TVQA (Lei et al.”
MethodIconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language ReasoningPreprint
Lu, P., Qiu, L., Chen, J., Xia, T., Zhao, Y., Zhang, W., Zhou, Y., Liang, X., & Zhu, S. C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.13214 · Cited by 57 · see citing works
“Some more works have proposed datasets of middle school math or science problems in more practical and complex scenarios [46, 27, 43, 44, 37].”
BackgroundAn Exploration of Data Augmentation and Sampling Techniques for Domain-Agnostic Question Answering
Longpre, S., Lu, Y., Tu, Z., & DuBois, C. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5829 · Cited by 34 · see citing works
“85 TextbookQA (Kembhavi et al., 2017) 56.”
Adversarial Multimodal Network for Movie Story Question Answering
Yuan, Z., Sun, S., Duan, L., Li, C., Wu, X. Y., & Xu, C. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3002667 · Cited by 30 · see citing works
“…clips and subtitles for story comprehension [20], [22], [38], textbook question answering with diagrams and languages for lessons understanding [21], [39]–[41], and visual dialogue which requires an agent to hold a meaningful dialog with humans in natural, conversational language about visual…”
BackgroundViQuAE, a Dataset for Knowledge-based Visual Question Answering about Named Entities
Lerner, P. J., Ferret, O., Guinaudeau, C., Borgne, H. L., Besançon, R., Moreno, J. G., & Lovón-Melgarejo, J. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531753 · Cited by 29 · see citing works
“On the contrary, cross-modal QA [6, 24, 40, 42, 47] can be seen as RC across multiple modalities (e.”
EduQG: A Multi-Format Multiple-Choice Dataset for the Educational Domain
Hadifar, A., Bitew, S. K., Deleu, J., Develder, C., & Demeester, T. (2023) · IEEE Access · doi:10.1109/access.2023.3248790 · Cited by 30 · see citing works
“ARC [41] and TQA [42] are collections of MCQs for students.”
BackgroundOn Making Reading Comprehension More Comprehensive
Gardner, M., Berant, J., Hajishirzi, H., Talmor, A., & Min, S. (2019) · Workshop on Machine Reading for Question Answering (MRQA) · doi:10.18653/v1/d19-5815 · Cited by 29 · see citing works
“…arithmetic reasoning (Dua et al., 2019), textbook question answering which requires understanding various forms of knowledge (Clark et al., 2018; Kembhavi et al., 2017) and multi-hop question answering which requires reading multiple distinct pieces of evidence (Talmor and Berant, 2018; Yang et…”
BackgroundSelf-citationCRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense
Gao, D., Wang, R., Shan, S., & Chen, X. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3210780 · Cited by 33 · see citing works
“[29] is to expand the scope of the questions by requiring some commonsense-related abilities.”
BackgroundCNN for Text-Based Multiple Choice Question Answering
Chaturvedi, A., Pandit, O., & Garain, U. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2044 · Cited by 28 · see citing works
“The accuracy of our proposed model on validation set of TQA and SciQ dataset (Kembhavi et al., 2017; Welbl et al., 2017) is given in Table 1 and Table 2.”
BackgroundMethodICDAR 2021 Competition on Document Visual Question Answering
Tito, R., Mathew, M., Jawahar, C. V., Valveny, E., & Karatzas, D. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86337-1_42 · Cited by 27 · see citing works
“Textbook QA [15] is a QA task where questions are based on a context of text and diagrams and images.”
BackgroundDynamic Graph Generation Network: Generating Relational Knowledge from Diagrams
Kim, D., Yoo, Y., Kim, J., Lee, S., & Kwak, N. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00438 · Cited by 26 · see citing works
Textbook Question Answering Under Instructor Guidance with Memory Networks
Li, J., Su, H., Zhu, J., Wang, S., & Zhang, B. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00385 · Cited by 25 · see citing works
“We record the Guidance in the memory, which is different from the previous methods [16, 21] that exploit input data or instructional rules as repository.”
BackgroundMethodInfluentialGómez-Pérez, J. M., & Ortega, R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.441 · Cited by 23 · see citing works
CAiRE-COVID: A Question Answering and Multi-Document Summarization System for COVID-19 Research.
Su, D., Xu, Y., Yu, T., Siddique, F. B., Barezi, E. J., & Fung, P. (2020) · Europe PMC (PubMed Central) · Cited by 24 · see citing works
Kaushik, D., Kiela, D., Lipton, Z. C., & Yih, W.-T. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.517 · Cited by 26 · see citing works
Reddy, R. G., Rui, X., Li, M., Lin, X., Wen, H., Cho, J., Huang, L., Bansal, M., Sil, A., Chang, S., et al. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21370 · Cited by 26 · see citing works
Discourse in Multimedia: A Case Study in Extracting Geometry Knowledge from Textbooks
Sachan, M., Dubey, A., Hovy, E. H., Mitchell, T. M., Roth, D., & Xing, E. P. (2019) · Computational Linguistics · doi:10.1162/coli_a_00360 · Cited by 23 · see citing works
“Other Related Tasks: Our work is also related to Textbook Question Answering (Kembhavi et al. 2017), which proposes the task of multimodal machine comprehension where the context needed to answer questions composes of both text and images.”
BackgroundQuestion Answering Infused Pre-training of General-Purpose Contextualized Representations
Jia, R., Lewis, M., & Zettlemoyer, L. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.59 · Cited by 23 · see citing works
Baradaran, R., & Amirkhani, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.08.095 · Cited by 24 · see citing works
Weakly Supervised Learning for Textbook Question Answering
Ma, J., Chai, Q., Huang, J., Liu, J., You, Y., & Zheng, Q. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3180563 · Cited by 22 · see citing works
Unsupervised Discovery of Multimodal Links in Multi-image, Multi-sentence Documents
Hessel, J., Lee, L., & Mimno, D. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1210 · Cited by 22 · see citing works
Contextualized Representations Using Textual Encyclopedic KnowledgePreprint
Joshi, M., Lee, K., Luan, Y., & Toutanova, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.12006 · Cited by 27 · see citing works
V-Doc : Visual questions answers with Documents
Ding, Y., Huang, Z., Wang, R., Zhang, Y., Chen, X., Ma, Y., Chung, H., & Han, S. C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02083 · Cited by 20 · see citing works
“Recently, there are several Document-VQA datasets proposed by using website [15], textbook slides [10], scanned documents [7] or cross-domain documents [13].”
BackgroundAlawwad, H. A., Alhothali, A., Naseem, U., Alkhathlan, A., & Jamal, A. T. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4761601 · Cited by 20 · see citing works
Automatically Generating Cause-and-Effect Questions from Passages
Stasaski, K., Rathod, M., Tu, T., Xiao, Y., & Hearst, M. A. (2021) · Workshop on Innovative Use of NLP for Building Educational Applications · Cited by 19 · see citing works
Improving Pretrained Language Model Fine-Tuning With Noise Stability Regularization
Hua, H., Li, X., Dou, D., Xu, C., & Luo, J. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3330926 · Cited by 22 · see citing works
Survey of Natural Language Processing for Education: Taxonomy, Systematic Review, and Future Trends
Lan, Y., Li, X., Du, H., Lu, X., Gao, M. Z., Qian, W., & Zhou, A. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3621181 · Cited by 43 · see citing works
“Mesophilic organisms have important uses in food preparation, including cheese, yogurt, beer, and wine.”
Benchmarking Machine Reading Comprehension: A Psychological Perspective
Sugawara, S., Stenetorp, P., & Aizawa, A. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.137 · Cited by 21 · see citing works
“Kembhavi et al. (2017) propose a dataset based on science text-books, which contain questions with passages, diagrams, and images.”
BackgroundSampat, S. K., Kumar, A., Yang, Y., & Baral, C. R. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.289 · Cited by 18 · see citing works
Single-dataset Experts for Multi-dataset Question Answering
Friedman, D., Dodge, B., & Chen, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.495 · Cited by 20 · see citing works
Chatterjee, S., & Dietz, L. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531944 · Cited by 18 · see citing works
“This subset is based partly on a Wikipedia dump from 2018 and partly on the Textbook Question Answering [28] dataset (questions from middle school science curriculum).”
MethodMELINDA: A Multimodal Dataset for Biomedical Experiment Method Classification
Wu, T.-L., Singh, S., Paul, S., Burns, G., & Peng, N. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i16.17657 · Cited by 18 · see citing works
PROST: Physical Reasoning about Objects through Space and Time
Aroca-Ouellette, S., Paik, C., Roncone, A., & Kann, K. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.404 · Cited by 17 · see citing works
Dynamic dual graph networks for textbook question answering
Wang, Y., Liu, J., Ma, J., Zeng, H., Zhang, L., & Li, J. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109441 · Cited by 17 · see citing works
Holistic Multi-Modal Memory Network for Movie Question Answering
Wang, A., Luu, A. T., Foo, C.-S., Zhu, H., Tay, Y., & Chandrasekhar, V. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2931534 · Cited by 17 · see citing works
“Several QA datasets incorporating multiple data modalities have recently been developed towards this end [12], [26], [13].”
BackgroundXTQA: Span-Level Explanations for Textbook Question Answering
Ma, J., Chai, Q., Liu, J., Yin, Q., Wang, P., & Zheng, Q. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3294991 · Cited by 20 · see citing works
“Recently, a new task called textbook question answering (TQA) [5] was proposed and it requires a system to answer diagram and nondiagram (ND) questions automatically given large multimodal contexts consisting of abundant essays and diagrams.”
BackgroundMethodInfluentialAnswering Science Exam Questions Using Query Reformulation with Background Knowledge
Musa, R., Wang, X., Fokoue, A., Mattei, N., Chang, M., Kapanipathi, P., Makni, B., Talamadupula, K., & Witbrock, M. (2019) · Automated Knowledge Base Construction · Cited by 15 · see citing works
Entity Retrieval Using Fine-Grained Entity Aspects
Chatterjee, S., & Dietz, L. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3463035 · Cited by 15 · see citing works
“This benchmark is based partly on a Wikipedia dump from 2018 and partly on the Textbook Question Answering [15] dataset.”
MethodMoCA: Incorporating domain pretraining and cross attention for textbook question answering
Xu, F., Lin, Q., Liu, J., Zhang, L., Zhao, T., Chai, Q., Pan, Y., Huang, Y., & Wang, Q. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109588 · Cited by 17 · see citing works
Effect of Visual Extensions on Natural Language Understanding in Vision-and-Language Models
Iki, T., & Aizawa, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.167 · Cited by 19 · see citing works
“Recent V&L tasks, such as multi-modal reading comprehension (Kembhavi et al., 2017; Yagcioglu et al., 2018; Hannan et al., 2020; Tanaka et al., 2021) and dialogue (Ilinykh et al., 2019; Haber et al., 2019; Udagawa and Aizawa, 2019), require a deeper NLU as well as the grounding.”
BackgroundSpatial-Semantic Collaborative Graph Network for Textbook Question Answering
Wang, Y., Wei, B., Liu, J., Lin, Q., Zhang, L., & Wu, Y. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3231463 · Cited by 15 · see citing works
“In recent years, there have been some works on TQA task, which can be roughly divided into the following three categories: graph-based [8], [33], [34], [35], [36], [37], pretraining-based [38], [39], [40], and interpretability-based [41].”
BackgroundMethodInfluentialNEWTON: Are Large Language Models Capable of Physical Reasoning?
Wang, Y., Duan, J., Fox, D., & Srinivasa, S. S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.652 · Cited by 20 · see citing works
Boateng, G., John, S., Boateng, S. T., Badu, P., Agyeman-Budu, P., & Kumbol, V. W.-A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-64299-9_9 · Cited by 14 · see citing works
- Authors
- Kembhavi, A., Seo, M., Schwenk, D., Choi, J., Farhadi, A., & Hajishirzi, H.
- Affiliations
- Allen Institute for Artificial Intelligence (US)
- University of Washington (US)
- Venue
- 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2017-07
- Pages
- 5376-5384
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2746097825
- Cited by
- 244
- References
- 34
- Citations by year
- 2026: 14 · 2025: 32 · 2024: 27 · 2023: 25 · 2022: 34 · 2021: 41 · 2020: 25 · 2019: 27 · 2018: 18 · 2017: 1
- FWCI
- 5.59
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Seoul National University Open Repository (Seoul National University) (submittedVersion)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Handwritten Text Recognition Techniques
- Keywords
- junior high school science, text parsing, visual question answering, question answering dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
