The DOI matches its Crossref record; 47 of the 49 references it cites were found in a registry or an index.
TVQA: Localized, Compositional Video Question Answering
Open accessAbstract
Recent years have witnessed an increasing interest in image-based question-answering (QA) tasks.However, due to data limitations, there has been much less work on video-based QA.In this paper, we present TVQA, a largescale video QA dataset based on 6 popular TV shows.TVQA consists of 152,545 QA pairs from 21,793 clips, spanning over 460 hours of video.Questions are designed to be compositional in nature, requiring systems to jointly localize relevant moments within a clip, comprehend subtitle-based dialogue, and recognize relevant visual concepts.We provide analyses of this new dataset as well as several baselines and a multi-stream end-to-end trainable neural network framework for the TVQA task.The dataset is publicly available at http://tvqa.cs.unc.edu.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Overall, they obtained promising results on the TVQA [129] and ActivityNetQA [130] datasets.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [129]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“We also use ResNet101 (He et al., 2016) trained on ImageNet (Deng et al., 2009) to extract whole image features.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 5×
“Visual Question Answering: Several image-based VQA datasets have recently been constructed, e.g., DAQUAR (Malinowski and Fritz, 2014), VQA (Antol et al., 2015), COCO-Q (Ren et al., 2015a), FM-IQA (Gao et al., 2015), Visual Madlibs (Yu et al., 2015), Visual7W (Zhu et al., 2016), CLEVR (Johnson et al., 2017), etc. Addit…”
BackgroundMethodInfluentialFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 5×
“ImageNet Features: We extract the pooled 2048D feature of the last block of ResNet101.”
BackgroundMethodInfluentialGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 4×
“A subtitle S, which contains a set of sentences, is flattened into a long sequence of words and GloVe (Pennington et al., 2014) is used to embed the words.”
BackgroundMethodInfluentialSQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 2×
“With the same goal in mind, Rajpurkar et al. (2016) introduced the SQuAD dataset, but their answers are specific spans from long passages.”
BackgroundVQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Gool, L. V. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_2 · Cited by 4,232
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,855 · cited 1×
“Text Question Answering: The related task of text-based question answering has been extensively explored (Richardson et al., 2013; Weston et al., 2015; Rajpurkar et al., 2016; Hermann et al., 2015; Hill et al., 2015).”
BackgroundCLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.215 · Cited by 1,781
Research works printed but not linked: 8
- Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Mar- garet Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question an- swering. ICCV, pages 2425-2433.
- Ronghang Hu, Huazhe Xu, Marcus Rohrbach, Jiashi Feng, Kate Saenko, and Trevor Darrell. 2016. Natu- ral language object retrieval. CVPR.
- Tegan Maharaj, Nicolas Ballas, Anna Rohrbach, Aaron Courville, and Christopher Pal. 2017b. A dataset and exploration of models for understanding video data through fill-in-the-blank question-answering. CVPR.
- Paul Over, Jon Fiscus, Greg Sanders, David Joy, Mar- tial Michel, George Awad, Alan Smeaton, Wessel Kraaij, and Georges Quénot. 2014. Trecvid 2014-an overview of the goals, tasks, data, evaluation mech- anisms and metrics. In Proceedings of TRECVID, page 52.
- Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. 2015. Flickr30k entities: Collecting region-to-phrase correspondences for richer image- to-sentence models. In ICCV.
- Licheng Yu, Zhe Lin, Xiaohui Shen, Jimei Yang, Xin Lu, Mohit Bansal, and Tamara L Berg. 2018b. Mat- tnet: Modular attention network for referring expres- sion comprehension. In CVPR.
- Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. 2016. Modeling context in referring expressions. In ECCV.
- Licheng Yu, Hao Tan, Mohit Bansal, and Tamara L Berg. 2017. A joint speakerlistener-reinforcer model for referring expressions. In CVPR.
Cited by
Sorted by citations
Items 501–534 of 534
Story-Context-Aware Audio Description Generation via Hierarchical Narrative Modeling
Yang, Y., & Liu, X. (2025) · International Conference on Virtual Reality and Visualization · doi:10.1109/icvrv67992.2025.00049
“TV-AD, on the other hand, consists of 34k AD annotations derived from 13 TV series, with its test set sourced from TVQA [21], featuring 3k ADs.”
Thayyil, S., & Hameed, I. A. (2025) · International Conference on Modern Educational Technology (ICMET) · doi:10.1109/icmet67594.2025.11451923
“This was extended to the video domain with complex datasets like MovieQA by Tapaswi et al. [10] and TVQA by Lei et al. [8], which aim to evaluate deep story comprehension.”
Transforming Frame-Based Video Descriptions into Text Using Large Language Models
Gül, Ö. S., & Sayar, A. (2026) · Gazi University Journal of Science Part A Engineering and Innovation · doi:10.54287/gujsa.1839173
Large-scale learning from video and natural language
Miech, A. (2020) · theses.fr (ABES) · doi:10.70675/e1c0d395ze48fz4d9bz821ezf11900bb536d
“In contrast to MovieQA, [Lei et al., 2018] propose the TVQA dataset a TV shows based dataset which instead focuses on the understanding of the visual content rather than the plot.”
Multimodal and interactive models for visually grounded language learning
Strub, F. (2020) · theses.fr (ABES) · doi:10.70675/565f6b44z5446z433azb85dz57d7ca36bccf
Video question answering with limited supervision
Engin, D. (2024) · theses.fr (ABES) · doi:10.70675/c9c7dcf2z4ca3z4e91z9d27zdacd00f76581
Multimodal deep learning for audiovisual production
Ouenniche, K. (2023) · theses.fr (ABES) · doi:10.70675/a01becf4z4ed9z4955z940cz49c821c1b8a8
Liao, C.-T., Xiao, X., Meng, C., Chen, Z., Qiao, Y., Zhou, W., Wang, T., Xu, Z., & Cao, X. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37235-2_25 · Cited by 10
Evaluating the Capability of Video Question Generation for Expert Knowledge Elicitation
Zhang, H., Hashimoto, A., & Hirasawa, T. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00298
“When evaluating the quality of generated questions, human assessment is commonly used [8, 18].”
CoreCaption: Core Caption based Text-to-Video Retrieval
Jang, J. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00638
“This surge in video data has brought the task of text-video retrieval to the fore-front of computer vision and multimedia analysis research [32, 42, 62].”
Shin, J., Hur, C., Cho, S., Choi, J., & Park, H. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00738
DiffLabel: Robust Pseudo Label Generation for Semi-Supervised Temporal Language Grounding
Song, X., Wang, L., Zhou, S., Sun, X., LIU, M. T. H., Xia, K., & Hua, G. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3694564
Wu, Z., Liu, Z., Chen, A., Zhang, J., Li, R., Ge, H., Wang, Z., Xiao, C., & Liang, C. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02898-w
“Secondly, as illustrated in Figure 3, the majority of current DVU datasets (Tapaswi et al., 2016; Lei et al., 2018, 2020; Song et al., 2024; Curtis et al., 2020) lack a balanced distribution of fine-grained topics.”
RA-SSU: Toward Fine-Grained Audio-Visual Learning With Region-Aware Sound Source Understanding
Sun, M., Wang, Y., Wang, H., Chen, S., Zhang, M., Qi, X., Li, Q., & Sun, Z. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3702530
“In this paper, wefocusonafine-grainedaudio-visuallearningtasktoempower videounderstandingwithamoredetailedperception.”
Zhenpeng, Z., Wu, X., Wang, X., Yu, Q., Wang, Y., & Liu, Z. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810669
REAP: Relevance-Guided Evidence-Unit Keyframe Extraction for Long Video Question Answering
Yang, M., Qu, H., & Zhao, J. Z. (2026) · International Symposium on Computer Technology and Information Science (ISCTIS) · doi:10.1109/isctis70043.2026.11572599
ViS2T: A vision and scene graph to text model for captive giant panda video captioning
Ma, C., Duan, C., Zhang, K., Chen, P., He, M., Wang, Z., Zhang, P., & Zhu, C. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.133544
StoryTeller: Training-Free Narrative Grounding for Long-Form Audio Description
Hahm, S. H., Dinh, M. T., & Jin, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37435-6_2
A Multi-Granularity Game-Theoretic Approach to Weakly Supervised Temporal Article Grounding
He, S., Wen, H., Zou, P., Zhou, S., & Kong, Q. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809835
“Video language grounding [1, 9] aims to localize target clips in an untrimmed video according to a given text query, which is essential for various multi-modal tasks, such as video question answering [19], highlight detection [30], video storytelling [21], and text-to-video retrieval [5, 8].”
Agentic-VideoRAG: A Cost-Aware and Explainable Framework for Video Retrieval
Rajarajeswari, S., & Swaroop, M. (2026) · International Conference on Secure Cyber Computing and Communications (ICSCCC) · doi:10.1109/icsccc69031.2026.11600045
TECL: Time-Equivariant Contrastive Learning for weakly-supervised Grounded Video Question Answering
Liu, W., Zhu, Z., Lu, Q., Zhang, S., Zhao, L., Zhao, D., Jiang, X., Zhu, M., & Wang, Y. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116687
Scene Graph for Vietnamese Video Understanding: An Agentic Approach with Reasoning
Vo, Q. N. T., Phan, D. T., & Quan, T. T. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-2600-9_12
Long Story Short: Story-level Video Understanding from 20K Short Films
Ghermi, R., Wang, X., Kalogeiton, V. S., & Laptev, I. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02940-x · Cited by 22
“Figure 3 showcases this issue, showing that modern LLMs can achieve high accuracy on questions from major movie datasets using only movie titles.”
InfluentialTemporal Coherence in Video-Language Models for Long-Form Narrative Understanding
M, T. S. M. (2026) · Eduschool Journal of Artificial Intelligence Research (EJAIR) · doi:10.63090/ejair/3139.7336.0006
Ask, Then Perceive: Question-Guided Cross-Modal Perception for Audio-Visual Question Answering
Ye, Q., Wang, J., Chen, C., Yu, S., Ma, H., Lin, X., & Yu, Z. (2026) · International Conference on Bioinformatics and Intelligent Computing (BIC) · doi:10.1145/3809986.3810022
“Among them, the [21] and [56] datasets collect video data in real environments that contain complex audio-visual pairs, which generate a large amount of redundant information prone to misclassification of the discriminative model.”
Wavelet-Based Intra-Video Counterfactual Reasoning for Video Question Grounding
Yuan, J., & Wei, W. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37098-3_35
Training-Free Event-Aware Memory Enhancement for Streaming Video Question Answering
Song, Y. (2026) · International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc70586.2026.11686687
“Current approaches to streaming VideoQA predominantly rely on text-centric memory architectures that compress video frames into textual captions before storage [1], [2].”
Li, Y., Yang, Y., Yu, F., Hu, J., Fang, Z., & Huang, L. (2026) · International Conference on Big Data and Information Analytics (BigDIA) · doi:10.1109/bigdia71496.2026.11684568
“Subsequent work promotes evidence-grounded CoT by adopting vision-aware prompting and training strategies to enhance reasoning faithfulness [15].”
Clue Matters: Empower Video Reasoning with Brain-Inspired Latent Clue Learning
Zhang, K., Li, X., Li, J., Wu, H., Zhao, X., Fan, Z., & Wang, L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37174-4_12
Cross-Modal-Domain Generalization Through Semantically Aligned Discrete Representations
Sen, S. N., Younis, R., & Ahmadi, Z. (2026) · doi:10.1109/cvprf71909.2026.00584
“Discrete spaces compress high-dimensional multimodal features into finite sets of generalizable latent codes [22, 34], enabling tasks such as visual question answering [3, 18, 19], query-based segmentation [10, 29], and audio-visual event localization [32, 40].”
One Identity, Many Roles: Multimodal Entity Coreference for Enhanced Video Situation Recognition
Darur, B., Garg, A., & Tapaswi, M. (2026) · doi:10.1109/cvprf71909.2026.00794
“…[15, 21, 55, 83], spatio-temporal action detection [17, 58], and visual tracking [4, 11, 12, 47, 74]; and (ii) video-language tasks such as dense video captioning [22, 72, 84], video question answering [13, 20, 29, 59, 77], referring expression grounding and segmentation [24, 32, 41, 50, 71].”
V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning
Cheng, Z., Hu, J., Liu, Z., Si, C., Li, W., & Gong, S. (2026) · doi:10.1109/cvprf71909.2026.00878
“Some partially cover on “when” [2, 4, 16, 24, 36] or “where” [27], without complete spatio-temporal reasoning chain.”
Causal Chain-Guided Reasoning for Modular and Explainable Causal-Why Video Question Answering
Parmar, P., Peh, E., & Fernando, B. (2026) · doi:10.1109/cvprf71909.2026.00563
Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21783 · Cited by 505
Self-citation
Metadata
- Authors
- Lei, J., Yu, L., Bansal, M., & Berg, T.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 1369-1379
- OpenAlex
- W2963541336
- Cited by
- 541
- References
- 49
- Citations by year
- 2026: 39 · 2025: 50 · 2024: 65 · 2023: 93 · 2022: 90 · 2021: 101 · 2020: 70 · 2019: 32 · 2018: 1
- FWCI
- 15.13
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1405822, 1562098, 1633295)
- Army Research Office (W911NF)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- video question answering, video localization, television series, end-to-end learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
