The DOI matches its Crossref record; 47 of the 49 references it cites were found in a registry or an index.
TVQA: Localized, Compositional Video Question Answering
Open accessAbstract
Recent years have witnessed an increasing interest in image-based question-answering (QA) tasks.However, due to data limitations, there has been much less work on video-based QA.In this paper, we present TVQA, a largescale video QA dataset based on 6 popular TV shows.TVQA consists of 152,545 QA pairs from 21,793 clips, spanning over 460 hours of video.Questions are designed to be compositional in nature, requiring systems to jointly localize relevant moments within a clip, comprehend subtitle-based dialogue, and recognize relevant visual concepts.We provide analyses of this new dataset as well as several baselines and a multi-stream end-to-end trainable neural network framework for the TVQA task.The dataset is publicly available at http://tvqa.cs.unc.edu.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Overall, they obtained promising results on the TVQA [129] and ActivityNetQA [130] datasets.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [129]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“We also use ResNet101 (He et al., 2016) trained on ImageNet (Deng et al., 2009) to extract whole image features.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 5×
“Visual Question Answering: Several image-based VQA datasets have recently been constructed, e.g., DAQUAR (Malinowski and Fritz, 2014), VQA (Antol et al., 2015), COCO-Q (Ren et al., 2015a), FM-IQA (Gao et al., 2015), Visual Madlibs (Yu et al., 2015), Visual7W (Zhu et al., 2016), CLEVR (Johnson et al., 2017), etc. Addit…”
BackgroundMethodInfluentialFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 5×
“ImageNet Features: We extract the pooled 2048D feature of the last block of ResNet101.”
BackgroundMethodInfluentialGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 4×
“A subtitle S, which contains a set of sentences, is flattened into a long sequence of words and GloVe (Pennington et al., 2014) is used to embed the words.”
BackgroundMethodInfluentialSQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 2×
“With the same goal in mind, Rajpurkar et al. (2016) introduced the SQuAD dataset, but their answers are specific spans from long passages.”
BackgroundVQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Gool, L. V. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_2 · Cited by 4,232
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,855 · cited 1×
“Text Question Answering: The related task of text-based question answering has been extensively explored (Richardson et al., 2013; Weston et al., 2015; Rajpurkar et al., 2016; Hermann et al., 2015; Hill et al., 2015).”
BackgroundCLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.215 · Cited by 1,781
Research works printed but not linked: 8
- Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Mar- garet Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question an- swering. ICCV, pages 2425-2433.
- Ronghang Hu, Huazhe Xu, Marcus Rohrbach, Jiashi Feng, Kate Saenko, and Trevor Darrell. 2016. Natu- ral language object retrieval. CVPR.
- Tegan Maharaj, Nicolas Ballas, Anna Rohrbach, Aaron Courville, and Christopher Pal. 2017b. A dataset and exploration of models for understanding video data through fill-in-the-blank question-answering. CVPR.
- Paul Over, Jon Fiscus, Greg Sanders, David Joy, Mar- tial Michel, George Awad, Alan Smeaton, Wessel Kraaij, and Georges Quénot. 2014. Trecvid 2014-an overview of the goals, tasks, data, evaluation mech- anisms and metrics. In Proceedings of TRECVID, page 52.
- Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. 2015. Flickr30k entities: Collecting region-to-phrase correspondences for richer image- to-sentence models. In ICCV.
- Licheng Yu, Zhe Lin, Xiaohui Shen, Jimei Yang, Xin Lu, Mohit Bansal, and Tamara L Berg. 2018b. Mat- tnet: Modular attention network for referring expres- sion comprehension. In CVPR.
- Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. 2016. Modeling context in referring expressions. In ECCV.
- Licheng Yu, Hao Tan, Mohit Bansal, and Tamara L Berg. 2017. A joint speakerlistener-reinforcer model for referring expressions. In CVPR.
Cited by
Sorted by citations
From Recognition to Cognition: Visual Commonsense Reasoning
Zellers, R., Bisk, Y., Farhadi, A., & Choi, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00688 · Cited by 769
“While past work has asked humans to write several counterfactual choices for each correct answer [75, 46], this process is expensive.”
BackgroundLess is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
Lei, J., Li, L., Zhou, L., Gan, Z., Berg, T. L., Bansal, M., & Liu, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00725 · Cited by 639
“Popular video-andlanguage tasks include text-to-video retrieval [75, 28, 54], video captioning [75, 82, 28, 54, 40], video question answering [74, 23, 33], and moment retrieval [2, 18, 35].”
BackgroundInfluentialSelf-citationLearning 2D Temporal Adjacent Networks for Moment Localization with Natural Language
Zhang, S., Peng, H., Fu, J., & Luo, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6984 · Cited by 513
Zhao, F., Zhang, C., & Geng, B. (2024) · ACM Computing Surveys · doi:10.1145/3649447 · Cited by 480
A Corpus for Reasoning about Natural Language Grounded in Photographs
Suhr, A., Zhou, S., Zhang, A. Q., Zhang, I., Bai, H., & Artzi, Y. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1644 · Cited by 628
“, 2018) Visual Question Answering TVQA (Lei et al., 2018) Video Question Answering 4 4”
BackgroundActBERT: Learning Global-Local Video-Text Representations
Zhu, L., & Yang, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00877 · Cited by 425
“, video question answering [36, 10, 18, 54], video captioning [46, 52], textvideo retrieval [47, 41, 25], video grounding [50].”
BackgroundHERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
Li, L., Chen, Y.-C., Cheng, Y., Gan, Z., Yu, L., & Liu, J. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.161 · Cited by 408
Self-citationCross-Modal Causal Relational Reasoning for Event-Level Visual Question Answering
Liu, Y., Li, G., & Lin, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3284038 · Cited by 373
“First, existing visual question answering methods usually focus on relatively simple events like movie [21], TVshow [22] or synthetic videos [23] where temporal understanding and causality discovery are simply not required to perform well, and ignore more complex and challenging events that require in-depth understand…”
BackgroundInfluentialSwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning
Lin, K., Li, L., Lin, C.-C., Ahmed, F., Gan, Z., Liu, Z., Lu, Y., & Wang, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01742 · Cited by 333
“Although it has shown great success in video-and-language understanding tasks, such as video question answering [24] and text-tovideo retrieval [25, 56], it remains unclear whether these sparsely sampled video frames are sufficient to generate rich and descriptive captions.”
BackgroundVL-ADAPTER: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks
Sung, Y.-L., Cho, J., & Bansal, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00516 · Cited by 329
“Table 6 shows the performance of the representative approaches used in Table 2 on TVQA, How2QA, TVC, and YC2C.”
MethodInfluentialSelf-citationLocal-Global Video-Text Interactions for Temporal Grounding
Mun, J., Cho, M., & Han, B. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01082 · Cited by 324
“Furthermore, with the recent advances of deep learning on top of large-scale datasets [2, 16, 17, 23], research on video content understanding is moving towards multi-modal problems (e.”
BackgroundNExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions
Xiao, J., Shang, X., Yao, A., & Chua, T. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00965 · Cited by 391
“Regarding visual feature representations, motion feature are important but naively concatenating appearance and motion features usually results in sub-optimal results (refer to EVQA [2], PSAC+ [26] and STVQA [27]).”
BackgroundMethodInfluentialVLP: A Survey on Vision-language Pre-training
Chen, F., Zhang, D., Han, M., Chen, X.-Y., Shi, J. C., Xu, S., & Xu, B. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1369-5 · Cited by 330
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark
Li, K., Wang, Y., He, Y., Li, Y., Wang, Y., Liu, Y., Wang, Z., Xu, J., Chen, G., Lou, P., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02095 · Cited by 300
“More results on NExT-QA [84], STAR [82], TVQA [35], EgoSchema [52] and IntentQA [40] can be found in the appendix.”
BackgroundResultTVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval
Lei, J., Yu, L., Berg, T. L., & Bansal, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_27 · Cited by 287
Self-citationHierarchical Conditional Relation Networks for Video Question Answering
Le, T. M., Le, V., Venkatesh, S., & Tran, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00999 · Cited by 260
BERT applications in natural language processing: a review
Gardazi, N. M., Daud, A. Y., Malik, M. K., Bukhari, A. A., Alsahfi, T., & Alshemaimri, B. K. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11162-5 · Cited by 354
Unifying Vision-and-Language Tasks via Text GenerationPreprint
Cho, J., Lei, J., Tan, H. X., & Bansal, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.02779 · Cited by 291
“, 2018b) in a wide range of discriminative (Goyal et al., 2019; Hudson & Manning, 2019; Lei et al., 2018; Mao et al., 2016; Xu et al., 2016; Zhou et al., 2018) and generative tasks (Chen et al.”
BackgroundSelf-citationVid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning
Yang, A., Nagrani, A., Seo, P. H., Miech, A., Pont-Tuset, J., Laptev, I., Šivic, J., & Schmid, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01032 · Cited by 296
CLEVRER: CoLlision Events for Video REpresentation and ReasoningPreprint
Yi, K., Gan, C., Li, Y., Kohli, P., Wu, J., Torralba, A. B., & Tenenbaum, J. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01442 · Cited by 477
“The VCR dataset (Zellers (Tapaswi et al., 2016) × × × MarioQA (Mun et al., 2017) × × × TVQA (Lei et al., 2018) × × × × Social-IQ (Zadeh et al., 2019) × × × ×”
BackgroundInfluentialLarge-scale Multi-modal Pre-trained Models: A Comprehensive Survey
Wang, X., Chen, G., Qian, G., Gao, P., Wei, X.-Y., Wang, Y., Tian, Y., & Gao, W. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1410-8 · Cited by 272
“• TVQA [90] is build based on six longrunning TV shows from 3 genres, including sitcoms, medical dramas, and crime drama.”
BackgroundJust Ask: Learning to Answer Questions from Millions of Narrated Videos
Yang, A., Miech, A., Šivic, J., Laptev, I., & Schmid, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00171 · Cited by 232
MERLOT: Multimodal Neural Script Knowledge ModelsPreprint
Zellers, R., Lu, X., Hessel, J., Yu, Y., Park, J. S., Cao, J., Farhadi, A., & Choi, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02636 · Cited by 239
“The tasks we report on are: TVQA [53], TVQA(+) [54], VLEP [55], MSRVTT-QA [105], MSRVTTMultichoice [109], LSMDC-Multichoice, LSMDC fill-in-the-blank QA [97, 81], ActivityNetQA [110, 37], TGIFQA [44], and DramaQA [18]5.”
MethodInfluentialFoundations & Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions
Liang, P. P., Zadeh, A., & Morency, L. (2024) · ACM Computing Surveys · doi:10.1145/3656580 · Cited by 438
“…recent progress in large-scale generative models has yielded impressive results in text-to-image [273, 278], text-to-video [302], audio-to-image [146], text-to-speech [276], speech-to-gesture [10], speaker-to-listener [240], language to pose [11], and speech and music generation [7, 72, 245].”
BackgroundVideo-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis
Fu, C., Dai, Y., Luo, Y., Li, L., Ren, S., Zhang, R., Wang, Z., Zhou, C., Shen, Y., Zhang, M., et al. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.02245 · Cited by 291
Reasoning with Heterogeneous Graph Alignment for Video Question Answering
Pin, J., & Han, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6767 · Cited by 191
VideoChat: chat-centric video understanding
Li, K., He, Y., Wang, Y., Li, Y., Wang, W., Luo, P., Wang, L., Wang, L., & Qiao, Y. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4321-9 · Cited by 1,181
“Please use the sequence adverbs "first", "next", "then" and "finally" to describe this video in detail, but don’t mention the specific time.”
TVQA+: Spatio-Temporal Grounding for Video Question Answering
Lei, J., Yu, L., Berg, T. L., & Bansal, M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.730 · Cited by 225
Self-citationAlign and Prompt: Video-and-Language Pre-training with Entity Prompts
Li, D., Li, J., Li, H., Niebles, J. C., & Hoi, S. C. H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00490 · Cited by 187
“Most prior methods on text-video retrieval [13,31,42,52,54] and videoQA [11, 14, 24, 26] employ pre-trained visual backbones and extract video features for each frame densely yet offline.”
MethodNuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario
Qian, T., Chen, J., Zhuo, L., Jiao, Y., & Jiang, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28253 · Cited by 254
Location-Aware Graph Convolutional Networks for Video Question Answering
Huang, D. Y., Chen, P., Zeng, R., Du, Q., Tan, M., & Gan, C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6737 · Cited by 178
“Based on the visual sources, we can classify the VQA tasks into two categories: image QA (Goyal et al. 2017; Gan et al. 2017) and video QA (Lei et al. 2018; Yi et al. 2019).”
BackgroundMethodAudio Visual Scene-Aware Dialog
Alamri, H., Cartillier, V., Das, A., Wang, J., Cherian, A., Essa, I. A., Batra, D., Marks, T. K., Hori, C., Anderson, P., et al. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00774 · Cited by 177
“While AVSD has fewer unique video clips compared to TVQA and MovieQA, which are curated from television and film, our videos are more naturalistic.”
BackgroundSupport-set bottlenecks for video-text representation learningPreprint
Patrick, M., Huang, P.-Y., Asano, Y. M., Metze, F., Hauptmann, A. G., Henriques, J., & Vedaldi, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02824 · Cited by 214
“0 JSFusion (Yu et al., 2018) 10 .”
BackgroundBoundary Proposal Network for Two-stage Natural Language Video Localization
Xiao, S., Chen, L., Zhang, S., Ji, W., Shao, J., Ye, L., & Xiao, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i4.16406 · Cited by 164
QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension
Rogers, A., Gardner, M., & Augenstein, I. (2022) · ACM Computing Surveys · doi:10.1145/3560260 · Cited by 212
“, [132, 330]), and the source of text used for generating those question matters a lot: the audio descriptions tend to focus on visual features, and text summaries focus on the plot [159].”
BackgroundMethodMERLOT RESERVE: Neural Script Knowledge through Vision and Language and Sound
Zellers, R., Lu, J., Lu, X., Yu, Y., Zhao, Y., Salehi, M., Kusupati, A., Hessel, J., Farhadi, A., & Choi, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01589 · Cited by 179
“[74] Jie Lei, Licheng Yu, Mohit Bansal, and Tamara L Berg.”
BackgroundMethodInfluentialVision-Language Pre-Training: Basics, Recent Advances, and Future Trends
Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., & Gao, J. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000105 · Cited by 155
Interventional Video Grounding with Dual Contrastive Learning
Nan, G., Qiao, R., Xiao, Y., Liu, J., Leng, S., Zhang, H., & Lu, W. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00279 · Cited by 154
Revisiting the “Video” in Video-Language Understanding
Buch, S., Eyzaguirre, C., Gaidon, A., Wu, J., Fei-Fei, L., & Niebles, J. C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00293 · Cited by 154
“Standard video-language tasks include both discriminative tasks, such as video question answering [12, 19, 27, 28, 30, 54, 56, 58, 59], text-to-video/moment retrieval [16,23,43,56,62], and generative tasks, such as video cap-tioning [4, 23] and open-ended VQA [54, 56].”
BackgroundMulla, N., & Gharpure, P. (2023) · Progress in Artificial Intelligence · doi:10.1007/s13748-023-00295-9 · Cited by 132
“Overall, they obtained promising results on the TVQA [129] and ActivityNetQA [130] datasets.”
ResultKaleido-BERT: Vision-Language Pre-training on Fashion Domain
Zhuge, M., Gao, D., Fan, D.-P., Jin, L., Chen, B., Zhou, H., Qiu, M., & Shao, L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01246 · Cited by 130
Learning to Answer Questions in Dynamic Audio-Visual Scenarios
Li, G., Wei, Y., Tian, Y., Xu, C., Wen, J.-R., & Hu, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01852 · Cited by 153
“Most video QA datasets, like ActivityNet-QA [54], TVQA [29], only contain visual question and provide limited possibility to explore audio-visual correlation.”
MethodInfluentialVideo Understanding With Large Language Models: A Survey
Tang, Y., Bi, J., Xu, S., Song, L., Liang, S., Wang, T., Zhang, D., An, J., Lin, J., Zhu, R., et al. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3566695 · Cited by 288
“TVQA [197].”
TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment
Yang, J., Bisk, Y., & Gao, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01136 · Cited by 115
“Realistic application scenarios around videos have prompted emergence of various video-language tasks, such as text-video retrieval [29, 54, 52], video question answering [20, 26], video captioning [53, 58], etc.”
BackgroundCausal Inference Meets Deep Learning: A Comprehensive Survey
Jiao, L., Wang, Y., Liu, X., Li, L., Liu, F., Ma, W., Guo, Y., Chen, P., Yang, S., & Hou, B. (2024) · Research · doi:10.34133/research.0467 · Cited by 139
BERT Representations for Video Question Answering
Yang, Z., García, N., Chu, C., Otani, M., Nakashima, Y., & Takemura, H. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093596 · Cited by 111
“For comparison, we report results on TVQA [21], STAGE [22] and MDAM [17].”
BackgroundMethodInfluentialUncertainty Guided Collaborative Training for Weakly Supervised Temporal Action Detection
Yang, W., Zhang, T., Yu, X., Tian, Q. C., Zhang, Y., & FengWu, F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00012 · Cited by 110
“Because of its broad application potentials in video summarization [22, 16], video surveillance [17], visual question answering [23], and others, it has attracted more and more attention from both academia and industry in recent years.”
BackgroundBridge to Answer: Structure-aware Graph Interaction Network for Video Question Answering
Park, J., Lee, J., & Sohn, K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01527 · Cited by 105
Video Graph Transformer for Video Question Answering
Xiao, J., Zhou, P., Chua, T., & Yan, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_3 · Cited by 108
Yu, J., Zhang, W., Lu, Y., Qin, Z., Hu, Y., Tan, J., & Wu, Q. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2972830 · Cited by 96
Ishmam, M. F., Shovon, M. S. H., Mridha, M. F., & Dey, N. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102270 · Cited by 107
“Being a multi-modal domain, visual question answering naturally works with visual and textual modalities but also works with auditory input as seen in multimodal video question answering [5].”
BackgroundMethodInfluentialSocial-IQ: A Question Answering Benchmark for Artificial Social Intelligence
Zadeh, A., Chan, M., Liang, P. P., Tong, E., & Morency, L. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00901 · Cited by 107
“In machine learning, this form of probing has a well established precedence in multiple different areas ranging from understanding books and text [25], to understanding events in the movies [26].”
BackgroundMethodInfluentialMIST : Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answering
Gao, D., Zhou, L., Ji, L., Zhu, L., Yang, Y., & Shou, M. Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01419 · Cited by 108
“Some others, such as TVQA [25], MovieQA [39], and Social-IQ [50], mainly focus on long videos cropped from movies or TV series for evaluating the understanding of the plot and social interactions, where subtitles play an essential role.”
BackgroundVIOLET : End-to-End Video-Language Transformers with Masked Visual-token ModelingPreprint
Fu, T.-J., Li, L., Gan, Z., Lin, K., Wang, W. Y., Wang, L., & Liu, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12681 · Cited by 212
Fine-grained Iterative Attention Network for Temporal Language Localization in Videos
Qu, X., Tang, P., Zou, Z., Cheng, Y., Dong, J., Zhou, P., & Xu, Z. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3414053 · Cited by 92
“rategy to better predict the temporal boundary. Extensive experiments on three real-world datasets validate the effectiveness of our method. Thefutureworkincludesapply FIANtomorecomplicatedbenchmarks [17, 20]. Combining our method with some pre-training model [18] could also further boost the performance. 7 ACKNOWLEDG…”
MethodTrends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
Mogadala, A., Kalimuthu, M., & Klakow, D. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11688 · Cited by 146
“2 TVQA The TVQA28 (Lei et al., 2018) is created using six different TV shows such as Friends, The Big Bang Theory, How I Met Your Mother, House M.”
MethodProgressive Attention Memory Network for Movie Story Question Answering
Kim, J., Ma, M., Kim, K., Kim, S.-J., & Yoo, C. D. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00853 · Cited by 91
“To this end, various video story QA benchmarks such as PororoQA [16], MeMexQA [12], TVQA [17] and MovieQA [26] have been suggested.”
BackgroundMethodAGQA: A Benchmark for Compositional Spatio-Temporal Reasoning
Grunde-McLaughlin, M., Krishna, R., & Agrawala, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01113 · Cited by 89
Xu, L., Huang, H., & Liu, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00975 · Cited by 99
“In this manner, generally, a fixed computation architecture [29, 12, 21, 10] can be applied over the whole video for tackling each question.”
BackgroundMethodInfluentialGrounding-Tracking-Integration
Yang, Z., Kumar, T., Chen, T., Su, J., & Luo, J. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3038720 · Cited by 88
Video Question Answering: Datasets, Algorithms and Challenges
Zhong, Y., Ji, W., Xiao, J., Li, Y., Deng, W., & Chua, T. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.432 · Cited by 133
Modality Shifting Attention Network for Multi-Modal Video Question Answering
Kim, J., Ma, M., Pham, T. X., Kim, K., & Yoo, C. D. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01012 · Cited by 84
Reading-Strategy Inspired Visual Representation Learning for Text-to-Video Retrieval
Dong, J., Wang, Y., Chen, X., Qu, X., Li, X., He, Y., & Wang, X. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3150959 · Cited by 89
“Additionally, comparing the models pre-trained on TV dataset [71]”
MethodInformation fusion in visual question answering: A Survey
Zhang, D., Cao, R., & Wu, S. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.03.005 · Cited by 82
Loss Re-Scaling VQA: Revisiting the Language Prior Problem From a Class-Imbalance View
Guo, Y., Nie, L., Cheng, Z., Tian, Q., & Zhang, M. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3128322 · Cited by 83
Dual-Evidential Learning for Weakly-supervised Temporal Action Localization
Chen, M., Gao, J., Yang, S., & Xu, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19772-7_12 · Cited by 80
“It has attracted significant attention from both academia and industry, due to the great potential for video retrieval [9, 41], summarization [24], surveillance [18, 51], anomaly detection [50], visual question answering [25], to name a few.”
BackgroundPano-AVQA: Grounded Audio-Visual Question Answering on 360° Videos
Yun, H., Yu, Y., Yang, W., Lee, K., & Kim, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00204 · Cited by 87
VideoLLM-online: Online Video Large Language Model for Streaming Video
Chen, J., Lv, Z., Wu, S., Lin, K. Q., Song, C., Gao, D., Liu, J.-W., Gao, Z., Mao, D., & Shou, M. Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01742 · Cited by 118
“Typical video understanding benchmarks, such as action recognition [10], temporal action localization [31], video question answering [41], and video dialogue [2], typically allow models to access entire video frames to make predictions, a setting referred to as ”offline”.”
BackgroundFrozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval
Bain, M., Nagrani, A., Varol, G., & Zisserman, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00175 · Cited by 76
“Joint visual-text models have become increasingly popular as they enable a wide suite of downstream tasks, including text-to-visual retrieval [29, 31, 36, 59], visual captioning [24, 58, 66], and visual question and answering [4, 27].”
BackgroundWebQA: Multihop and Multimodal QA
Chang, Y., Cao, G., Narang, M., Gao, J., Suzuki, H., & Bisk, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01600 · Cited by 110
“We provide annotators with four snippets to prevent and allow them to contribute facts they researched to help answer the question.”
MethodRevealing Single Frame Bias for Video-and-Language Learning
Lei, J., Berg, T. L., & Bansal, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.29 · Cited by 146
Self-citationLearnable Aggregating Net with Diversity Learning for Video Question Answering
Li, X., Gao, L., Wang, X., Liu, W., Xu, X., Shen, H. T., & Song, J. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350971 · Cited by 73
“Firstly, we extend single-path co-attention to multi-path pyramid co-attention with diversity learning, which explicitly encourage attended more diverse.”
BackgroundKnowIT VQA: Answering Knowledge-Based Questions about Videos
García, N., Otani, M., Chu, C., & Nakashima, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6713 · Cited by 6
TEVAD: Improved video anomaly detection with captions
Chen, W., Ma, K. T., Yew, Z. J., Hur, M., & Khoo, D. A.-A. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00587 · Cited by 82
“More recently, with the success of transformer models [10, 52] in natural language processing fields, the computer vision community has tried to apply the ideas on different downstream tasks and achieved promising results [5, 18, 28, 29, 33, 67].”
BackgroundTarget Adaptive Context Aggregation for Video Scene Graph Generation
Teng, Y., Wang, L., Li, Z., & Wu, G. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01343 · Cited by 74
“This structured video representation, known as video scene graph [27], can contribute to more accurate action recognition [10] and allow for our vision models to tackle high-level and complex inference tasks, such as video caption [7, 36], video retrieval [4, 16], and video question answering [33, 15].”
BackgroundLearning Audio-Video Modalities from Image Captions
Nagrani, A., Seo, P. H., Seybold, B., Hauth, A., Manén, S., Sun, C., & Schmid, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19781-9_24 · Cited by 82
Proposal-Based Multiple Instance Learning for Weakly-Supervised Temporal Action Localization
Ren, H., Yang, W., Zhang, T., & Zhang, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00237 · Cited by 69
“TAL has recently received increasing attention from the research community due to its broad application potentials, such as intelligent surveillance [44], video summarization [22], highlight detection [49], and visual question answering [23].”
Multi-Scale 2D Temporal Adjacency Networks for Moment Localization With Natural Language
Zhang, S., Peng, H., Fu, J., Lu, Y., & Luo, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3120745 · Cited by 74
“Video moment localization with natural language has a wide range of applications, such as video question answering [14], video content retrieval [15], dense video captioning [16], aswell as video storytelling [17].”
VLG-Net: Video-Language Graph Matching Network for Video Grounding
Soldan, M., Xu, M., Qu, S., Tegner, J. N., & Ghanem, B. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00361 · Cited by 73
“This novel inter-disciplinary task has gained momentum within the vision and language communities for its relevance and possible applications in video retrieval [13, 52, 75], video question answering [21, 28], human-computer interaction [82], and video storytelling [19].”
BackgroundCan I Trust Your Answer? Visually Grounded Video Question Answering
Xiao, J., Yao, A., Li, Y., & Chua, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01254 · Cited by 101
“TVQA [25]) or limited to few objects ( e.g .”
BackgroundMethodInfluentialVindLU: A Recipe for Effective Video-and-Language Pretraining
Cheng, F., Wang, X., Lei, J., Crandall, D., Bansal, M., & Bertasius, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01034 · Cited by 76
“In Table 6, we also present our results for the video question-answering task on ActivityNet-QA [85], MSRVTT-QA [75], MSRVTTMC [84] and TVQA [33].”
BackgroundResultInfluentialSelf-citationMultimodal Emotion-Cause Pair Extraction in Conversations
Wang, F., Ding, Z., Xia, R., Li, Z., & Yu, J. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3226559 · Cited by 71
“For example, exploring visual commonsense reasoning [42], [43], [44], a new direction in computer vision, might be helpful for MECPE.”
BackgroundSlideVQA: A Dataset for Document Visual Question Answering on Multiple Images
Tanaka, R., Nishida, K., Nishida, K., Hasegawa, T., Saito, I., & Saito, K. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26598 · Cited by 114
Violin: A Large-Scale Dataset for Video-and-Language Inference
Liu, J., Chen, W., Cheng, Y., Gan, Z., Yu, L., Yang, Y., & Liu, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01091 · Cited by 69
Self-citationVisual Semantic Role Labeling for Video Understanding
Sadhu, A., Gupta, T., Yatskar, M., Nevatia, R., & Kembhavi, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00554 · Cited by 63
“Segments ActivityNet[24], Vatex[77], YouCook[13], MSR-VTT [82] , LSMDC [60] Video QA Q/A, Subtitle or Script (optional) MSRVTT-QA[81], VideoQA[86], ActivityNetQA[85], TVQA[37], MovieQA[70] Text to Video Retrieval Text Query, ASR output (optional) HowTo100M[50], TVR[38], DiDeMo[25], Charades-STA[17] Video Object Ground…”
MethodLi, J., Niu, L., & Zhang, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02059 · Cited by 73
“This is different from the popular TVQA [27], MovieQA [40], and CLEVRER [52] dataset that are based on some unrealistic scenes (e.”
MethodInfluentialVideo Face Clustering With Unknown Number of Clusters
Tapaswi, M., Law, M. T., & Fidler, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00513 · Cited by 62
“Understanding characters also has a direct influence on important research such as video captioning [34, 35], question-answering [22, 44], studying social situations [45] and 4D effects [56].”
BackgroundVideo2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning
Fang, Z., Gokhale, T., Banerjee, P., Baral, C. R., & Yang, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.61 · Cited by 69
Attend What You Need: Motion-Appearance Synergistic Networks for Video Question Answering
Seo, A., Kang, G.-C., Park, J., & Zhang, B. (2021) · doi:10.18653/v1/2021.acl-long.481 · Cited by 67
“In particular, video question answering (video QA) tasks (Xu et al., 2016; Jang et al., 2017; Lei et al., 2018; Yu et al., 2019; Choi et al., 2020) have been proposed with the goal of reasoning over higher-level vision-”
Learning Interactions and Relationships Between Movie Characters
Kukleva, A., Tapaswi, M., & Laptev, I. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00987 · Cited by 65
Document Understanding Dataset and Evaluation (DUDE)
Landeghem, J. V., Powalski, R., Tito, R., Jurkiewicz, D., Blaschko, M. B., Borchmann, Ł., Coustaty, M., Moens, S., Pietruszka, M., Ackaert, B., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01789 · Cited by 95
“For example, the question-answering approach is combined with modalities such as videos [44, 13, 14, 28, 17], images [98, 3, 29, 68, 7, 8], speech [99, 43], knowledge graphs [93, 84, 80, 22, 37], and maps [70, 15].”
MethodEmoSen: Generating Sentiment and Emotion Controlled Responses in a Multimodal Dialogue System
Firdaus, M., Chauhan, H., Ekbal, A., & Bhattacharyya, P. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3015491 · Cited by 62
“Similar to [53], we sample dialogue from a given scene while constraining video clip length to 60 seconds for the comedy based TV series and 90 seconds for drama TV series.”
BackgroundMethodA Qualitative Comparison of CoQA, SQuAD 2.0 and QuAC
Yatskar, M. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1241 · Cited by 60
“Other work considers multi-modal contexts where interactions are a single turn (Tapaswi et al., 2016; Antol et al., 2015; Lei et al., 2018) or multi-turn (Das et al.”
BackgroundVX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs
Lin, X., Bertasius, G., Wang, J., Chang, S., Parikh, D., & Torresani, L. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00693 · Cited by 58
“Prime examples include datasets for image or video captioning [10, 38, 51, 24, 56, 28], question answering (QA) [5, 13, 54, 58, 19, 36, 46, 26], as well as audio-visual dialog [11, 1].”
BackgroundMethodResultInfluentialA Survey on Relation Extraction
Detroja, K., Bhensdadia, C. K., & Bhatt, B. (2022) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2023.200244 · Cited by 64
Embracing Uncertainty: Decoupling and De-bias for Robust Temporal Grounding
Zhou, H., Zhang, C., Luo, Y., Chen, Y., & Hu, C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00834 · Cited by 54
“video captioning [26, 41, 5], video object segmentation [7, 13] and video question answering [19, 14, 35].”
BackgroundRelation-aware Video Reading Comprehension for Temporal Language Grounding
Gao, J., Sun, X., Xu, M., Zhou, X., & Ghanem, B. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.324 · Cited by 67
“It has also drawn great attention from industry due to its various applications such as video question answering (Huang et al., 2020; Lei et al., 2018), video content retrieval (Dong et al.”
An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling
Fu, T.-J., Li, L., Gan, Z., Lin, K., Wang, W. Y., Wang, L., & Liu, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02193 · Cited by 70
“Researchers have explored such capability on VidL tasks including text-to-video retrieval [80, 30, 59, 38, 40], video question answering [26, 79, 36, 37], moment retrieval [24, 20, 30, 38], and video captioning [74, 87, 80, 59].”
BackgroundSupport-Set Based Cross-Supervision for Video Grounding
Ding, X., Wang, N., Zhang, S., Cheng, D., Li, X., Huang, Z., Tang, M., & Gao, X. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01137 · Cited by 53
“Some works use the contrastive loss to improve video-text representations based on HowTo100M for tasks such as video caption [53], video retrieval [2] and video question answering [24].”
MethodInteractive Question Answering Systems: Literature Review
Biancofiore, G. M., Deldjoo, Y., Noia, T. D., Sciascio, E. D., & Narducci, F. (2024) · ACM Computing Surveys · doi:10.1145/3657631 · Cited by 61
“[63] to be compositional in nature, requiring systems to recognize relevant moments within clips, comprehend subtitles and visual concepts to adequately respond.”
Background
Metadata
- Authors
- Lei, J., Yu, L., Bansal, M., & Berg, T.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 1369-1379
- OpenAlex
- W2963541336
- Cited by
- 541
- References
- 49
- Citations by year
- 2026: 39 · 2025: 50 · 2024: 65 · 2023: 93 · 2022: 90 · 2021: 101 · 2020: 70 · 2019: 32 · 2018: 1
- FWCI
- 15.13
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1405822, 1562098, 1633295)
- Army Research Office (W911NF)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- video question answering, video localization, television series, end-to-end learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
