The DOI matches its Crossref record; 47 of the 49 references it cites were found in a registry or an index.
TVQA: Localized, Compositional Video Question Answering
Open accessAbstract
Recent years have witnessed an increasing interest in image-based question-answering (QA) tasks.However, due to data limitations, there has been much less work on video-based QA.In this paper, we present TVQA, a largescale video QA dataset based on 6 popular TV shows.TVQA consists of 152,545 QA pairs from 21,793 clips, spanning over 460 hours of video.Questions are designed to be compositional in nature, requiring systems to jointly localize relevant moments within a clip, comprehend subtitle-based dialogue, and recognize relevant visual concepts.We provide analyses of this new dataset as well as several baselines and a multi-stream end-to-end trainable neural network framework for the TVQA task.The dataset is publicly available at http://tvqa.cs.unc.edu.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Overall, they obtained promising results on the TVQA [129] and ActivityNetQA [130] datasets.”
From Automatic question generation: a review of methodologies, datasets, evaluation metrics, and applications (2023) · cites this paper as [129]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“We also use ResNet101 (He et al., 2016) trained on ImageNet (Deng et al., 2009) to extract whole image features.”
MethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 5×
“Visual Question Answering: Several image-based VQA datasets have recently been constructed, e.g., DAQUAR (Malinowski and Fritz, 2014), VQA (Antol et al., 2015), COCO-Q (Ren et al., 2015a), FM-IQA (Gao et al., 2015), Visual Madlibs (Yu et al., 2015), Visual7W (Zhu et al., 2016), CLEVR (Johnson et al., 2017), etc. Addit…”
BackgroundMethodInfluentialFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 5×
“ImageNet Features: We extract the pooled 2048D feature of the last block of ResNet101.”
BackgroundMethodInfluentialGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 4×
“A subtitle S, which contains a set of sentences, is flattened into a long sequence of words and GloVe (Pennington et al., 2014) is used to embed the words.”
BackgroundMethodInfluentialSQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 2×
“With the same goal in mind, Rajpurkar et al. (2016) introduced the SQuAD dataset, but their answers are specific spans from long passages.”
BackgroundVQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Gool, L. V. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_2 · Cited by 4,232
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,855 · cited 1×
“Text Question Answering: The related task of text-based question answering has been extensively explored (Richardson et al., 2013; Weston et al., 2015; Rajpurkar et al., 2016; Hermann et al., 2015; Hill et al., 2015).”
BackgroundCLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
Johnson, J., Hariharan, B., van der Maaten, L., Fei-Fei, L., Zitnick, C. L., & Girshick, R. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.215 · Cited by 1,781
Flagged first, then by citations
2 of the 49 references couldn’t be matched to a known paper.
Research works printed but not linked: 8
- Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Mar- garet Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question an- swering. ICCV, pages 2425-2433.
- Ronghang Hu, Huazhe Xu, Marcus Rohrbach, Jiashi Feng, Kate Saenko, and Trevor Darrell. 2016. Natu- ral language object retrieval. CVPR.
- Tegan Maharaj, Nicolas Ballas, Anna Rohrbach, Aaron Courville, and Christopher Pal. 2017b. A dataset and exploration of models for understanding video data through fill-in-the-blank question-answering. CVPR.
- Paul Over, Jon Fiscus, Greg Sanders, David Joy, Mar- tial Michel, George Awad, Alan Smeaton, Wessel Kraaij, and Georges Quénot. 2014. Trecvid 2014-an overview of the goals, tasks, data, evaluation mech- anisms and metrics. In Proceedings of TRECVID, page 52.
- Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. 2015. Flickr30k entities: Collecting region-to-phrase correspondences for richer image- to-sentence models. In ICCV.
- Licheng Yu, Zhe Lin, Xiaohui Shen, Jimei Yang, Xin Lu, Mohit Bansal, and Tamara L Berg. 2018b. Mat- tnet: Modular attention network for referring expres- sion comprehension. In CVPR.
- Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. 2016. Modeling context in referring expressions. In ECCV.
- Licheng Yu, Hao Tan, Mohit Bansal, and Tamara L Berg. 2017. A joint speakerlistener-reinforcer model for referring expressions. In CVPR.
Cited by
Sorted by citations
Items 201–300 of 534
PERCEIVER-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention
Tang, Z., Cho, J., Lei, J., & Bansal, M. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00439 · Cited by 17
“Following this success, imagetext [69, 52, 9, 47, 86, 44, 41, 12, 59, 30] and videotext [68, 88, 45, 72, 84, 83, 70, 71] multi-modal transformers have achieved improvements on various visionand-language tasks [3, 8, 80, 87, 42].”
BackgroundMethodSelf-citationText-Guided Object Detector for Multi-modal Video Question Answering
Shen, R., Inoue, N., & Shinoda, K. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00109 · Cited by 16
“The early studies [8, 19, 9, 20] usually used Word2Vec and GloVe as the textual information extractor to extract static word embeddings, whereas recently, contextual word embeddings extracted by BERT [28, 4, 29, 7] are more preferred because of its excellent performance.”
BackgroundMethodInfluentialGuo, K., Tian, D., Hu, Y., Lin, C., Sun, Y., Zhou, J., Duan, X., Gao, J., & Yin, B. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3409453 · Cited by 20
“These include VIS + LSTM [79], I3D + LSTM [53], TVQA [55], BERT-VQA [42], HCRN [56], Eclipse [27], VQAC [59], DualVGR [61], CLIP + Unsupervised [33], CLIP + Finetuning [32], CLIP + LORA [80], CLIP + Adapter [81], ATP [70], CMCIR [28], and Tem-Adapter [32].”
Pairwise VLAD Interaction Network for Video Question Answering
Wang, H., Guo, D., Hua, X., & Wang, M. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475620 · Cited by 15
“Intelligent conversation referring to both vision and language has emerged as a prospective research topic in the computer vision community, such as visual question answering (VQA) [1, 37], visual dialog [4, 12, 13, 22] and video question answering (VideoQA) [18, 24, 31, 33].”
BackgroundVLN-Video: Utilizing Driving Videos for Outdoor Vision-and-Language Navigation
Li, J., Padmakumar, A., Sukhatme, G. S., & Bansal, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29813 · Cited by 16
Self-citationYu, T., Fu, K., Zhang, J., Huang, Q., & Yu, J. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3390984 · Cited by 19
“Concurrently emerging with VideoQA [37], MovieQA presents distinctive challenges in reasoning across diverse modalities, involving not only the visual content but also additional textual resources like subtitles and plots of the movies [38], as well as TV shows [39], [40], among others.”
BackgroundQuestion Answering Survey: Directions, Challenges, Datasets, Evaluation Matrices
Pandya, H. A., & Bhatt, B. (2021) · Journal of Xidian University · doi:10.37896/jxu15.4/014 · Cited by 42
“Figure 5 indicates examples of visual question answering derived from the image dataset COCOQA[89] and TVQA[64], a dataset for video based question answering.”
InfluentialMulti-step Joint-Modality Attention Network for Scene-Aware Dialogue SystemPreprint
Chu, Y.-W., Lin, K.-Y., Hsu, C.-C., & Ku, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06206 · Cited by 15
“Much research (Tapaswi et al. 2015; Lei et al. 2018; Jang et al. 2017) The AVSD dataset provides multimodal features, including vision, audio, and dialogue history, for videos.”
Learning visual language models for video understanding
Yang, A. (2023) · theses.fr (ABES) · doi:10.70675/802872a7z8092z4556za1fdz2c7af45f0e72 · Cited by 26
About Time: Advances, Challenges, and Outlooks of Action Understanding
Stergiou, A., & Poppe, R. W. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02478-4 · Cited by 17
VGNMN: Video-grounded Neural Module Networks for Video-Grounded Dialogue Systems
Le, H., Chen, N. F., & Hoi, S. C. H. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.247 · Cited by 18
“Extended from image-grounded tasks, e.g. (Antol et al., 2015), recently Jang et al. (2017); Lei et al. (2018) propose to use video as the grounding features.”
BackgroundMeasuring Compositional Consistency for Video Question Answering
Gandhi, M., Gul, M. O., Prakash, E., Grunde-McLaughlin, M., Krishna, R., & Agrawala, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00499 · Cited by 14
“Despite the popularity of video question answering as a benchmark task [10, 12, 15, 23, 39, 46, 47], questions in several prominent benchmarks rely on dialogue and plot summaries instead of a video’s visual contents [18, 23, 39, 49], focus on short video clips or only a handful of objects [29, 45], or suffer from bias…”
BackgroundMethodNExT-OOD: Overcoming Dual Multiple-Choice VQA Biases
Zhang, X., Zhang, F., & Xu, C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3269429 · Cited by 15
“…past five years, and conclude that numerous VQA datasets have been constructed in the IID setting [9], [13], [16], [19], [25], [26], [27], [28], [29], [30], [31], while a limited number of OOD benchmarks have been developed in open-ended VQA [32], [33], [34], [35], [36] and multiple-choice VQA [24].”
BackgroundCross-Modality Time-Variant Relation Learning for Generating Dynamic Scene Graphs
Wang, J., Huang, J., Zhang, C., & Deng, Z. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10161478 · Cited by 14
Semi-supervised Active Learning for Video Action Detection
Singh, A., Rana, A. J., Kumar, A. S., Vyas, S., & Rawat, Y. S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28292 · Cited by 14
Video Question Answering with Procedural Programs
Choudhury, R., Niinuma, K., Kitani, K., & Jeni, L. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72920-1_18 · Cited by 19
AutoAD-Zero: A Training-Free Framework for Zero-Shot Audio Description
Xie, J., Han, T., Bain, M., Nagrani, A., Varol, G., Xie, W., & Zisserman, A. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0908-6_5 · Cited by 16
Episodic Memory Reader: Learning What to Remember for Question Answering from Streaming Data
Han, M., Kang, M., Jung, H., & Hwang, S. J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1434 · Cited by 19
“We validate our proposed memory network on a large-scale QA task (Trivia QA) and video question answering task (TVQA) where the context is too large to fit into the external memory, against rule-based and an RL-based scheduling method without consideration of relative importance be-tween memories.”
InfluentialAdaptive Context Reading Network for Movie Scene Detection
Liu, D., Kamath, N., Bhattacharya, S., & Puri, R. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3042476 · Cited by 15
On the hidden treasure of dialog in video question answering
Engin, D., Schnitzler, F., Duong, N. Q. K., & Avrithis, Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00207 · Cited by 15
“vised temporal localization [20] biases system design towards two-stage localization→answering [21, 16]; fixed question structure focusing on temporal localization [20] often results in mere alignment of questions with subtitles and matching answers with the discovered context [14], providing little progress on the ma…”
BackgroundMethodInfluentialA Survey on VQA: Datasets and Approaches
Zou, Y., & Xie, Q. (2020) · International Conference on Information Technology and Computer Application (ITCA) · doi:10.1109/itca52113.2020.00069 · Cited by 20
“When image datasets have been widely explored, video datasets were also published, such as MovieQA[3], PororoQA[27], TVQA[28], etc.”
BackgroundInfluentialRelation-aware Hierarchical Attention Framework for Video Question Answering
Li, F., Bai, T., Cao, C., Liu, Z., Yan, C., & Wu, B. (2021) · International Conference on Multimedia Retrieval · doi:10.1145/3460426.3463635 · Cited by 14
“For example, in Figure 1, given a question as "What is behind Leonard when he is talking to Leslie about his relationship with Penny?", previous methods [19, 20] incorporate all information into modeling, including the relationship between "Leslie" and "Penny".”
BackgroundMethodInfluentialLinguistic issues behind visual question answering
Bernardi, R., & Pezzelle, S. (2021) · Language and Linguistics Compass · doi:10.1111/lnc3.12417 · Cited by 14
“The largest‐scale Video‐QA dataset currently available is TVQA (Lei et al., 2018, 2020), which contains questions about popular TV shows.”
BackgroundMethodOverview of the MedVidQA 2022 Shared Task on Medical Video Question-Answering
Gupta, D. K., & Demner‐Fushman, D. (2022) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2022.bionlp-1.25 · Cited by 14
“The majority of the existing studies (Lei et al., 2018; Xue et al., 2018; Li et al., 2020a; Chadha et al., 2020) on video question answering are focused on open-domain videos such as movies (Tapaswi et al., 2016), TV shows (Lei et al., 2018, 2020a), and games (Mun et al., 2017).”
BackgroundLearning To Locate Visual Answer In Video Corpus Using Question
Li, B., Weng, Y., Sun, B., & Shutao Li, A. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096391 · Cited by 14
“Different from traditional video questionanswering [4, 5], visual answer localization (VAL) is more efficient, which aims to provide a visual answer and has received extensive attention from researchers [6, 7, 8].”
BackgroundMultimodal Intent Discovery from Livestream Videos
Maharana, A., Tran, Q., Dernoncourt, F., Yoon, S., Bui, T., Chang, W. H., & Bansal, M. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.36 · Cited by 13
“Lei et al. (2018); Kim et al. (2017); Maharaj et al. (2017); Jang et al. (2017); Tapaswi et al. (2016) and Yi et al. (2020) introduce video-based question answering datasets created from various sources of creative visual content, i.e. movies, TV shows, GIFs etc. Lei et al. (2020b) and Lei et al.…”
BackgroundMethodSelf-citationModal-specific Pseudo Query Generation for Video Corpus Moment Retrieval
Jung, M., Choi, S., Kim, J. C., Kim, J.-H., & Zhang, B. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.530 · Cited by 15
How You Feelin’? Learning Emotions and Mental States in Movie Scenes
Srivastava, D., Singh, A. K., & Tapaswi, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00248 · Cited by 12
FunQA: Towards Surprising Video Comprehension
Xie, B., Zhang, S., Zhou, Z., Li, B., Zhang, Y., Hessel, J., Yang, J., & Liu, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73232-4_3 · Cited by 13
PACS: A Dataset for Physical Audiovisual CommonSense Reasoning
Yu, S., Wu, P., Liang, P. P., Salakhutdinov, R., & Morency, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_17 · Cited by 18
Representing Movie Characters in Dialogues
Azab, M., Kojima, N., Deng, J., & Mihalcea, R. F. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1010 · Cited by 13
“This model relies on bidirectional attention between context (represented by subtitles and/or visual content) and question answer pairs as queries to predict the correct answer (Lei et al., 2018).”
BackgroundInfluentialCLIP-Powered TASS: Target-Aware Single-Stream Network for Audio-Visual Question Answering
Jiang, Y., & Yin, J. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02289-z · Cited by 12
“Audio-Visual Question Answering (AVQA), unlike previous question answering (Lei et al., 2018; You et al., 2021; Chen et al., 2021b; Wang et al., 2021; Jiang et al., 2023a), which exploits the natural multimodal medium of video, is attracting increasing attention from researchers…”
Attention in Reasoning: Dataset, Analysis, and Modeling
Chen, S., Jiang, M. X., Yang, J., & Zhao, Q. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3114582 · Cited by 13
“Several visual reasoning datasets [13], [57], [58], [59], [60], [61], [62], [63], [64], [65], [66] have been collected in the form of VQA.”
MethodToward Long Form Audio-Visual Video Understanding
Hou, W., Li, G., Tian, Y., & Hu, D. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3672079 · Cited by 14
“They usually span several minutes, covering multiple related events in different categories.”
ViLA: Efficient Video-Language Alignment for Video Question Answering
Wang, X., Liang, J., Wang, C.-K., Deng, K., Lou, Y., Lin, M. C., & Yang, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73033-7_11 · Cited by 20
“This includes the NExT-QA [56], STAR [54], How2QA [31], TVQA [25] and VLEP [26].”
MethodInfluentialVideo Question Answering with Phrases via Semantic Roles
Sadhu, A., Chen, K., & Nevatia, R. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.196 · Cited by 16
Generalized pyramid co-attention with learnable aggregation net for video question answering
Gao, L., Chen, T., Li, X., Zeng, P., Zhao, L., & Li, Y.-F. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108145 · Cited by 12
ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videos
Zhou, Y., Zheng, L., Zhao, Z., Wu, F. Q., Jianping Fan, B., Ren, K., & Yu, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02221 · Cited by 13
“To comprehensively evaluate the capabilities of existing VideoQA models, several prominent benchmarks have been established [12, 22, 31, 35, 40, 44, 45].”
BackgroundBe Different to Be Better! A Benchmark to Leverage the Complementarity of Language and Vision
Pezzelle, S., Greco, C., Gandolfi, G., Gualdoni, E., & Bernardi, R. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.248 · Cited by 15
“, 2015); answering questions from videos and subtitles (Lei et al., 2018); having a dialogue on objects (De Vries et al.”
BackgroundA deep co-attentive hand-based video question answering framework using multi-view skeleton
Rastgoo, R., Kiani, K., & Escalera, S. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13573-w · Cited by 12
Qiu, Y., Nagasaki, Y., Hara, K., Kataoka, H., Suzuki, R., Iwata, K., & Satoh, Y. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00335 · Cited by 11
“Recently, a range of tasks that require detailed video semantics have been proposed, such as video question answering [4, 5, 6], video captioning [7, 8], spatio-temporal action localization [9, 10].”
BackgroundLearning Situation Hyper-Graphs for Video Question Answering
Khan, A. U., Kuehne, H., Wu, B., Chheu, K., Bousselham, W., Gan, C., Lobo, N. D. V., & Shah, M. A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01429 · Cited by 11
“Several video QA benchmarks are introduced to evaluate this task from varying perspectives entailing description [34, 51, 57, 61], temporal reasoning [17, 53, 64], causal structures [48, 53], visual-language comprehension [26, 46, 52], relational reasoning [46], and measuring social intelligence [58].”
BackgroundDynamic Multistep Reasoning based on Video Scene Graph for Video Question Answering
Mao, J., Jiang, W., Wang, X., Feng, Z., Lyu, Y., Liu, H., & Yong, Z. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.286 · Cited by 11
“Existing meth-ods for video QA conduct direct answering selection based on the multimodal encoding of questions and videos (Jang et al., 2017; Lei et al., 2018, 2020).”
MethodVideo-Text Pre-training with Learned Regions for Retrieval
Yan, R., Shou, M. Z., Ge, Y., Wang, J., Lin, X., Cai, G., & Tang, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25414 · Cited by 29
Transcript to Video: Efficient Clip Sequencing from Texts
Xiong, Y., Heilbron, F. C., & Lin, D. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548268 · Cited by 11
“Visual Semantic Embeddings (VSE) [17, 14] are widely adopted approaches for cross-modality tasks [59, 35, 2, 62, 32, 27] to learn joint visual-text representations.”
BackgroundMethodSemantic Association Network for Video Corpus Moment Retrieval
Kim, D., Yoon, S., Hong, J. W., & Yoo, C. D. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747523 · Cited by 11
“With pretrainig, we pretrain SAN under HowTo100M[7] and a large-scale TV dataset [2, 20, 6, 21] as [7].”
BackgroundMethodInfluentialDynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of ResourcesPreprint
Lin, H., Zhang, H., Ma, Y., He, T., Zhang, Z., Zha, S., & Li, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12043 · Cited by 18
MCQA: Multimodal Co-attention Based Network for Question AnsweringPreprint
Kumar, A., Mittal, T., & Manocha, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.12238 · Cited by 11
“Lei et al. (2018) use a context matching module and BiLSTM to model the inputs in a combined manner for multimodal question answering task.”
BackgroundMethodResultInfluentialCharacter Matters: Video Story Understanding with Character-Aware RelationsPreprint
Geng, S., Zhang, J., Fu, Z., Gao, P., Zhang, H., & de Melo, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.08646 · Cited by 11
“Several recent works (Anderson et al. 2018; Lei et al. 2018; Yao et al. 2018) have followed this direction and ex-a plored various approaches to incorporate grounded visual features for questions answering tasks.”
InfluentialMMVU: Measuring Expert-Level Multi-Discipline Video Understanding
Zhao, Y., Zhang, H., Xie, L., Hu, T., Gan, G., Long, Y., Hu, Z., Chen, W., Li, C., Xu, Z., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00793 · Cited by 80
“The rise of video-based foundation models [41, 66, 135, 165] has driven the development of new benchmarks that include diverse video comprehension tasks for more comprehensive evaluation [48, 76, 90, 92, 115, 156, 160].”
Explainable Video Entailment with Grounded Visual Evidence
Chen, J., & Golisano, Y. K. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00203 · Cited by 10
“Bridging the gap between computer vision and natural language processing is a rapid growing research area in various tasks including visual captioning [40, 34], VQA [20, 1, 33], and visual-textual retrieval [22, 23].”
BackgroundInfluentialLLQA - Lifelog Question Answering Dataset
Tran, L.-D., Ho, T. C., Pham, L. A., Nguyen, B. T., Gurrin, C. G., & Zhou, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-98358-1_18 · Cited by 11
“Because of the similarity to Video QA task, we implemented TVQA, the original TVQA [17] model, trained on TVQA dataset.”
BackgroundMethodInfluential2PESNet: Towards online processing of temporal action localization
Kim, Y. H., Nam, S., & Kim, S. J. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108871 · Cited by 10
Ли, Б., Weng, Y., Xia, F., Sun, B., & Shutao Li, A. (2022) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2022.bionlp-1.21 · Cited by 10
“In recent years, the gap between language and visual understanding has narrowed (Guo et al., 2016; Lu et al., 2019) due to the development of pre-trained models (Devlin et al., 2018) and the introduction of large-scale language-vision datasets (Lei et al., 2018, 2020a,b).”
BackgroundNewsKVQA: Knowledge-Aware News Video Question Answering
Gupta, P., & Gupta, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05981-0_1 · Cited by 11
Video Question Generation via Semantic Rich Cross-Modal Self-Attention Networks Learning
Wang, Y.-S., Su, H.-T., Chang, C.-H., Liu, Z., & Hsu, W. H. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053476 · Cited by 10
“Moreover, the *: Both authors contributed equally to this research. ability to ask can also aid the development in popular vision-language fields, such as Visual Dialogue task[2], Video Question Answering[3], and Embodied Question Answering[4] by decreasing the expensive questions annotation cost.”
BackgroundMethodInfluentialLAD: A Hybrid Deep Learning System for Benign Paroxysmal Positional Vertigo Disorders Diagnostic
Pham, T. X., Choi, J. W., Mina, R. J. L., Nguyen, T., Madjid, S. R., & Yoo, C. D. (2022) · IEEE Access · doi:10.1109/access.2022.3215625 · Cited by 12
“Deep learning has made a huge impact in computer vision that includes image classification [14], [15], [16], video question and answering [17], [18], [19], object recognition [20], and natural language processing [21].”
BackgroundDramaQA: Character-Centered Video Story Understanding with Hierarchical QAPreprint
Choi, S.-H., On, K.-W., Heo, Y., Seo, A., Jang, Y., Lee, M., & Zhang, B. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.03356 · Cited by 10
“TVQA aims to utilize both visual and language information from 60-90 second video clips.”
BackgroundMethodInfluentialExperts Collaboration Learning for Continual Multi-Modal Reasoning
Xu, L. H., & Liu, J. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3310336 · Cited by 9
“We also follow [1], [11], [42], and use a BiLSTM to extract textual input features (It ) of each sample.”
BackgroundMethodInfluentialSiamese Learning with Joint Alignment and Regression for Weakly-Supervised Video Paragraph Grounding
Tan, C., Lai, J., Zheng, W., & Hu, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01288 · Cited by 11
“…which has received increasing attention due to its wide range of real-world applications such as video retrieval [3, 17, 21, 23, 80, 94], video summarization [50, 55, 59, 60, 83, 99], action segmentation [19, 27, 35, 37, 45, 72], video question answering [30, 36, 38, 41, 82, 89], etc.”
Yu, T., Fu, K., Wang, S., Huang, Q., & Yu, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3475510 · Cited by 13
Goldfish: Vision-Language Understanding of Arbitrarily Long Videos
Ataallah, K., Shen, X., Abdelrahman, E., Sleiman, E., Zhuge, M., Ding, J., Zhu, D., Schmidhuber, J., & Elhoseiny, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73397-0_15 · Cited by 35
Live: Learning Video LLM with Streaming Speech Transcription at Scale
Chen, J., Zeng, Z., Lin, Y., Li, W., Ma, Z., & Shou, M. Z. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02708 · Cited by 13
“However, this paradigm does not align well with many real-time applications ( e.g ., AR glasses).”
Complementary spatiotemporal network for video question answering
Li, X., Wu, A., & Han, Y. (2021) · Multimedia Systems · doi:10.1007/s00530-021-00805-6 · Cited by 9
“The traditional works [13, 19, 32] that represent video are using the variants of the RNN which evolve LSTM [4], GRU [2], and Bi-LSTM [15].”
MethodGITAAR-GIT based Abnormal Activity Recognition on UCF Crime Dataset
Sirisha, U., & Chandana, B. S. (2023) · International Conference on Smart Systems and Inventive Technology (ICSSIT) · doi:10.1109/icssit55814.2023.10061116 · Cited by 9
“By us-ing this hybrid model GITAAR, the performance is evaluated on these 5 datasets MSVD [6],MSRVTT [7], YouCook2 [13] VATEX [17], and TVC [18].”
MethodGhosal, D., Majumder, N., Mihalcea, R. F., & Poria, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.691 · Cited by 10
Answer Mining from a Pool of Images: Towards Retrieval-Based Visual Question Answering
Penamakuri, A. S., Gupta, M., Gupta, M. D., & Mishra, A. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/146 · Cited by 17
Identifying Visible Actions in Lifestyle Vlogs
Ignat, O., Burdick, L., Deng, J., & Mihalcea, R. F. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1643 · Cited by 12
“Similar to previous research on multimodal methods (Lei et al., 2018; Xu et al., 2015; Wu et al., 2013; Jang et al., 2017), we also perform feature ablation to determine the role played by each modality in solving the task.”
Are we Asking the Right Questions in MovieQA?
Jasani, B. A., Girdhar, R., & Ramanan, D. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00235 · Cited by 10
CRAFT: A Benchmark for Causal Reasoning About Forces and inTeractions
Ates, T., Atesoglu, M. S., Yiğit, Ç., Kesen, İ., Kobaş, M., Erdem, E., Erdem, A., Göksun, T., & Yüret, D. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.205 · Cited by 14
mTVR: Multilingual Moment Retrieval in Videos
Lei, J., Berg, T. L., & Bansal, M. (2021) · doi:10.18653/v1/2021.acl-short.92 · Cited by 11
“The TVR (Lei et al., 2020) dataset contains 108,965 high-quality English queries from 21,793 videos from 6 long-running TV shows (provided by TVQA (Lei et al., 2018)).”
BackgroundSelf-citationDVD: A Diagnostic Dataset for Multi-step Reasoning in Video Grounded Dialogue
Le, H., Sankar, C., Moon, S., Beirami, A., Geramifard, A., & Kottur, S. (2021) · doi:10.18653/v1/2021.acl-long.439 · Cited by 12
“…, same as (4) but with access to visual information which is encoded by pretrained CNN models and temporal attention (TA) (Jang et al., 2017; Lei et al., 2018; Hori et al., 2019); (6) TF(C+Q+V) , which uses a Transformer-based architecture to encode visual and language information (Schwartz…”
BackgroundMethodInfluentialVideoNavQA: Bridging the Gap between Visual and Embodied Question AnsweringPreprint
Cangea, C., Belilovsky, E., Lió, P., & Courville, A. C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.04950 · Cited by 17
“Several datasets have been proposed that also consider video question answering in settings such as movies [19, 22, 33].”
BackgroundTAB-VCR: Tags and Attributes based Visual Commonsense Reasoning BaselinesPreprint
Lin, J., Jain, U., & Schwing, A. G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.14671 · Cited by 24
“Similarly related are question answering datasets built on videos [84, 64, 51, 52] and those based on visual embodied agents [31, 22].”
BackgroundSOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge
Wang, A., Wu, B., Chen, S., Chen, Z., Guan, H., Lee, W.-N., Li, L. E., & Gan, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01271 · Cited by 13
“…efforts have been directed towards evaluating multimodal commonsense reasoning, transitioning from purely language-based models to those incorporating visual comprehension, such as Visual Question Answering (VQA) [1, 15, 19, 28] or Visual Commonsense Reasoning (VCR) [10, 14, 17, 18, 47, 52].”
BackgroundRanking Distillation for Open-Ended Video Question Answering with Insufficient Labels
Liang, T., Tan, C., Xia, B., Zheng, W., & Hu, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01250 · Cited by 8
“There are mainly two types of tasks in this domain: the multiple-choice task MC-VQA [24, 26] offers several options, typically up to five, for each question and requires selecting the single correct one, while the open-ended task OE-VQA [20, 50, 52, 59] provides a global vocabulary of over 1K…”
BackgroundLi, G., Ding, X., Cheng, D., Li, J., Wang, N., & Gao, X. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3521758 · Cited by 8
“Compared to its fully-supervised counterpart, weakly supervised approaches simplify the data collection process and avoid annotation biases, finding vast applications in video action analysis [1], video question and answer [2], [3], and human-computer interaction systems [4], [5].”
Dang, R., Yuan, Y. F., Zhang, W., Xin, Y., Zhang, B., Li, L., Wang, L., Zeng, Q., Li, X., & Bing, L. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02290 · Cited by 17
“Robotot-ttt-tttttCentric”
Language Models are Causal Knowledge Extractors for Zero-shot Video Question Answering
Su, H.-T., Niu, Y., Lin, X., Hsu, W. H., & Chang, S. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00523 · Cited by 8
“Subsequent VidQA datasets [16, 17, 40] stepped forward to temporal relations of successive events.”
BackgroundRelational Space-Time Query in Long-Form Videos
Yang, X., Chu, F.-J., Feiszli, M., Goyal, R., Torresani, L., & Tran, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00619 · Cited by 8
“After that, various QA benchmarks have been proposed by collecting human-generated questions on either video clips extracted from TV series ( e.g ., TVQA [33]), GIFs ( e.g ., TGIF-QA [25] or internet videos ( e.g ., ActivityNet-QA [66], Next-QA [58], Just-Ask [61]).”
BackgroundMethodVideo Question Answering Using Clip-Guided Visual-Text Attention
Ye, S., Kong, W., Yao, C., Ren, J., & Jiang, X. (2023) · Proceedings - International Conference on Image Processing · doi:10.1109/icip49359.2023.10222286 · Cited by 10
“Benefiting from the additional general domain knowledge that can describe unseen answers in the target application domain, pre-trained models significantly boost the performance of downstream VideoQA tasks and achieve state-of-the-art results on many VideoQA datasets [6, 17, 18].”
BackgroundFinding the Right Moment: Human-Assisted Trailer Creation via Task Composition
Papalampidi, P., Keller, F., & Lapata, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3323030 · Cited by 13
“…understanding approaches have mainly focused on isolated video clips, and tasks such as the alignment between movie scenes and book chapters [8], question answering ( [9], [10]), video captioning for movie shots [11] or clips from TV episodes [12], and text-to-video retrieval ( [12], [13], [14]).”
EgoTV: Egocentric Task Verification from Natural Language Task Descriptions
Hazra, R., Chen, B., Rai, A., Kamra, N., & Desai, R. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01414 · Cited by 12
Knowledge Proxy Intervention for Deconfounded Video Question Answering
Li, J., Niu, L., & Zhang, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00261 · Cited by 8
“To this end, the VideoQA benchmarks start from the problem of description [66, 24, 29] and then build more challenging datasets towards temporal reasoning [6], physical reasoning [71], evidence reasoning [61], and commonsense reasoning [32].”
BackgroundMultimodal Dialogue State Tracking
Le, H., Chen, N. F., & Hoi, S. C. H. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.248 · Cited by 14
Face-to-Face Contrastive Learning for Social Intelligence Question-Answering
Wilf, A., Ma, M. Q., Liang, P. P., Zadeh, A., & Morency, L. (2023) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg57933.2023.10042612 · Cited by 9
GEM: A General Evaluation Benchmark for Multimodal Tasks
Su, L., Duan, N., Cui, E., Ji, L., Wu, C., Luo, H., Liu, Y., Zhong, M., Bharti, T., & Sacheti, A. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.229 · Cited by 15
Video-Text Pre-training with Learned RegionsPreprint
Yan, R., Shou, M. Z., Ge, Y., Wang, A. J., Lin, X., Cai, G., & Tang, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.01194 · Cited by 8
“It enables a series of downstream video-text tasks, such as video-text retrieval [32, 56, 73], video question answering [25,35,36], and video captioning [56,73,78].”
BackgroundGazeVQA: A Video Question Answering Dataset for Multiview Eye-Gaze Task-Oriented Collaborations
Ilaslan, M. F., Song, C., Chen, J., Gao, D., Lei, W., Xu, Q., Lim, J., & Shou, M. Z. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.648 · Cited by 8
“Moreover, there are several VQA studies, such as TutorialVQA (Colas et al., 2020), AGQA (Grunde-McLaughlin et al., 2021), TVQA (Lei et al., 2018), KnowIT VQA (Garcia et al., 2020), EgoVQA (Fan, 2019), Ego-TaskQA (Jia et al., 2022), GQA (Hudson and Man-10462 ning, 2019), DramaQA (Choi et al., 2021),…”
BackgroundDiffVsgg: Diffusion-Driven Online Video Scene Graph Generation
Chen, M., Li, L., Wang, W., & Yang, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02715 · Cited by 7
“Video Scene Graph Generation (VSGG) is receiving growing attention as it benefits a wide range of downstream tasks ( e.g ., video caption [34, 91], video retrieval [31, 50], and visual question answering [49, 87]).”
TEVL: Trilinear Encoder for Video-language Representation Learning
Man, X., Shao, J., Chen, F., Zhang, M., & Shen, H. T. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3585388 · Cited by 8
“TVQA [24] is a dataset collected under multiple-choice settings from TV videos.”
BackgroundMethodInfluentialReading Between the Lanes: Text VideoQA on the Road
Tom, G., Mathew, M., Garcia-Bordils, S., Karatzas, D., & Jawahar, C. V. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-41731-3_9 · Cited by 17
“Multiple-Choice datasets[36,24,43] consist of a pre-defined set of options for answers.”
BackgroundMethodAMEGO: Active Memory from Long EGOcentric Videos
Goletto, G., Nagarajan, T., Averta, G., & Damen, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72624-8_6 · Cited by 7
VQuAD: Video Question Answering Diagnostic Dataset
Gupta, V., Patro, B. N., Parihar, H., & Namboodiri, V. P. (2022) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw54805.2022.00034 · Cited by 7
“Several datasets have been proposed to do reasoning over video content, e.g., MovieFIB [23], VideoQA [48], LSMDC [31, 36], MovieQA [35], PororoQA [17], Mari-oQA [28], and TVQA [18].”
BackgroundYe, M., You, Q., & Ma, F. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00256 · Cited by 7
“We refer to image question answering [4, 5, 8, 18] and video question answering [12, 21, 35] tasks as traditional VQAs.”
BackgroundMethodZero-Shot and Few-Shot Video Question Answering with Multi-Modal Prompts
Engin, D., & Avrithis, Y. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00298 · Cited by 10
“Adapting pretrained vision-language models to down-stream tasks relies on fully supervised fi ne-tuning on VideoQA datasets in general [50, 53, 21, 29, 58, 33, 14].”
Ko, D., Lee, J. S., Choi, M., Chu, J., Park, J., & Kim, H. J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00288 · Cited by 8
“Typical VideoQA benchmarks take two formats: multiple-choice [3, 54] and openended [45, 48, 46, 47].”
BackgroundCo-Attentional Transformers for Story-Based Video Understanding
Bebensee, B., & Zhang, B. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413868 · Cited by 7
“One such dataset for video question answering is the TVQA dataset which is build around short 60 to 90 seconds long video clips and questions bridging vision and language clues [7].”
BackgroundSupport-Set Based Cross-Supervision for Video GroundingPreprint
Ding, X., Wang, N., Zhang, S., Cheng, D., Li, X., Huang, Z., Tang, M., & Gao, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.10576 · Cited by 7
“Some works use the contrastive loss to improve video-text representations based on HowTo100M for tasks such as video caption [53], video retrieval [2] and video question answering [24].”
MethodQuerying as Prompt: Parameter-Efficient Learning for Multimodal Language Model
Liang, T., Huang, J., Kong, M. C., Chen, L., & Zhu, Q. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02536 · Cited by 7
“For the other two tasks, we conducted experiments on the How2QA [20] and TVQA datasets [15] for VideoQA, as well as the CMU-MOSEI dataset [55] for MSA.”
Method
Metadata
- Authors
- Lei, J., Yu, L., Bansal, M., & Berg, T.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 1369-1379
- OpenAlex
- W2963541336
- Cited by
- 541
- References
- 49
- Citations by year
- 2026: 39 · 2025: 50 · 2024: 65 · 2023: 93 · 2022: 90 · 2021: 101 · 2020: 70 · 2019: 32 · 2018: 1
- FWCI
- 15.13
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1405822, 1562098, 1633295)
- Army Research Office (W911NF)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- video question answering, video localization, television series, end-to-end learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
