Cited by 293Multimodal Machine Learning ApplicationsHuman Pose and Action RecognitionVisual Attention and Saliency Detection

The DOI matches its Crossref record; 37 of the 38 references it cites were found in a registry or an index.

Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering

Open access

Fan, Chenyou, Zhang, Xiaofan, Zhang, Shu, Wang, Wensheng, Zhang, Chi, Huang, Heng·2019·2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

DOI 10.1109/cvpr.2019.00210

Open this paper in CoChat — free account Open PDF

Abstract

In this paper, we propose a novel end-to-end trainable Video Question Answering (VideoQA) framework with three major components: 1) a new heterogeneous memory which can effectively learn global context information from appearance and motion features; 2) a redesigned question memory which helps understand the complex semantics of question and highlights queried subjects; and 3) a new multimodal fusion layer which performs multi-step reasoning by attending to relevant visual and textual hints with self-updated attention. Our VideoQA model firstly generates the global context-aware visual and textual features respectively by interacting current inputs with memory contents. After that, it makes the attentional fusion of the multimodal visual and textual representations to infer the correct answer. Multiple cycles of reasoning can be made to iteratively refine attention weights of the multimodal data and improve the final representation of the QA pair. Experimental results demonstrate our approach achieves state-of-the-art performance on four VideoQA benchmark datasets.

Integrity signals

Registry

Crossref · 10.1109/cvpr.2019.00210

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Self-citation
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Untitled work
(2017) · Cited by 11,625
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00636 · Cited by 5,416
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487

Show all 38 references

Research works printed but not linked: 2

kembhavi, 2017, Textbook question answering for multimodal machine comprehension, CVPR
paszke, 2017, On Automatic Differentiation

Sorted by citations

Learning Memory-Guided Normality for Anomaly Detection
Park, H., Noh, J., & Ham, B. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01438 · Cited by 1,034
Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
Lei, J., Li, L., Zhou, L., Gan, Z., Berg, T. L., Bansal, M., & Liu, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00725 · Cited by 639
The multi-modal fusion in visual question answering: a review of attention mechanisms
Lu, S., Liu, M., Yin, L., Yin, Z., Liu, X., & Zheng, W. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1400 · Cited by 492
A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456
Cross-Modal Causal Relational Reasoning for Event-Level Visual Question Answering
Liu, Y., Li, G., & Lin, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3284038 · Cited by 373
NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions
Xiao, J., Shang, X., Yao, A., & Chua, T. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00965 · Cited by 397
Hierarchical Conditional Relation Networks for Video Question Answering
Le, T. M., Le, V., Venkatesh, S., & Tran, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00999 · Cited by 260
CLEVRER: CoLlision Events for Video REpresentation and ReasoningPreprint
Yi, K., Gan, C., Li, Y., Kohli, P., Wu, J., Torralba, A. B., & Tenenbaum, J. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.01442 · Cited by 477
MeGA-CDA: Memory Guided Attention for Category-Aware Unsupervised Domain Adaptive Object Detection
Vibashan, V. S., Gupta, V., Oza, P., Sindagi, V. A., & Patel, V. M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00449 · Cited by 228
Just Ask: Learning to Answer Questions from Millions of Narrated Videos
Yang, A., Miech, A., Šivic, J., Laptev, I., & Schmid, C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00171 · Cited by 232
Divide-and-Assemble: Learning Block-wise Memory for Unsupervised Anomaly Detection
Hou, J., Zhang, Y., Zhong, Q., Xie, D., Pu, S., & Zhou, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00867 · Cited by 213
Reasoning with Heterogeneous Graph Alignment for Video Question Answering
Pin, J., & Han, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6767 · Cited by 191
URVOS: Unified Referring Video Object Segmentation Network with a Large-Scale Benchmark
Seo, S., Lee, J., & Han, B. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58555-6_13 · Cited by 182
Align and Prompt: Video-and-Language Pre-training with Entity Prompts
Li, D., Li, J., Li, H., Niebles, J. C., & Hoi, S. C. H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00490 · Cited by 187
Location-Aware Graph Convolutional Networks for Video Question Answering
Huang, D. Y., Chen, P., Zeng, R., Du, Q., Tan, M., & Gan, C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6737 · Cited by 178
HVPR: Hybrid Voxel-Point Representation for Single-stage 3D Object Detection
Noh, J., Lee, S., & Ham, B. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01437 · Cited by 170
All in One: Exploring Unified Video-Language Pre-Training
Wang, J., Ge, Y., Yan, R., Ge, Y., Lin, K. Q., Tsutsui, S., Lin, X., Cai, G., Wu, J., Shan, Y., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00638 · Cited by 199
Improving Nowcasting of Convective Development by Incorporating Polarimetric Radar Variables Into a Deep‐Learning Model
Pan, X., Lu, Y., Zhao, K., Huang, H., Wang, M., & Chen, H. (2021) · Geophysical Research Letters · doi:10.1029/2021gl095302 · Cited by 158
Self-citation
Enriching Local and Global Contexts for Temporal Action Localization
Zhu, Z., Tang, W., Wang, L., Zheng, N., & Hua, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01326 · Cited by 140
Learning to Answer Questions in Dynamic Audio-Visual Scenarios
Li, G., Wei, Y., Tian, Y., Xu, C., Wen, J.-R., & Hu, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01852 · Cited by 153
Video as Conditional Graph Hierarchy for Multi-Granular Question Answering
Xiao, J., Yao, A., Liu, Z., Li, Y., Ji, W., & Chua, T. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20184 · Cited by 124
Divide and Conquer: Question-Guided Spatio-Temporal Contextual Attention for Video Question Answering
Jiang, J., Chen, Z., Lin, H., Zhao, X., & Gao, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6766 · Cited by 108
Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answering
Park, J., Lee, J., & Sohn, K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01527 · Cited by 105
Hierarchical Representation Network With Auxiliary Tasks for Video Captioning and Video Question Answering
Gao, L., Yu, L., Zeng, P., Song, J., Wang, M., & Shen, H. T. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3120867 · Cited by 106
Video Graph Transformer for Video Question Answering
Xiao, J., Zhou, P., Chua, T., & Yan, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_3 · Cited by 108
DualVGR: A Dual-Visual Graph Reasoning Unit for Video Question Answering
Wang, J., Bao, B., & Xu, C. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3097171 · Cited by 100
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token ModelingPreprint
Fu, T.-J., Li, L., Gan, Z., Lin, K., Wang, W. Y., Wang, L., & Liu, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12681 · Cited by 212
Self-citation
AGQA: A Benchmark for Compositional Spatio-Temporal Reasoning
Grunde-McLaughlin, M., Krishna, R., & Agrawala, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01113 · Cited by 89
SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Events
Xu, L., Huang, H., & Liu, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00975 · Cited by 99
Self-citation
Video Question Answering: Datasets, Algorithms and Challenges
Zhong, Y., Ji, W., Xiao, J., Li, Y., Deng, W., & Chua, T. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.432 · Cited by 133
Modality Shifting Attention Network for Multi-Modal Video Question Answering
Kim, J., Ma, M., Pham, T. X., Kim, K., & Yoo, C. D. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01012 · Cited by 84
EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone
Pramanick, S., Song, Y., Nag, S., Lin, K. Q., Shah, H., Shou, M. Z., Chellappa, R., & Zhang, P. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00487 · Cited by 106
Modality-specific and shared generative adversarial network for cross-modal retrieval
Wu, F., Jing, X., Wu, Z., Ji, Y., Dong, X., Luo, X., Huang, Q., & Wang, R. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107335 · Cited by 78
Temporal Memory Attention for Video Semantic Segmentation
Wang, H., Wang, W., & Liu, J. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506731 · Cited by 74
Self-citation
AVQA: A Dataset for Audio-Visual Question Answering on Videos
Yang, P., Wang, X. Y., Duan, X., Chen, H., Hou, R., Jin, C., & Zhu, W. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548291 · Cited by 64
Hierarchical Object-oriented Spatio-Temporal Reasoning for Video Question Answering
Dang, L. H., Le, T. M., Le, V., & Tran, T. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/88 · Cited by 73
Visual Anomaly Detection via Partition Memory Bank Module and Error Estimation
Xing, P., & Li, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3237562 · Cited by 67
Long-Term Video Question Answering via Multimodal Hierarchical Memory Attentive Networks
Yu, T., Yu, J., Zhou, Y., Huang, Q., & Tian, Q. C. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.2995959 · Cited by 63
From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answering
Li, J., Niu, L., & Zhang, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.02059 · Cited by 73
Attend What You Need: Motion-Appearance Synergistic Networks for Video Question Answering
Seo, A., Kang, G.-C., Park, J., & Zhang, B. (2021) · doi:10.18653/v1/2021.acl-long.481 · Cited by 67
Contrastive Video Question Answering via Video Graph Transformer
Xiao, J., Zhou, P., Yao, A., Li, Y., Hong, R., Yan, S., & Chua, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3292266 · Cited by 73
Look Before you Speak: Visually Contextualized Utterances
Seo, P. H., Nagrani, A., & Schmid, C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01660 · Cited by 57
HAIR: Hierarchical Visual-Semantic Relational Reasoning for Video Question Answering
Liu, F., Liu, J., Wang, N., & Lu, H. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00172 · Cited by 56
An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling
Fu, T.-J., Li, L., Gan, Z., Lin, K., Wang, W. Y., Wang, L., & Liu, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02193 · Cited by 70
Self-citation
Discovering the Real Association: Multimodal Causal Reasoning in Video Question Answering
Zang, C., Wang, H., Pei, M., & Liang, W. G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01824 · Cited by 60
Action-Centric Relation Transformer Network for Video Question Answering
Zhang, J., Shao, J., Cao, R., Gao, L., Xu, X., & Shen, H. T. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3048440 · Cited by 50
Cross-Modal Dynamic Networks for Video Moment Retrieval With Text Query
Wang, G., Xu, X., Shen, F., Lu, H., Ji, Y., & Shen, H. T. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3142420 · Cited by 49
Progressive Graph Attention Network for Video Question Answering
Peng, L., Yang, S., Bin, Y., & Wang, G. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475193 · Cited by 49
Clover: Towards A Unified Video-Language Alignment and Fusion Model
Huang, J., Li, Y., Feng, J., Wu, X., Sun, X., & Ji, R. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01427 · Cited by 59
DeCEMBERT: Learning from Noisy Instructional Videos via Dense Captions and Entropy Minimization
Tang, Z., Lei, J., & Bansal, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.193 · Cited by 48
Progressive Spatio-temporal Perception for Audio-Visual Question Answering
Li, G., Hou, W., & Hu, D. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612293 · Cited by 43
Adaptive Spatio-Temporal Graph Enhanced Vision-Language Representation for Video QA
Jin, W., Zhao, Z., Cao, X., Zhu, J., He, X., & Zhuang, Y. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3076556 · Cited by 41
Memorial GAN With Joint Semantic Optimization for Unpaired Image Captioning
Song, P., Guo, D., Zhou, J., Xu, M., & Wang, M. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2022.3175012 · Cited by 41
Cross-Attentional Spatio-Temporal Semantic Graph Networks for Video Question Answering
Liu, Y., Zhang, X., Huang, F., Zhang, B., & Li, Z. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3142526 · Cited by 42
Self-citation
End-to-End Video Question-Answer Generation With Generator-Pretester Network
Su, H.-T., Chang, C.-H., Shen, P.-W., Wang, Y.-S., Chang, Y.-L., Chang, Y.-C., Cheng, P.-J., & Hsu, W. H. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3051277 · Cited by 38
Self-trained prediction model and novel anomaly score mechanism for video anomaly detection
Guo, A., Guo, L., Zhang, R., Wang, Y., & Gao, S. (2022) · Image and Vision Computing · doi:10.1016/j.imavis.2022.104391 · Cited by 37
Video Question Answering With Prior Knowledge and Object-Sensitive Learning
Zeng, P., Zhang, H., Gao, L., Song, J., & Shen, H. T. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3205212 · Cited by 39
Adaptive Memory Broad Learning System for Unsupervised Time Series Anomaly Detection
Zhong, Z., Yu, Z., Fan, Z., Chen, C. L. P., & Yang, K. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3415621 · Cited by 40
(2.5+1)D Spatio-Temporal Scene Graphs for Video Question Answering
Cherian, A., Hori, C., Marks, T. K., & Roux, J. L. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19922 · Cited by 37
Contextual Attention Network for Emotional Video Captioning
Song, P., Guo, D., Cheng, J. S., & Wang, M. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3183402 · Cited by 36
Visual Commonsense-Aware Representation Network for Video Captioning
Zeng, P., Zhang, H., Gao, L., Li, X., Qian, J., & Shen, H. T. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3323491 · Cited by 39
Recurrent Relational Memory Network for Unsupervised Image Captioning
Guo, D., Wang, Y., Song, P., & Wang, M. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/128 · Cited by 38
Counting Bites and Recognizing Consumed Food from Videos for Passive Dietary Monitoring
Qiu, J., Lo, F. P.-W., Jiang, S., Tsai, Y.-Y., Sun, Y., & Lo, B. P. L. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3022815 · Cited by 36
Equivariant and Invariant Grounding for Video Question Answering
Li, Y., Wang, X., Xiao, J., & Chua, T. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548035 · Cited by 34
Locate Before Answering: Answer Guided Question Localization for Video Question Answering
Qian, T., Cui, R., Chen, J., Peng, P., Guo, X., & Jiang, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3323878 · Cited by 37
MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Models
Ko, D., Choi, J., Choi, H. K., On, K.-W., Roh, B., & Kim, H. J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01925 · Cited by 38
Visual Causal Scene Refinement for Video Question Answering
Wei, Y.-S., Liu, Y., Yan, H., Li, G., & Lin, L. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611873 · Cited by 40
BiST: Bi-directional Spatio-Temporal Reasoning for Video-Grounded Dialogues
Le, H., Sahoo, D., Chen, N. F., & Hoi, S. C. H. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.145 · Cited by 33
Multilevel Hierarchical Network with Multiscale Sampling for Video Question Answering
Peng, M., Wang, C., Gao, Y., Shi, Y., & Zhou, X.-D. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/178 · Cited by 31
Learning from Inside: Self-driven Siamese Sampling and Reasoning for Video Question Answering
Yu, W., Zheng, H., Li, M., Ji, L., Wu, L., Xiao, N., & Duan, N. (2021) · neural information processing systems · Cited by 30
EgoVQA - An Egocentric Video Question Answering Benchmark Dataset
Fan, C. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00536 · Cited by 30
Self-citation
Long video question answering: A Matching-guided Attention Model
Wang, W., Huang, Y., & Wang, L. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107248 · Cited by 29
Self-citation
Fake News Detection via Multi-Modal Topic Memory Network
Ying, L., Yu, H. L., Wang, J., Ji, Y., & Qian, S. (2021) · IEEE Access · doi:10.1109/access.2021.3113981 · Cited by 29
iPerceive: Applying Common-Sense Reasoning to Multi-Modal Dense Video Captioning and Video Question AnsweringPreprint
Chadha, A. R., Arora, G., & Kaloty, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.07735 · Cited by 36
IntentQA: Context-aware Video Intent Reasoning
Li, J., Wei, P., Han, W., & Fan, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01099 · Cited by 35
Visual Question Answering on Image Sets
Bansal, A., Zhang, Y., & Chellappa, R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_4 · Cited by 29
Hierarchical Conditional Relation Networks for Multimodal Video Question Answering
Le, T. M., Le, V., Venkatesh, S., & Tran, T. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01514-3 · Cited by 27
Learning to Answer Visual Questions From Web Videos
Yang, A., Miech, A., Šivic, J., Laptev, I., & Schmid, C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3173208 · Cited by 36
Video Question Answering With Semantic Disentanglement and Reasoning
Liu, J., Wang, G., Xie, J., Zhou, F., & Xu, H. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3317447 · Cited by 29
Weakly Supervised Segmentation of Small Buildings with Point Labels
Lee, J.-H., Kim, C., & Sull, S. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00731 · Cited by 26
Geometry Sensitive Cross-Modal Reasoning for Composed Query Based Image Retrieval
Zhang, F., Xu, M., & Xu, C. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3138302 · Cited by 26
Question-Aware Global-Local Video Understanding Network for Audio-Visual Question Answering
Chen, Z., Wang, L., Wang, P., & Gao, P. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3318220 · Cited by 26
Discovering Spatio-Temporal Rationales for Video Question Answering
Li, Y., Xiao, J., Feng, C., Wang, X., & Chua, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01275 · Cited by 35
Embedded Heterogeneous Attention Transformer for Cross-Lingual Image Captioning
Song, Z., Hu, Z., Zhou, Y., Zhao, Y., Hong, R., & Wang, M. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3384678 · Cited by 25
Multi-Scale Progressive Attention Network for Video Question Answering
Guo, Z., Zhao, J., Jiao, L., Liu, X., & Li, L. (2021) · doi:10.18653/v1/2021.acl-short.122 · Cited by 25
Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learning
Amrani, E., Ben‐Ari, R., Rotman, D., & Bronstein, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16822 · Cited by 35
COCA: COllaborative CAusal Regularization for Audio-Visual Question Answering
Lao, M., Pu, N., Liu, Y., He, K., Bakker, E. M., & Lew, M. S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i11.26527 · Cited by 24
3-D Relation Network for visual relation recognition in videos
Cao, Q., Huang, H., Shang, X., Wang, B., & Chua, T. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.12.029 · Cited by 23
Self-citation
LiVLR: A Lightweight Visual-Linguistic Reasoning Framework for Video Question Answering
Jiang, J., Liu, Z., & Zheng, N. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3185900 · Cited by 26
ERM: Energy-Based Refined-Attention Mechanism for Video Question Answering
Zhang, F., Wang, R., Zhou, F., & Luo, Y. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3212463 · Cited by 23
Multi-Granularity Interaction and Integration Network for Video Question Answering
Wang, Y., Liu, M., Wu, J., & Nie, L. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3278492 · Cited by 25
Video Question Answering: a Survey of Models and Datasets
Sun, G., Liang, L., Li, T., Yu, B., Wu, M., & Zhang, B. (2021) · Mobile Networks and Applications · doi:10.1007/s11036-020-01730-0 · Cited by 25
Dynamic Spatio-Temporal Graph Reasoning for VideoQA With Self-Supervised Event Recognition
Nie, J., Wang, X. Y., Hou, R., Li, G., Chen, H., & Zhu, W. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3411448 · Cited by 27
Robust video question answering via contrastive cross-modality representation learning
Yang, X., Zeng, J., Guo, D., Wang, S., Dong, J., & Wang, M. (2024) · Science China Information Sciences · doi:10.1007/s11432-023-4084-6 · Cited by 25
Document Collection Visual Question Answering
Tito, R., Karatzas, D., & Valveny, E. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86331-9_50 · Cited by 24
Video Question-Answering Techniques, Benchmark Datasets and Evaluation Metrics Leveraging Video Captioning: A Comprehensive Survey
Khurana, K., & Deshpande, U. A. (2021) · IEEE Access · doi:10.1109/access.2021.3058248 · Cited by 26
Stereoscopic Vision Recalling Memory for Monocular 3D Object Detection
Kim, J. U., Kim, H.-I., & Ro, Y. M. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3274479 · Cited by 21
Reducing Vision-Answer Biases for Multiple-Choice VQA
Zhang, X., Zhang, F., & Xu, C. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3302162 · Cited by 21
Self-citation
Tem-adapter: Adapting Image-Text Pretraining for Video Question Answer
Chen, G., Liu, X., Wang, G., Zhang, K., Torr, P. H. S., Zhang, X.-P. S., & Tang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01282 · Cited by 28
Self-citation
Boosting Audio Visual Question Answering via Key Semantic-Aware Cues
Li, G., Du, H., & Hu, D. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680803 · Cited by 20

Authors: Fan, C., Zhang, X., Zhang, S., Wang, W., Zhang, C., & Huang, H.
Affiliations: Jingdong (China) (CN)
University of Pittsburgh (US)
ORCID: Fan, Chenyou
Zhang, Xiaofan
Zhang, Shu
Wang, Wensheng
Zhang, Chi
Huang, Heng
Venue: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher: IEEE
Published: 2019-06
Pages: 1999-2007
ISSN: 1063-6919 (ISSN-L)
DOI: 10.1109/cvpr.2019.00210
OpenAlex: W2954199749
Cited by: 293
References: 38
Citations by year: 2026: 16 · 2025: 23 · 2024: 49 · 2023: 65 · 2022: 46 · 2021: 59 · 2020: 31 · 2019: 4
FWCI: 11.42
Type: conference paper
Language: en
Open access: green
arxiv.org
Licence: doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
Keywords: video question answering, heterogeneous memory, multimodal attention, multimodal fusion, global context modeling, appearance and motion features, multi-step reasoning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks