Cited by 293Multimodal Machine Learning ApplicationsHuman Pose and Action RecognitionVisual Attention and Saliency Detection

The DOI matches its Crossref record; 37 of the 38 references it cites were found in a registry or an index.

Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering

Open access

Fan, Chenyou, Zhang, Xiaofan, Zhang, Shu, Wang, Wensheng, Zhang, Chi, Huang, Heng·2019·2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

DOI 10.1109/cvpr.2019.00210

Open this paper in CoChat — free account Open PDF

Abstract

In this paper, we propose a novel end-to-end trainable Video Question Answering (VideoQA) framework with three major components: 1) a new heterogeneous memory which can effectively learn global context information from appearance and motion features; 2) a redesigned question memory which helps understand the complex semantics of question and highlights queried subjects; and 3) a new multimodal fusion layer which performs multi-step reasoning by attending to relevant visual and textual hints with self-updated attention. Our VideoQA model firstly generates the global context-aware visual and textual features respectively by interacting current inputs with memory contents. After that, it makes the attentional fusion of the multimodal visual and textual representations to infer the correct answer. Multiple cycles of reasoning can be made to iteratively refine attention weights of the multimodal data and improve the final representation of the QA pair. Experimental results demonstrate our approach achieves state-of-the-art performance on four VideoQA benchmark datasets.

Integrity signals

Registry

Crossref · 10.1109/cvpr.2019.00210

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Self-citation
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Untitled work
(2017) · Cited by 11,625
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
Anderson, P., He, X., Buehler, C., Teney, D., Johnson, M., Gould, S. J., & Zhang, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00636 · Cited by 5,416
VQA: Visual Question Answering
Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487

Flagged first, then by citations

Show all 38 references

1 of the 38 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

kembhavi, 2017, Textbook question answering for multimodal machine comprehension, CVPR
paszke, 2017, On Automatic Differentiation

Sorted by citations

Items 101–200 of 293

CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios
Ye, Q., Yu, Z., Shao, R., Xie, X., Torr, P. H. S., & Cao, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72684-2_9 · Cited by 37
Video Question Answering Using Language-Guided Deep Compressed-Domain Video Feature
Kim, N., Ha, S. J., & Kang, J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00173 · Cited by 21
Defending against adversarial attacks using spherical sampling-based variational auto-encoder
Yin, S., Zhang, X., & Zuo, L.-Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2021.12.080 · Cited by 19
Self-citation
Feature Augmented Memory with Global Attention Network for VideoQA
Cai, J., Yuan, C., Shi, C., Li, L., Cheng, Y., & Shan, Y. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/139 · Cited by 29
Multi-Granularity Relational Attention Network for Audio-Visual Question Answering
Li, L., Jin, T., Wang, L., Jiang, H., Pan, W., Wang, J., Xiao, S., Xia, Y., Jiang, W., & Zhao, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3264524 · Cited by 18
Video Question Answering with Iterative Video-Text Co-tokenization
Piergiovanni, A., Morton, K., Kuo, W., Ryoo, M. S., & Angelova, A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20059-5_5 · Cited by 18
CAT+: Investigating and Enhancing Audio-Visual Understanding in Large Language Models
Ye, Q., Yu, Z., Shao, R., Cui, Y., Kang, X., Liu, X., Torr, P. H. S., & Cao, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3582389 · Cited by 21
Instance-sequence reasoning for video question answering
Liu, R., & Han, Y. (2022) · Frontiers of Computer Science · doi:10.1007/s11704-021-1248-1 · Cited by 17
MS-DETR: Natural Language Video Localization with Sampling Moment-Moment Interaction
Wang, J., Sun, A., Zhang, H., & Li, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.77 · Cited by 21
Exploiting Structural Consistency of Chest Anatomy for Unsupervised Anomaly Detection in Radiography Images
Xiang, T., Zhang, Y., Lu, Y., Yuille, A., Zhang, C., Cai, W., & Zhou, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3382009 · Cited by 17
Self-citation
Recent Advances in Video Question Answering: A Review of Datasets and Methods
Patel, D., Parikh, R., & Shastri, Y. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68790-8_27 · Cited by 20
Dynamic Spatio-Temporal Modular Network for Video Question Answering
Qian, Z., Wang, X. Y., Duan, X., Chen, H., & Zhu, W. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548061 · Cited by 16
CFMMC-Align: Coarse-Fine Multi-Modal Contrastive Alignment Network for Traffic Event Video Question Answering
Guo, K., Tian, D., Hu, Y., Lin, C., Sun, Y., Zhou, J., Duan, X., Gao, J., & Yin, B. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3409453 · Cited by 20
Question-Guided Erasing-Based Spatiotemporal Attention Learning for Video Question Answering
Liu, F., Liu, J., Hong, R., & Lu, H. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3105280 · Cited by 17
Cross-modal Causal Relation Alignment for Video Question Grounding
Chen, W.-X., Liu, Y., Chen, B., Su, J.-D., Zheng, Y.-S., & Lin, L. (2025) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52734.2025.02243 · Cited by 30
Pairwise VLAD Interaction Network for Video Question Answering
Wang, H., Guo, D., Hua, X., & Wang, M. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475620 · Cited by 15
Multi-Granularity Contrastive Cross-Modal Collaborative Generation for End-to-End Long-Term Video Question Answering
Yu, T., Fu, K., Zhang, J., Huang, Q., & Yu, J. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3390984 · Cited by 19
Unsupervised Spectrum Anomaly Detection With Distillation and Memory Enhanced Autoencoders
Qi, P., Jiang, T., Xu, J., He, J., Zheng, S., & Li, Z. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3424837 · Cited by 15
Heterogeneous memory enhanced graph reasoning network for cross-modal retrieval
Ji, Z., Chen, K., He, Y., Pang, Y., & Li, X. (2022) · Science China Information Sciences · doi:10.1007/s11432-021-3367-y · Cited by 15
ContextLoc++: A Unified Context Model for Temporal Action Localization
Zhu, Z., Wang, L., Tang, W., Zheng, N., & Hua, G. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3237597 · Cited by 15
Learning visual language models for video understanding
Yang, A. (2023) · theses.fr (ABES) · doi:10.70675/802872a7z8092z4556za1fdz2c7af45f0e72 · Cited by 26
About Time: Advances, Challenges, and Outlooks of Action Understanding
Stergiou, A., & Poppe, R. W. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02478-4 · Cited by 17
VGNMN: Video-grounded Neural Module Networks for Video-Grounded Dialogue Systems
Le, H., Chen, N. F., & Hoi, S. C. H. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.247 · Cited by 18
Measuring Compositional Consistency for Video Question Answering
Gandhi, M., Gul, M. O., Prakash, E., Grunde-McLaughlin, M., Krishna, R., & Agrawala, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00499 · Cited by 14
A multi-scale self-supervised hypergraph contrastive learning framework for video question answering
Wang, Z., Wu, B., Ota, K., Dong, M., & Li, H. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.08.057 · Cited by 14
CAD - Contextual Multi-modal Alignment for Dynamic AVQA
Nadeem, A., Hilton, A., Dawes, R., Thomas, G., & Mustafa, A. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00709 · Cited by 16
Video Question Answering with Procedural Programs
Choudhury, R., Niinuma, K., Kitani, K., & Jeni, L. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72920-1_18 · Cited by 19
Adaptive Context Reading Network for Movie Scene Detection
Liu, D., Kamath, N., Bhattacharya, S., & Puri, R. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3042476 · Cited by 15
On the hidden treasure of dialog in video question answering
Engin, D., Schnitzler, F., Duong, N. Q. K., & Avrithis, Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00207 · Cited by 15
Dynamic self-attention with vision synchronization networks for video question answering
Liu, Y., Zhang, X., Huang, F., Shen, S., Tian, P., Li, L., & Li, Z. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108959 · Cited by 13
Self-citation
Transformer-Empowered Invariant Grounding for Video Question Answering
Li, Y., Wang, X., Xiao, J., Ji, W., & Chua, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3303451 · Cited by 17
Enhancing video anomaly detection with learnable memory network: A new approach to memory-based auto-encoders
Wang, Z., Gu, X., Gu, X., & Hu, J. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.103946 · Cited by 14
TG-VQA: Ternary Game of Video Question Answering
Li, H., Jin, P., Cheng, Z., Zhang, S., Chen, K., Wang, Z., Liu, C., & Chen, J. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/116 · Cited by 13
Self-citation
Amalgamation of protein sequence, structure and textual information for improving protein-protein interaction identification
Dutta, P., & Saha, S. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.570 · Cited by 15
CLIP-Powered TASS: Target-Aware Single-Stream Network for Audio-Visual Question Answering
Jiang, Y., & Yin, J. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02289-z · Cited by 12
LiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling
Chen, D., Tao, C., Hou, L., Shang, L., Jiang, X., & Liu, Q. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.545 · Cited by 18
ViLA: Efficient Video-Language Alignment for Video Question Answering
Wang, X., Liang, J., Wang, C.-K., Deng, K., Lou, Y., Lin, M. C., & Yang, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73033-7_11 · Cited by 20
Neural Reasoning, Fast and Slow, for Video Question Answering
Le, T. M., Le, V., Venkatesh, S., & Tran, T. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207580 · Cited by 19
Generalized pyramid co-attention with learnable aggregation net for video question answering
Gao, L., Chen, T., Li, X., Zeng, P., Zhao, L., & Li, Y.-F. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108145 · Cited by 12
ReGR: Relation-aware graph reasoning framework for video question answering
Wang, Z., Li, F., Ota, K., Dong, M., & Wu, B. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103375 · Cited by 12
A deep co-attentive hand-based video question answering framework using multi-view skeleton
Rastgoo, R., Kiani, K., & Escalera, S. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13573-w · Cited by 12
Flexible Frame Selection for Efficient Video Reasoning
Buch, S., Nagrani, A., Arnab, A., & Schmid, C. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02707 · Cited by 11
Learning Situation Hyper-Graphs for Video Question Answering
Khan, A. U., Kuehne, H., Wu, B., Chheu, K., Bousselham, W., Gan, C., Lobo, N. D. V., & Shah, M. A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01429 · Cited by 11
Event Graph Guided Compositional Spatial–Temporal Reasoning for Video Question Answering
Bai, Z., Wang, R., Gao, D., & Chen, X. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3358726 · Cited by 11
Dynamic Multistep Reasoning based on Video Scene Graph for Video Question Answering
Mao, J., Jiang, W., Wang, X., Feng, Z., Lyu, Y., Liu, H., & Yong, Z. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.286 · Cited by 11
Advancing Video Question Answering with a Multi-modal and Multi-layer Question Enhancement Network
Liu, M., Zhang, F., Luo, X., Liu, F., Wei, Y., & Nie, L. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612239 · Cited by 11
A Neural Turing Machine-based approach to Remaining Useful Life Estimation
Falcon, A., D’Agostino, G., Serra, G., Brajnik, G., & Tasso, C. (2020) · IEEE Conference on Prognostics and Health Management · doi:10.1109/icphm49022.2020.9187043 · Cited by 10
VSRN: Visual-Semantic Relation Network for Video Visual Relation Inference
Cao, Q., & Huang, H. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3068214 · Cited by 10
Self-citation
NewsKVQA: Knowledge-Aware News Video Question Answering
Gupta, P., & Gupta, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05981-0_1 · Cited by 11
Efficient End-to-End Video Question Answering with Pyramidal Multimodal Transformer
Peng, M., Wang, C., Shi, Y., & Zhou, X.-D. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i2.25296 · Cited by 14
Experts Collaboration Learning for Continual Multi-Modal Reasoning
Xu, L. H., & Liu, J. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3310336 · Cited by 9
Prompting Video-Language Foundation Models With Domain-Specific Fine-Grained Heuristics for Video Question Answering
Yu, T., Fu, K., Wang, S., Huang, Q., & Yu, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3475510 · Cited by 13
Complementary spatiotemporal network for video question answering
Li, X., Wu, A., & Han, Y. (2021) · Multimedia Systems · doi:10.1007/s00530-021-00805-6 · Cited by 9
Simple Baselines for Interactive Video Retrieval with Questions and Answers
Liang, K., & Albanie, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01018 · Cited by 12
Multimodal Data Fusion based on the Global Workspace Theory
Bao, C., Fountas, Z., Olugbade, T., & Bianchi‐Berthouze, N. (2020) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3382507.3418849 · Cited by 11
Unsupervised domain adaptation via reliable pseudolabeling based memory module and dynamic distance threshold learning
Wei, G., Li, X., Huang, L., Nie, J., & Wei, Z. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110667 · Cited by 9
Multimodal Data Fusion based on the Global Workspace Theory
Bao, C., Fountas, Z., Olugbade, T., & Bianchi-Berthouze, N. (2020) · arXiv · doi:10.48550/arxiv.2001.09485 · Cited by 9
SHMamba: Structured Hyperbolic State Space Model for Audio-Visual Question Answering
Yang, Z., Li, W., & Cheng, G. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3597461 · Cited by 8
Analyzing Effect of Multi-Modality in Predicting Protein-Protein Interactions
Jha, K., & Saha, S. (2022) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2022.3157531 · Cited by 8
Video Question Answering Using Clip-Guided Visual-Text Attention
Ye, S., Kong, W., Yao, C., Ren, J., & Jiang, X. (2023) · Proceedings - International Conference on Image Processing · doi:10.1109/icip49359.2023.10222286 · Cited by 10
Knowledge Proxy Intervention for Deconfounded Video Question Answering
Li, J., Niu, L., & Zhang, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00261 · Cited by 8
Object-Centric Representation Learning for Video Question Answering
Dang, L. H., Le, T. M., Le, V., & Tran, T. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533961 · Cited by 8
Knowledge-Constrained Answer Generation for Open-Ended Video Question Answering
Jin, Y., Niu, G., Xiao, X., Zhang, J., Peng, X., & Yu, J. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i7.25983 · Cited by 8
Temporal Memory Attention for Video Semantic SegmentationPreprint
Wang, H., Wang, N., & Liu, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.08643 · Cited by 8
DiffVsgg: Diffusion-Driven Online Video Scene Graph Generation
Chen, M., Li, L., Wang, W., & Yang, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02715 · Cited by 7
Self-citation
A Dual-Stream architecture based on Neural Turing Machine and Attention for the Remaining Useful Life Estimation problem
Falcon, A., D’Agostino, G., Serra, G., Brajnik, G., & Tasso, C. (2020) · PHM Society European Conference · doi:10.36001/phme.2020.v5i1.1227 · Cited by 7
Attention Guided Relation Detection Approach for Video Visual Relation Detection
Cao, Q., & Huang, H. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3109430 · Cited by 7
Self-citation
Co-Attentional Transformers for Story-Based Video Understanding
Bebensee, B., & Zhang, B. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413868 · Cited by 7
HN-MUM: heterogeneous video anomaly detection network with multi-united-memory module
Li, H., Wang, Y., Chen, M., & Li, J. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15154-x · Cited by 7
STAIR: Spatial-Temporal Reasoning with Auditable Intermediate Results for Video Question Answering
Wang, Y., Wang, Y., Chen, K., & Zhao, D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29890 · Cited by 7
TR-Adapter: Parameter-Efficient Transfer Learning for Video Question Answering
Wang, Y., Liu, M., Song, X., & Nie, L. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3521708 · Cited by 11
Guiding Audio-Visual Question Answering with Collective Question Reasoning
Pei, B., Huang, Y., Chen, G., Xu, J., Wang, L., Wang, L., Lü, T., Qiao, Y., & Wu, F. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02510-7 · Cited by 6
PSAM: Parameter-Free Spatiotemporal Attention Mechanism for Video Question Answering
Zhang, F., Wang, R., Zhou, F., Luo, Y., & Li, J. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3333192 · Cited by 6
Exploring coherence from heterogeneous representations for OCR image captioning
Zhang, Y., Song, Z., & Hu, Z. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01470-1 · Cited by 6
Dual Hierarchical Temporal Convolutional Network with QA-Aware Dynamic Normalization for Video Story Question Answering
Liu, F., Liu, J., Zhu, X., Hong, R., & Lu, H. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413649 · Cited by 6
Video Visual Relation Detection via 3D Convolutional Neural Network
Qu, M., Cui, J., Su, T., Deng, G., & Shao, W. (2022) · IEEE Access · doi:10.1109/access.2022.3154423 · Cited by 6
Memory-Guided Image De-Raining Using Time-Lapse Data
Cho, J., Kim, S., & Sohn, K. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3180561 · Cited by 6
Confidence-Based Event-Centric Online Video Question Answering on a Newly Constructed ATBS Dataset
Kong, W., Ye, S., Yao, C., & Ren, J. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095044 · Cited by 6
Rethinking Multi-Modal Alignment in Multi-Choice VideoQA from Feature and Sample Perspectives
Xiao, S., Chen, L., Gao, K., Wang, Z., Yang, Y., Zhang, Z., & Xiao, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.561 · Cited by 12
Sentence Attention Blocks for Answer Grounding
Khoshsirat, S., & Kambhamettu, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00559 · Cited by 6
Learning to Reason with Relational Video Representation for Question Answering.
Le, T. M., Le, V., Venkatesh, S., & Tran, T. (2019) · Cited by 6
Anomaly Detection based on Pose Estimation and GRU-FFN
Fan, B., Li, P., Shu, J., & Wang, Z. (2021) · IEEE Sustainable Power and Energy Conference (iSPEC) · doi:10.1109/ispec53008.2021.9736129 · Cited by 6
Temporal Pyramid Transformer with Multimodal Interaction for Video Question AnsweringPreprint
Peng, M., Wang, C., Gao, Y., Shi, Y., & Zhou, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.04735 · Cited by 6
Video Question Answering: A survey of the state-of-the-art
Jeshmol, P. J., & Kovoor, B. C. (2024) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2024.104320 · Cited by 9
FashionNTM: Multi-turn Fashion Image Retrieval via Cascaded Memory
Pal, A., Wadhwa, S., Jaiswal, A., Zhang, X., Wu, Y., Chada, R., Natarajan, P., & Christensen, H. I. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01040 · Cited by 7
Self-citation
Spatio-Temporal Graph Convolution Transformer for Video Question Answering
Tang, J., Hu, J., Huang, W., Shen, S., Pan, J., Wang, D.-M., & Ding, Y. (2024) · IEEE Access · doi:10.1109/access.2024.3445636 · Cited by 5
Language-aware Visual Semantic Distillation for Video Question Answering
Zou, B., Chao, Y., Qiao, Y., Quan, C., & Zhao, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02560 · Cited by 9
Modality-specific adaptive scaling and attention network for cross-modal retrieval
Ke, X., Chen, B., Cai, Y., Liu, H., Guo, W., & Chen, W. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128664 · Cited by 5
Video Language Co-Attention with Multimodal Fast-Learning Feature Fusion for VideoQA
Abdessaied, A., Sood, E., & Bulling, A. (2022) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2022.repl4nlp-1.15 · Cited by 5
Video question answering supported by a multi-task learning objective
Falcon, A., Serra, G., & Lanz, O. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14333-0 · Cited by 5
Few-Shot Multi-Agent Perception With Ranking-Based Feature Learning
Fan, C., Hu, J., & Huang, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3285755 · Cited by 5
Self-citation
Heterogeneous Interactive Graph Network for Audio–Visual Question Answering
Zhao, Y., Xi, W., Bai, G., Liu, X., & Zhao, J. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112165 · Cited by 5
Learning Question-Guided Video Representation for Multi-Turn Video Question Answering
Chao, G.-L., Rastogi, A., Yavuz, S., Hakkani-Tür, D. Z., Chen, J., & Lane, I. R. (2019) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w19-5926 · Cited by 6
Prediction of Protein-Protein Interactions using Deep Multi-Modal Representations
Jha, K., Saha, S., & Saha, S. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533478 · Cited by 6
Dense but Efficient VideoQA for Intricate Compositional Reasoning
Lee, J., Kang, W., & Kim, E. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00117 · Cited by 5
Language-Guided Visual Aggregation Network for Video Question Answering
Xiao, L., Wang, D., Wang, Q., Wan, B., An, L., & He, L. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3613909 · Cited by 5
MISSRAG: Addressing the Missing Modality Challenge in Multimodal Large Language Models
Pipoli, V., Saporita, A., Bolelli, F., Cornia, M., Baraldi, L., Grana, C., Cucchiara, R., & Ficarra, E. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00308 · Cited by 8
Cross-Modal-Aware Representation Learning with Syntactic Hypergraph Convolutional Network for VideoQA
Xu, Z., Zhong, W., Su, Q., & Zhang, F. (2023) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme55011.2023.00073 · Cited by 4
Cascade transformers with dynamic attention for video question answering
Jiang, Y., Yan, T., Yao, M., Wang, H., & Liu, W. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.103983 · Cited by 4
Video Entailment via Reaching a Structure-Aware Cross-modal Consensus
Yao, X., Gao, J., Chen, M., & Xu, C. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612345 · Cited by 4

Authors: Fan, C., Zhang, X., Zhang, S., Wang, W., Zhang, C., & Huang, H.
Affiliations: Jingdong (China) (CN)
University of Pittsburgh (US)
ORCID: Fan, Chenyou
Zhang, Xiaofan
Zhang, Shu
Wang, Wensheng
Zhang, Chi
Huang, Heng
Venue: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher: IEEE
Published: 2019-06
Pages: 1999-2007
ISSN: 1063-6919 (ISSN-L)
DOI: 10.1109/cvpr.2019.00210
OpenAlex: W2954199749
Cited by: 293
References: 38
Citations by year: 2026: 16 · 2025: 23 · 2024: 49 · 2023: 65 · 2022: 46 · 2021: 59 · 2020: 31 · 2019: 4
FWCI: 11.42
Type: conference paper
Language: en
Open access: green
arxiv.org
Licence: doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
Keywords: video question answering, heterogeneous memory, multimodal attention, multimodal fusion, global context modeling, appearance and motion features, multi-step reasoning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks