Cited by 401Multimodal Machine Learning ApplicationsVideo Analysis and SummarizationHuman Pose and Action Recognition

The DOI matches its Crossref record; 36 of the 38 references it cites were found in a registry or an index.

Attention-Based Multimodal Fusion for Video Description

Open access

Hori, Chiori, Hori, Takaaki, Lee, Teng-Yok, Zhang, Ziming, Harsham, Bret, Hershey, John R., Marks, Tim K., Sumi, Kazuhiko·2017·2017 IEEE International Conference on Computer Vision (ICCV)

DOI 10.1109/iccv.2017.450

Open this paper in CoChat — free account Open PDF

Abstract

Current methods for video description are based on encoder-decoder sentence generation using recurrent neural networks (RNNs). Recent work has demonstrated the advantages of integrating temporal attention mechanisms into these models, in which the decoder network predicts each word in the description by selectively giving more weight to encoded features from specific time frames. Such methods typically use two different types of features: image features (from an object classification model), and motion features (from an action recognition model), combined by naive concatenation in the model input. Because different feature modalities may carry task-relevant information at different times, fusing them by naive concatenation may limit the model's ability to dynamically determine the relevance of each type of feature to different parts of the description. In this paper, we incorporate audio features in addition to the image and motion features. To fuse these three modalities, we introduce a multimodal attention model that can selectively utilize features from different modalities for each word in the output description. Combining our new multimodal attention model with standard temporal attention outperforms state-of-the-art methods on two standard datasets: YouTube2Text and MSR-VTT.

Integrity signals

Registry

Crossref · 10.1109/iccv.2017.450

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Long Short-Term Memory
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014) · SSST@EMNLP · doi:10.3115/v1/w14-4012 · Cited by 7,251

Flagged first, then by citations

Show all 38 references

2 of the 38 references couldn’t be matched to a known paper.

Research works printed but not linked: 3

chorowski, 2015, Advances in Neural IInformation Processing Systems, 577
tokui, 2015, Chainer: a next-generation open source framework for deep learning, Proceedings of Workshop on Machine Learning Systems (LearningSys) in The Twenty-ninth Annual Conference on Neural Information Processing Systems (NIPS)
tieleman, 2012, Lecture 6.5-RmsProp: Divide the gradient by a running average of its recent magnitude, COURSERA Neural Networks for Machine Learning

Sorted by citations

Items 201–300 of 397

A Non-Invasive Fetal QRS Complex Detection Method Based on a Multi-Feature Fusion Neural Network
Huang, Z., Yu, J., Shan, Y., & Wang, X. (2024) · Applied Sciences · doi:10.3390/app14198987 · Cited by 10
Multi-guiding long short-term memory for video captioning
Xu, N., Liu, A.-A., Nie, W., & Su, Y. (2018) · Multimedia Systems · doi:10.1007/s00530-018-0598-5 · Cited by 9
Capturing Long-Range Dependencies in Video Captioning
Lee, J.-Y., Lee, Y., Seong, S., Kim, K., Kim, S.-J., & Kim, J. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803143 · Cited by 9
Multimodal Data Fusion based on the Global Workspace Theory
Bao, C., Fountas, Z., Olugbade, T., & Bianchi‐Berthouze, N. (2020) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3382507.3418849 · Cited by 11
Predicting and attending to damaging collisions for placing everyday objects in photo-realistic simulations
Magassouba, A., Sugiura, K., Nakayama, A., Hirakawa, T., Yamashita, T., Fujiyoshi, H., & Kawai, H. (2021) · Advanced Robotics · doi:10.1080/01691864.2021.1913446 · Cited by 9
Hierarchical Context-aware Network for Dense Video Event Captioning
Ji, L., Guo, X., Huang, H., & Chen, X. (2021) · doi:10.18653/v1/2021.acl-long.156 · Cited by 10
Sports Video Captioning by Attentive Motion Representation based Hierarchical Recurrent Neural Networks
Qi, M., Wang, Y., Li, A., & Luo, J. (2018) · ACM International Workshop on Multimedia Content Analysis in Sports (MMSports) · doi:10.1145/3265845.3265851 · Cited by 10
Multimodal Multi-Speaker Merger & Acquisition Financial Modeling: A New Task, Dataset, and Neural Baselines
Sawhney, R., Goyal, M., Goel, P., Mathur, P., & Shah, R. R. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.526 · Cited by 8
Semantic similarity on multimodal data: A comprehensive survey with applications
Ihnaini, B., Abuhaija, B., Mills, E. A., & Mahmuddin, M. (2024) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2024.102263 · Cited by 15
Driving Behavior Aware Caption Generation for Egocentric Driving Videos Using In-Vehicle Sensors
Zhang, H., Takeda, K., Sasano, R., Adachi, Y., & Ohtani, K. (2021) · IEEE Intelligent Vehicles Symposium Workshops (IV Workshops) · doi:10.1109/ivworkshops54471.2021.9669259 · Cited by 8
Combinatorial Analysis of Deep Learning and Machine Learning Video Captioning Studies: A Systematic Literature Review
Kehkashan, T., Alsaeedi, A., Yafooz, W. M. S., Ismail, N. A., & Al-Dhaqm, A. (2024) · IEEE Access · doi:10.1109/access.2024.3357980 · Cited by 8
Smart IoT Multimodal Emotion Recognition System Using Deep Learning Networks
Aiswaryadevi, V. J., Priyanka, G., Bama, S. S., Kiruthika, S. U., Soundarya, S., Sruthi, M. S., & Nataraj, N. (2021) · Studies in big data · doi:10.1007/978-981-33-6400-4_1 · Cited by 8
A multi‐modal heterogeneous data mining algorithm using federated learning
Wei, X. (2021) · The Journal of Engineering · doi:10.1049/tje2.12049 · Cited by 8
UAT: Universal Attention Transformer for Video Captioning
Im, H., & Choi, Y. S. (2022) · Sensors · doi:10.3390/s22134817 · Cited by 9
Speech Summarization of Long Spoken Document: Improving Memory Efficiency of Speech/Text Encoders
Kano, T., Ogawa, A., Delcroix, M., Sharma, R., Matsuura, K., & Watanabe, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095019 · Cited by 8
PassREfinder: Credential Stuffing Risk Prediction by Representing Password Reuse between Websites on a Graph
Kim, J., Song, M., Seo, M., Jin, Y., & Shin, S. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00020 · Cited by 8
Cross-Modal Attention-Guided Convolutional Network for Multi-modal Cardiac Segmentation
Zhou, Z., Guo, X., Yang, W., Shi, Y., Zhou, L., Wang, L., & Yang, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32692-0_69 · Cited by 8
Foundation Models for Video Understanding: A SurveyPreprint
Madan, N., Møgelmose, A., Modi, R., Rawat, Y. S., & Moeslund, T. B. (2024) · doi:10.36227/techrxiv.171769139.99464428/v2 · Cited by 11
A comprehensive review of on-board action recognition models in public transportation systems
Meurie, C., & Lézoray, O. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128311 · Cited by 9
Video captioning using Semantically Contextual Generative Adversarial Network
Hemalatha, M., & Chandra, S. C. (2022) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2022.103453 · Cited by 7
Panel-Page-Aware Comic Genre Understanding
Xu, C., Xu, X., Zhao, N., Cai, W., Zhang, H., Li, C., & Liu, X. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3270105 · Cited by 8
A comprehensive survey on deep-learning-based visual captioning
Xin, B., Xu, N., Zhai, Y., Zhang, T., Lu, Z., Liu, J., Nie, W., Li, X., & Liu, A.-A. (2023) · Multimedia Systems · doi:10.1007/s00530-023-01175-x · Cited by 8
DuKA: A Dual-Keyless-Attention Model for Multi-Modality EHR Data Fusion and Organ Failure Prediction
Liu, Z., Wu, X., Yang, Y., & Clifton, D. A. (2023) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2023.3331305 · Cited by 7
Learning With Correlation-Guided Attention for Multienergy Consumption Forecasting
Park, J. S., Park, J.-H., Choi, J., & Kwon, H.-Y. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3424336 · Cited by 7
Few-Shot Multi-Agent Perception
Fan, C., Hu, J., & Huang, J. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475315 · Cited by 7
Fall risk assessment through a synergistic multi-source DNN learning model
Andreeva, O., Ding, W. Y., Leveille, S. G., Cai, Y., & Chen, P. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102280 · Cited by 7
Sequential Embedding Induced Text Clustering, a Non-parametric Bayesian Approach
Duan, T., Lou, Q., Srihari, S. N., & Xie, X. S. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-16142-2_6 · Cited by 7
Attention-based Visual-Audio Fusion for Video Caption Generation
Guo, N., Liu, H., & Jiang, L. (2019) · International Conference on Advanced Robotics and Mechatronics (ICARM) · doi:10.1109/icarm.2019.8834066 · Cited by 7
SelectFusion: A Generic Framework to Selectively Learn Multisensory Fusion.Preprint
Chen, C., Rosa, S., Lu, C. X., Trigoni, N., & Markham, A. (2019) · arXiv (Cornell University) · Cited by 7
Transfer learning for multimodal dialog
Palaskar, S., Sanabria, R., & Metze, F. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101093 · Cited by 6
Seeing and hearing too: Audio representation for video captioning
Chuang, S.-P., Wan, C.-H., Huang, P., Yang, C.-Y., & Lee, H.-Y. (2017) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2017.8268961 · Cited by 6
Diversified realistic face image generation GAN for human subjects in multimedia content creation
Kumar, L., & Singh, D. K. (2024) · Computer Animation and Virtual Worlds · doi:10.1002/cav.2232 · Cited by 6
Attention-driven Multi-sensor Selection
Braun, S., Neil, D., Anumula, J., Ceolini, E., & Liu, S. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852396 · Cited by 6
Two-Stream Hybrid Attention Network for Multimodal Classification
Chen, Q., Zhenyu, S., Zuo, Z. C., Fu, J., & Sun, Y. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506177 · Cited by 6
Video captioning in Vietnamese using deep learning
Phuc, D. T., Trieu, T. Q., Tính, N. V., & Hieu, D. S. (2022) · International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering · doi:10.11591/ijece.v12i3.pp3092-3103 · Cited by 6
Sentence Attention Blocks for Answer Grounding
Khoshsirat, S., & Kambhamettu, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00559 · Cited by 6
Sensor Fusion-Based Cooperative Trail Following for Autonomous Multi-Robot System
Geng, M., Liu, S., & Wu, Z. (2019) · Sensors · doi:10.3390/s19040823 · Cited by 6
Attention-Based Convolutional LSTM for Describing Video
Liu, Z., Chen, T., Ding, E., Liu, Y., & Yu, W. (2020) · IEEE Access · doi:10.1109/access.2020.3010872 · Cited by 6
Video captioning with global and local text attention
Peng, Y., Wang, C., Pei, Y., & Li, Y. (2021) · The Visual Computer · doi:10.1007/s00371-021-02294-0 · Cited by 6
Evaluation metrics for video captioning: A survey
Inácio, A. D. S., & Lopes, H. S. (2023) · Machine Learning with Applications · doi:10.1016/j.mlwa.2023.100488 · Cited by 9
Recognizing Food Places in Egocentric Photo-Streams Using Multi-Scale Atrous Convolutional Networks and Self-Attention Mechanism
Sarker, M. M. K., Rashwan, H. A., Akram, F., Talavera, E., Banu, S. F., Radeva, P. I., & Puig, D. (2019) · IEEE Access · doi:10.1109/access.2019.2902225 · Cited by 6
A region-based convolutional fusion network for typhoon intensity estimation in satellite images
Yin, P., Chen, H., Huang, H., Su, H., Wu, Q., & Wan, Q. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108671 · Cited by 5
Graph Neural Network for 3‐Dimensional Structures Including Dihedral Angles for Molecular Property Prediction
Sangala, S. A. R., & Raghunathan, S. (2025) · Journal of Computational Chemistry · doi:10.1002/jcc.70121 · Cited by 6
Towards Bridging Video and Language by Caption Generation and Sentence Localization
Chen, S. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3481032 · Cited by 5
Unimodal Face Classification with Multimodal Training
Teng, W., & Bai, C. (2021) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg52635.2021.9666965 · Cited by 5
Guiding Attention using Partial-Order Relationships for Image Captioning
Popattia, M., Rafi, M., Qureshi, R., & Nawaz, S. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00513 · Cited by 5
Countering Modal Redundancy and Heterogeneity: A Self-Correcting Multimodal Fusion
Wang, P., Wang, X., Wang, B., Zhang, Y., Bai, L., & Wang, Y. (2022) · Proceedings · doi:10.1109/icdm54844.2022.00062 · Cited by 5
Video question answering supported by a multi-task learning objective
Falcon, A., Serra, G., & Lanz, O. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14333-0 · Cited by 5
Few-Shot Multi-Agent Perception With Ranking-Based Feature Learning
Fan, C., Hu, J., & Huang, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3285755 · Cited by 5
MHRN: A Multimodal Hierarchical Reasoning Network for Topic Detection
Li, J., Wang, Y., & Li, W. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3358696 · Cited by 6
Video Captioning in Compressed Video
Zhu, M. (2021) · International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc52351.2021.9526927 · Cited by 6
A Survey of Driver Behavior Perception Methods for Human-Computer Hybrid Enhancement of Intelligent Driving
Yi, J., Du, A., Zhu, Z., & Ding, H. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-3842-9_58 · Cited by 5
Knowledge graph incremental embedding for unseen modalities
Wei, Y., Chen, W., Wen, S., Liu, A., & Zhao, L. (2023) · Knowledge and Information Systems · doi:10.1007/s10115-023-01868-9 · Cited by 5
Multimodal-enhanced hierarchical attention network for video captioning
Zhong, M., Chen, Y., Zhang, H., Xiong, H., & Wang, Z. (2023) · Multimedia Systems · doi:10.1007/s00530-023-01130-w · Cited by 5
Partners in Crime: Multi-view Sequential Inference for Movie Understanding
Papasarantopoulos, N., Frermann, L., Lapata, M., & Cohen, S. B. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1212 · Cited by 5
Spectrogram Analysis Via Self-Attention for Realizing Cross-Model Visual-Audio Generation
Tan, H., Wu, G., Zhao, P., & Chen, Y. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9052918 · Cited by 5
Cross-modal Learning for Multi-modal Video CategorizationPreprint
Goyal, P., Sahu, S., Ghosh, S., & Lee, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.03501 · Cited by 5
Evaluation of Video Masked Autoencoders’ Performance and Uncertainty Estimations for Driver Action and Intention Recognition
Vellenga, K., Steinhauer, H. J., Falkman, G., & Björklund, T. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00726 · Cited by 7
Investigating topics, audio representations and attention for multimodal scene-aware dialog
Kumar, S. H., Okur, E., Sahay, S., Huang, J., & Nachman, L. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101102 · Cited by 4
Data Augmentation Techniques for the Video Question Answering Task
Falcon, A., Lanz, O., & Serra, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-66415-2_33 · Cited by 6
SASRM: A Semantic and Attention Spatio-temporal Recurrent Model for Next Location Prediction
Zhang, X., Li, B., Song, C., Huang, Z., & Li, Y. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206935 · Cited by 4
Focus and Align: Learning Tube Tokens for Video-Language Pre-Training
Zhu, Y., Li, X., Zheng, M., Yang, J., Wang, Z., Guo, X., Chai, Z., Yuan, Y., & Jiang, S. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3231108 · Cited by 4
Tri-CLT: Learning Tri-Modal Representations with Contrastive Learning and Transformer for Multimodal Sentiment Recognition
Yang, Z., Li, Z., Zhu, D., & Zhou, Y. (2024) · Information Technology And Control · doi:10.5755/j01.itc.53.1.35060 · Cited by 4
Characterizing the Impact of Using Features Extracted from Pre-trained Models on the Quality of Video Captioning Sequence-to-Sequence Models
Hammad, M., Hammad, M., & Elshenawy, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59830-3_21 · Cited by 4
Attention Based Multi-Modal Fusion Architecture for Open-Ended Video Question Answering Systems
Pendurkar, S., Kolpekwar, S., Dhoot, S., Haribhakta, Y. V., & Banerjee, B. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.04.047 · Cited by 4
Integrating Multiple ASR Systems into NLP Backend with Attention Fusion
Kano, T., Ogawa, A., Delcroix, M., & Watanabe, S. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746699 · Cited by 4
Cricket Video Highlight Generation Methods: A Review
Shingrakhia, H., & Patel, H. N. (2022) · ELCVIA Electronic Letters on Computer Vision and Image Analysis · doi:10.5565/rev/elcvia.1465 · Cited by 4
Improving Video Captioning with Non-Local Neural Networks
Lee, J.-Y., & Kim, J. (2018) · IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia) · doi:10.1109/icce-asia.2018.8552140 · Cited by 4
Attend-Fusion: Efficient Audio-Visual Fusion for Video Classification
Awan, M., Nadeem, A., Awan, M. J., Mustafa, A., & Husain, S. S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93806-1_15 · Cited by 5
Chapter-Llama: Efficient Chaptering in Hour-Long Videos with LLMs
Ventura, L., Yang, A., Schmid, C., & Varol, G. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01765 · Cited by 3
Deep learning-based phenotypic analysis and intelligent environmental control in edible mushrooms: Advances, challenges, and prospects
Zhao, R., Luo, C., Zhang, Q., Wei, Q., Zhang, X., Wang, M., Zhang, J., & Li, H. (2025) · Smart Agricultural Technology · doi:10.1016/j.atech.2025.101330 · Cited by 3
Through the Eyes of Emotion: A Multi-faceted Eye Tracking Dataset for Emotion Recognition in Virtual Reality
Yang, T., Regmi, B., Du, L., Bulling, A., Zhang, X., & Lan, G. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749545 · Cited by 3
Toward Few-Shot Multimodal Voice Pathology Classification on an Embedded Edge Device
Kanagachalam, S., & Kim, D. (2025) · IEEE Access · doi:10.1109/access.2025.3606943 · Cited by 3
TCN BERT Attention Guided Stock Market Movement Prediction
Zhang, Z., Matsuda, Y., & Liu, Y. H. (2024) · International Conference on Computer Information and Big Data Applications (CIBDA) · doi:10.1145/3671151.3671336 · Cited by 3
Self-citation
Video Captioning Method Based on Semantic Topic Association
Fu, Y., Yang, Y., & Ye, O. (2025) · Electronics · doi:10.3390/electronics14050905 · Cited by 3
Survey of Dense Video Captioning: Techniques, Resources, and Future Perspectives
Liu, Z., & Song, R. (2025) · Applied Sciences · doi:10.3390/app15094990 · Cited by 3
Hierarchical multimodal attention for end-to-end audio-visual scene-aware dialogue response generation
Le, H., Sahoo, D., Chen, N. F., & Hoi, S. C. H. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101095 · Cited by 3
A Dynamic Feature Weighting Method for Mangrove Pests Image Classification with Heavy-tailed Distributions
Wang, Z., Gong, W., & Li, W. (2020) · International Conference on Vision, Image and Signal Processing (ICVISP) · doi:10.1145/3448823.3448826 · Cited by 3
NeuroVI-based new datasets and space attention network for the recognition and falling detection of delivery packages
Liu, X., Yang, Z.-X., Xu, Z., & Yan, X. (2022) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2022.934260 · Cited by 3
Automatic Captioning based on Visible and Infrared Images
Wang, Y., Lou, S., Wang, K., Wang, Y., Yuan, X., & Liu, H. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610654 · Cited by 3
An Analysis on Conversational AI: The Multimodal Frontier in Chatbot System Advancements
Nikhil, G., Yeligatla, D. R., Chaparala, T. C., Chalavadi, V., Kaur, H., & Singh, V. K. (2024) · International Conference on Inventive Computing and Informatics (ICICI) · doi:10.1109/icici62254.2024.00069 · Cited by 3
Multi-Modal Graph-Based Sentiment Analysis via Hybrid Contrastive Learning
Zhang, J. (2024) · IEEE International Conference on Sensors, Electronics and Computer Engineering (ICSECE) · doi:10.1109/icsece61636.2024.10729439 · Cited by 3
Visual Oriented Encoder: Integrating Multimodal and Multi-Scale Contexts for Video Captioning
Yang, B., & Zou, Y. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412315 · Cited by 3
Dense Video Captioning Using Video-Audio Features and Topic Modeling Based on Caption
Palivela, L. H., Swetha, S., Guhan, M. N., & Venkatesh, M. P. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-7753-4_40 · Cited by 3
CLASSIFICATION OF SEVERELY OCCLUDED IMAGE SEQUENCES VIA CONVOLUTIONAL RECURRENT NEURAL NETWORKS
Zheng, J., Wang, Y., Zhang, X., & Li, X. (2018) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip.2018.8646539 · Cited by 3
Expectation Learning for Stimulus Prediction Across Modalities Improves Unisensory Classification
Barros, P. V. A., Eppe, M., Parisi, G. I., Liu, X., & Wermter, S. (2019) · Frontiers in Robotics and AI · doi:10.3389/frobt.2019.00137 · Cited by 3
Real-Time Environment Description Application for Visually Challenged People
Bhile, A. A., & Hole, V. (2020) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-030-37051-0_38 · Cited by 3
Intelligence Is beyond Learning: A Context-Aware Artificial Intelligent System for Video Understanding
Ghozia, A., Attiya, G., Adly, E., & El-Fishawy, N. A. (2020) · Computational Intelligence and Neuroscience · doi:10.1155/2020/8813089 · Cited by 4
Equilibrated Recurrent Neural Network: Neuronal Time-Delayed Self-Feedback Improves Accuracy and StabilityPreprint
Zhang, Z., Kag, A., Sullivan, A., & Saligrama, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.00755 · Cited by 6
Self-citation
FDCT: Frequency-Aware Decomposition and Cross-Modal Token-Alignment for Multisensor Target Classification
Sami, S. M., Hasan, M. M., Nasrabadi, N. M., & Rao, R. (2025) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2025.3550474 · Cited by 2
Multimodal Graph Representation Learning over Arbitrary Sets of Modalities
Patapati, S., & Srinivasan, T. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00686 · Cited by 2
Estimation of Ewe Live Weight and Carcass Traits Using Advanced Hybrid Deep Learning and Multimodal Feature Fusion
Shalaldeh, A., Safa, M., Logan, C. M., & Othman, M. (2026) · Biology · doi:10.3390/biology15100815 · Cited by 2
Utilizing Text-based Augmentation to Enhance Video Captioning
Li, S., Yang, B., & Zou, Y. (2022) · International Conference on Artificial Intelligence and Big Data (ICAIBD) · doi:10.1109/icaibd55127.2022.9820499 · Cited by 2
ERDBF: Embedding-Regularized Double Branches Fusion for Multi-Modal Age Estimation
Wu, B., Lu, H., Chen, Z., Zhu, C., & Xu, S. (2023) · IEEE Access · doi:10.1109/access.2023.3275765 · Cited by 2
Bidirectional transformer with knowledge graph for video captioning
Zhong, M., Chen, Y., Zhang, H., Xiong, H., & Wang, Z. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17822-4 · Cited by 2
AutoAMS: Automated attention-based multi-modal graph learning architecture search
Al-Sabri, R., Gao, J., Chen, J., Olouladé, B. M., & Wu, Z. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106427 · Cited by 5
A Hierarchical Graph-Based Approach for Recognition and Description Generation of Bimanual Actions in Videos
Ziaeetabar, F., Tamošiūnaitė, M., & Wörgötter, F. (2024) · IEEE Access · doi:10.1109/access.2024.3509674 · Cited by 7
A Transformer-based Multimodal Feature Fusion Model for Video Captioning
Wang, Y. (2024) · International Conference on Electronic Information Engineering and Computer Science (EIECS) · doi:10.1109/eiecs63941.2024.10799977 · Cited by 2
Adaptively forget with crossmodal and textual distillation for class-incremental video captioning
Xiong, H., Wang, L., Qiu, H., Zhao, T., Qiu, B., & Li, H. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129388 · Cited by 2
Watch, Reason and Code
Duan, X., Wu, Q., Gan, C., Zhang, Y., Huang, W., van den Hengel, A., & Zhu, W. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351094 · Cited by 2

Authors: Hori, C., Hori, T., Lee, T.-Y., Zhang, Z., Harsham, B., Hershey, J. R., Marks, T. K., & Sumi, K.
Affiliations: Mitsubishi Electric (United States) (US)
Aoyama Gakuin University (JP)
ORCID: Hori, Chiori
Hori, Takaaki
Lee, Teng-Yok
Zhang, Ziming
Marks, Tim K.
Sumi, Kazuhiko
Venue: 2017 IEEE International Conference on Computer Vision (ICCV)
Publisher: IEEE
Published: 2017-10
Pages: 4203-4212
DOI: 10.1109/iccv.2017.450
OpenAlex: W2584992898
Cited by: 401
References: 38
Citations by year: 2026: 11 · 2025: 24 · 2024: 41 · 2023: 47 · 2022: 54 · 2021: 55 · 2020: 63 · 2019: 71 · 2018: 30 · 2017: 4
FWCI: 13.22
Type: conference paper
Language: en
Open access: green
arxiv.org
Locations: IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
Keywords: audio description, multimodal fusion, multimodal attention, temporal attention, audio features, imaging features, motion features, video captioning, MSR-VTT

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Attention-Based Multimodal Fusion for Video Description

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks