The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Describing Video With Attention-Based Bidirectional LSTM
Abstract
Video captioning has been attracting broad research attention in the multimedia community. However, most existing approaches heavily rely on static visual information or partially capture the local temporal knowledge (e.g., within 16 frames), thus hardly describing motions accurately from a global view. In this paper, we propose a novel video captioning framework, which integrates bidirectional long-short term memory (BiLSTM) and a soft attention mechanism to generate better global representations for videos as well as enhance the recognition of lasting motions in videos. To generate video captions, we exploit another long-short term memory as a decoder to fully explore global contextual information. The benefits of our proposed method are two fold: 1) the BiLSTM structure comprehensively preserves global temporal and visual information and 2) the soft attention mechanism enables a language decoder to recognize and focus on principle targets from the complex content. We verify the effectiveness of our proposed video captioning framework on two widely used benchmarks, that is, microsoft video description corpus and MSR-video to text, and the experimental results demonstrate the superiority of the proposed approach compared to several state-of-the-art methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172 · cited 1×
“Schuster and Paliwal [37] constructed a bidirectional RNN for speech recognition, which merges two separate expert networks with linear and logarithmic fusion, respectively, and comprehensively utilizes all of the input information.”
Background(2004) · Cited by 9,718
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,672
Meteor Universal: Language Specific Translation Evaluation for Any Target Language
Denkowski, M., & Lavie, A. (2014) · Proceedings of the Ninth Workshop on Statistical Machine Translation · doi:10.3115/v1/w14-3348 · Cited by 1,952
MSR-VTT: A Large Video Description Dataset for Bridging Video and Language
Xu, J., Mei, T., Yao, T., & Rui, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.571 · Cited by 1,759
Static Analysis of Shape in TensorFlow Programs
The Theano Development Team, Al‐Rfou, R., Alain, G., Almahairi, A., Angermueller, C., Bahdanau, D., Ballas, N., Bastien, F., Bayer, J., Belikov, A., et al. (2016) · arXiv (Cornell University) · doi:10.4230/lipics.ecoop.2020.15 · Cited by 1,589
Sequence to Sequence -- Video to Text
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R., Darrell, T., & Saenko, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.515 · Cited by 1,352
Srivastava, R. K., Greff, K., & Schmidhuber, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.06228 · Cited by 1,246
DenseCap: Fully Convolutional Localization Networks for Dense Captioning
Johnson, J., Karpathy, A., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.494 · Cited by 1,137
Describing Videos by Exploiting Temporal Structure
Yao, L., Torabi, A., Cho, K., Ballas, N., Pal, C., Larochelle, H., & Courville, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.512 · Cited by 1,035
Translating Videos to Natural Language Using Deep Recurrent Neural Networks
Venugopalan, S., Xu, H., Donahue, J., Rohrbach, M., Mooney, R., & Saenko, K. (2015) · Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.3115/v1/n15-1173 · Cited by 959
Collecting Highly Parallel Data for Paraphrase Evaluation
Chen, D. J., & Dolan, W. B. (2011) · Cited by 904
Adversarial Cross-Modal Retrieval
Wang, B., Yang, Y., Xu, X., Hanjalic, A., & Shen, H. T. (2017) · Proceedings of the 25th ACM international conference on Multimedia · doi:10.1145/3123266.3123326 · Cited by 779
Self-citationAction Recognition in Video Sequences using Deep Bi-Directional LSTM With CNN Features
Ullah, A., Ahmad, J., Muhammad, K., Sajjad, M., & Baik, S. W. (2018) · IEEE Access · doi:10.1109/access.2017.2778011 · Cited by 756
Video Captioning With Attention-Based LSTM and Semantic Consistency
Gao, L., Guo, Z., Zhang, H., Xu, X., & Shen, H. T. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2729019 · Cited by 637
Self-citationJointly Modeling Embedding and Translation to Bridge Video and Language
Pan, Y., Mei, T., Yao, T., Li, H., & Rui, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.497 · Cited by 563
Guadarrama, S., Krishnamoorthy, N., Malkarnenkar, G., Venugopalan, S., Mooney, R., Darrell, T., & Saenko, K. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.337 · Cited by 500
Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioning
Pan, P., Xu, Z., Yang, Y., Wu, F., & Zhuang, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.117 · Cited by 412
Self-citationUnsupervised Deep Hashing with Similarity-Adaptive and Discrete Optimization
Shen, F., Xu, Y., Liu, L., Yang, Y., Huang, Z., & Shen, H. T. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2789887 · Cited by 407
Self-citationRobust Joint Graph Sparse Coding for Unsupervised Spectral Feature Selection
Zhu, X., Li, X., Zhang, S., Ju, C., & Wu, X. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2521602 · Cited by 339
Self-citationBlock-Row Sparse Multiview Multilabel Learning for Image Classification
Zhu, X., Li, X., & Zhang, S. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2403356 · Cited by 324
Self-citationXu, R., Xiong, C., Chen, W., & Corso, J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9512 · Cited by 298
Generating Natural-Language Video Descriptions Using Text-Mined Knowledge
Krishnamoorthy, N., Malkarnenkar, G., Mooney, R., Saenko, K., & Guadarrama, S. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8679 · Cited by 232
Integrating Language and Vision to Generate Natural Language Descriptions of Videos in the Wild
Thomason, J., Venugopalan, S., Guadarrama, S., Saenko, K., & Mooney, R. J. (2014) · Cited by 173
Visual-Patch-Attention-Aware Saliency Detection
Jian, M., Lam, K.-M., Dong, J., & Shen, L. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2356200 · Cited by 170
Multitask Spectral Clustering by Exploring Intertask Correlation
Yang, Y., Ma, Z., Yang, Y., Nie, F., & Shen, H. T. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2344015 · Cited by 170
Self-citationImproving LSTM-based Video Description with Linguistic Knowledge Mined from Text
Venugopalan, S., Hendricks, L. A., Mooney, R. J., & Saenko, K. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1204 · Cited by 138
Optimized Graph Learning Using Partial Tags and Multiple Features for Image and Video Annotation
Song, J., Gao, L., Nie, F., Shen, H. T., Yan, Y., & Sebe, N. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2601260 · Cited by 130
Self-citationExploiting Web Images for Semantic Video Indexing Via Robust Sample-Specific Loss
Yang, Y., Zha, Z.-J., Gao, Y., Zhu, X., & Chua, T. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2323014 · Cited by 125
Self-citationAsymmetric Binary Coding for Image Search
Shen, F., Yang, Y., Liu, L., Liu, W., Tao, D., & Shen, H. T. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2699863 · Cited by 124
Self-citationRobust discrete code modeling for supervised hashing
Luo, Y., Yang, Y., Shen, F., Huang, Z. H., Zhou, P., & Shen, H. T. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.02.034 · Cited by 117
Self-citationCoherent Semantic-Visual Indexing for Large-Scale Image Retrieval in the Cloud
Hong, R., Li, L., Cai, J., Tao, D., Wang, M., & Tian, Q. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2710635 · Cited by 106
Hashing with Angular Reconstructive Embeddings
Hu, M., Yang, Y., Shen, F., Xie, N., & Shen, H. T. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2749147 · Cited by 103
Self-citationRobust Discrete Spectral Hashing for Large-Scale Image Semantic Indexing
Yang, Y., Shen, F., Shen, H. T., Li, H., & Li, X. (2015) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2016.2516024 · Cited by 95
Self-citationFrame- and Segment-Level Features and Candidate Pool Evaluation for Video Caption Generation
Shetty, R., & Laaksonen, J. T. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2984062 · Cited by 89
Bidirectional Long-Short Term Memory for Video Description
Bin, Y., Yang, Y., Shen, F., Xu, X., & Shen, H. T. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2967258 · Cited by 87
Self-citationJoint Attention by Gaze Interpolation and Saliency
Yücel, Z., Salah, A. A., Meriçli, Ç., Meriçli, T., Valenti, R., & Gevers, T. (2012) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2216979 · Cited by 74
Video-Based Pedestrian Re-Identification by Adaptive Spatio-Temporal Appearance Model
Zhang, W., Ma, B., Liu, K., & Huang, R. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2672440 · Cited by 73
A Multi-scale Multiple Instance Video Description NetworkPreprint
Xu, H., Venugopalan, S., Ramanishka, V., Rohrbach, M., & Saenko, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05914 · Cited by 71
Robust Web Image Annotation via Exploring Multi-Facet and Structural Knowledge
Hu, M., Yang, Y., Shen, F., Zhang, L., Shen, H. T., & Li, X. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2717185 · Cited by 54
Self-citationAdaptively Attending to Visual Attributes and Linguistic Knowledge for Captioning
Bin, Y., Yang, Y., Zhou, J., Huang, Z. H., & Shen, H. T. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123391 · Cited by 41
Self-citationRecurrent attention network using spatial-temporal relations for action recognition
Zhang, M., Yang, Y., Ji, Y., Xie, N., & Shen, F. (2017) · Signal Processing · doi:10.1016/j.sigpro.2017.12.008 · Cited by 30
Self-citationPerceptually Guided Photo Retargeting
Xia, Y., Zhang, L., Hong, R., et al. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2520959 · Cited by 15
Cited by
Predicting COVID-19 in China Using Hybrid AI Model
Zheng, N., Du, S., Wang, J., Zhang, H., Cui, W., Kang, Z., Yang, T., Lou, B., Chi, Y., Long, H., et al. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.2990162 · Cited by 313
Attention, please! A survey of neural attention models in deep learning
de Santana Correia, A., & Colombini, E. L. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10148-x · Cited by 298
Cross-Modal Attention With Semantic Consistence for Image–Text Matching
Xu, X., Wang, T., Yang, Y., Zuo, L., Shen, F., & Shen, H. T. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2967597 · Cited by 279
Self-citationA comprehensive review of AI-Based detection of Arrhythmia using Electrocardiogram (ECG)
Ayyub, A., Politis, C., & Usman, M. A. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110594 · Cited by 18
Metadata
- Authors
- Bin, Y., Yang, Y., Shen, F., Xie, N., Shen, H. T., & Li, X.
- Venue
- IEEE Transactions on Cybernetics
- Published
- 2019-07
- Volume
- 49
- Issue
- 7
- Pages
- 2631-2641
- ISSN
- 2168-2267 (ISSN-L)
- 2168-2275
- OpenAlex
- W2803620531
- Cited by
- 270
- References
- 57
- FWCI
- 17.20
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.