The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Video Summarization With Attention-Based Encoder–Decoder Networks
Open accessAbstract
This paper addresses the problem of supervised video summarization by formulating it as a sequence-to-sequence learning problem, where the input is a sequence of original video frames, and the output is a keyshot sequence. Our key idea is to learn a deep summarization network with attention mechanism to mimic the way of selecting the keyshots of human. To this end, we propose a novel video summarization framework named attentive encoder-decoder networks for video summarization (AVS), in which the encoder uses a bidirectional long short-term memory (BiLSTM) to encode the contextual information among the input video frames. As for the decoder, two attention-based LSTM networks are explored by using additive and multiplicative objective functions, respectively. Extensive experiments are conducted on two video summarization benchmark datasets, i.e., SumMe and TVSum. The results demonstrate the superiority of the proposed AVS-based approaches against the state-of-the-art approaches, with remarkable improvements on both datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Sequence to Sequence -- Video to Text
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R., Darrell, T., & Saenko, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.515 · Cited by 1,352
Video abstraction: A systematic review and classification
Truong, B. T., & Venkatesh, S. (2007) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/1198302.1198305 · Cited by 833
Creating Summaries from User Videos
Gygli, M., Grabner, H., Riemenschneider, H., & Van Gool, L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_33 · Cited by 752
Video Summarization with Long Short-Term Memory
Zhang, K., Chao, W.-L., Sha, F., & Grauman, K. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46478-7_47 · Cited by 739
VSUMM: A mechanism designed to produce static video summaries and a novel evaluation method
de Avila, S. E. F., Lopes, A. P. B., da Luz, A., & de Albuquerque Araújo, A. (2011) · Pattern Recognition Letters · doi:10.1016/j.patrec.2010.08.004 · Cited by 679
Discovering important people and objects for egocentric video summarization
Lee, Y. J., Ghosh, J., & Grauman, K. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247820 · Cited by 658
TVSum: Summarizing web videos using titles
Song, Y., Vallmitjana, J., Stent, A., & Jaimes, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299154 · Cited by 623
Framewise phoneme classification with bidirectional LSTM networks
Graves, A., & Schmidhuber, J. (2006) · Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. · doi:10.1109/ijcnn.2005.1556215 · Cited by 618
Unsupervised Video Summarization with Adversarial LSTM Networks
Mahasseni, B., Lam, M., & Todorovic, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.318 · Cited by 607
A user attention model for video summarization
Ma, Y.-F., Lu, L., Zhang, H.-J., & Li, M. (2002) · Proceedings of the tenth ACM international conference on Multimedia · doi:10.1145/641007.641116 · Cited by 551
Category-Specific Video Summarization
Potapov, D., Douze, M., Harchaoui, Z., & Schmid, C. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10599-4_35 · Cited by 481
Video summarization by learning submodular mixtures of objectives
Gygli, M., Grabner, H., & Van Gool, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298928 · Cited by 397
Diverse Sequential Subset Selection for Supervised Video Summarization
Gong, B., Chao, W., Grauman, K., et al. (2014) · Cited by 397
Video summarization and scene detection by graph modeling
Ngo, C.-W., Ma, Y.-F., & Zhang, H.-J. (2005) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2004.841694 · Cited by 345
Pairwise Relationship Guided Deep Hashing for Cross-Modal Retrieval
Yang, E., Deng, C., Liu, W., Liu, X., Tao, D., & Gao, X. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10719 · Cited by 331
Towards Scalable Summarization of Consumer Videos Via Sparse Dictionary Selection
Cong, Y., Yuan, J., & Luo, J. (2011) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2011.2166951 · Cited by 310
Information theory-based shot cut/fade detection and video summarization
Černeková, Z., Pitas, I., & Nikou, C. (2005) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2005.856896 · Cited by 301
Video Summarization Using Fully Convolutional Sequence Networks
Rochan, M., Ye, L., & Wang, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01258-8_22 · Cited by 287
Large-Scale Video Summarization Using Web-Image Priors
Khosla, A., Hamid, R., Lin, C.-J., & Sundaresan, N. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.348 · Cited by 283
Event Driven Web Video Summarization by Tag Localization and Key-Shot Identification
Wang, M., Hong, R., Li, G., Zha, Z.-J., Yan, S., & Chua, T.-S. (2012) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2012.2185041 · Cited by 263
HSA-RNN: Hierarchical Structure-Adaptive RNN for Video Summarization
Zhao, B., Li, X., & Lu, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00773 · Cited by 237
Self-citationSummary Transfer: Exemplar-Based Subset Selection for Video Summarization
Zhang, K., Chao, W.-L., Sha, F., & Grauman, K. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.120 · Cited by 220
Multi-View Video Summarization
Fu, Y., Guo, Y., Zhu, Y., Liu, F., Song, C., & Zhou, Z.-H. (2010) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2010.2052025 · Cited by 194
Video summarization via minimum sparse reconstruction
Mei, S., Guan, G., Wang, Z., Wan, S., He, M., & Dagan Feng, D. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2014.08.002 · Cited by 180
Learning deep event models for crowd anomaly detection
Feng, Y., Yuan, Y., & Lu, X. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.09.063 · Cited by 180
Unsupervised Extraction of Video Highlights via Robust Recurrent Auto-Encoders
Yang, H., Wang, B., Lin, S., Wipf, D., Guo, M., & Guo, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.526 · Cited by 172
Retrospective Encoders for Video Summarization
Zhang, K., Grauman, K. L., & Sha, F. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_24 · Cited by 133
A General Framework for Edited Video and Raw Video Summarization
Li, X., Zhao, B., & Lu, X. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2695887 · Cited by 128
Self-citationLearning Multilayer Channel Features for Pedestrian Detection
Cao, J., Pang, Y., & Li, X. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2694224 · Cited by 121
Self-citationCoherent Semantic-Visual Indexing for Large-Scale Image Retrieval in the Cloud
Hong, R., Li, L., Cai, J., Tao, D., Wang, M., & Tian, Q. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2710635 · Cited by 106
Motion-State-Adaptive Video Summarization via Spatiotemporal Analysis
Zhang, Y., Tao, R., & Wang, Y. (2016) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2016.2539638 · Cited by 60
A Memory Network Approach for Story-Based Temporal Summarization of 360° Videos
Lee, S., Sung, J., Yu, Y., & Kim, G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00153 · Cited by 60
Query-aware sparse coding for web multi-video summarization
Ji, Z., Ma, Y., Pang, Y., & Li, X. (2018) · Information Sciences · doi:10.1016/j.ins.2018.09.050 · Cited by 55
Self-citationImproving Sequential Determinantal Point Processes for Supervised Video Summarization
Sharghi, A., Borji, A., Li, C., Yang, T., & Gong, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01219-9_32 · Cited by 52
Li, Y., Wang, L., Yang, T., & Gong, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_10 · Cited by 45
Cascade Learning by Optimally Partitioning
Pang, Y., Cao, J., & Li, X. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2601438 · Cited by 35
Self-citationLearning Visual Storylines with Skipping Recurrent Neural Networks
Sigurdsson, G. A., Chen, X., & Gupta, A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46454-1_5 · Cited by 35
Perceptual Attributes Optimization for Multivideo Summarization
Nie, L., Hong, R., Zhang, L., Xia, Y., Tao, D., & Sebe, N. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2493558 · Cited by 26
Cited by
Learning a Deep Multi-Scale Feature Ensemble and an Edge-Attention Guidance for Image Fusion
Liu, J., Fan, X., Jiang, J., Liu, R., & Luo, Z. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3056725 · Cited by 431
Video Summarization Using Deep Neural Networks: A Survey
Apostolidis, E., Adamantidou, E., Metsai, A. I., Mezaris, V., & Patras, I. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3117472 · Cited by 350
Attention, please! A survey of neural attention models in deep learning
de Santana Correia, A., & Colombini, E. L. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10148-x · Cited by 298
Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning
Mascharka, D., Tran, P., Soklaski, R., & Majumdar, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00519 · Cited by 203
“Many authors [1, 4, 6, 12, 17, 18, 26, 27, 23, 28, 33] use a spatial softmax to compute attention weights.”
Real-Time Video Emotion Recognition Based on Reinforcement Learning and Domain Knowledge
Zhang, K., Li, Y., Wang, J., Cambria, E., & Li, X. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3072412 · Cited by 184
“People are inclined to express their emotions implicitly through some habitual ways during interpersonal conversations, which are different from the official circumstances[4].”
BackgroundSelf-citation
Metadata
- Authors
- Ji, Z., Xiong, K., Pang, Y., & Li, X.
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2020-06
- Volume
- 30
- Issue
- 6
- Pages
- 1709-1717
- ISSN
- 1051-8215 (ISSN-L)
- 1558-2205
- OpenAlex
- W2964167369
- Cited by
- 392
- References
- 42
- FWCI
- 21.45
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.