The DOI matches its DataCite record; 29 of the 30 references it cites were found in a registry or an index.
Unsupervised Learning of Video Representations using LSTMs
Open accessAbstract
We use multilayer Long Short Term Memory (LSTM) networks to learn representations of video sequences. Our model uses an encoder LSTM to map an input sequence into a fixed length representation. This representation is decoded using single or multiple decoder LSTMs to perform different tasks, such as reconstructing the input sequence, or predicting the future sequence. We experiment with two kinds of input sequences - patches of image pixels and high-level representations ("percepts") of video frames extracted using a pretrained convolutional net. We explore different design choices such as whether the decoder LSTMs should condition on the generated output. We analyze the outputs of the model qualitatively to see how well the model can extrapolate the learned video representation into the future and into the past. We try to visualize and interpret the learned features. We stress test the model by running it on longer time scales and on out-of-domain data. We further evaluate the representations by finetuning them for a supervised learning problem - human action recognition on the UCF-101 and HMDB-51 datasets. We show that the representations help improve classification accuracy, especially when there are only a few training examples. Even models pretrained on unrelated datasets (300 hours of YouTube videos) can help action recognition performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This is the first time that motion prediction with cross-dimensional learning has been used for self-supervised learning in the context of HAR.”
From Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion (2021) · cites this paper as [23]
In the citing paper
“While motion prediction has been used in the field of computer vision for self-supervised learning [23, 26], our model is novel in that the cross-dimensional aspect results in more effective learning.”
From Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion (2021) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,313
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Research works printed but not linked: 1
- Hochreiter, Sepp and Schmidhuber, J¨ urgen. Long short-term memory. Neural Computation , 9(8):1735-1780, 1997.
Cited by
Sorted by citations
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., & Woo, W.-C. (2015) · arXiv · doi:10.48550/arxiv.1506.04214 · Cited by 7,709
A Closer Look at Spatiotemporal Convolutions for Action Recognition
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., & Paluri, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00675 · Cited by 3,672
Social LSTM: Human Trajectory Prediction in Crowded Spaces
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., & Savarese, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.110 · Cited by 3,675
Convolutional Two-Stream Network Fusion for Video Action Recognition
Feichtenhofer, C., Pinz, A. J., & Zisserman, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.213 · Cited by 2,842
TSM: Temporal Shift Module for Efficient Video Understanding
Lin, J., Gan, C., & Han, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00718 · Cited by 2,209
Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks
Gupta, A., Johnson, J., Fei-Fei, L., Savarese, S., & Alahi, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00240 · Cited by 2,466
Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Jing, L., & Tian, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992393 · Cited by 2,036
Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks
Qiu, Z., Yao, T., & Mei, T. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.590 · Cited by 1,944
Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges
Hesamian, M. H., Jia, W., He, X., & Kennedy, P. J. (2019) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-019-00227-x · Cited by 1,710
Sequence to Sequence -- Video to Text
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R. J., Darrell, T., & Saenko, K. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.515 · Cited by 1,448
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition
Liu, J., Shahroudy, A., Xu, D., & Wang, G. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46487-9_50 · Cited by 1,369
Structural-RNN: Deep Learning on Spatio-Temporal Graphs
Jain, A., Zamir, A. R., Savarese, S., & Saxena, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.573 · Cited by 1,133
MoCoGAN: Decomposing Motion and Content for Video Generation
Tulyakov, S., Liu, M.-Y., Yang, X., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00165 · Cited by 1,076
Describing Videos by Exploiting Temporal Structure
Yao, L., Torabi, A., Cho, K., Ballas, N., Pal, C., Larochelle, H., & Courville, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.512 · Cited by 1,035
Unsupervised Learning of Visual Representations Using Videos
Wang, X., & Gupta, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.320 · Cited by 1,005
Semi-supervised Sequence Learning
Dai, A. M., & Le, Q. V. (2015) · arXiv · doi:10.48550/arxiv.1511.01432 · Cited by 994
Photographic Image Synthesis with Cascaded Refinement Networks
Chen, Q., & Koltun, V. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.168 · Cited by 887
A Comprehensive Survey on Graph Anomaly Detection With Deep Learning
Ma, X., Wu, J. X., Xue, S., Yang, J. Y., Zhou, C., Sheng, Q. Z., Xiong, H., & Akoglu, L. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3118815 · Cited by 939
Random synaptic feedback weights support error backpropagation for deep learning
Lillicrap, T., Cownden, D., Tweed, D., & Akerman, C. J. (2016) · Nature Communications · doi:10.1038/ncomms13276 · Cited by 971
Accelerating Very Deep Convolutional Networks for Classification and Detection
Zhang, X., Zou, J., He, K., & Sun, J. Z. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2502579 · Cited by 871
Sequence-Level Knowledge Distillation
Kim, Y., & Rush, A. M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1139 · Cited by 836
Videos as Space-Time Region Graphs
Wang, X., & Gupta, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01228-1_25 · Cited by 793
What makes for good views for contrastive learning
Tian, Y., Sun, C., Poole, B., Krishnan, D., Schmid, C., & Isola, P. (2020) · arXiv · doi:10.48550/arxiv.2005.10243 · Cited by 773
Video Frame Synthesis Using Deep Voxel Flow
Liu, Z., Yeh, R. A., Tang, X., Liu, Y., & Agarwala, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.478 · Cited by 766
Video Frame Interpolation via Adaptive Separable Convolution
Niklaus, S., Mai, L., & Liu, F. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.37 · Cited by 746
PredRNN: recurrent neural networks for predictive learning using spatiotemporal LSTMs
Wang, Y., Long, M., Wang, J., Gao, Z., & Yu, P. S. (2017) · neural information processing systems · Cited by 734
Ground, Proximal, and Satellite Remote Sensing of Soil Moisture
Babaeian, E., Sadeghi, M., Jones, S. B., Montzka, C., Vereecken, H., & Tuller, M. (2019) · Reviews of Geophysics · doi:10.1029/2018rg000618 · Cited by 728
Spatiotemporal Multiplier Networks for Video Action Recognition
Feichtenhofer, C., Pinz, A. J., & Wildes, R. P. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.787 · Cited by 692
Human Action Recognition From Various Data Modalities: A Review
Sun, Z., Ke, Q., Rahmani, H., Bennamoun, M., Wang, G., & Liu, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3183112 · Cited by 719
PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning
Wang, Y., Wu, H., Zhang, J., Gao, Z., Wang, J., Yu, P. S., & Long, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3165153 · Cited by 677
Global Context-Aware Attention LSTM Networks for 3D Action Recognition
Liu, J., Wang, G., Hu, P., Duan, L., & Kot, A. C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.391 · Cited by 682
Going deeper into action recognition: A survey
Herath, S., Harandi, M. T., & Porikli, F. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.01.010 · Cited by 656
Patient Subtyping via Time-Aware LSTM Networks
Baytaş, İ. M., Xiao, C., Zhang, X., Wang, F., Jain, A. K., & Zhou, J. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3097997 · Cited by 682
Luo, W., Yang, B., & Urtasun, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00376 · Cited by 644
Unsupervised Video Summarization with Adversarial LSTM Networks
Mahasseni, B., Lam, M., & Todorović, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.318 · Cited by 630
YouTube-VOS: Sequence-to-Sequence Video Object Segmentation
Xu, N., Yang, L., Fan, Y., Yang, J., Yue, D., Liang, Y., Price, B. L., Cohen, S. D., & Huang, T. S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01228-1_36 · Cited by 641
Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks
Liu, J., Wang, G., Duan, L., Abdiyeva, K., & Kot, A. C. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2785279 · Cited by 612
Spatio-Temporal AutoEncoder for Video Anomaly Detection
Zhao, Y., Deng, B., Shen, C., Liu, Y., Lu, H., & Hua, X. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123451 · Cited by 609
Action Recognition using Visual AttentionPreprint
Sharma, S., Kiros, R., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04119 · Cited by 623
Self-citationFirst Order Motion Model for Image Animation
Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., & Sebe, N. (2020) · arXiv · doi:10.48550/arxiv.2003.00196 · Cited by 585
Dynamic Image Networks for Action Recognition
Bilen, H., Fernando, B., Gavves, E., Vedaldi, A., & Gould, S. J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.331 · Cited by 579
Temporal Relational Ranking for Stock Prediction
Feng, F., He, X., Wang, X., Luo, C., Liu, Y., & Chua, T.-S. (2019) · ACM Transactions on Information Systems · doi:10.1145/3309547 · Cited by 553
Nonlinear Dynamic Soft Sensor Modeling With Supervised Long Short-Term Memory Network
Yuan, X., Li, L., & Wang, Y. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2902129 · Cited by 561
Anticipating Visual Representations from Unlabeled Video
Vondrick, C., Pirsiavash, H., & Torralba, A. B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.18 · Cited by 538
Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates
Liu, J., Shahroudy, A., Xu, D., Kot, A. C., & Wang, G. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2771306 · Cited by 548
Unsupervised Representation Learning by Sorting Sequences
Lee, H.-Y., Huang, J., Singh, M., & Yang, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.79 · Cited by 541
Wang, Y., Zhang, J., Zhu, H., Long, M., Wang, J., & Yu, P. S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00937 · Cited by 536
Vision-based human activity recognition: a survey
Beddiar, D. R., Nini, B., Sabokrou, M., & Hadid, A. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09004-3 · Cited by 536
TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering
Jang, Y., Song, Y., Yu, Y., Kim, Y., & Kim, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.149 · Cited by 527
VideoLSTM convolves, attends and flows for action recognition
Li, Z., Gavrilyuk, K., Gavves, E., Jain, M., & Snoek, C. G. M. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.10.011 · Cited by 492
Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification
Wu, Z., Wang, X., Jiang, Y., Ye, H., & Xue, X. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806222 · Cited by 490
Colorization as a Proxy Task for Visual Understanding
Larsson, G., Maire, M., & Shakhnarovich, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.96 · Cited by 481
Wang, H., & Wang, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.387 · Cited by 477
An Uncertain Future: Forecasting from Static Images Using Variational Autoencoders
Walker, J., Doersch, C., Gupta, A., & Hebert, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46478-7_51 · Cited by 475
Image and Video Compression With Neural Networks: A Review
Ma, S., Zhang, X., Jia, C., Zhao, Z., Wang, S., & Wang, S. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2910119 · Cited by 484
Spatiotemporal Contrastive Video Representation Learning
Qian, R., Meng, T., Gong, B., Yang, M., Wang, H., Belongie, S., & Cui, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00689 · Cited by 489
Learning Correspondence From the Cycle-Consistency of Time
Wang, X., Jabri, A., & Efros, A. A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00267 · Cited by 452
SELF-LABELLING VIA SIMULTANEOUS CLUSTERING AND REPRESENTATION LEARNINGPreprint
Asano, Y. M., Rupprecht, C., & Vedaldi, A. (2019) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.1911.05371 · Cited by 523
A review of Convolutional-Neural-Network-based action recognition
Yao, G., Lei, T., & Zhong, J. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.05.018 · Cited by 437
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
Tong, Z., Song, Y., Wang, J., & Wang, L. (2022) · arXiv · doi:10.48550/arxiv.2203.12602 · Cited by 909
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and TextPreprint
Akbari, H., Yuan, L., Qian, R., Chuang, W.-H., Chang, S., Cui, Y., & Gong, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.11178 · Cited by 469
SimVP: Simpler yet Better Video Prediction
Gao, Z., Tan, C., Wu, L., & Li, S. Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00317 · Cited by 466
Self-Supervised Video Representation Learning with Odd-One-Out Networks
Fernando, B., Bilen, H., Gavves, E., & Gould, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.607 · Cited by 421
Self-Supervised Spatiotemporal Learning via Video Clip Order Prediction
Xu, D., Xiao, J., Zhao, Z., Shao, J., Xie, D., & Zhuang, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01058 · Cited by 426
Context-Aware Synthesis for Video Frame Interpolation
Niklaus, S., & Liu, F. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00183 · Cited by 415
Temporal Generative Adversarial Nets with Singular Value Clipping
Saito, M., Matsumoto, E., & Saito, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.308 · Cited by 422
Hard Negative Mixing for Contrastive LearningPreprint
Kalantidis, Y., Sarıyıldız, M. B., Pion, N., Weinzaepfel, P., & Larlus, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01028 · Cited by 423
RGB-D-based human motion recognition with deep learning: A survey
Wang, P., Li, W., Ogunbona, P., Wan, J., & Escalera, S. (2018) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2018.04.007 · Cited by 412
Every Moment Counts: Dense Detailed Labeling of Actions in Complex Videos
Yeung, S. Y., Russakovsky, O., Jin, N., Andriluka, M., Mori, G., & Fei-Fei, L. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-017-1013-y · Cited by 421
Jiang, Y., Wu, Z., Wang, J., Xue, X., & Chang, S. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2670560 · Cited by 411
Self-Attention ConvLSTM for Spatiotemporal Prediction
Lin, Z., Li, M., Zheng, Z., Cheng, Y., & Yuan, C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6819 · Cited by 399
Detecting Cyber Attacks in Industrial Control Systems Using Convolutional Neural Networks
Kravchik, M., & Shabtai, A. (2018) · Workshop on Cyber-Physical Systems Security and PrivaCy (CPS-SPC) · doi:10.1145/3264888.3264896 · Cited by 421
Deep Learning for Medical Image-Based Cancer Diagnosis
Jiang, X., Hu, Z., Wang, S., & Zhang, Y. (2023) · Cancers · doi:10.3390/cancers15143608 · Cited by 409
Video Representation Learning by Dense Predictive Coding
Han, T., Xie, W., & Zisserman, A. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00186 · Cited by 393
Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks
Xue, T., Wu, J., Bouman, K. L., & Freeman, W. T. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2854726 · Cited by 381
Advances in De Novo Drug Design: From Conventional to Machine Learning Methods
Mouchlis, V. D., Afantitis, A., Serra, A., Fratello, M., Papadiamantis, A. G., Aidinis, V., Lynch, I., Greco, D., & Melagraki, G. (2021) · International Journal of Molecular Sciences · doi:10.3390/ijms22041676 · Cited by 381
Self-Supervised Learning in Remote Sensing: A review
Wang, Y., Albrecht, C. M., Braham, N. A. A., Mou, L., & Zhu, X. X. (2022) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2022.3198244 · Cited by 364
Minimal gated unit for recurrent neural networks
Zhou, G., Wu, J., Zhang, C.-L., & Zhou, Z. (2016) · Machine Intelligence Research · doi:10.1007/s11633-016-1006-2 · Cited by 374
Deep and Confident Prediction for Time Series at Uber
Zhu, L., & Laptev, N. (2017) · 2017 IEEE International Conference on Data Mining Workshops (ICDMW) · doi:10.1109/icdmw.2017.19 · Cited by 362
Compressed Video Action Recognition
Wu, C.-Y., Zaheer, M., Hu, H., Manmatha, R., Smola, A. J., & Krähenbühl, P. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00631 · Cited by 374
Stochastic Adversarial Video PredictionPreprint
Lee, A. X., Zhang, R., Ebert, F., Abbeel, P., Finn, C., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.01523 · Cited by 440
A survey on deep learning in medicine: Why, how and when?
Piccialli, F., Somma, V. D., Giampaolo, F., Cuomo, S., & Fortino, G. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.09.006 · Cited by 369
Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videos
de Morais, R. F. A. B., Le, V., Tran, T., Saha, B., Mansour, M. R., & Venkatesh, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01227 · Cited by 351
Animating Arbitrary Objects via Deep Motion Transfer
Siarohin, A., Lathuilière, S., Tulyakov, S., Ricci, E., & Sebe, N. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00248 · Cited by 358
Data-driven prediction of unsteady flow over a circular cylinder using deep learning
Lee, S., & You, D. (2019) · Journal of Fluid Mechanics · doi:10.1017/jfm.2019.700 · Cited by 346
Dual Motion GAN for Future-Flow Embedded Video Prediction
Liang, X., Lee, L., Dai, W., & Xing, E. P. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.194 · Cited by 349
Learning Video Object Segmentation with Visual Memory
Tokmakov, P., Alahari, K., & Schmid, C. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.480 · Cited by 348
Channel Attention Is All You Need for Video Frame Interpolation
Choi, M., Kim, H., Han, B., Xu, N., & Lee, K. M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6693 · Cited by 350
Learning to detect violent videos using convolutional long short-term memory
Sudhakaran, S., & Lanz, O. (2017) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2017.8078468 · Cited by 346
The Pose Knows: Video Forecasting by Generating Pose Futures
Walker, J., Marino, K., Gupta, A., & Hebert, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.361 · Cited by 341
Endo, Y., Kanamori, Y., & Mitani, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130834 · Cited by 337
HP-GAN: Probabilistic 3D Human Motion Prediction via GAN
Barsoum, E., Kender, J. R., & Liu, Z. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00191 · Cited by 344
Unsupervised Deep Anomaly Detection for Multi-Sensor Time-Series Signals
Zhang, Y., Chen, Y., Wang, J., & Pan, Z. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3102110 · Cited by 338
Disentangling Physical Dynamics From Unknown Factors for Unsupervised Video Prediction
Guen, V. L., & Thome, N. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01149 · Cited by 314
Rank Pooling for Action Recognition
Fernando, B., Gavves, E., Jose Oramas, M., Ghodrati, A., & Tuytelaars, T. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2558148 · Cited by 311
Unsupervised Scalable Representation Learning for Multivariate Time Series
Franceschi, J.-Y., Dieuleveut, A., & Jaggi, M. (2019) · arXiv · doi:10.48550/arxiv.1901.10738 · Cited by 305
Charte, D., Charte, F., García, S., del Jesus, M. J., & Herrera, F. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.12.007 · Cited by 308
Deep learning-based clustering approaches for bioinformatics
Karim, M. R., Beyan, O., Zappa, A., Costa, I. G., Rebholz‐Schuhmann, D., Cochez, M., & Decker, S. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz170 · Cited by 319
Gait Recognition via Disentangled Representation Learning
Zhang, Z., Tran, L., Yin, X. P., Atoum, Y., Liu, X., Wan, J., & Wang, N. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00484 · Cited by 302
Metadata
- Authors
- Srivastava, N., Mansimov, E., & Salakhutdinov, R.
- Affiliations
- University of Toronto (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- Pages
- 843-852
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1502.04681
- OpenAlex
- W2116435618
- Cited by
- 1,861
- References
- 30
- Citations by year
- 2026: 3 · 2025: 56 · 2024: 128 · 2023: 148 · 2022: 183 · 2021: 279 · 2020: 308 · 2019: 286 · 2018: 238 · 2017: 146 · 2016: 62 · 2015: 23
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Machine Learning
- Funders
- Raytheon Company
- Nvidia
- Samsung
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- video representation learning, unsupervised learning, long short-term memory networks, LSTM encoder-decoder, video reconstruction, future frame prediction, perceptual features, feature visualization, human action recognition, transfer learning, few-shot learning, out-of-domain generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
