The DOI matches its DataCite record; 29 of the 30 references it cites were found in a registry or an index.
Unsupervised Learning of Video Representations using LSTMs
Open accessAbstract
We use multilayer Long Short Term Memory (LSTM) networks to learn representations of video sequences. Our model uses an encoder LSTM to map an input sequence into a fixed length representation. This representation is decoded using single or multiple decoder LSTMs to perform different tasks, such as reconstructing the input sequence, or predicting the future sequence. We experiment with two kinds of input sequences - patches of image pixels and high-level representations ("percepts") of video frames extracted using a pretrained convolutional net. We explore different design choices such as whether the decoder LSTMs should condition on the generated output. We analyze the outputs of the model qualitatively to see how well the model can extrapolate the learned video representation into the future and into the past. We try to visualize and interpret the learned features. We stress test the model by running it on longer time scales and on out-of-domain data. We further evaluate the representations by finetuning them for a supervised learning problem - human action recognition on the UCF-101 and HMDB-51 datasets. We show that the representations help improve classification accuracy, especially when there are only a few training examples. Even models pretrained on unrelated datasets (300 hours of YouTube videos) can help action recognition performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This is the first time that motion prediction with cross-dimensional learning has been used for self-supervised learning in the context of HAR.”
From Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion (2021) · cites this paper as [23]
In the citing paper
“While motion prediction has been used in the field of computer vision for self-supervised learning [23, 26], our model is novel in that the cross-dimensional aspect results in more effective learning.”
From Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion (2021) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,313
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Research works printed but not linked: 1
- Hochreiter, Sepp and Schmidhuber, J¨ urgen. Long short-term memory. Neural Computation , 9(8):1735-1780, 1997.
Cited by
Sorted by citations
Items 801–900 of 1,005
Deep Multi-Sensory Object Category Recognition Using Interactive Behavioral Exploration
Tatiya, G., & Sinapov, J. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8794095 · Cited by 24
Dual attention convolutional network for action recognition
Li, X., Xie, M., Zhang, Y., Ding, G., & Tong, W. (2020) · IET Image Processing · doi:10.1049/iet-ipr.2019.0963 · Cited by 26
LAC: LSTM AUTOENCODER with Community for Insider Threat Detection
Paul, S., & Mishra, S. (2020) · International Conference Big Data Research · doi:10.1145/3445945.3445958 · Cited by 24
Reciprocal Twin Networks for Pedestrian Motion Learning and Future Path Prediction
Sun, H., Zhao, Z., Yin, Z., & He, Z. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3076078 · Cited by 25
A Review of Artificial Intelligence Technologies for Early Prediction of Alzheimer's DiseasePreprint
Yang, K., & Mohammed, E. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.01781 · Cited by 24
Cross-Agent Action Recognition
Wang, H., & Wang, L. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2746092 · Cited by 23
Deep Learning Propagation Models over Irregular Terrain
Ribero, M., Heath, R. W., Vikalo, H., Chizhik, D., & Valenzuela, R. A. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682491 · Cited by 23
UAV Anomaly Detection with Distributed Artificial Intelligence Based on LSTM-AE and AE
Bae, G., & Joe, I. (2019) · Lecture notes in electrical engineering · doi:10.1007/978-981-32-9244-4_43 · Cited by 24
Point-to-Point Video Generation
Wang, T.-H. J., Cheng, Y.-C., Lin, C. H., Chen, H.-T., & Sun, M. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01059 · Cited by 23
Hierarchically Decoupled Spatial-Temporal Contrast for Self-supervised Video Representation Learning
Zhang, Z., & Crandall, D. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00105 · Cited by 23
AttEF: Convolutional LSTM Encoder-Forecaster with Attention Module for Precipitation Nowcasting
Fang, W., Pang, L., Yi, W., & Sheng, V. S. (2021) · Intelligent Automation & Soft Computing · doi:10.32604/iasc.2021.016589 · Cited by 23
CloudCast: A Satellite-Based Dataset and Baseline for Forecasting Clouds
Nielsen, A. H., Iosifidis, A., & Karstoft, H. (2021) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2021.3062936 · Cited by 25
Adaptive Spatial Location With Balanced Loss for Video Captioning
Li, L., Zhang, Y., Tang, S., Xie, L., Li, X., & Tian, Q. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3045735 · Cited by 23
Representation Learning with Video Deep InfoMaxPreprint
Hjelm, R. D., & Bachman, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.13278 · Cited by 28
Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion
Rahimi Taghanaki, S., Rainbow, M. J., & Etemad, A. (2021) · 2021 International Symposium on Wearable Computers · doi:10.1145/3460421.3480417 · Cited by 22
“While motion prediction has been used in the field of computer vision for self-supervised learning [23, 26], our model is novel in that the cross-dimensional aspect results in more effective learning.”
MethodWu, S., Wang, H., & Luo, K. H. (2023) · Energy and AI · doi:10.1016/j.egyai.2023.100333 · Cited by 22
Unsupervised Learning of Video Representations via Dense Trajectory Clustering
Tokmakov, P., Hebert, M., & Schmid, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-66096-3_28 · Cited by 22
Video frame interpolation via optical flow estimation with image inpainting
Liu, X., Liu, H., & Lin, Y. (2020) · International Journal of Intelligent Systems · doi:10.1002/int.22285 · Cited by 22
Autoregressive Predictive Coding: A Comprehensive Study
Yang, G.-P., Yeh, S.-L., Chung, Y.-A., Glass, J. R., & Tang, H. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3203608 · Cited by 24
Multimodality Multi-Lead ECG Arrhythmia Classification using Self-Supervised Learning
Phan, T., Le, D., Patel, B. D., Adjeroh, D., Wu, J., Jensen, M. Ø., & Le, N. (2022) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi56158.2022.9926925 · Cited by 22
A Universal Event-Based Plug-In Module for Visual Object Tracking in Degraded Conditions
Zhang, J., Dong, B., Fu, Y., Wang, Y., Wei, X., Yin, B., & Yang, X. S. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01959-8 · Cited by 22
Ihsan, M. A., Eram, A. F., Nahar, L., & Kadir, M. A. (2024) · IEEE Access · doi:10.1109/access.2024.3370684 · Cited by 22
Self-Paced Video Data Augmentation by Generative Adversarial Networks with Insufficient Samples
Zhang, Y., Jia, G., Chen, L., Zhang, M., & Yong, J. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3414003 · Cited by 22
Learning Scene Dynamics from Point Cloud Sequences
He, P. L., Emami, P., Ranka, S., & Rangarajan, A. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-021-01551-y · Cited by 24
PDE-Driven Spatiotemporal DisentanglementPreprint
Donà, J., Franceschi, J.-Y., Lamprier, S., & Gallinari, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.01352 · Cited by 25
Siamese Networks for Weakly Supervised Human Activity Recognition
Sheng, T., & Huber, M. (2019) · 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC) · doi:10.1109/smc.2019.8914045 · Cited by 21
Unsupervised End-to-End Deep Model for Newborn and Infant Activity Recognition
Jun, K., & Choi, S. (2020) · Sensors · doi:10.3390/s20226467 · Cited by 21
Stock trend prediction based on industry relationships driven hypergraph attention networks
Han, H., Xie, L., Chen, S., & Xu, H. (2023) · Applied Intelligence · doi:10.1007/s10489-023-05035-z · Cited by 23
Investigation of Pig Activity Based on Video Data and Semi-Supervised Neural Networks
Wutke, M., Schmitt, A. O., Traulsen, I., & Gültaş, M. (2020) · AgriEngineering · doi:10.3390/agriengineering2040039 · Cited by 21
Modelling Neighbor Relation in Joint Space-Time Graph for Video Correspondence Learning
Zhao, Z., Jin, Y., & Heng, P. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00981 · Cited by 21
Ship Pitch Prediction Based on Bi-ConvLSTM-CA Model
Fu, H., Gu, Z., & Wang, Y. (2022) · Journal of Marine Science and Engineering · doi:10.3390/jmse10070840 · Cited by 22
Discriminatively Learned Hierarchical Rank Pooling Networks
Fernando, B., & Gould, S. J. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-017-1030-x · Cited by 23
Learning Object-Centric Transformation for Video Prediction
Chen, X., Wang, W., Wang, J., & Li, W. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123349 · Cited by 21
Bi-direction hierarchical LSTM with spatial-temporal attention for action recognition
Yang, H., Zhang, J., Li, S., & Luo, T. (2018) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-18209 · Cited by 21
A Novel Dynamic Model Capturing Spatial and Temporal Patterns for Facial Expression Analysis
Wang, S., Zheng, Z., Yin, S. W., Yang, J., & Ji, Q. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2911937 · Cited by 22
Driver Fatigue Detection using Recurrent Neural Networks
Ed-doughmi, Y., & Idrissi, N. (2019) · International Conferences on Networking, Information Systems & Security · doi:10.1145/3320326.3320376 · Cited by 22
Sequence-to-Segments Networks for Detecting Segments in Videos
Wei, Z., Wang, B., Hoai, M., Zhang, J., Shen, X., Lin, Z., Měch, R., & Samaras, D. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2940225 · Cited by 21
Dimensionality reduction to maximize prediction generalization capability
Isomura, T., & Toyoizumi, T. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00306-1 · Cited by 22
DNA Privacy: Analyzing Malicious DNA Sequences Using Deep Neural Networks
Bae, H., Min, S., Choi, H.-S., & Yoon, S. (2020) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2020.3017191 · Cited by 21
Dynamic Motion Estimation and Evolution Video Prediction Network
Kim, N., & Kang, J. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3035281 · Cited by 21
Learning to Anticipate Future with Dynamic Context Removal
Xu, X., Li, Y., & Lu, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01240 · Cited by 22
From Here to There: Video Inbetweening Using Direct 3D ConvolutionsPreprint
Li, Y., Roblek, D., & Tagliasacchi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10240 · Cited by 21
Wang, P., Zhang, X., Chen, Y., & Cao, Z. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.104013 · Cited by 21
Unsupervised Learning of Face Representations
Datta, S., Sharma, G., & Jawahar, C. V. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00029 · Cited by 20
Lechner, M., Hasani, R., Rus, D. L., & Grosu, R. (2020) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra40945.2020.9196608 · Cited by 20
Deep learning based HEVC in-loop filter and noise reduction
Kuanar, S. P., Rao, K. R., Conly, C., & Gorey, N. (2021) · Signal Processing Image Communication · doi:10.1016/j.image.2021.116409 · Cited by 20
Predicting humans future motion trajectories in video streams using generative adversarial network
Hassan, A. M., Khan, M. U. G., Iqbal, R., Riaz, O., Bashir, A. K., & Tariq, U. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11457-z · Cited by 21
Traffic Anomaly Prediction Based on Joint Static-Dynamic Spatio-Temporal Evolutionary Learning
Liu, X., Zhang, Z., Lyu, L., Zhang, Z., Xiao, S., Shen, C. Q., & Yu, P. S. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3150272 · Cited by 21
Liu, J., Rangwala, M., Ahluwalia, K. S., Ghajar, S., Dhami, H., Tokekar, P., Tracy, B. F., & Williams, R. K. (2022) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2022.3211873 · Cited by 22
STUNNER: Radar Echo Extrapolation Model Based on Spatiotemporal Fusion Neural Network
Fang, W., Pang, L., Sheng, V. S., & Wang, Q. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3268187 · Cited by 20
Fast Fourier Inception Networks for Occluded Video Prediction
Li, P., Zhang, C., & Xu, X. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3310330 · Cited by 20
Hierarchical temporal prediction captures motion processing along the visual pathway
Singer, Y., Taylor, L., Willmore, B. D. B., King, A. J., & Harper, N. S. (2023) · eLife · doi:10.7554/elife.52599 · Cited by 25
Unsupervised Hierarchical Dynamic Parsing and Encoding for Action Recognition
Su, B., Zhou, J., Ding, X., & Wu, Y. N. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2745212 · Cited by 20
Jointly Modeling Static Visual Appearance and Temporal Pattern for Unsupervised Video Hashing
Li, C., Yang, Y., Cao, J., & Huang, Z. H. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3133030 · Cited by 20
Hierarchical and Spatio-Temporal Sparse Representation for Human Action Recognition
Tian, Y., Kong, Y., Ruan, Q., An, G., & Fu, Y. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2788196 · Cited by 20
Unsupervised Learning for Surgical Motion by Learning to Predict the Future
DiPietro, R. S., & Hager, G. D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00937-3_33 · Cited by 20
Structure-Constrained Motion Sequence Generation
Yan, Y., Ni, B., Zhang, W., Xu, J., & Yang, X. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2885235 · Cited by 20
Sky Image forecasting with Generative Adversarial Networks for cloud coverage prediction
Andrianakos, G., Tsourounis, D., Oikonomou, S., Kastaniotis, D., Economou, G., & Kazantzidis, A. (2019) · International Conference on Information, Intelligence, Systems & Applications (IISA) · doi:10.1109/iisa.2019.8900774 · Cited by 20
Deep Unsupervised Binary Coding Networks for Multivariate Time Series Retrieval
Zhu, D., Song, D., Chen, Y., Lumezanu, C., Cheng, W., Zong, B., Ni, J., Mizoguchi, T., Yang, T., & Chen, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i02.5497 · Cited by 20
Long-Term Video Prediction via Criticization and Retrospection
Chen, X., Xu, C., Yang, X., & Tao, D. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2998297 · Cited by 20
Understanding Road Layout From Videos as a Whole
Liu, B., Zhuang, B., Schulter, S., Ji, P., & Chandraker, M. K. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00447 · Cited by 20
Self-Supervised Deep TripleNet for Video Object Segmentation
Xu, K., Wen, L., Li, G., & Huang, Q. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3026913 · Cited by 20
SinGAN-GIF: Learning a Generative Video Model from a Single GIF
Arora, R., & Lee, Y. J. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00135 · Cited by 20
Phan, H. T. H., Kumar, A., Feng, D. D., FULHAM, M. J., & Kim, J. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2885572 · Cited by 21
Mobile Video Action RecognitionPreprint
Huo, Y., Xu, X., Lu, Y., Niu, Y., Lu, Z., & Wen, J.-R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.10155 · Cited by 20
Precipitation Nowcasting with Star-Bridge NetworksPreprint
Cao, Y., Li, Q., Shan, H., Huang, Z., Chen, L., Ma, L., & Zhang, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.08069 · Cited by 20
Transformers predicting the future. Applying attention in next-frame and time series forecastingPreprint
Cholakov, R., & Kolev, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.08224 · Cited by 22
Video Prediction via Selective Sampling
Xu, J., Ni, B., & Yang, X. (2018) · neural information processing systems · Cited by 19
Toward Automatically Labeling Situations in Soccer
Fassmeyer, D., Anzer, G., Bauer, P., & Brefeld, U. (2021) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2021.725431 · Cited by 21
SS-3DCAPSNET: Self-Supervised 3d Capsule Networks for Medical Segmentation on Less Labeled Data
Tran, M., Ly, L., Hua, B., & Le, N. (2022) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi52829.2022.9761627 · Cited by 19
Multi-Frame Pyramid Refinement Network for Video Frame Interpolation
Zhang, H., Wang, R., & Zhao, Y. (2019) · IEEE Access · doi:10.1109/access.2019.2940510 · Cited by 19
Multi-stream dynamic video Summarization
Elfeki, M., Wang, L., & Borji, A. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00026 · Cited by 21
Multimodal Image Synthesis with Conditional Implicit Maximum Likelihood Estimation
Li, K., Peng, S., Zhang, T., & Malik, J. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01325-y · Cited by 22
Large Scale Autonomous Driving Scenarios Clustering with Self-supervised Feature Extraction
Zhao, J., Fang, J., Ye, Z., & Zhang, L. (2021) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv48863.2021.9575644 · Cited by 19
Unsupervised Representation Learning for Proteochemometric Modeling
Kim, P. T., Winter, R., & Clevert, D.-A. (2021) · International Journal of Molecular Sciences · doi:10.3390/ijms222312882 · Cited by 19
Self-Supervised Motion Perception for Spatiotemporal Representation Learning
Liu, C., Yao, Y., Luo, D., Zhou, Y., & Ye, Q. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3160860 · Cited by 19
Pedestrian Path Forecasting in Crowd
Li, Y. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123287 · Cited by 19
Learning Unsupervised Visual Grounding Through Semantic Self-Supervision
Javed, S. A., Saxena, S., & Gandhi, V. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/112 · Cited by 23
Recurrent Auto-Encoder Model for Large-Scale Industrial Sensor Signal Analysis
Wong, T., & Luo, Z. (2018) · Communications in computer and information science · doi:10.1007/978-3-319-98204-5_17 · Cited by 20
Empirical Activation Function Effects on Unsupervised Convolutional LSTM Learning
Elsayed, N., Maida, A. S., & Bayoumi, M. (2018) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai.2018.00060 · Cited by 20
TwoStreamVAN: Improving Motion Modeling in Video Generation
Sun, X., Xu, H., & Saenko, K. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093557 · Cited by 19
Revisiting Hierarchical Approach for Persistent Long-Term Video PredictionPreprint
Lee, W., Jung, W., Zhang, H., Chen, T., Koh, J. Y., Huang, T., Yoon, H.-S., Lee, H., & Hong, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.06697 · Cited by 22
Clockwork Variational AutoencodersPreprint
Saxena, V., Ba, J. L., & Hafner, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.09532 · Cited by 27
Hou, Y., Wang, S., Wang, P., Gao, Z., & Li, W. (2017) · IEEE Access · doi:10.1109/access.2017.2782258 · Cited by 18
An efficient and sparse approach for large scale human action recognition in videos
Beaudry, C., Péteri, R., & Mascarilla, L. (2016) · Machine Vision and Applications · doi:10.1007/s00138-016-0760-z · Cited by 18
An Intelligent Building Occupancy Detection System Based on Sparse Auto-Encoder
Liu, Z., Zhang, J., & Geng, L. (2017) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw.2017.10 · Cited by 18
Video2vec: Learning semantic spatio-temporal embeddings for video representation
Hu, S.-H., Li, Y., & Li, B. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7899735 · Cited by 18
From Thumbnails to Summaries-A Single Deep Neural Network to Rule Them All
Gu, H., & Swaminathan, V. (2018) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2018.8486533 · Cited by 18
Learning to Reproduce Visually Similar Movements by Minimizing Event-Based Prediction Error
Kaiser, J., Melbaum, S., Tieck, J. C. V., Roennau, A., Butz, M. V., & Dillmann, R. (2018) · IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob) · doi:10.1109/biorob.2018.8487959 · Cited by 18
Reduced-Gate Convolutional LSTM Architecture for Next-Frame Video Prediction Using Predictive Coding
Elsayed, N., Maida, A. S., & Bayoumi, M. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852480 · Cited by 20
Uni-and-Bi-Directional Video Prediction via Learning Object-Centric Transformation
Chen, X., & Wang, W. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2946475 · Cited by 18
PredNet and Predictive Coding: A Critical Review
Rane, R. P., Szügyi, E., Saxena, V., Ofner, A., & Stober, S. (2020) · International Conference on Multimedia Retrieval · doi:10.1145/3372278.3390694 · Cited by 19
High-Dimensional Similarity Search with Quantum-Assisted Variational Autoencoder
Gao, N., Wilson, M., Vandal, T., Vinci, W., Nemani, R., & Rieffel, E. G. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403138 · Cited by 18
A Novel Signal Detection Scheme Based on Adaptive Ensemble Deep Learning Algorithm in SC-FDE Systems
Qiao, Y., Li, J., He, B., Li, W., & Xin, T. (2020) · IEEE Access · doi:10.1109/access.2020.3006265 · Cited by 18
Lightweight Action Recognition in Compressed Videos
Huo, Y., Xu, X., Lu, Y., Niu, Y., Ding, M., Lu, Z., Xiang, T., & Wen, J.-R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-66096-3_24 · Cited by 18
A Reversible-Logic Based Architecture for Long Short-Term Memory (LSTM) Network
Khalil, K., Dey, B., Kumar, A., & Bayoumi, M. (2021) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas51556.2021.9401395 · Cited by 19
CoCon: Cooperative-Contrastive Learning
Rai, N., Adeli, E., Lee, K.-H., Gaidon, A., & Niebles, J. C. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00377 · Cited by 18
Attention Augmented ConvLSTM for Environment Prediction
Lange, B., Itkina, M., & Kochenderfer, M. J. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636386 · Cited by 18
Dubatovka, A., & Buhmann, J. M. (2022) · BME Frontiers · doi:10.34133/2022/9813062 · Cited by 22
Temporal Representation Learning on Monocular Videos for 3D Human Pose Estimation
Honari, S., Constantin, V., Rhodin, H., Salzmann, M., & Fua, P. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3215307 · Cited by 18
Metadata
- Authors
- Srivastava, N., Mansimov, E., & Salakhutdinov, R.
- Affiliations
- University of Toronto (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- Pages
- 843-852
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1502.04681
- OpenAlex
- W2116435618
- Cited by
- 1,861
- References
- 30
- Citations by year
- 2026: 3 · 2025: 56 · 2024: 128 · 2023: 148 · 2022: 183 · 2021: 279 · 2020: 308 · 2019: 286 · 2018: 238 · 2017: 146 · 2016: 62 · 2015: 23
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Machine Learning
- Funders
- Raytheon Company
- Nvidia
- Samsung
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- video representation learning, unsupervised learning, long short-term memory networks, LSTM encoder-decoder, video reconstruction, future frame prediction, perceptual features, feature visualization, human action recognition, transfer learning, few-shot learning, out-of-domain generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
