The DOI matches its DataCite record; 29 of the 30 references it cites were found in a registry or an index.
Unsupervised Learning of Video Representations using LSTMs
Open accessAbstract
We use multilayer Long Short Term Memory (LSTM) networks to learn representations of video sequences. Our model uses an encoder LSTM to map an input sequence into a fixed length representation. This representation is decoded using single or multiple decoder LSTMs to perform different tasks, such as reconstructing the input sequence, or predicting the future sequence. We experiment with two kinds of input sequences - patches of image pixels and high-level representations ("percepts") of video frames extracted using a pretrained convolutional net. We explore different design choices such as whether the decoder LSTMs should condition on the generated output. We analyze the outputs of the model qualitatively to see how well the model can extrapolate the learned video representation into the future and into the past. We try to visualize and interpret the learned features. We stress test the model by running it on longer time scales and on out-of-domain data. We further evaluate the representations by finetuning them for a supervised learning problem - human action recognition on the UCF-101 and HMDB-51 datasets. We show that the representations help improve classification accuracy, especially when there are only a few training examples. Even models pretrained on unrelated datasets (300 hours of YouTube videos) can help action recognition performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This is the first time that motion prediction with cross-dimensional learning has been used for self-supervised learning in the context of HAR.”
From Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion (2021) · cites this paper as [23]
In the citing paper
“While motion prediction has been used in the field of computer vision for self-supervised learning [23, 26], our model is novel in that the cross-dimensional aspect results in more effective learning.”
From Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion (2021) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,313
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Research works printed but not linked: 1
- Hochreiter, Sepp and Schmidhuber, J¨ urgen. Long short-term memory. Neural Computation , 9(8):1735-1780, 1997.
Cited by
Sorted by citations
Items 201–300 of 1,005
A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications
Wu, F. L., Wang, Q., Bian, J., Ding, N., Lu, F., Cheng, J. S., Dou, D., & Xiong, H. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3232034 · Cited by 167
Temporal Pyramid Pooling-Based Convolutional Neural Network for Action Recognition
Wang, P., Cao, Y., Shen, C., Liu, L., & Shen, H. T. (2016) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2016.2576761 · Cited by 152
Learning effective Gait features using LSTM
Feng, Y., Li, Y., & Luo, J. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7899654 · Cited by 154
Correlational Convolutional LSTM for human action recognition
Majd, M., & Safabakhsh, R. (2019) · Neurocomputing · doi:10.1016/j.neucom.2018.10.095 · Cited by 151
Asynchronous Temporal Fields for Action Recognition
Sigurdsson, G. A., Divvala, S., Farhadi, A., & Gupta, A. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.599 · Cited by 150
Ali, Z., Kim, J., Li, X., & Cong, G. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/519 · Cited by 150
Deep Uncertainty Quantification
Wang, B., Jie Lu, B., Yan, Z., Luo, H., Li, T., Zheng, Y., & Zhang, G. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330704 · Cited by 149
Unsupervised Deep Video Hashing via Balanced Code for Large-Scale Video Retrieval
Wu, G., Han, J., Guo, Y., Liu, L., Ding, G., Ni, Q., & Shao, L. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2882155 · Cited by 148
Grounded Human-Object Interaction Hotspots From Video
Nagarajan, T., Feichtenhofer, C., & Grauman, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00878 · Cited by 147
Li, W., & Hsu, C.-Y. (2022) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi11070385 · Cited by 147
MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics
Yan, X., Rastogi, A., Villegas, R., Sunkavalli, K., Shechtman, E., Hadap, S., Yumer, E., & Lee, H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01228-1_17 · Cited by 153
Sequential Attend, Infer, Repeat: Generative Modelling of Moving ObjectsPreprint
Kosiorek, A. R., Kim, H., Posner, I., & Teh, Y. W. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.01794 · Cited by 158
Actor and Observer: Joint Modeling of First and Third-Person Videos
Sigurdsson, G. A., Gupta, A., Schmid, C., Farhadi, A., & Alahari, K. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00772 · Cited by 146
Yang, S., Chen, D., Li, S., & Wang, W. (2020) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2020.137117 · Cited by 151
Representation Learning of Temporal Dynamics for Skeleton-Based Action Recognition
Du, Y., Fu, Y., & Wang, L. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2552404 · Cited by 151
ContextVP: Fully Context-Aware Video Prediction
Byeon, W., Wang, Q., Srivastava, R. K., & Koumoutsakos, P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_46 · Cited by 152
Prediction of 3-D Ocean Temperature by Multilayer Convolutional LSTM
Zhang, K., Geng, X., & Yan, X. (2020) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2019.2947170 · Cited by 147
Video Processing Using Deep Learning Techniques: A Systematic Literature Review
Sharma, V., Gupta, M., Kumar, A., & Mishra, D. (2021) · IEEE Access · doi:10.1109/access.2021.3118541 · Cited by 153
Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation
Zhang, D. J., Wu, J. Z., Liu, J., Zhao, R., Ran, L., Gu, Y., Gao, D., & Shou, M. Z. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02271-9 · Cited by 275
Folded Recurrent Neural Networks for Future Video Prediction
Oliu, M., Selva, J., & Escalera, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_44 · Cited by 141
Improved Conditional VRNNs for Video Prediction
Castrejón, L., Ballas, N., & Courville, A. C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00770 · Cited by 147
Recurrent Neural Networks: An Embedded Computing Perspective
Rezk, N. M., Purnaprajna, M., Nordström, T., & Ul-Abdin, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2982416 · Cited by 150
Feature Extraction Using an RNN Autoencoder for Skeleton-Based Abnormal Gait Recognition
Jun, K., Lee, D.-W., Lee, K., Lee, S., & Kim, M. S. (2020) · IEEE Access · doi:10.1109/access.2020.2967845 · Cited by 145
Learning for Video Compression With Recurrent Auto-Encoder and Recurrent Probability Model
Yang, R., Mentzer, F., Gool, L. V., & Timofte, R. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3043590 · Cited by 148
Deep Learning Application in Plant Stress Imaging: A Review
Gao, Z., Luo, Z., Zhang, W., Lv, Z., & Xu, Y. (2020) · AgriEngineering · doi:10.3390/agriengineering2030029 · Cited by 140
Pose Guided Human Video Generation
Yang, C., Wang, Z., Zhu, X., Huang, C., Shi, J., & Lin, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_13 · Cited by 142
SDC-Net: Video Prediction Using Spatially-Displaced Convolution
Reda, F. A., Liu, G., Shih, K. J., Kirby, R. M., Barker, J., Tarjan, D., Tao, A., & Catanzaro, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_44 · Cited by 138
Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models
Ge, S., Nah, S., Liu, G., Poon, T., Tao, A., Catanzaro, B., Jacobs, D. W., Huang, J., Liu, M.-Y., & Balaji, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02096 · Cited by 176
Deep learning for video classification and captioning
Wu, Z., Yao, T., Fu, Y., & Jiang, Y.-G. (2017) · doi:10.1145/3122865.3122867 · Cited by 149
VideoFlow: A Conditional Flow-Based Model for Stochastic Video GenerationPreprint
Kumar, M. N. A., Babaeizadeh, M., Erhan, D., Finn, C., Levine, S., Dinh, L., & Kingma, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.01434 · Cited by 172
Learning Temporal Transformations from Time-Lapse Videos
Zhou, Y., & Berg, T. L. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_16 · Cited by 137
Unified Spatio-Temporal Attention Networks for Action Recognition in Videos
Li, D., Yao, T., Duan, L., Mei, T., & Rui, Y. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2862341 · Cited by 136
Recurrent Mixture Density Network for Spatiotemporal Visual AttentionPreprint
Bazzani, L., Larochelle, H., & Torresani, L. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.08199 · Cited by 139
Deep Spatial–Spectral Representation Learning for Hyperspectral Image Denoising
Dong, W., Wang, H., Wu, F., Shi, G., & Li, X. (2019) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2019.2911881 · Cited by 134
Evolving Losses for Unsupervised Video Representation Learning
Piergiovanni, A., Angelova, A., & Ryoo, M. S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00021 · Cited by 140
A CNN-RNN Combined Structure for Real-World Violence Detection in Surveillance Cameras
Vosta, S., & Yow, K. (2022) · Applied Sciences · doi:10.3390/app12031021 · Cited by 138
Action Recognition by Learning Deep Multi-Granular Spatio-Temporal Video Representation
Li, Q., Qiu, Z., Yao, T., Mei, T., Rui, Y., & Luo, J. (2016) · International Conference on Multimedia Retrieval · doi:10.1145/2911996.2912001 · Cited by 133
A Generative Adversarial Gated Recurrent Unit Model for Precipitation Nowcasting
Tian, L., Li, X., Ye, Y., Xie, P., & Li, Y. (2019) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2019.2926776 · Cited by 137
Chen, C., Zhu, W., Steibel, J. P., Siegford, J. M., Wurtz, K. E., Han, J., & Norton, T. (2019) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2019.105166 · Cited by 132
Classification of drinking and drinker-playing in pigs by a video-based deep learning method
Chen, C., Zhu, W., Steibel, J. P., Siegford, J. M., Han, J., & Norton, T. (2020) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2020.05.010 · Cited by 136
Recurrent Environment SimulatorsPreprint
Chiappa, S., Racanière, S., Wierstra, D., & Mohamed, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.02254 · Cited by 154
An intelligent financial portfolio trading strategy using deep Q-learning
Park, H., Sim, M. K., & Choi, D. G. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113573 · Cited by 136
Two-Stream Collaborative Learning With Spatial-Temporal Attention for Video Classification
Peng, Y., Zhao, Y., & Zhang, J. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2808685 · Cited by 131
Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
Mogadala, A., Kalimuthu, M., & Klakow, D. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11688 · Cited by 146
Modeling Multimodal Clues in a Hybrid Deep Learning Framework for Video Classification
Jiang, Y., Wu, Z., Tang, J., Li, Z., Xue, X., & Chang, S. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2823900 · Cited by 130
Multimodal Keyless Attention Fusion for Video Classification
Long, X., Gan, C., de Melo, G., Liu, X., Li, Y., Li, F., & Wen, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12319 · Cited by 126
Han, Y., Wang, S., Ren, Y., Wang, C., Gao, P., & Chen, G. (2019) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi8060243 · Cited by 126
Augmenting physical models with deep networks for complex dynamics forecasting*
Yin, Y., Guen, V. L., Donà, J., de Bézenac, E., Ayed, I., Thome, N., & Gallinari, P. (2021) · Journal of Statistical Mechanics Theory and Experiment · doi:10.1088/1742-5468/ac3ae5 · Cited by 125
Automatic Modulation Classification Based on Deep Learning for Unmanned Aerial Vehicles
Zhang, D., Ding, W., Zhang, B., Xie, C., Li, H., Han, J., & Li, H. (2018) · Sensors · doi:10.3390/s18030924 · Cited by 126
Geometry Guided Convolutional Neural Networks for Self-Supervised Video Representation Learning
Gan, C., Gong, B., Liu, K., Su, H., & Guibas, L. J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00586 · Cited by 124
Pan, Y., Qiu, Z., Yao, T., Li, H., & Mei, T. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3127905 · Cited by 128
Cycle-SUM: Cycle-Consistent Adversarial LSTM Networks for Unsupervised Video Summarization
Yuan, L., Tay, F. E., Li, P., Zhou, L. P., & Feng, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019143 · Cited by 125
Self-supervised Video Transformer
Ranasinghe, K., Naseer, M., Khan, S. H., Khan, F. S., & Ryoo, M. S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00289 · Cited by 128
Sharma, A., Liu, X., & Yang, X. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.05.019 · Cited by 125
Flow-Grounded Spatial-Temporal Video Prediction from Still Images
Li, Y., Chen, F., Yang, J., Wang, Z., Lu, X., & Yang, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_37 · Cited by 122
Temporal Localization of Fine-Grained Actions in Videos by Domain Transfer from Web Images
Sun, C., Shetty, S., Sukthankar, R., & Nevatia, R. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806226 · Cited by 124
Learning to Segment Moving Objects
Tokmakov, P., Schmid, C., & Alahari, K. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1122-2 · Cited by 126
BayesSim: Adaptive Domain Randomization Via Probabilistic Inference for Robotics Simulators
Ramos, F. T., Possas, R., & Fox, D. (2019) · Robotics: Science and Systems · doi:10.15607/rss.2019.xv.029 · Cited by 166
Deep Analysis of CNN-based Spatio-temporal Representations for Action Recognition
Chen, C.-F. R., Panda, R., Ramakrishnan, K., Feris, R., Cohn, J., Oliva, A., & Fan, Q. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00610 · Cited by 125
Forecasting Human Dynamics from Static Images
Chao, Y.-W., Yang, J., Price, B. L., Cohen, S. D., & Deng, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.388 · Cited by 119
Recurrent Tubelet Proposal and Recognition Networks for Action Detection
Li, D., Qiu, Z., Dai, Q., Yao, T., & Mei, T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_19 · Cited by 119
Real-Time System for Driver Fatigue Detection Based on a Recurrent Neuronal Network
Ed-doughmi, Y., Idrissi, N., & Hbali, Y. (2020) · Journal of Imaging · doi:10.3390/jimaging6030008 · Cited by 100
Multimodal Federated Learning on IoT Data
Zhao, Y., Barnaghi, P. M., & Haddadi, H. (2022) · IEEE/ACM International Conference on Internet-of-Things Design and Implementation (IoTDI) · doi:10.1109/iotdi54339.2022.00011 · Cited by 124
Imagination improves Multimodal TranslationPreprint
Elliott, D., & Kádár, Á. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.04350 · Cited by 126
Video Prediction Recalling Long-term Motion Context via Memory Alignment Learning
Lee, S., Kim, H. G., Hwi Choi, D., Kim, H.-I., & Ro, Y. M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00307 · Cited by 129
A neural network trained for prediction mimics diverse features of biological neurons and perception
Lotter, W., Kreiman, G., & Cox, D. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0170-9 · Cited by 128
Abnormal Event Detection From Videos Using a Two-Stream Recurrent Variational Autoencoder
Yan, S., Smith, J. S., Lu, W., & Zhang, B. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2018.2883368 · Cited by 120
Saito, M., Saito, S., Koyama, M., & Kobayashi, S. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-020-01333-y · Cited by 116
Multilayer and Multimodal Fusion of Deep Neural Networks for Video Classification
Yang, X., Molchanov, P. A., & Kautz, J. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2964297 · Cited by 115
Beyond Expectation: Deep Joint Mean and Quantile Regression for Spatiotemporal Problems
Rodrigues, F., & Pereira, F. C. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2966745 · Cited by 122
Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundary
Asai, M., & Fukunaga, A. S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12077 · Cited by 120
Skeleton Cloud Colorization for Unsupervised 3D Action Representation Learning
Yang, S., Liu, J., Lu, S., Er, M. H., & Kot, A. C. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01317 · Cited by 115
A critical analysis of self-supervision, or what we can learn from a single image
Asano, Y. M., Rupprecht, C., & Vedaldi, A. (2019) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.48550/arxiv.1904.13132 · Cited by 129
Unsupervised Learning and Segmentation of Complex Activities from Video
Şener, F., & Yao, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00873 · Cited by 121
Computer network intrusion detection using sequential LSTM Neural Networks autoencoders
Mirza, A. H., & Coşan, S. (2018) · Signal Processing and Communications Applications Conference (SIU) · doi:10.1109/siu.2018.8404689 · Cited by 114
Human Action Monitoring for Healthcare Based on Deep Learning
Gao, Y., Xiang, X., Xiong, N. N., Huang, B., Lee, H. J., Alrifai, R., Jiang, X., & Fang, Z. (2018) · IEEE Access · doi:10.1109/access.2018.2869790 · Cited by 114
Towards Corner Case Detection for Autonomous Driving
Bolte, J.-A., Bär, A., Lipinski, D., & Fingscheidt, T. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8813817 · Cited by 114
Multi-Task Video Captioning with Video and Entailment Generation
Pasunuru, R., & Bansal, M. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1117 · Cited by 118
Video Jigsaw: Unsupervised Learning of Spatiotemporal Context for Video Action Recognition
Ahsan, U., Madhok, R., & Essa, I. A. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00025 · Cited by 113
Self-Supervised Learning of Point Clouds via Orientation Estimation
Poursaeed, O., Jiang, T., Qiao, H., Xu, N., & Kim, V. G. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00112 · Cited by 113
Jin, B., Hu, Y. H., Tang, Q., Niu, J., Shi, Z., Han, Y., & Li, X. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00461 · Cited by 116
Learning by Association — A Versatile Semi-Supervised Training Method for Neural Networks
Haeusser, P., Mordvintsev, A., & Cremers, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.74 · Cited by 114
DynamoNet: Dynamic Action and Motion Network
Diba, A., Sharma, V., Gool, L. V., & Stiefelhagen, R. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00629 · Cited by 111
A Lightweight Spatial and Temporal Multi-Feature Fusion Network for Defect Detection
Hu, B., Gao, B., Woo, W. L., Ruan, L., Jin, J., Yang, Y., & Yu, Y. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3036770 · Cited by 111
iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysis
Liu, X., Shi, H., Chen, H., Yu, Z., Li, X., & Zhao, G. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01049 · Cited by 127
Pixelwise Deep Sequence Learning for Moving Object Detection
Chen, Y., Wang, J., Zhu, B., Tang, M., & Lu, H. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2770319 · Cited by 110
Learning to Model Relationships for Zero-Shot Video Classification
Gao, J., Zhang, T., & Xu, C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2985708 · Cited by 111
3-D PersonVLAD: Learning Deep Global Representations for Video-Based Person Reidentification
Wu, L. Y., Wang, Y., Shao, L., & Wang, M. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2891244 · Cited by 109
Self-Supervised Remote Sensing Feature Learning: Learning Paradigms, Challenges, and Future Works
Tao, C., Qi, J., Guo, M., Zhu, Q., & Li, H. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3276853 · Cited by 116
You Lead, We Exceed: Labor-Free Video Concept Learning by Jointly Exploiting Web Videos and Images
Gan, C., Yao, T., Yang, K., Yang, Y., & Mei, T. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.106 · Cited by 109
Discriminative Hierarchical Rank Pooling for Activity Recognition
Fernando, B., Anderson, P., Hütter, M., & Gould, S. J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.212 · Cited by 108
Intra-Hour Photovoltaic Generation Forecasting Based on Multi-Source Data and Deep Learning Methods
Yao, T., Wang, J., Wu, H., Zhang, P., Li, S., Xu, K., Liu, X., & Chi, X. (2021) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2021.3123337 · Cited by 111
DROCC: Deep Robust One-Class ClassificationPreprint
Goyal, S., Raghunathan, A., Jain, M., Simhadri, H. V., & Jain, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.12718 · Cited by 112
Convolutional Tensor-Train LSTM for Spatio-temporal LearningPreprint
Su, J., Byeon, W., Kossaifi, J., Huang, F., Kautz, J., & Anandkumar, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09131 · Cited by 113
SeCo: Exploring Sequence Supervision for Unsupervised Representation Learning
Yao, T., Zhang, Y., Qiu, Z., Pan, Y., & Mei, T. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i12.17274 · Cited by 106
DepthNet: A Recurrent Neural Network Architecture for Monocular Depth Prediction
Kumar, A. C. S., Bhandarkar, S. M., & Prasad, M. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00066 · Cited by 112
Huang, J., Segura, L. J., Wang, T., Zhao, G., Sun, H., & Zhou, C. (2020) · Additive manufacturing · doi:10.1016/j.addma.2020.101197 · Cited by 105
MediaPipe’s Landmarks with RNN for Dynamic Sign Language Recognition
Samaan, G. H., Wadie, A. R., Attia, A. K., Asaad, A. M., Kamel, A. E., Slim, S. O., Abdallah, M. S., & Cho, Y. I. (2022) · Electronics · doi:10.3390/electronics11193228 · Cited by 105
Deep Multimodal Representation Learning from Temporal Data
Yang, X., Ramesh, P. S., Chitta, R., Madhvanath, S., Bernal, E. A., & Luo, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.538 · Cited by 104
Learning Generic Sentence Representations Using Convolutional Neural Networks
Gan, Z., Pu, Y., Henao, R., Li, C., He, X., & Carin, L. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1254 · Cited by 111
Metadata
- Authors
- Srivastava, N., Mansimov, E., & Salakhutdinov, R.
- Affiliations
- University of Toronto (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- Pages
- 843-852
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1502.04681
- OpenAlex
- W2116435618
- Cited by
- 1,861
- References
- 30
- Citations by year
- 2026: 3 · 2025: 56 · 2024: 128 · 2023: 148 · 2022: 183 · 2021: 279 · 2020: 308 · 2019: 286 · 2018: 238 · 2017: 146 · 2016: 62 · 2015: 23
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Machine Learning
- Funders
- Raytheon Company
- Nvidia
- Samsung
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- video representation learning, unsupervised learning, long short-term memory networks, LSTM encoder-decoder, video reconstruction, future frame prediction, perceptual features, feature visualization, human action recognition, transfer learning, few-shot learning, out-of-domain generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
