The DOI matches its DataCite record; 29 of the 30 references it cites were found in a registry or an index.
Unsupervised Learning of Video Representations using LSTMs
Open accessAbstract
We use multilayer Long Short Term Memory (LSTM) networks to learn representations of video sequences. Our model uses an encoder LSTM to map an input sequence into a fixed length representation. This representation is decoded using single or multiple decoder LSTMs to perform different tasks, such as reconstructing the input sequence, or predicting the future sequence. We experiment with two kinds of input sequences - patches of image pixels and high-level representations ("percepts") of video frames extracted using a pretrained convolutional net. We explore different design choices such as whether the decoder LSTMs should condition on the generated output. We analyze the outputs of the model qualitatively to see how well the model can extrapolate the learned video representation into the future and into the past. We try to visualize and interpret the learned features. We stress test the model by running it on longer time scales and on out-of-domain data. We further evaluate the representations by finetuning them for a supervised learning problem - human action recognition on the UCF-101 and HMDB-51 datasets. We show that the representations help improve classification accuracy, especially when there are only a few training examples. Even models pretrained on unrelated datasets (300 hours of YouTube videos) can help action recognition performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This is the first time that motion prediction with cross-dimensional learning has been used for self-supervised learning in the context of HAR.”
From Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion (2021) · cites this paper as [23]
In the citing paper
“While motion prediction has been used in the field of computer vision for self-supervised learning [23, 26], our model is novel in that the cross-dimensional aspect results in more effective learning.”
From Self-supervised Human Activity Recognition by Learning to Predict Cross-Dimensional Motion (2021) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,313
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Research works printed but not linked: 1
- Hochreiter, Sepp and Schmidhuber, J¨ urgen. Long short-term memory. Neural Computation , 9(8):1735-1780, 1997.
Cited by
Sorted by citations
Items 301–400 of 1,005
Sequential Video VLAD: Training the Aggregation Locally and Temporally
Xu, Y., Han, Y., Hong, R., & Tian, Q. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2846664 · Cited by 103
Explaining and Interpreting LSTMs
Arras, L., Arjona-Medina, J., Widrich, M., Montavon, G., Gillhofer, M., Müller, K.-R., Hochreiter, S., & Samek, W. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_11 · Cited by 111
Xing, Y., Caterina, G. D., & Soraghan, J. J. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.590164 · Cited by 106
PredCNN: Predictive Learning with Cascade Convolutions
Xu, Z., Wang, Y., Long, M., & Wang, J. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/408 · Cited by 106
Progressive Instance-Aware Feature Learning for Compositional Action Recognition
Yan, R., Xie, L., Shu, X., Zhang, L., & Tang, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3261659 · Cited by 117
Long-term Forecasting using Higher Order Tensor RNNsPreprint
Yu, R., Zheng, S., Anandkumar, A., & Yue, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00073 · Cited by 129
Illusory Motion Reproduced by Deep Neural Networks Trained for Prediction
Watanabe, E., Kitaoka, A., Sakamoto, K., Yasugi, M., & Tanaka, K. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00345 · Cited by 101
Morerio, P., Cavazza, J., Volpi, R., Vidal, R., & Murino, V. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.383 · Cited by 106
Song, H., Sun, C., Wu, X., Chen, M., & Jia, Y. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2950530 · Cited by 104
ElSaadani, M., Habib, E., Abdelhameed, A. M., & Bayoumi, M. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.636234 · Cited by 104
Video Generation From Single Semantic Label Map
Pan, J., Wang, C., Jia, X., Shao, J., Sheng, L., Yan, J., & Wang, X. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00385 · Cited by 99
Disentangling Propagation and Generation for Video Prediction
Gao, H., Xu, H., Cai, Q.-Z., Wang, R., Yu, F., & Darrell, T. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00910 · Cited by 103
Deep Multi-Modal Network Based Automated Depression Severity Estimation
Uddin, M. A., Joolee, J. B., & Sohn, K.-A. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3179478 · Cited by 102
Human Action Recognition from Various Data Modalities: A ReviewPreprint
Rahmani, H., Bennamoun, M., & Ke, Q. (2021) · doi:10.36227/techrxiv.13708270 · Cited by 99
Eltouny, K., & Liang, X. P. (2022) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12845 · Cited by 99
Anticipating the future by watching unlabeled video.Preprint
Vondrick, C., Pirsiavash, H., & Torralba, A. B. (2015) · arXiv (Cornell University) · Cited by 98
Skeleton-Aided Articulated Motion Generation
Yan, Y., Xu, J., Ni, B., Zhang, W., & Yang, X. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123277 · Cited by 97
Satellite Image Prediction Relying on GAN and LSTM Neural Networks
Xu, Z. W., Du, J., Wang, J., Jiang, C., & Ren, Y. (2019) · International Conference on Communications · doi:10.1109/icc.2019.8761462 · Cited by 97
Walk and Learn: Facial Attribute Representation Learning from Egocentric Video and Contextual Data
Wang, J., Cheng, Y., & Feris, R. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.252 · Cited by 99
2022 Review of Data-Driven Plasma Science
Anirudh, R., Archibald, R. K., Asif, M. S., Becker, M. M., Benkadda, S., Bremer, P., Budé, R. H. S., Chang, C. S., Chen, L., Churchill, R. M., et al. (2023) · IEEE Transactions on Plasma Science · doi:10.1109/tps.2023.3268170 · Cited by 105
Nishani, E., & Çiço, B. (2017) · Mediterranean Conference on Embedded Computing (MECO) · doi:10.1109/meco.2017.7977207 · Cited by 96
Webly-Supervised Video Recognition by Mutually Voting for Relevant Web Images and Web Video Frames
Gan, C., Sun, C., Duan, L., & Gong, B. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46487-9_52 · Cited by 96
Few-Shot Deep Adversarial Learning for Video-Based Person Re-Identification
Wu, L. Y., Wang, Y., Yin, H., Wang, M., & Shao, L. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2940684 · Cited by 97
Unsupervised object-level video summarization with online motion auto-encoder
Zhang, Y., Liang, X., Zhang, D., Tan, M. H., & Xing, E. P. (2018) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.07.030 · Cited by 97
Orderless Recurrent Models for Multi-Label Classification
Yazici, V. O., González-García, A., Arnau Ramisa, A., Twardowski, B., & van de Weijer, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01345 · Cited by 98
Yin, J., Shen, J., Gao, X., Crandall, D., & Yang, R. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3125981 · Cited by 95
Alaa, A. M., van Breugel, B., Saveliev, E., & van der Schaar, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.08921 · Cited by 124
Online anomaly detection with concept drift adaptation using recurrent neural networks
Saurav, S., Malhotra, P., Tv, V., Gugulothu, N., Vig, L., Agarwal, P., & Shroff, G. M. (2018) · Joint International Conference on Data Science & Management of Data (CODS-COMAD) · doi:10.1145/3152494.3152501 · Cited by 99
Unsupervised Learning of Edges
Li, Y., Paluri, M., Rehg, J. M., & Dollár, P. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.179 · Cited by 94
Autonomous aerial cinematography in unstructured environments with learned artistic decision‐making
Bonatti, R., Wang, W., Ho, C., Ahuja, A., Gschwindt, M., Camci, E., Kayacan, E., Choudhury, S., & Scherer, S. (2020) · Journal of Field Robotics · doi:10.1002/rob.21931 · Cited by 102
Future Video Synthesis With Object Motion Prediction
Wu, Y., Gao, R., Park, J., & Chen, Q. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00558 · Cited by 95
A Hierarchical Spatio-Temporal Graph Convolutional Neural Network for Anomaly Detection in Videos
Zeng, X., Jiang, Y., Ding, W., Li, H., Hao, Y., & Qiu, Z. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3134410 · Cited by 94
Scaling Autoregressive Video Models
Weissenborn, D., Täckström, O., & Uszkoreit, J. (2019) · arXiv · doi:10.48550/arxiv.1906.02634 · Cited by 93
Three-stream CNNs for action recognition
Wang, L., Ge, L., Li, R., & Fang, Y. (2017) · Pattern Recognition Letters · doi:10.1016/j.patrec.2017.04.004 · Cited by 93
Going in circles is the way forward: the role of recurrence in visual inference
van Bergen, R. S., & Kriegeskorte, N. (2020) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2020.11.009 · Cited by 95
Learning a State Representation and Navigation in Cluttered and Dynamic Environments
Hoeller, D., Wellhausen, L., Farshidian, F., & Hutter, M. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3068639 · Cited by 96
Phenaki: Variable Length Video Generation From Open Domain Textual Description
Villegas, R., Babaeizadeh, M., Kindermans, P.-J., Moraldo, H., Zhang, H., Saffar, M. T., Castro, S., Kunze, J., & Erhan, D. (2022) · arXiv · doi:10.48550/arxiv.2210.02399 · Cited by 92
“These papers include deterministic methods using a combination of recurrent and convolutional networks [36, 42, 13, 50], variational based stochastic meth-ods [2, 10, 46, 3] and more recently by learning a discrete representation [49, 33, 31], auto-regressive models [51, 55, 28, 57], diffusion…”
MethodDiverse Image Synthesis From Semantic Layouts via Conditional IMLE
Li, K., Zhang, T., & Malik, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00432 · Cited by 91
Flexible Spatio-Temporal Networks for Video Prediction
Lu, C., Hirsch, M., & Schölkopf, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.230 · Cited by 92
Zhang, L., Lü, L., Wang, X., Zhu, R. M., Bagheri, M. H., Summers, R. M., & Yao, J. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2943841 · Cited by 100
Deng, Z., Jonetzko, Y., Zhang, L., & Zhang, J. (2020) · Sensors · doi:10.3390/s20041050 · Cited by 91
A Robust Spatiotemporal Forecasting Framework for Photovoltaic Generation
Chai, S., Xu, Z., Jia, Y., & Wong, W. K. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.3006085 · Cited by 94
VisualCOMET: Reasoning About the Dynamic Context of a Still Image
Park, J. S., Bhagavatula, C., Mottaghi, R., Farhadi, A., & Choi, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58558-7_30 · Cited by 102
Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction
Liu, Z., Wu, S., Jin, S., Ji, S., Liu, Q., Lu, S., & Cheng, L. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3139918 · Cited by 91
Controllable Video Generation with Sparse Trajectories
Hao, Z., Huang, X., & Belongie, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00819 · Cited by 90
Redundancy and Attention in Convolutional LSTM for Gesture Recognition
Zhu, G., Zhang, L., Yang, L., Mei, L., Shah, S. A. A., Bennamoun, M., & Shen, P. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2919764 · Cited by 90
Broaden Your Views for Self-Supervised Video Learning
Recasens, A., Luc, P., Alayrac, J.-B., Wang, L., Strub, F., Tallec, C., Malinowski, M., Pătrăucean, V., Altché, F., Valko, M., et al. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00129 · Cited by 98
Boroumand, A., Ghose, S., Akin, B., Narayanaswami, R., Junior, G. F. D. O., Ma, X., Shiu, E., & Mutlu, O. (2021) · International Conference on Parallel Architectures and Compilation Techniques (PACT) · doi:10.1109/pact52795.2021.00019 · Cited by 107
Basketball action recognition based on the combination of YOLO and a deep fuzzy LSTM network
Khobdeh, S. B., Yamaghani, M. R., & Khodaparast, S. (2023) · The Journal of Supercomputing · doi:10.1007/s11227-023-05611-7 · Cited by 90
Human trajectory prediction and generation using LSTM models and GANs
Rossi, L., Paolanti, M., Pierdicca, R., & Frontoni, E. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108136 · Cited by 93
Zheng, M., Man, J., Wang, D., Chen, Y., Li, Q., & Liu, Y. (2023) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2023.109235 · Cited by 89
Maharaj, T., Ballas, N., Rohrbach, A., Courville, A. C., & Pal, C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.778 · Cited by 93
Temporal Dynamic Graph LSTM for Action-Driven Video Object Detection
Yuan, Y., Liang, X., Wang, X., Yeung, D., & Gupta, A. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.200 · Cited by 92
Spatio-Temporal Manifold Learning for Human Motions via Long-Horizon Modeling
Wang, H., Ho, E. S. L., Shum, H. P. H., & Zhu, Z. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2936810 · Cited by 90
Sun, C., Shrivastava, A., Vondrick, C., Sukthankar, R., Murphy, K. P., & Schmid, C. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00036 · Cited by 87
World-Consistent Video-to-Video Synthesis
Mallya, A., Wang, T.-C., Sapra, K., & Liu, M.-Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58598-3_22 · Cited by 87
High Fidelity Video Prediction with Large Stochastic Recurrent Neural NetworksPreprint
Villegas, R., Pathak, A., Kannan, H., Erhan, D., Le, Q. V., & Lee, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.01655 · Cited by 89
Abnormal Event Detection in Videos Using Hybrid Spatio-Temporal Autoencoder
Wang, L., Zhou, F., Li, Z., Zuo, W., & Tan, H. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451070 · Cited by 86
Im2Flow: Motion Hallucination from Static Images for Action Recognition
Gao, R., Xiong, B., & Grauman, K. L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00622 · Cited by 86
Advanced Intrusion Detection Combining Signature-Based and Behavior-Based Detection Methods
Kwon, H.-Y., Kim, T., & Lee, M.-K. (2022) · Electronics · doi:10.3390/electronics11060867 · Cited by 85
Bidirectional Multirate Reconstruction for Temporal Modeling in Videos
Zhu, L., Xu, Z., & Yang, Y. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.147 · Cited by 87
Accelerating Recurrent Neural Networks: A Memory-Efficient Approach
Wang, Z., Lin, J. T., & Wang, Z. (2017) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2017.2717950 · Cited by 85
Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks
Schreiber, M., Hoermann, S., & Dietmayer, K. (2019) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2019.8793582 · Cited by 90
Rakhimov, R., Volkhonskiy, D., Artemov, A., Zorin, D., & Burnaev, E. V. (2021) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0010241801010112 · Cited by 85
VideoFlow: A Flow-Based Generative Model for Video
Kumar, M. N. A., Babaeizadeh, M., Erhan, D., Finn, C., Levine, S., Dinh, L., & Kingma, D. (2019) · Cited by 84
TSM: Temporal Shift Module for Efficient and Scalable Video Understanding on Edge Devices
Lin, J., Gan, C., Wang, K., & Han, S. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3029799 · Cited by 88
Li, H., & Fan, Y. (2019) · 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) · doi:10.1109/isbi.2019.8759397 · Cited by 83
Predicting Future Instance Segmentation by Forecasting Convolutional Features
Luc, P., Couprie, C., LeCun, Y., & Verbeek, J. J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_36 · Cited by 82
Hierarchical Graph Convolutional Networks for Action Quality Assessment
Zhou, K., Ma, Y., Shum, H. P. H., & Liang, X. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3281413 · Cited by 82
Luo, H., Ji, L., Shi, B., Huang, H., Duan, N., Li, T., Chen, X., & Zhou, M. (2020) · arXiv (Cornell University) · Cited by 82
Self-Supervised Relative Depth Learning for Urban Scene Understanding
Jiang, H., Larsson, G., Maire, M., Shakhnarovich, G., & Miller, E. G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_2 · Cited by 83
PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processing
Fan, H., & Yang, Y. (2019) · arXiv · doi:10.48550/arxiv.1910.08287 · Cited by 80
RATM: Recurrent Attentive Tracking Model
Kahou, S. E., Michalski, V., Memisevic, R., Pal, C., & Vincent, P. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.206 · Cited by 85
Temporal Residual Networks for Dynamic Scene Recognition
Feichtenhofer, C., Pinz, A. J., & Wildes, R. P. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.786 · Cited by 80
Collaborative Deep Metric Learning for Video Understanding
Lee, J., Abu-El-Haija, S., Varadarajan, B., & Natsev, A. (2018) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3219819.3219856 · Cited by 83
Liu, H., Li, Y., Tsang, M., & Liu, Y. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330881 · Cited by 80
Deep Learning in Next-Frame Prediction: A Benchmark Review
Zhou, Y., Dong, H., & Saddik, A. E. (2020) · IEEE Access · doi:10.1109/access.2020.2987281 · Cited by 87
Analyzing human–human interactions: A survey
Stergiou, A., & Poppe, R. W. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2019.102799 · Cited by 86
Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling
Zou, Y., Ji, P., Tran, Q.-H., Huang, J., & Chandraker, M. K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58568-6_42 · Cited by 83
LSTM, VADER and TF-IDF based Hybrid Sentiment Analysis Model
Chiny, M., Chihab, M., Bencharef, O., & Chihab, Y. (2021) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2021.0120730 · Cited by 86
Body Joint Guided 3-D Deep Convolutional Descriptors for Action Recognition
Cao, C., Zhang, Y., Zhang, C., & Lu, H. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2756840 · Cited by 78
Action Anticipation with RBF Kernelized Feature Mapping RNN
Shi, Y., Fernando, B., & Hartley, R. I. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_19 · Cited by 81
Learning to Simulate Dynamic Environments With GameGAN
Kim, S. W., Zhou, Y., Philion, J., Torralba, A. B., & Fidler, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00131 · Cited by 83
Stand-Alone Inter-Frame Attention in Video Models
Long, F., Qiu, Z., Pan, Y., Yao, T., Luo, J., & Mei, T. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00319 · Cited by 78
Medina-Quero, J., Zhang, S., Nugent, C., & Espinilla, M. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.07.068 · Cited by 80
E-LSTM: An Efficient Hardware Architecture for Long Short-Term Memory
Wang, M., Wang, Z., Lu, J., Lin, J. T., & Wang, Z. (2019) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2019.2911739 · Cited by 80
Aerial Image Road Extraction Based on an Improved Generative Adversarial Network
Zhang, X., Han, X., Li, C., Tang, X., Zhou, H., & Jiao, L. (2019) · Remote Sensing · doi:10.3390/rs11080930 · Cited by 77
Adversarial Self-supervised Learning for Semi-supervised 3D Action Recognition
Si, C., Nie, X., Wang, W., Wang, L., Tan, T., & Feng, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58571-6_3 · Cited by 77
DwNet: Dense warp-based network for pose-guided human video generationPreprint
Zablotskaia, P., Siarohin, A., Zhao, B., & Sigal, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.09139 · Cited by 77
ManifoldNet: A Deep Neural Network for Manifold-Valued Data With Applications
Chakraborty, R., Bouza, J., Manton, J. H., & Vemuri, B. C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3003846 · Cited by 83
Deep unsupervised multi-modal fusion network for detecting driver distraction
Zhang, Y., Chen, Y., & Gao, C. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.09.023 · Cited by 78
Chen, C.-W., Tseng, S.-P., Kuan, T.-W., & Wang, J.-F. (2020) · Information · doi:10.3390/info11020106 · Cited by 78
Probabilistic Future Prediction for Video Scene Understanding
Hu, A., Cotter, F., Mohan, N. C., Gurău, C., & Kendall, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_45 · Cited by 79
A 3D-CNN and LSTM Based Multi-Task Learning Architecture for Action Recognition
Ouyang, X., Xu, S., Zhang, C., Zhou, P., Yang, Y., Liu, G., & Li, X. (2019) · IEEE Access · doi:10.1109/access.2019.2906654 · Cited by 76
Ohn-Bar, E., Prakash, A., Behl, A., Chitta, K., & Geiger, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01131 · Cited by 76
Survey on deep learning methods in human action recognition
Koohzadi, M., & Charkari, N. M. (2017) · IET Computer Vision · doi:10.1049/iet-cvi.2016.0355 · Cited by 75
A Deep Spatiotemporal Perspective for Understanding Crowd Behavior
Li, Y. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2834873 · Cited by 74
Loy-Benitez, J., Heo, S., & Yoo, C. (2020) · Control Engineering Practice · doi:10.1016/j.conengprac.2020.104330 · Cited by 77
CATER: A diagnostic dataset for Compositional Actions and TEmporal ReasoningPreprint
Girdhar, R., & Ramanan, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.04744 · Cited by 116
Semi-supervised Multi-modal Emotion Recognition with Cross-Modal Distribution Matching
Liang, J., Li, R., & Jin, Q. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413579 · Cited by 73
Metadata
- Authors
- Srivastava, N., Mansimov, E., & Salakhutdinov, R.
- Affiliations
- University of Toronto (CA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2015
- Pages
- 843-852
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1502.04681
- OpenAlex
- W2116435618
- Cited by
- 1,861
- References
- 30
- Citations by year
- 2026: 3 · 2025: 56 · 2024: 128 · 2023: 148 · 2022: 183 · 2021: 279 · 2020: 308 · 2019: 286 · 2018: 238 · 2017: 146 · 2016: 62 · 2015: 23
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Machine Learning
- Funders
- Raytheon Company
- Nvidia
- Samsung
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- video representation learning, unsupervised learning, long short-term memory networks, LSTM encoder-decoder, video reconstruction, future frame prediction, perceptual features, feature visualization, human action recognition, transfer learning, few-shot learning, out-of-domain generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
