The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
SimVP: Simpler yet Better Video Prediction
Open accessAbstract
From CNN, RNN, to ViT, we have witnessed remarkable advancements in video prediction, incorporating auxiliary inputs, elaborate neural architectures, and sophisticated training strategies. We admire these progresses but are confused about the necessity: is there a simple method that can perform comparably well? This paper proposes SimVp, a simple video prediction model that is completely built upon CNN and trained by MSE loss in an end-to-end fashion. Without introducing any additional tricks and complicated strategies, we can achieve state-of-the-art performance on five benchmark datasets. Through extended experiments, we demonstrate that SimVP has strong generalization and extensibility on real-world datasets. The significant reduction of training cost makes it easier to scale to complex scenarios. We believe SimVP can serve as a solid baseline to stimulate the further development of video prediction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., & Woo, W.-C. (2015) · arXiv · doi:10.48550/arxiv.1506.04214 · Cited by 7,709
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 1×
“6 [24] is a complex human pose dataset with 3.6 million samples, recording different activities.”
BackgroundInfluentialRecognizing human actions: a local SVM approach
Schuldt, C., Laptev, I., & Caputo, B. (2004) · Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. · doi:10.1109/icpr.2004.1334462 · Cited by 3,010
ViViT: A Video Vision Transformer
Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lucic, M., & Schmid, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00676 · Cited by 2,767
Neural Discrete Representation Learning
Oord, A. V. D., Vinyals, O., & Kavukcuoglu, K. (2017) · arXiv · doi:10.48550/arxiv.1711.00937 · Cited by 2,729
Liu, Z., Ning, J., Cao, Y., Wei, Y., Zhang, Z., Lin, S., & Hu, H. (2021) · arXiv · doi:10.48550/arxiv.2106.13230 · Cited by 2,294
Liu, Z., Ning, J., Cao, Y., Wei, Y., Zhang, Z., Lin, S., & Hu, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00320 · Cited by 2,294
Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction
Zhang, J., Zheng, Y., & Qi, D. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10735 · Cited by 2,282
Unsupervised Learning of Video Representations using LSTMs
Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1502.04681 · Cited by 1,861
Is Space-Time Attention All You Need for Video Understanding?
Bertasius, G., Wang, H., & Torresani, L. (2021) · arXiv · doi:10.48550/arxiv.2102.05095 · Cited by 1,663
Multiscale vision transformers
Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00675 · Cited by 1,407
Pedestrian detection: A benchmark
Dollar, P., Wojek, C., Schiele, B., & Perona, P. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206631 · Cited by 1,313
A guide to convolution arithmetic for deep learning
Dumoulin, V., & Visin, F. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.07285 · Cited by 1,172
MoCoGAN: Decomposing Motion and Content for Video Generation
Tulyakov, S., Liu, M.-Y., Yang, X., & Kautz, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00165 · Cited by 1,076
Generating Videos with Scene Dynamics
Vondrick, C., Pirsiavash, H., & Torralba, A. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.02612 · Cited by 785
Video Frame Synthesis Using Deep Voxel Flow
Liu, Z., Yeh, R. A., Tang, X., Liu, Y., & Agarwala, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.478 · Cited by 743
PredRNN: recurrent neural networks for predictive learning using spatiotemporal LSTMs
Wang, Y., Long, M., Wang, J., Gao, Z., & Yu, P. S. (2017) · neural information processing systems · Cited by 734
Self-citationPredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning
Wang, Y., Wu, H., Zhang, J., Gao, Z., Wang, J., Yu, P. S., & Long, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3165153 · Cited by 677
Self-citationNeimark, D., Bar, O., Zohar, M., & Asselmann, D. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00355 · Cited by 551
Decomposing Motion and Content for Natural Video Sequence Prediction
Villegas, R., Yang, J., Hong, S., Lin, X., & Lee, H. (2017) · arXiv · doi:10.48550/arxiv.1706.08033 · Cited by 488
Wang, Y., Zhang, J., Zhu, H., Long, M., Wang, J., & Yu, P. S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00937 · Cited by 480
Brabandere, B. D., Jia, X., Tuytelaars, T., & Gool, L. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09673 · Cited by 461
Stochastic Variational Video PredictionPreprint
Babaeizadeh, M., Finn, C., Erhan, D., Campbell, R. H., & Levine, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.11252 · Cited by 455
Temporal Generative Adversarial Nets with Singular Value Clipping
Saito, M., Matsumoto, E., & Saito, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.308 · Cited by 407
Axial Attention in Multidimensional Transformers
Ho, J., Kalchbrenner, N., Weissenborn, D., & Salimans, T. (2019) · arXiv · doi:10.48550/arxiv.1912.12180 · Cited by 370
Dual Motion GAN for Future-Flow Embedded Video Prediction
Liang, X., Lee, L., Dai, W., & Xing, E. P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.194 · Cited by 328
PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
Wang, Y., Gao, Z., Long, M., Wang, J., & Yu, P. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.06300 · Cited by 324
Self-citationDisentangling Physical Dynamics From Unknown Factors for Unsupervised Video Prediction
Le Guen, V., & Thome, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01149 · Cited by 306
A Review on Deep Learning Techniques for Video Prediction
Oprea, S., Martinez-Gonzalez, P., Garcia-Garcia, A., Castro-Vargas, J. A., Orts-Escolano, S., Garcia-Rodriguez, J., & Argyros, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3045007 · Cited by 301
Stochastic Video Generation with a Learned Prior
Denton, E., & Fergus, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.07687 · Cited by 278
Spatio-temporal video autoencoder with differentiable memory
Pătrăucean, V., Handa, A., & Cipolla, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06309 · Cited by 211
Unsupervised Learning of Disentangled Representations from Video
Denton, E., & Birodkar, V. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10915 · Cited by 205
Eidetic 3D LSTM: A Model for Video Prediction and Beyond
Wang, Y., Jiang, L., Yang, M. H., Li, L.-J., Long, M., & Fei-Fei, L. (2019) · International Conference on Learning Representations · Cited by 199
Stochastic Adversarial Video Prediction
Lee, A. X., Zhang, R., Ebert, F., Abbeel, P., Finn, C., & Levine, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.01523 · Cited by 184
Predicting Future Frames Using Retrospective Cycle GAN
Kwon, Y.-H., & Park, M.-G. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00191 · Cited by 184
Adversarial Video Generation on Complex Datasets
Clark, A., Donahue, J., & Simonyan, K. (2019) · arXiv · doi:10.48550/arxiv.1907.06571 · Cited by 157
ContextVP: Fully Context-Aware Video Prediction
Byeon, W., Wang, Q., Srivastava, R. K., & Koumoutsakos, P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01270-0_46 · Cited by 152
Folded Recurrent Neural Networks for Future Video Prediction
Oliu, M., Selva, J., & Escalera, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_44 · Cited by 150
Improved Conditional VRNNs for Video Prediction
Castrejón, L., Ballas, N., & Courville, A. C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00770 · Cited by 147
SDC-Net: Video Prediction Using Spatially-Displaced Convolution
Reda, F. A., Liu, G., Shih, K. J., Kirby, R. M., Barker, J., Tarjan, D., Tao, A., & Catanzaro, B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_44 · Cited by 142
Rakhimov, R., Volkhonskiy, D., Artemov, A., Zorin, D., & Burnaev, E. V. (2021) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0010241801010112 · Cited by 132
Jin, B., Hu, Y. H., Tang, Q., Niu, J., Shi, Z., Han, Y., & Li, X. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00461 · Cited by 116
PredCNN: Predictive Learning with Cascade Convolutions
Xu, Z., Wang, Y., Long, M., & Wang, J. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/408 · Cited by 106
Disentangling Propagation and Generation for Video Prediction
Gao, H., Xu, H., Cai, Q.-Z., Wang, R., Yu, F., & Darrell, T. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00910 · Cited by 104
Hierarchical Long-term Video Prediction without SupervisionPreprint
Wichers, N., Villegas, R., Erhan, D., & Lee, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.04768 · Cited by 96
Efficient and Information-Preserving Future Frame Prediction and Beyond
Yu, W., Lu, Y., Easterbrook, S., & Fidler, S. (2020) · International Conference on Learning Representations · Cited by 93
Flexible Spatio-Temporal Networks for Video Prediction
Lu, C., Hirsch, M., & Schölkopf, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.230 · Cited by 92
Controllable Video Generation with Sparse Trajectories
Hao, Z., Huang, X., & Belongie, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00819 · Cited by 92
Probabilistic Future Prediction for Video Scene Understanding
Hu, A., Cotter, F., Mohan, N. C., Gurău, C., & Kendall, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_45 · Cited by 79
Towards High Resolution Video Generation with Progressive Growing of Sliced Wasserstein GANsPreprint
Acharya, D., Huang, Z.-W., Paudel, D., & van Gool, L. (2018) · arXiv.org · doi:10.48550/arxiv.1810.02419 · Cited by 74
Transformation-Based Models of Video SequencesPreprint
van Amersfoort, J., Kannan, A., Ranzato, M., Szlam, A. D., Tran, D., & Chintala, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.08435 · Cited by 61
Chiu, H.-K., Adeli, E., & Niebles, J. C. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2992184 · Cited by 52
Photo-Realistic Video Prediction on Natural Videos of Largely Changing FramesPreprint
Shouno, O. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.08635 · Cited by 37
Predicting Future Instance Segmentation with Contextual Pyramid ConvLSTMs
Sun, J., Xie, J., Hu, J., Lin, Z., Lai, J., Zeng, W., & Zheng, W. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350949 · Cited by 36
TGANv2: Efficient Training of Large Models for Video Generation with Multiple Subsampling LayersPreprint
Saito, M., & Saito, S. (2018) · arXiv (Cornell University) · Cited by 35
VarNet: Exploring Variations for Unsupervised Video Prediction
Jin, B., Hu, Y. H., Zeng, Y., Tang, Q., Liu, S., & Ye, J. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8594264 · Cited by 35
Prediction Under Uncertainty with Error-Encoding NetworksPreprint
Henaff, M., Zhao, J., & LeCun, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04994 · Cited by 26
Z-Order Recurrent Neural Networks for Video Prediction
Zhang, J., Wang, Y., Long, M., Jianmin, W., & Yu, P. S. (2019) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2019.00048 · Cited by 26
Mutual Suppression Network for Video Prediction using Disentangled FeaturesPreprint
Lee, J., Lee, J., Lee, S., & Yoon, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.04810 · Cited by 17
Cited by
A review of convolutional neural networks in computer vision
Zhao, X., Wang, L., Zhang, Y., Han, X., Deveci, M., & Parmar, M. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10721-6 · Cited by 1,148
VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecasting
Tang, Y., Dong, P., Tang, Z., Chu, X., & Liang, J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00575 · Cited by 70
Metadata
- Authors
- Gao, Z., Tan, C., Wu, L., & Li, S. Z.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 3160-3170
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4313009682
- Cited by
- 403
- References
- 62
- FWCI
- 28.10
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.