The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
Open accessAbstract
Representing and rendering dynamic scenes has been an important but challenging task. Especially, to accurately model complex motions, high efficiency is usually hard to guarantee. To achieve real-time dynamic scene rendering while also enjoying high training and storage efficiency, we propose 4D Gaussian Splatting (4D-GS) as a holistic representation for dynamic scenes rather than applying 3D-GS for each individual frame. In 4D-GS, a novel explicit representation containing both 3D Gaussians and 4D neural voxels is proposed. A decomposed neural voxel encoding algorithm inspired by HexPlane is proposed to efficiently build Gaussian features from 4D neural voxels and then a lightweight MLP is applied to predict Gaussian deformations at novel timestamps. Our 4D-GS method achieves real-time rendering under high resolutions, 82 FPS at an 800x800 resolution on an RTX 3090 GPU while maintaining comparable or better quality than previous state- of-the-art methods. More demos and code are available at https://guanjunwu.github.io/4dgs/.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · doi:10.1109/cvpr.1993 · Cited by 17,667
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
NeRF: representing scenes as neural radiance fields for view synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2022) · Communications of the ACM · doi:10.1145/3503250 · Cited by 6,852
Structure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Kerbl, B., Kopanas, G., Leimkuehler, T., & Drettakis, G. (2023) · ACM Transactions on Graphics · doi:10.1145/3592433 · Cited by 6,025
Instant neural graphics primitives with a multiresolution hash encoding
Müller, T., Evans, A., Schied, C., & Keller, A. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530127 · Cited by 4,114
Proceedings of IEEE International Conference on Computer Vision
(1995) · doi:10.1109/iccv.1995 · Cited by 2,280
Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., & Srinivasan, P. P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00580 · Cited by 1,921
Drebin, R. A., Carpenter, L., & Hanrahan, P. (1988) · ACM SIGGRAPH Computer Graphics · doi:10.1145/378456.378484 · Cited by 1,612
Plenoxels: Radiance Fields without Neural Networks
Fridovich-Keil, S., Yu, A., Tancik, M., Chen, Q., Recht, B., & Kanazawa, A. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00542 · Cited by 1,388
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections
Martin-Brualla, R., Radwan, N., Sajjadi, M. S. M., Barron, J. T., Dosovitskiy, A., & Duckworth, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00713 · Cited by 1,374
D-NeRF: Neural Radiance Fields for Dynamic Scenes
Pumarola, A., Corona, E., Pons-Moll, G., & Moreno-Noguer, F. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01018 · Cited by 1,355
Nerfies: Deformable Neural Radiance Fields
Park, K., Sinha, U., Barron, J. T., Bouaziz, S., Goldman, D. B., Seitz, S. M., & Martin-Brualla, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00581 · Cited by 1,059
Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction
Sun, C., Sun, M., & Chen, H.-T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00538 · Cited by 939
PU-Net: Point Cloud Upsampling Network
Yu, L., Li, X., Fu, C.-W., Cohen-Or, D., & Heng, P.-A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00295 · Cited by 697
Zwicker, M., Pfister, H., van Baar, J., & Gross, M. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383300 · Cited by 666
HyperNeRF: a higher-dimensional representation for topologically varying neural radiance fields
Park, K., Sinha, U., Hedman, P., Barron, J. T., Bouaziz, S., Goldman, D. B., Martin-Brualla, R., & Seitz, S. M. (2021) · ACM Transactions on Graphics · doi:10.1145/3478513.3480487 · Cited by 618
High-quality streamable free-viewpoint video
Collet, A., Chuang, M., Sweeney, P., Gillett, D., Evseev, D., Calabrese, D., Hoppe, H., Kirk, A., & Sullivan, S. (2015) · ACM Transactions on Graphics · doi:10.1145/2766945 · Cited by 573
Point-NeRF: Point-based Neural Radiance Fields
Xu, Q., Xu, Z., Philip, J., Bi, S., Shu, Z., Sunkavalli, K., & Neumann, U. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00536 · Cited by 572
K-Planes: Explicit Radiance Fields in Space, Time, and Appearance
Fridovich-Keil, S., Meanti, G., Warburg, F. R., Recht, B., & Kanazawa, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01201 · Cited by 560
NeRF++: Analyzing and Improving Neural Radiance Fields
Zhang, K., Riegler, G., Snavely, N., & Koltun, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.07492 · Cited by 487
Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis
Luiten, J., Kopanas, G., Leibe, B., & Ramanan, D. (2024) · 2024 International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00044 · Cited by 469
HexPlane: A Fast Representation for Dynamic Scenes
Cao, A., & Johnson, J. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00021 · Cited by 435
DeepView: View Synthesis With Learned Gradient Descent
Flynn, J., Broxton, M., Debevec, P., DuVall, M., Fyffe, G., Overbeck, R., Snavely, N., & Tucker, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00247 · Cited by 424
Neural 3D Video Synthesis from Multi-view Video
Li, T., Slavcheva, M., Zollhoefer, M., Green, S., Lassner, C., Kim, C., Schmidt, T., Lovegrove, S., Goesele, M., Newcombe, R., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00544 · Cited by 387
Dynamic View Synthesis from Dynamic Monocular Video
Gao, C., Saraf, A., Kopf, J., & Huang, J.-B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00566 · Cited by 346
Fast Dynamic Radiance Fields with Time-Aware Neural Voxels
Fang, J., Yi, T., Wang, X., Xie, L., Zhang, X., Liu, W., Nießner, M., & Tian, Q. (2022) · SIGGRAPH Asia 2022 Conference Papers · doi:10.1145/3550469.3555383 · Cited by 285
Self-citationNeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction
Wang, Y., Han, Q., Habermann, M., Daniilidis, K., Theobalt, C., & Liu, L. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00305 · Cited by 281
ADOP: approximate differentiable one-pixel point rendering
Rückert, D., Franke, L., & Stamminger, M. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530122 · Cited by 273
Immersive light field video with a layered mesh representation
Broxton, M., Flynn, J., Overbeck, R., Erickson, D., Hedman, P., Duvall, M., Dourgarian, J., Busch, J., Whalen, M., & Debevec, P. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392485 · Cited by 266
NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields
Song, L., Chen, A., Li, Z., Chen, Z., Chen, L., Yuan, J., Xu, Y., & Geiger, A. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3247082 · Cited by 254
Differentiable surface splatting for point-based geometry processing
Yifan, W., Serena, F., Wu, S., Öztireli, C., & Sorkine-Hornung, O. (2019) · ACM Transactions on Graphics · doi:10.1145/3355089.3356513 · Cited by 251
Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis
Li, Z., Chen, Z., Li, Z., & Xu, Y. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00813 · Cited by 208
Tensor4D: Efficient Neural 4D Decomposition for High-Fidelity Dynamic Reconstruction and Rendering
Shao, R., Zheng, Z., Tu, H., Liu, B., Zhang, H., & Liu, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01596 · Cited by 180
Robust Dynamic Radiance Fields
Liu, Y.-L., Gao, C., Meuleman, A., Tseng, H.-Y., Saraf, A., Kim, C., Chuang, Y., Kopf, J., & Huang, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00010 · Cited by 180
HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling
Attal, B., Huang, J., Richardt, C., Zollhöfer, M., Kopf, J., O’Toole, M., & Kim, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01594 · Cited by 127
Mixed Neural Voxels for Fast Multi-view Video Synthesis
Wang, F., Tan, S., Li, X., Tian, Z., Song, Y., & Liu, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01805 · Cited by 119
RobustFusion: Human Volumetric Capture with Data-Driven Visual Cues Using a RGBD Camera
Su, Z., Xu, L., Zheng, Z., Yu, T., Liu, Y., & Fang, L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58548-8_15 · Cited by 118
Multi-Scale Geometric Consistency Guided and Planar Prior Assisted Multi-View Stereo
Xu, Q., Kong, W., Tao, W., & Pollefeys, M. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3200074 · Cited by 102
Representing Volumetric Videos as Dynamic MLP Maps
Peng, S., Yan, Y., Shuai, Q., Bao, H., & Zhou, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00414 · Cited by 75
MPS-NeRF: Generalizable 3D Human Rendering From Multiview Images
Gao, X., Yang, J., Kim, J., Peng, S., Liu, Z., & Tong, X. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3205910 · Cited by 59
V4D: Voxel for 4D Novel View Synthesis
Gan, W., Xu, H., Huang, Y., Chen, S., & Yokoya, N. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3312127 · Cited by 52
High-Fidelity and Real-Time Novel View Synthesis for Dynamic Scenes
Lin, H., Peng, S., Xu, Z., Xie, T., He, X., Bao, H., & Zhou, X. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618142 · Cited by 44
MonoNeRF: Learning a Generalizable Dynamic Radiance Field from Monocular Videos
Tian, F., Du, S., & Duan, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01641 · Cited by 42
HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars
Hu, T., Yu, T., Zheng, Z., Zhang, H., Liu, Y., & Zwicker, M. (2022) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv57658.2022.00032 · Cited by 32
Robust 3D Human Motion Reconstruction via Dynamic Template Construction
Li, Z., Ji, Y., Yang, W. H., Ye, J., & Jingyi Yu, A. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00063 · Cited by 21
4D Human Body Correspondences from Panoramic Depth Maps
Li, Z., Wu, M., Zhou, W., & Jingyi Yu, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00304 · Cited by 10
Point-DynRF: Point-based Dynamic Radiance Fields from a Monocular Video
Park, B., & Kim, C. (2024) · IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv57701.2024.00314 · Cited by 9
Cited by
Matsuki, H., Murai, R., Kelly, P. H. J., & Davison, A. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01708 · Cited by 493
SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM
Keetha, N., Karhade, J., Jatavallabhula, K. M., Yang, G., Scherer, S., Ramanan, D., & Luiten, J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02018 · Cited by 462
GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting
Yan, C., Qu, D., Xu, D., Zhao, B., Wang, Z., Wang, D., & Li, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01853 · Cited by 390
LangSplat: 3D Language Gaussian Splatting
Qin, M., Li, W., Zhou, J., Wang, H., & Pfister, H. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01895 · Cited by 268
DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes
Zhou, X., Lin, Z., Shan, X., Wang, Y., Sun, D., & Yang, M.-H. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02044 · Cited by 248
Metadata
- Authors
- Wu, G., Yi, T., Fang, J., Xie, L., Zhang, X., Wei, W., Liu, W., Tian, Q., & Wang, X.
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 20310-20320
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402727915
- Cited by
- 806
- References
- 49
- FWCI
- 231.47
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.