The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction
Open accessAbstract
Implicit neural representation has paved the way for new approaches to dynamic scene reconstruction. Nonetheless, cutting-edge dynamic neural rendering methods rely heavily on these implicit representations, which frequently struggle to capture the intricate details of objects in the scene. Furthermore, implicit methods have difficulty achieving real-time rendering in general dynamic scenes, limiting their use in a variety of tasks. To address the issues, we propose a deformable 3D Gaussians splatting method that reconstructs scenes using 3D Gaussians and learns them in canonical space with a deformation field to model monocular dynamic scenes. We also introduce an annealing smoothing training mechanism with no extra overhead, which can mitigate the impact of inaccurate poses on the smoothness of time interpolation tasks in real-world scenes. Through a differential Gaussian rasterizer, the deformable 3D Gaussians not only achieve higher rendering quality but also real-time rendering speed. Experiments show that our method outperforms existing methods significantly in terms of both rendering quality and speed, making it well-suited for tasks such as novel-view synthesis, time interpolation, and real-time rendering. Our code is available at https://github.com/ingra14m/Deformable-3D-Gaussians.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
NeRF: representing scenes as neural radiance fields for view synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2022) · Communications of the ACM · doi:10.1145/3503250 · Cited by 6,852
Structure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433
International Conference on Learning Representations (ICLR 2013)
松尾, 豊. (2013) · 人工知能学会誌 = Journal of Japanese Society for Artificial Intelligence · Cited by 6,286
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Kerbl, B., Kopanas, G., Leimkuehler, T., & Drettakis, G. (2023) · ACM Transactions on Graphics · doi:10.1145/3592433 · Cited by 6,025
Instant neural graphics primitives with a multiresolution hash encoding
Müller, T., Evans, A., Schied, C., & Keller, A. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530127 · Cited by 4,114
Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields
Barron, J. T., Mildenhall, B., Verbin, D., Srinivasan, P. P., & Hedman, P. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00539 · Cited by 1,935
Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., & Srinivasan, P. P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00580 · Cited by 1,921
D-NeRF: Neural Radiance Fields for Dynamic Scenes
Pumarola, A., Corona, E., Pons-Moll, G., & Moreno-Noguer, F. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01018 · Cited by 1,355
TensoRF: Tensorial Radiance Fields
Chen, A., Xu, Z., Geiger, A., Yu, J., & Su, H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19824-3_20 · Cited by 1,184
Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields
Barron, J. T., Mildenhall, B., Verbin, D., Srinivasan, P. P., & Hedman, P. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01804 · Cited by 1,126
Nerfies: Deformable Neural Radiance Fields
Park, K., Sinha, U., Barron, J. T., Bouaziz, S., Goldman, D. B., Seitz, S. M., & Martin-Brualla, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00581 · Cited by 1,059
Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction
Sun, C., Sun, M., & Chen, H.-T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00538 · Cited by 939
PlenOctrees for Real-time Rendering of Neural Radiance Fields
Yu, A., Li, R., Tancik, M., Li, H., Ng, R., & Kanazawa, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00570 · Cited by 854
Depth-supervised NeRF: Fewer Views and Faster Training for Free
Deng, K., Liu, A., Zhu, J.-Y., & Ramanan, D. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01254 · Cited by 841
KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs
Reiser, C., Peng, S., Liao, Y., & Geiger, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01407 · Cited by 668
BARF: Bundle-Adjusting Neural Radiance Fields
Lin, C.-H., Ma, W.-C., Torralba, A., & Lucey, S. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00569 · Cited by 628
Virtualized reality: constructing virtual worlds from real scenes
Kanade, T., Rander, P., & Narayanan, P. J. (1997) · IEEE Multimedia · doi:10.1109/93.580394 · Cited by 624
HyperNeRF: a higher-dimensional representation for topologically varying neural radiance fields
Park, K., Sinha, U., Hedman, P., Barron, J. T., Bouaziz, S., Goldman, D. B., Martin-Brualla, R., & Seitz, S. M. (2021) · ACM Transactions on Graphics · doi:10.1145/3478513.3480487 · Cited by 618
High-quality streamable free-viewpoint video
Collet, A., Chuang, M., Sweeney, P., Gillett, D., Evseev, D., Calabrese, D., Hoppe, H., Kirk, A., & Sullivan, S. (2015) · ACM Transactions on Graphics · doi:10.1145/2766945 · Cited by 573
K-Planes: Explicit Radiance Fields in Space, Time, and Appearance
Fridovich-Keil, S., Meanti, G., Warburg, F. R., Recht, B., & Kanazawa, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01201 · Cited by 560
FastNeRF: High-Fidelity Neural Rendering at 200FPS
Garbin, S. J., Kowalski, M., Johnson, M., Shotton, J., & Valentin, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01408 · Cited by 541
Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis
Luiten, J., Kopanas, G., Leibe, B., & Ramanan, D. (2024) · 2024 International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00044 · Cited by 469
Surface Capture for Performance-Based Animation
Starck, J., & Hilton, A. (2007) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2007.68 · Cited by 452
Baking Neural Radiance Fields for Real-Time View Synthesis
Hedman, P., Srinivasan, P. P., Mildenhall, B., Barron, J. T., & Debevec, P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00582 · Cited by 451
Neuralangelo: High-Fidelity Neural Surface Reconstruction
Li, Z., Müller, T., Evans, A., Taylor, R. H., Unberath, M., Liu, M.-Y., & Lin, C.-H. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00817 · Cited by 441
NeRD: Neural Reflectance Decomposition from Image Collections
Boss, M., Braun, R., Jampani, V., Barron, J. T., Liu, C., & Lensch, H. P. A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01245 · Cited by 438
HexPlane: A Fast Representation for Dynamic Scenes
Cao, A., & Johnson, J. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00021 · Cited by 435
Tretschk, E., Tewari, A., Golyanik, V., Zollhofer, M., Lassner, C., & Theobalt, C. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01272 · Cited by 432
Neural 3D Video Synthesis from Multi-view Video
Li, T., Slavcheva, M., Zollhoefer, M., Green, S., Lassner, C., Kim, C., Schmidt, T., Lovegrove, S., Goesele, M., Newcombe, R., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00544 · Cited by 387
Space-time Neural Irradiance Fields for Free-Viewpoint Video
Xian, W., Huang, J.-B., Kopf, J., & Kim, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00930 · Cited by 357
FreeNeRF: Improving Few-Shot Neural Rendering with Free Frequency Regularization
Yang, J., Pavone, M., & Wang, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00798 · Cited by 316
Chen, Z., Funkhouser, T., Hedman, P., & Tagliasacchi, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01590 · Cited by 314
Fast Dynamic Radiance Fields with Time-Aware Neural Voxels
Fang, J., Yi, T., Wang, X., Xie, L., Zhang, X., Liu, W., Nießner, M., & Tian, Q. (2022) · SIGGRAPH Asia 2022 Conference Papers · doi:10.1145/3550469.3555383 · Cited by 285
NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images
Liu, Y., Wang, P., Lin, C. C., Long, X., Wang, J., Liu, L., Komura, T., & Wang, W. (2023) · ACM Transactions on Graphics · doi:10.1145/3592134 · Cited by 270
NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields
Song, L., Chen, A., Li, Z., Chen, Z., Chen, L., Yuan, J., Xu, Y., & Geiger, A. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3247082 · Cited by 254
NeRF--: Neural Radiance Fields Without Known Camera Parameters
Wang, Z., Wu, S., Xie, W., Chen, M., & Prisacariu, V. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07064 · Cited by 245
NeRF-Editing: Geometry Editing of Neural Radiance Fields
Yuan, Y.-J., Sun, Y.-T., Lai, Y.-K., Ma, Y., Jia, R., & Gao, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01781 · Cited by 231
Zwicker, M., Pfister, H., van Baar, J., & Gross, M. (2005) · Proceedings Visualization, 2001. VIS '01. · doi:10.1109/visual.2001.964490 · Cited by 220
Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time
Wang, L., Zhang, J., Liu, X., Zhao, F., Zhang, Y., Zhang, Y., Wu, M., Jingyi Yu, A., & Xu, L. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01316 · Cited by 211
Self-citationTensor4D: Efficient Neural 4D Decomposition for High-Fidelity Dynamic Reconstruction and Rendering
Shao, R., Zheng, Z., Tu, H., Liu, B., Zhang, H., & Liu, Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01596 · Cited by 180
BAD-NeRF: Bundle Adjusted Deblur Neural Radiance Fields
Wang, P., Zhao, L., Ma, R., & Liu, P. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00406 · Cited by 171
Acorn: adaptive coordinate networks for neural scene representation
Martel, J. N. P., Lindell, D. B., Lin, C. Z., Chan, E. R., Monteiro, M., & Wetzstein, G. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459785 · Cited by 165
StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learning
Huang, Y.-H., He, Y., Yuan, Y.-J., Lai, Y.-K., & Gao, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01780 · Cited by 162
Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fields
Hu, W., Wang, Y., Ma, L., Yang, B., Gao, L., Liu, X., & Ma, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01811 · Cited by 145
Efficient Neural Radiance Fields for Interactive Free-viewpoint Video
Lin, H., Peng, S., Xu, Z., Yan, Y., Shuai, Q., Bao, H., & Zhou, X. (2022) · SIGGRAPH Asia Conference Papers · doi:10.1145/3550469.3555376 · Cited by 139
Temporally coherent completion of dynamic shapes
Li, H., Luo, L., Vlasic, D., Peers, P., Popović, J., Pauly, M. V., & Rusinkiewicz, S. M. (2012) · ACM Transactions on Graphics · doi:10.1145/2077341.2077343 · Cited by 123
NeRF-DS: Neural Radiance Fields for Dynamic Specular Objects
Yan, Z., Li, C., & Lee, G. H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00801 · Cited by 123
NeuRBF: A Neural Fields Representation with Adaptive Radial Basis Functions
Chen, Z., Li, Z., Song, L., Chen, L., Jingyi Yu, A., Yuan, J., & Xu, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00386 · Cited by 85
Representing Volumetric Videos as Dynamic MLP Maps
Peng, S., Yan, Y., Shuai, Q., Bao, H., & Zhou, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00414 · Cited by 75
Differentiable Point-Based Radiance Fields for Efficient View Synthesis
Zhang, Q., Baek, S., Rusinkiewicz, S. M., & Heide, F. (2022) · SIGGRAPH Asia Conference Papers · doi:10.1145/3550469.3555413 · Cited by 69
Approximate Differentiable Rendering with Algebraic Surfaces
Keselman, L., & Hebert, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19824-3_35 · Cited by 56
Neural Trajectory Fields for Dynamic Novel View SynthesisPreprint
Wang, C., Eckart, B., Lucey, S., & Gallo, O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.05994 · Cited by 32
NeRF-Texture: Texture Synthesis with Neural Radiance Fields
Huang, Y.-H., Cao, Y., Lai, Y., Shan, Y., & Gao, L. (2023) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3588432.3591484 · Cited by 24
Lyu, X., Dai, P., Li, Z., Yan, D., Lin, Y., Peng, Y., & Qi, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00821 · Cited by 15
Cited by
SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM
Keetha, N., Karhade, J., Jatavallabhula, K. M., Yang, G., Scherer, S., Ramanan, D., & Luiten, J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02018 · Cited by 462
GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting
Yan, C., Qu, D., Xu, D., Zhao, B., Wang, Z., Wang, D., & Li, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01853 · Cited by 390
LangSplat: 3D Language Gaussian Splatting
Qin, M., Li, W., Zhou, J., Wang, H., & Pfister, H. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01895 · Cited by 268
DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes
Zhou, X., Lin, Z., Shan, X., Wang, Y., Sun, D., & Yang, M.-H. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02044 · Cited by 248
DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation
Zhao, G., Ni, C., Wang, X., Zhu, Z., Zhang, X., Wang, Y., Huang, G., Chen, X., Wang, B., Zhang, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.13571 · Cited by 41
Self-citation
Metadata
- Authors
- Yang, Z., Gao, X., Zhou, W., Jiao, S., Zhang, Y., & Jin, X.
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 20331-20341
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402713101
- Cited by
- 452
- References
- 54
- FWCI
- 129.80
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.