The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis
Open accessAbstract
We present a method that simultaneously addresses the tasks of dynamic scene novel-view synthesis and six degree-of-freedom (6-DOF) tracking of all dense scene elements. We follow an analysis-by-synthesis framework, inspired by recent work that models scenes as a collection of 3D Gaussians which are optimized to reconstruct input images via differentiable rendering. To model dynamic scenes, we allow Gaussians to move and rotate over time while enforcing that they have persistent color, opacity, and size. By regularizing Gaussians’ motion and rotation with local-rigidity constraints, we show that our Dynamic 3D Gaussians correctly model the same area of physical space over time, including the rotation of that space. Dense 6-DOF tracking and dynamic reconstruction emerges naturally from persistent dynamic view synthesis, without requiring any correspondence or flow as input. We demonstrate a large number of downstream applications enabled by our representation, including first-person view synthesis, dynamic compositional scene synthesis, and 4D video editing.111. Project Website: dynamic3dgaussians.github.io
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Kerbl, B., Kopanas, G., Leimkuehler, T., & Drettakis, G. (2023) · ACM Transactions on Graphics · doi:10.1145/3592433 · Cited by 6,025
Self-citationNeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351
Instant neural graphics primitives with a multiresolution hash encoding
Müller, T., Evans, A., Schied, C., & Keller, A. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530127 · Cited by 4,114
A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation
Perazzi, F., Pont-Tuset, J., McWilliams, B., Van Gool, L., Gross, M., & Sorkine-Hornung, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.85 · Cited by 1,989
D-NeRF: Neural Radiance Fields for Dynamic Scenes
Pumarola, A., Corona, E., Pons-Moll, G., & Moreno-Noguer, F. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01018 · Cited by 1,355
The computation of optical flow
Beauchemin, S. S., & Barron, J. L. (1995) · ACM Computing Surveys · doi:10.1145/212094.212141 · Cited by 1,278
Hand Keypoint Detection in Single Images Using Multiview Bootstrapping
Simon, T., Joo, H., Matthews, I., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.494 · Cited by 1,119
Optical models for direct volume rendering
Max, N. (1995) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/2945.468400 · Cited by 1,059
Nerfies: Deformable Neural Radiance Fields
Park, K., Sinha, U., Barron, J. T., Bouaziz, S., Goldman, D. B., Seitz, S. M., & Martin-Brualla, R. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00581 · Cited by 1,059
DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time
Newcombe, R. A., Fox, D., & Seitz, S. M. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298631 · Cited by 951
Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes
Li, Z., Niklaus, S., Snavely, N., & Wang, O. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00643 · Cited by 674
Zwicker, M., Pfister, H., van Baar, J., & Gross, M. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383300 · Cited by 666
MOTS: Multi-Object Tracking and Segmentation
Voigtlaender, P., Krause, M., Osep, A., Luiten, J., Sekar, B. B. G., Geiger, A., & Leibe, B. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00813 · Cited by 658
Self-citationHyperNeRF: a higher-dimensional representation for topologically varying neural radiance fields
Park, K., Sinha, U., Hedman, P., Barron, J. T., Bouaziz, S., Goldman, D. B., Martin-Brualla, R., & Seitz, S. M. (2021) · ACM Transactions on Graphics · doi:10.1145/3478513.3480487 · Cited by 618
K-Planes: Explicit Radiance Fields in Space, Time, and Appearance
Fridovich-Keil, S., Meanti, G., Warburg, F. R., Recht, B., & Kanazawa, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01201 · Cited by 560
Panoptic Studio: A Massively Multiview System for Social Motion Capture
Joo, H., Liu, H., Tan, L., Gui, L., Nabbe, B., Matthews, I., Kanade, T., Nobuhara, S., & Sheikh, Y. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.381 · Cited by 546
HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video
Weng, C.-Y., Curless, B., Srinivasan, P. P., Barron, J. T., & Kemelmacher-Shlizerman, I. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01573 · Cited by 452
HexPlane: A Fast Representation for Dynamic Scenes
Cao, A., & Johnson, J. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00021 · Cited by 435
Vedula, S., Baker, S., Rander, P., Collins, R., & Kanade, T. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790293 · Cited by 426
Space-time Neural Irradiance Fields for Free-Viewpoint Video
Xian, W., Huang, J.-B., Kopf, J., & Kim, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00930 · Cited by 357
Robust Dynamic Radiance Fields
Liu, Y.-L., Gao, C., Meuleman, A., Tseng, H.-Y., Saraf, A., Kim, C., Chuang, Y., Kopf, J., & Huang, J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00010 · Cited by 326
PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking
Zheng, Y., Harley, A. W., Shen, B., Wetzstein, G., & Guibas, L. J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01818 · Cited by 326
Particle Video Revisited: Tracking Through Occlusions Using Point Trajectories
Harley, A. W., Fang, Z., & Fragkiadaki, K. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20047-2_4 · Cited by 321
Tracking Everything Everywhere All at Once
Wang, Q., Chang, Y., Cai, R., Li, Z., Hariharan, B., Holynski, A., & Snavely, N. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01813 · Cited by 250
Neural Radiance Flow for 4D View Synthesis and Video Processing
Du, Y., Zhang, Y., Yu, H.-X., Tenenbaum, J. B., & Wu, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01406 · Cited by 217
TAVA: Template-free Animatable Volumetric Actors
Li, R., Tanke, J., Vo, M., Zollhöfer, M., Gall, J., Kanazawa, A., & Lassner, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19824-3_25 · Cited by 217
CoTracker: It Is Better to Track Together
Karaev, N., Rocco, I., Graham, B., Neverova, N., Vedaldi, A., & Rupprecht, C. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-73033-7_2 · Cited by 212
HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion
Işık, M., Rünz, M., Georgopoulos, M., Khakhulin, T., Starck, J., Agapito, L., & Nießner, M. (2023) · ACM Transactions on Graphics · doi:10.1145/3592415 · Cited by 187
KillingFusion: Non-rigid 3D Reconstruction without Correspondences
Slavcheva, M., Baust, M., Cremers, D., & Ilic, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.581 · Cited by 177
The Ninth Visual Object Tracking VOT2021 Challenge Results
Kristan, M., Matas, J., Leonardis, A., Felsberg, M., Pflugfelder, R. P., Kämäräinen, J., Chang, H. J., Danelljan, M., Zajc, L. Č., Lukežič, A., et al. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00305 · Cited by 164
Self-citationTAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement
Doersch, C., Yang, Y., Vecerik, M., Gokay, D., Gupta, A., Aytar, Y., Carreira, J., & Zisserman, A. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00923 · Cited by 161
BANMo: Building Animatable 3D Neural Models from Many Casual Videos
Yang, G., Vo, M., Neverova, N., Ramanan, D., Vedaldi, A., & Joo, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00288 · Cited by 152
Self-citationRobust Dense Mapping for Large-Scale Dynamic Environments
Bârsan, I. A., Liu, P., Pollefeys, M., & Geiger, A. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8462974 · Cited by 139
SUDS: Scalable Urban Dynamic Scenes
Turki, H., Zhang, J. Y., Ferroni, F., & Ramanan, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01191 · Cited by 132
Self-citationTrack to Reconstruct and Reconstruct to Track
Luiten, J., Fischer, T., & Leibe, B. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.2969183 · Cited by 131
Self-citationDifferentiable Point-Based Radiance Fields for Efficient View Synthesis
Zhang, Q., Baek, S., Rusinkiewicz, S. M., & Heide, F. (2022) · SIGGRAPH Asia Conference Papers · doi:10.1145/3550469.3555413 · Cited by 69
Approximate Differentiable Rendering with Algebraic Surfaces
Keselman, L., & Hebert, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19824-3_35 · Cited by 56
Prokudin, S., Ma, Q., Raafat, M., Valentin, J., & Tang, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00732 · Cited by 50
MAP Visibility Estimation for Large-Scale Dynamic 3D Reconstruction
Joo, H., Park, H. S., & Sheikh, Y. A. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.147 · Cited by 32
Total-Recon: Deformable Scene Reconstruction for Embodied View Synthesis
Song, C., Yang, G., Deng, K., Zhu, J.-Y., & Ramanan, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01620 · Cited by 16
Self-citationNeural Pixel Composition for 3D-4D View Synthesis from Multi-Views
Bansal, A., & Zollhöfer, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00036 · Cited by 10
Cited by
4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
Wu, G., Yi, T., Fang, J., Xie, L., Zhang, X., Wei, W., Liu, W., Tian, Q., & Wang, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01920 · Cited by 806
Matsuki, H., Murai, R., Kelly, P. H. J., & Davison, A. J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01708 · Cited by 493
SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM
Keetha, N., Karhade, J., Jatavallabhula, K. M., Yang, G., Scherer, S., Ramanan, D., & Luiten, J. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02018 · Cited by 462
Self-citationDeformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction
Yang, Z., Gao, X., Zhou, W., Jiao, S., Zhang, Y., & Jin, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01922 · Cited by 452
GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting
Yan, C., Qu, D., Xu, D., Zhao, B., Wang, Z., Wang, D., & Li, X. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01853 · Cited by 390
Metadata
- Authors
- Luiten, J., Kopanas, G., Leibe, B., & Ramanan, D.
- Venue
- 2024 International Conference on 3D Vision (3DV)
- Publisher
- IEEE
- Published
- 2024-03-18
- Pages
- 800-809
- OpenAlex
- W4399574574
- Cited by
- 469
- References
- 44
- FWCI
- 134.40
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.