CoChat
Sign Up
Cited by 395

PixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction

Charatan, David, Li, Sizhe Lester, Tagliasacchi, Andrea, Sitzmann, Vincent·2024·2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr52733.2024.01840
Open this paper in CoChat — free account

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr52733.2024.01840

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Single-image 3D view synthesis is particularly challenging, typically requiring the learning of a strong 3D prior model from multi-view image datasets.

    “Within this line of research, single-image 3D view synthesis (Charatan et al., 2024; Lin et al., 2023; Tucker and Snavely, 2020; Wiles et al., 2020; Yu et al., 2021) is particularly challenging, typically requiring the learning of a strong 3D prior model from multi-view image datasets.”

    From Cosmos World Foundation Model Platform for Physical AI (2025) · cites this paper as Charatan et al., 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 54

  1. Kara-Ali Aliev, Artem Sevastopolsky, Maria Kolos, Dmitry Ulyanov, and Victor Lempitsky. Neural point-based graphics. In Proceedings of the European Conference on Computer Vision (ECCV), pages 696-712. Springer, 2020. 2
  2. Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P. Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the International Confer- ence on Computer Vision (ICCV), 2021. 2
  3. Eric R Chan, Koki Nagano, Matthew A Chan, Alexander W Bergman, Jeong Joon Park, Axel Levy, Miika Aittala, Shalini De Mello, Tero Karras, and Gordon Wetzstein. Generative novel view synthesis with 3d-aware diffusion models. Pro- ceedings of the International Conference on 3D Vision (3DV), 2023. 2
  4. Anpei Chen, Zexiang Xu, Fuqiang Zhao, Xiaoshuai Zhang, Fanbo Xiang, Jingyi Yu, and Hao Su. Mvsnerf: Fast general- izable radiance field reconstruction from multi-view stereo. In Proceedings of the International Conference on Computer Vision (ICCV), pages 14124-14133, 2021.
  5. Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. arXiv preprint arXiv:2203.09517, 2022. 2 · arXiv
  6. Julian Chibane, Aayush Bansal, Verica Lazova, and Gerard Pons-Moll. Stereo radiance fields (srf): Learning view syn- thesis from sparse views of novel scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021. 2
  7. Arthur P Dempster, Nan M Laird, and Donald B Rubin. Max- imum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society: series B (methodologi- cal), 39(1):1-22, 1977. 2
  8. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Transform- ers for image recognition at scale. Proceedings of the Interna- tional Conference on Learning Representations (ICLR), 2021. 6
  9. Yilun Du, Cameron Smith, Ayush Tewari, and Vincent Sitz- mann. Learning to render novel views from wide-baseline stereo pairs. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), 2023. 1, 2, 5, 6, 7, 12, 15, 16
  10. David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep network. In Advances in Neural Information Processing Systems (NeurIPS), 2014. 2, 3
  11. Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), pages 5501-5510, 2022. 2
  12. Clément Godard, Oisin Mac Aodha, Michael Firman, and Gabriel J Brostow. Digging into self-supervised monocular depth estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 2, 3
  13. Pengsheng Guo, Miguel Angel Bautista, Alex Colburn, Liang Yang, Daniel Ulbricht, Joshua M Susskind, and Qi Shan. Fast and explicit neural view synthesis. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 3791-3800, 2022. 2
  14. Yihui He, Rui Yan, Katerina Fragkiadaki, and Shoou-I Yu. Epipolar transformers. In Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2020. 2, 3
  15. Chi Jin, Yuchen Zhang, Sivaraman Balakrishnan, Martin J Wainwright, and Michael I Jordan. Local maxima in the likelihood of gaussian mixture models: Structural results and algorithmic consequences. Advances in Neural Information Processing Systems (NeurIPS), 29, 2016. 2
  16. Mohammad Mahdi Johari, Yann Lepoittevin, and Franc ¸ois Fleuret. Geonerf: Generalizing nerf with geometry priors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 18365-18375, 2022. 2
  17. Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. 6
  18. Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes. Proceedings of the International Conference on Learning Representations (ICLR), 2014. 5
  19. Georgios Kopanas, Julien Philip, Thomas Leimkühler, and George Drettakis. Point-based neural rendering with per-view optimization. In Computer Graphics Forum, pages 29-43. Wiley Online Library, 2021. 2
  20. Kai-En Lin, Lin Yen-Chen, Wei-Sheng Lai, Tsung-Yi Lin, Yi-Chang Shih, and Ravi Ramamoorthi. Vision transformer for nerf-based view synthesis from a single input image. Pro- ceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2022. 2
  21. Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Maka- dia, Noah Snavely, and Angjoo Kanazawa. Infinite nature: Perpetual view generation of natural scenes from a single image. In Proceedings of the International Conference on Computer Vision (ICCV), 2021. 6
  22. Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. Neural sparse voxel fields. Advances in Neural Information Processing Systems (NeurIPS), 2020. 2
  23. Yuan Liu, Sida Peng, Lingjie Liu, Qianqian Wang, Peng Wang, Christian Theobalt, Xiaowei Zhou, and Wenping Wang. Neural rays for occlusion-aware image-based rendering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 7824-7833, 2022. 2
  24. Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, and Yaser Sheikh. Neural volumes: Learning dynamic renderable volumes from images. ACM Trans. Graph., 2019. 2
  25. Ricardo Martin-Brualla, Noha Radwan, Mehdi SM Sajjadi, Jonathan T Barron, Alexey Dosovitskiy, and Daniel Duck- worth. Nerf in the wild: Neural radiance fields for uncon- strained photo collections. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 7210-7219, 2021. 2
  26. Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthe- sis. In Proceedings of the European Conference on Computer Vision (ECCV), pages 405-421, 2020. 1, 2
  27. Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolu- tion hash encoding. ACM Trans. Graph., 41(4):102:1-102:15, 2022.
  28. Thu H Nguyen-Phuoc, Chuan Li, Stephen Balaban, and Yongliang Yang. Rendernet: A deep convolutional network for differentiable rendering from 3d shapes. Advances in Neural Information Processing Systems (NeurIPS), 31, 2018. 2
  29. Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. Differentiable volumetric rendering: Learn- ing implicit 3d representations without 3d supervision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3504-3515, 2020. 2
  30. René Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. 2, 3
  31. Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Com- mon objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the International Conference on Computer Vision (ICCV), pages 10901-10911, 2021. 2
  32. Gernot Riegler and Vladlen Koltun. Free view synthesis. In Proceedings of the European Conference on Computer Vision (ECCV), 2020. 2
  33. Mehdi SM Sajjadi, Henning Meyer, Etienne Pot, Urs Bergmann, Klaus Greff, Noha Radwan, Suhani Vora, Mario Lucic, Daniel Duckworth, Alexey Dosovitskiy, et al. Scene representation transformer: Geometry-free novel view synthe- sis through set-latent scene representations. arXiv preprint arXiv:2111.13152, 2021. 1, 2 · arXiv
  34. Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Herrmann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry La- gun, Li Fei-Fei, Deqing Sun, et al. Zeronvs: Zero-shot 360- degree view synthesis from a single real image. arXiv preprint arXiv:2310.17994, 2023. 2 · arXiv
  35. Prafull Sharma, Ayush Tewari, Yilun Du, Sergey Zakharov, Rares Andrei Ambrus, Adrien Gaidon, William T Freeman, Fredo Durand, Joshua B Tenenbaum, and Vincent Sitzmann. Neural groundplans: Persistent neural scene representations from a single image. In Proceedings of the International Conference on Learning Representations (ICLR), 2022. 2
  36. Vincent Sitzmann, Justus Thies, Felix Heide, Matthias Nießner, Gordon Wetzstein, and Michael Zollhöfer. Deepvox- els: Learning persistent 3d feature embeddings. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 1, 2
  37. Vincent Sitzmann, Michael Zollhöfer, and Gordon Wetzstein. Scene representation networks: Continuous 3d-structure- aware neural scene representations. Advances in Neural In- formation Processing Systems (NeurIPS), 32, 2019. 1, 2
  38. Vincent Sitzmann, Julien N.P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wetzstein. Implicit neural rep- resentations with periodic activation functions. In Advances in Neural Information Processing Systems (NeurIPS), 2020. 2
  39. Cameron Smith, Yilun Du, Ayush Tewari, and Vincent Sitz- mann. Flowcam: Training generalizable 3d radiance fields without camera poses via pixel-aligned scene flow. Advances in Neural Information Processing Systems (NeurIPS), 2023. 8
  40. Pratul P Srinivasan, Richard Tucker, Jonathan T Barron, Ravi Ramamoorthi, Ren Ng, and Noah Snavely. Pushing the bound- aries of view extrapolation with multiplane images. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 175-184, 2019. 2
  41. Mohammed Suhail, Carlos Esteves, Leonid Sigal, and Ameesh Makadia. Generalizable patch-based neural render- ing. In Proceedings of the European Conference on Computer Vision (ECCV). Springer, 2022. 2, 6, 7, 12, 15, 16
  42. Mohammed Suhail, Carlos Esteves, Leonid Sigal, and Ameesh Makadia. Light field neural rendering. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 8269-8279, 2022. 1, 2, 5
  43. Stanislaw Szymanowicz, Christian Rupprecht, and Andrea Vedaldi. Viewset diffusion:(0-) image-conditioned 3d genera- tive models from 2d data. Proceedings of the International Conference on Computer Vision (ICCV), 2023. 8
  44. Andrea Tagliasacchi and Ben Mildenhall. Volume rendering digest (for nerf). arXiv preprint arXiv:2209.02417, 2022. 2 · arXiv
  45. Ayush Tewari, Justus Thies, Ben Mildenhall, Pratul Srini- vasan, Edgar Tretschk, W Yifan, Christoph Lassner, Vincent Sitzmann, Ricardo Martin-Brualla, Stephen Lombardi, et al. Advances in neural rendering. In Computer Graphics Forum, pages 703-735. Wiley Online Library, 2022. 1, 2
  46. Ayush Tewari, Tianwei Yin, George Cazenavette, Semon Rezchikov, Joshua B Tenenbaum, Frédo Durand, William T Freeman, and Vincent Sitzmann. Diffusion with forward models: Solving stochastic inverse problems without direct supervision. Advances in Neural Information Processing Systems (NeurIPS), 2023. 2, 8
  47. Alex Trevithick and Bo Yang. Grf: Learning a general radi- ance field for 3d representation and rendering. In Proceedings of the International Conference on Computer Vision (ICCV), pages 15182-15192, 2021. 2
  48. Richard Tucker and Noah Snavely. Single-view view synthe- sis with multiplane images. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 551-560, 2020. 2
  49. Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul Srini- vasan, Howard Zhou, Jonathan T. Barron, Ricardo Martin- Brualla, Noah Snavely, and Thomas Funkhouser. Ibrnet: Learning multi-view image-based rendering. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 2
  50. Olivia Wiles, Georgia Gkioxari, Richard Szeliski, and Justin Johnson. Synsin: End-to-end view synthesis from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 7467-7477, 2020. 2
  51. Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelNeRF: Neural radiance fields from one or few images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 1, 2, 6, 7, 12, 15, 16
  52. Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep networks as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), 2018. 6
  53. Yunzhi Zhang and Jiajun Wu. Video extrapolation in space and time. arXiv e-prints, pages arXiv-2205, 2022. 2
  54. Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view syn- thesis using multiplane images. ACM Trans. Graph. (Proc. SIGGRAPH), 37, 2018. 2, 6

Cited by

Sorted by citations

Authors
Charatan, D., Li, S. L., Tagliasacchi, A., & Sitzmann, V.
Affiliations
  • Massachusetts Institute of Technology
  • Simon Fraser University, University of Toronto
Venue
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2024-06-16
Pages
19457-19467
DOI
10.1109/cvpr52733.2024.01840
Cited by
395
References
6
Licence
Funders
National Science Foundation (2211259)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review