The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Soft 3D reconstruction for view synthesis
Open accessAbstract
We present a novel algorithm for view synthesis that utilizes a soft 3D reconstruction to improve quality, continuity and robustness. Our main contribution is the formulation of a soft 3D representation that preserves depth uncertainty through each stage of 3D reconstruction and rendering. We show that this representation is beneficial throughout the view synthesis pipeline. During view synthesis, it provides a soft model of scene geometry that provides continuity across synthesized views and robustness to depth uncertainty. During 3D reconstruction, the same robust estimates of scene visibility can be applied iteratively to improve depth estimation around object edges. Our algorithm is based entirely on O(1) filters, making it conducive to acceleration and it works with structured or unstructured sets of input views. We compare with recent classical and learning-based algorithms on plenoptic lightfields, wide baseline captures, and lightfield videos produced from camera arrays.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
(2015) · doi:10.1109/cvpr31182.2015 · Cited by 3,737
Photo tourism: exploring photo collections in 3D
Snavely, N., Seitz, S. M., & Szeliski, R. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141964 · Cited by 2,866
Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach
Debevec, P. E., Taylor, C. J., & Malik, J. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237191 · Cited by 1,905
He, K., Sun, J., & Tang, X. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15549-9_1 · Cited by 1,565
Efficient Belief Propagation for Early Vision
Felzenszwalb, P. F., & Huttenlocher, D. P. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-006-7899-4 · Cited by 1,345
Stereo matching using belief propagation
Sun, J., Zheng, N.-N., & Shum, H.-Y. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1206509 · Cited by 1,271
Plenoptic modeling: an image-based rendering system
McMillan, L., & Bishop, G. (1995) · Proceedings of the 22nd annual conference on Computer graphics and interactive techniques - SIGGRAPH '95 · doi:10.1145/218380.218398 · Cited by 1,262
Computing visual correspondence with occlusions using graph cuts
Kolmogorov, V., & Zabih, R. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937668 · Cited by 1,119
View interpolation for image synthesis
Chen, S. E., & Williams, L. (1993) · Proceedings of the 20th annual conference on Computer graphics and interactive techniques · doi:10.1145/166117.166153 · Cited by 1,097
Cited by
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Kerbl, B., Kopanas, G., Leimkuehler, T., & Drettakis, G. (2023) · ACM Transactions on Graphics · doi:10.1145/3592433 · Cited by 6,025
“Volumetric representations for novel-view synthesis were initiated by Soft3D [Penner and Zhang 2017]; deep-learning techniques coupled with volumetric ray-marching were subsequently proposed [Henzler et al. 2019; Sitzmann et al. 2019] building on a continuous differentiable density field to…”
MethodNeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351
Plenoxels: Radiance Fields without Neural Networks
Fridovich-Keil, S., Yu, A., Tancik, M., Chen, Q., Recht, B., & Kanazawa, A. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00542 · Cited by 1,388
Deferred neural rendering: image synthesis using neural textures
Thies, J., Zollhöfer, M., & Nießner, M. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323035 · Cited by 1,356
Local light field fusion: practical view synthesis with prescriptive sampling guidelines
Mildenhall, B., Srinivasan, P. P., Ortiz-Cayon, R., Kalantari, N. K., Ramamoorthi, R., Ng, R., & Kar, A. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322980 · Cited by 1,175
PlenOctrees for Real-time Rendering of Neural Radiance Fields
Yu, A., Li, R., Tancik, M., Li, H., Ng, R., & Kanazawa, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00570 · Cited by 854
MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo
Chen, A., Xu, Z., Zhao, F., Zhang, X., Xiang, F., Yu, J., & Su, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01386 · Cited by 797
Stereo magnification: learning view synthesis using multiplane images
Zhou, T., Tucker, R., Flynn, J., Fyffe, G., & Snavely, N. (2018) · ACM Transactions on Graphics · doi:10.1145/3197517.3201323 · Cited by 776
IBRNet: Learning Multi-View Image-Based Rendering
Wang, Q., Wang, Z., Genova, K., Srinivasan, P., Zhou, H., Barron, J. T., Martin-Brualla, R., Snavely, N., & Funkhouser, T. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00466 · Cited by 725
Neural volumes: learning dynamic renderable volumes from images
Lombardi, S., Simon, T., Saragih, J., Schwartz, G., Lehrmann, A., & Sheikh, Y. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323020 · Cited by 693
Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes
Li, Z., Niklaus, S., Snavely, N., & Wang, O. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00643 · Cited by 674
KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs
Reiser, C., Peng, S., Liao, Y., & Geiger, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01407 · Cited by 668
Peng, S., Zhang, Y., Xu, Y., Wang, Q., Shuai, Q., Bao, H., & Zhou, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00894 · Cited by 626
Deep blending for free-viewpoint image-based rendering
Hedman, P., Philip, J., Price, T., Frahm, J.-M., Drettakis, G., & Brostow, G. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275084 · Cited by 617
DeepVoxels: Learning Persistent 3D Feature Embeddings
Sitzmann, V., Thies, J., Heide, F., Nießner, M., Wetzstein, G., & Zollhöfer, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00254 · Cited by 558
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.08934 · Cited by 524
Baking Neural Radiance Fields for Real-Time View Synthesis
Hedman, P., Srinivasan, P. P., Mildenhall, B., Barron, J. T., & Debevec, P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00582 · Cited by 451
DeepView: View Synthesis With Learned Gradient Descent
Flynn, J., Broxton, M., Debevec, P., DuVall, M., Fyffe, G., Overbeck, R., Snavely, N., & Tucker, R. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00247 · Cited by 424
SynSin: End-to-End View Synthesis From a Single Image
Wiles, O., Gkioxari, G., Szeliski, R., & Johnson, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00749 · Cited by 418
Space-time Neural Irradiance Fields for Free-Viewpoint Video
Xian, W., Huang, J.-B., Kopf, J., & Kim, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00930 · Cited by 357
Dynamic View Synthesis from Dynamic Monocular Video
Gao, C., Saraf, A., Kopf, J., & Huang, J.-B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00566 · Cited by 346
Single-View View Synthesis With Multiplane Images
Tucker, R., & Snavely, N. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00063 · Cited by 324
Pushing the Boundaries of View Extrapolation With Multiplane Images
Srinivasan, P. P., Tucker, R., Barron, J. T., Ramamoorthi, R., Ng, R., & Snavely, N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00026 · Cited by 322
Chen, Z., Funkhouser, T., Hedman, P., & Tagliasacchi, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01590 · Cited by 314
3D Photography Using Context-Aware Layered Depth Inpainting
Shih, M.-L., Su, S.-Y., Kopf, J., & Huang, J.-B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00805 · Cited by 280
Immersive light field video with a layered mesh representation
Broxton, M., Flynn, J., Overbeck, R., Erickson, D., Hedman, P., Duvall, M., Dourgarian, J., Busch, J., Whalen, M., & Debevec, P. (2020) · ACM Transactions on Graphics · doi:10.1145/3386569.3392485 · Cited by 266
Neural Articulated Radiance Field
Noguchi, A., Sun, X., Lin, S., & Harada, T. (2021) · arXiv · doi:10.48550/arxiv.2104.03110 · Cited by 177
Geometry-Free View Synthesis: Transformers and no 3D Priors
Rombach, R., Esser, P., & Ommer, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01409 · Cited by 75
Stereo Magnification: Learning View Synthesis using Multiplane Images
Zhou, T., Tucker, R., Flynn, J., Fyffe, G., & Snavely, N. (2018) · arXiv · doi:10.48550/arxiv.1805.09817 · Cited by 150
Metadata
- Authors
- Penner, E., & Zhang, L.
- Venue
- ACM Transactions on Graphics
- Published
- 2017-11-20
- Volume
- 36
- Issue
- 6
- Pages
- 1-11
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2758175304
- Cited by
- 358
- References
- 35
- FWCI
- 11.68
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.