The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction
Open accessAbstract
Neural implicit 3D representations have emerged as a powerful paradigm for reconstructing surfaces from multi-view images and synthesizing novel views. Unfortunately, existing methods such as DVR or IDR require accurate per-pixel object masks as supervision. At the same time, neural radiance fields have revolutionized novel view synthesis. However, NeRF’s estimated volume density does not admit accurate surface reconstruction. Our key insight is that implicit surface models and radiance fields can be formulated in a unified way, enabling both surface and volume rendering using the same model. This unified perspective enables novel, more efficient sampling procedures and the ability to reconstruct accurate surfaces without input masks. We compare our method on the DTU, BlendedMVS, and a synthetic indoor dataset. Our experiments demonstrate that we outperform NeRF in terms of reconstruction quality while performing on par with IDR without requiring masks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multiple View Geometry in Computer Vision
Hartley, R., & Zisserman, A. (2004) · doi:10.1017/cbo9780511811685 · Cited by 21,070
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_24 · Cited by 5,351
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00025 · Cited by 3,422
A volumetric method for building complex models from range images
Curless, B., & Levoy, M. (1996) · Proceedings of the 23rd annual conference on Computer graphics and interactive techniques · doi:10.1145/237170.237269 · Cited by 3,124 · cited 1×
“The former approaches usually have a complex pipeline requiring additional steps like fusing depth information [9, 31] and meshing [18, 19], while the latter ones are limited to low resolution due to cubic memory requirements.”
BackgroundOccupancy Networks: Learning 3D Reconstruction in Function Space
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00459 · Cited by 2,708
Self-citationA Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457
Pixelwise View Selection for Unstructured Multi-View Stereo
Schönberger, J. L., Zheng, E., Frahm, J.-M., & Pollefeys, M. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_31 · Cited by 2,315
Screened poisson surface reconstruction
Kazhdan, M., & Hoppe, H. (2013) · ACM Transactions on Graphics · doi:10.1145/2487228.2487237 · Cited by 2,231
Poisson Surface Reconstruction
Kazhdan, M. M., Bolitho, M., & Hoppe, H. (2006) · doi:10.2312/sgp/sgp06/061-070 · Cited by 2,125
MVSNet: Depth Inference for Unstructured Multi-view Stereo
Yao, Y., Luo, Z., Li, S., Fang, T., & Quan, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_47 · Cited by 1,455
Marr, D., & Poggio, T. (1976) · Science · doi:10.1126/science.968482 · Cited by 1,377
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections
Martin-Brualla, R., Radwan, N., Sajjadi, M. S. M., Barron, J. T., Dosovitskiy, A., & Duckworth, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00713 · Cited by 1,374
D-NeRF: Neural Radiance Fields for Dynamic Scenes
Pumarola, A., Corona, E., Pons-Moll, G., & Moreno-Noguer, F. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01018 · Cited by 1,355
A Theory of Shape by Space Carving
Kutulakos, K. N., & Seitz, S. M. (2000) · International Journal of Computer Vision · doi:10.1023/a:1008191222954 · Cited by 1,254
PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
Saito, S., Huang, Z., Natsume, R., Morishima, S., Li, H., & Kanazawa, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00239 · Cited by 1,228
Local light field fusion: practical view synthesis with prescriptive sampling guidelines
Mildenhall, B., Srinivasan, P. P., Ortiz-Cayon, R., Kalantari, N. K., Ramamoorthi, R., Ng, R., & Kar, A. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322980 · Cited by 1,175
Learning to compare image patches via convolutional neural networks
Zagoruyko, S., & Komodakis, N. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299064 · Cited by 1,140
Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision
Niemeyer, M., Mescheder, L., Oechsle, M., & Geiger, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00356 · Cited by 834
Self-citationConvolutional Occupancy Networks
Peng, S., Niemeyer, M., Mescheder, L., Pollefeys, M., & Geiger, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58580-8_31 · Cited by 801
Self-citationEfficient Deep Learning for Stereo Matching
Luo, W., Schwing, A. G., & Urtasun, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.614 · Cited by 741
PatchMatch Stereo - Stereo Matching with Slanted Support Windows
Bleyer, M., Rhemann, C., & Rother, C. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.14 · Cited by 736
GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields
Niemeyer, M., & Geiger, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01129 · Cited by 731
Self-citationLarge Scale Multi-view Stereopsis Evaluation
Jensen, R., Dahl, A., Vogiatzis, G., Tola, E., & Aanaes, H. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.59 · Cited by 729
Neural volumes: learning dynamic renderable volumes from images
Lombardi, S., Simon, T., Saragih, J., Schwartz, G., Lehrmann, A., & Sheikh, Y. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323020 · Cited by 693
DeMoN: Depth and Motion Network for Learning Monocular Stereo
Ummenhofer, B., Zhou, H., Uhrig, J., Mayer, N., Ilg, E., Dosovitskiy, A., & Brox, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.596 · Cited by 692
Large-Scale Data for Multiple-View Stereopsis
Aanæs, H., Jensen, R. R., Vogiatzis, G., Tola, E., & Dahl, A. B. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0902-9 · Cited by 672
Recurrent MVSNet for High-Resolution Multi-View Stereo Depth Inference
Yao, Y., Luo, Z., Li, S., Shen, T., Fang, T., & Quan, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00567 · Cited by 666
Peng, S., Zhang, Y., Xu, Y., Wang, Q., Shuai, Q., Bao, H., & Zhou, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00894 · Cited by 626
Self-citationPhotorealistic scene reconstruction by voxel coloring
Seitz, S. M., & Dyer, C. R. (2002) · Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1997.609462 · Cited by 543
Multi-view Supervision for Single-View Reconstruction via Differentiable Ray Consistency
Tulsiani, S., Zhou, T., Efros, A. A., & Malik, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.30 · Cited by 519
NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis
Srinivasan, P. P., Deng, B., Zhang, X., Tancik, M., Mildenhall, B., & Barron, J. T. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00741 · Cited by 515
DeepMVS: Learning Multi-view Stereopsis
Huang, P.-H., Matzen, K., Kopf, J., Ahuja, N., & Huang, J.-B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00298 · Cited by 493
NeRF++: Analyzing and Improving Neural Radiance Fields
Zhang, K., Riegler, G., Snavely, N., & Koltun, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.07492 · Cited by 487
BlendedMVS: A Large-Scale Dataset for Generalized Multi-View Stereo Networks
Yao, Y., Luo, Z., Li, S., Zhang, J., Ren, Y., Zhou, L., Fang, T., & Quan, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00186 · Cited by 478
NeRD: Neural Reflectance Decomposition from Image Collections
Boss, M., Braun, R., Jampani, V., Barron, J. T., Liu, C., & Lensch, H. P. A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01245 · Cited by 438
Real-Time Visibility-Based Fusion of Depth Maps
Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J.-M., Yang, R., Nister, D., & Pollefeys, M. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408984 · Cited by 358
Learning Shape Templates With Structured Implicit Functions
Genova, K., Cole, F., Vlasic, D., Sarna, A., Freeman, W., & Funkhouser, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00725 · Cited by 323
GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
Schwarz, K., Liao, Y., Niemeyer, M., & Geiger, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02442 · Cited by 314
Self-citationSceneNet RGB-D: Can 5M Synthetic Images Beat Generic ImageNet Pre-training on Indoor Segmentation?
McCormac, J., Handa, A., Leutenegger, S., & Davison, A. J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.292 · Cited by 291
Texture Fields: Learning Texture Representations in Function Space
Oechsle, M., Mescheder, L., Niemeyer, M., Strauss, T., & Geiger, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00463 · Cited by 252
Self-citationDIST: Rendering Deep Implicit Signed Distance Function With Differentiable Sphere Tracing
Liu, S., Zhang, Y., Peng, S., Shi, B., Pollefeys, M., & Cui, Z. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00209 · Cited by 250
Self-citationOccupancy Flow: 4D Reconstruction by Learning Particle Dynamics
Niemeyer, M., Mescheder, L., Oechsle, M., & Geiger, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00548 · Cited by 246
Self-citationDISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction
Xu, Q., Wang, W., Ceylan, D., Měch, R., & Neumann, U. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10711 · Cited by 234
Implicit Surface Representations As Layers in Neural Networks
Michalkiewicz, M., Pontes, J. K., Jack, D., Baktashmotlagh, M., & Eriksson, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00484 · Cited by 227
A probabilistic framework for space carving
Broadhurst, A., Drummond, T. W., & Cipolla, R. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937544 · Cited by 225
Learning to Infer Implicit Surfaces without 3D SupervisionPreprint
Liu, S., Saito, S., Chen, W., & Li, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00767 · Cited by 186
OctNetFusion: Learning Depth Fusion from Data
Riegler, G., Ulusoy, A. O., Bischof, H., & Geiger, A. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00017 · Cited by 177
Self-citationControlling Neural Level SetsPreprint
Atzmon, M., Haim, N., Yariv, L., Israelov, O., Maron, H., & Lipman, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11911 · Cited by 116
RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials
Paschalidou, D., Ulusoy, A. O., Schmitt, C., Gool, L. V., & Geiger, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00410 · Cited by 102
Self-citationShape Reconstruction Using Volume Sweeping and Learned Photoconsistency
Leroy, V., Franco, J.-S., & Boyer, E. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_48 · Cited by 63
Poxels: Probabilistic Voxelized Volume Reconstruction
Bonet, J. S. D. (1999) · Cited by 60
Towards Probabilistic Volumetric Reconstruction Using Ray Potentials
Ulusoy, A. O., Geiger, A., & Black, M. J. (2015) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2015.9 · Cited by 56
Self-citationLearning Implicit Surface Light Fields
Oechsle, M., Niemeyer, M., Reiser, C., Mescheder, L., Strauss, T., & Geiger, A. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00055 · Cited by 49
Self-citationLearning Non-Volumetric Depth Fusion Using Successive Reprojections
Donné, S., & Geiger, A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00782 · Cited by 40
Self-citation
Cited by
Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields
Barron, J. T., Mildenhall, B., Verbin, D., Srinivasan, P. P., & Hedman, P. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00539 · Cited by 1,935
TensoRF: Tensorial Radiance Fields
Chen, A., Xu, Z., Geiger, A., Yu, J., & Su, H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19824-3_20 · Cited by 1,184
Self-citationEfficient Geometry-aware 3D Generative Adversarial Networks
Chan, E. R., Lin, C. Z., Chan, M. A., Nagano, K., Pan, B., de Mello, S., Gallo, O., Guibas, L., Tremblay, J., Khamis, S., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01565 · Cited by 1,093
NICE-SLAM: Neural Implicit Scalable Encoding for SLAM
Zhu, Z., Peng, S., Larsson, V., Xu, W., Bao, H., Cui, Z., Oswald, M. R., & Pollefeys, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01245 · Cited by 825
Self-citationRegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs
Niemeyer, M., Barron, J. T., Mildenhall, B., Sajjadi, M. S. M., Geiger, A., & Radwan, N. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00540 · Cited by 609
Self-citation
Metadata
- Authors
- Oechsle, M., Peng, S., & Geiger, A.
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 5569-5579
- OpenAlex
- W3156022514
- Cited by
- 622
- References
- 56
- FWCI
- 87.08
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.