The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Efficient Geometry-aware 3D Generative Adversarial Networks
Open accessAbstract
Unsupervised generation of high-quality multi-view-consistent images and 3D shapes using only collections of single-view 2D photographs has been a long-standing challenge. Existing 3D GANs are either compute intensive or make approximations that are not 3D-consistent; the former limits quality and resolution of the generated images and the latter adversely affects multi-view consistency and shape quality. In this work, we improve the computational efficiency and image quality of 3D GANs without overly relying on these approximations. We introduce an expressive hybrid explicit implicit network architecture that, together with other design choices, synthesizes not only high-resolution multi-view-consistent images in real time but also produces high-quality 3D geometry. By decoupling feature generation and neural rendering, our framework is able to leverage state-of-the-art 2D CNN generators, such as StyleGAN2, and inherit their efficiency and expressiveness. We demonstrate state-of-the-art 3D-aware synthesis with FFHQ and AFHQ Cats, among other experiments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714
Self-citationArcFace: Additive Angular Margin Loss for Deep Face Recognition
Deng, J., Guo, J., Xue, N., & Zafeiriou, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00482 · Cited by 7,781
NeRF: representing scenes as neural radiance fields for view synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2022) · Communications of the ACM · doi:10.1145/3503250 · Cited by 6,852
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332
Self-citationDeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00025 · Cited by 3,422
Occupancy Networks: Learning 3D Reconstruction in Function Space
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00459 · Cited by 2,708
Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields
Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., & Srinivasan, P. P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00580 · Cited by 1,921
Tanks and temples: benchmarking large-scale scene reconstruction
Knapitsch, A., Park, J., Zhou, Q.-Y., & Koltun, V. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073599 · Cited by 1,707
StarGAN v2: Diverse Image Synthesis for Multiple Domains
Choi, Y., Uh, Y., Yoo, J., & Ha, J.-W. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00821 · Cited by 1,663
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Tancik, M., Srinivasan, P. P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ramamoorthi, R., Barron, J. T., & Ng, R. (2020) · arXiv · doi:10.48550/arxiv.2006.10739 · Cited by 1,459
Learning Implicit Fields for Generative Shape Modeling
Chen, Z., & Zhang, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00609 · Cited by 1,443
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
Wu, J., Zhang, C., Xue, T., Freeman, W. T., & Tenenbaum, J. B. (2016) · arXiv · doi:10.48550/arxiv.1610.07584 · Cited by 1,393 · cited 2×
“Voxel-based GANs directly extend the CNN generators used in 2D settings to 3D [14,21,47,48,68,74].”
BackgroundNeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections
Martin-Brualla, R., Radwan, N., Sajjadi, M. S. M., Barron, J. T., Dosovitskiy, A., & Duckworth, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00713 · Cited by 1,374
D-NeRF: Neural Radiance Fields for Dynamic Scenes
Pumarola, A., Corona, E., Pons-Moll, G., & Moreno-Noguer, F. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01018 · Cited by 1,355
Training Generative Adversarial Networks with Limited Data
Karras, T., Aittala, M., Hellsten, J., Laine, S., Lehtinen, J., & Aila, T. (2020) · arXiv · doi:10.48550/arxiv.2006.06676 · Cited by 1,170
Self-citationOptical models for direct volume rendering
Max, N. (1995) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/2945.468400 · Cited by 1,059
PlenOctrees for Real-time Rendering of Neural Radiance Fields
Yu, A., Li, R., Tancik, M., Li, H., Ng, R., & Kanazawa, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00570 · Cited by 854
Alias-Free Generative Adversarial Networks
Karras, T., Aittala, M., Laine, S., Härkönen, E., Hellsten, J., Lehtinen, J., & Aila, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.12423 · Cited by 840
Self-citationDifferentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision
Niemeyer, M., Mescheder, L., Oechsle, M., & Geiger, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00356 · Cited by 834
Convolutional Occupancy Networks
Peng, S., Niemeyer, M., Mescheder, L., Pollefeys, M., & Geiger, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58580-8_31 · Cited by 801
Learning Continuous Image Representation with Local Implicit Image Function
Chen, Y., Liu, S., & Wang, X. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00852 · Cited by 782
Which Training Methods for GANs do actually Converge?
Mescheder, L., Geiger, A., & Nowozin, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.04406 · Cited by 768
Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set
Deng, Y., Yang, J., Xu, S., Chen, D., Jia, Y., & Tong, X. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00038 · Cited by 763
GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields
Niemeyer, M., & Geiger, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01129 · Cited by 731
Neural volumes: learning dynamic renderable volumes from images
Lombardi, S., Simon, T., Saragih, J., Schwartz, G., Lehrmann, A., & Sheikh, Y. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3323020 · Cited by 693
pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis
Chan, E. R., Monteiro, M., Kellnhofer, P., Wu, J., & Wetzstein, G. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00574 · Cited by 689
Self-citationKiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs
Reiser, C., Peng, S., Liao, Y., & Geiger, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01407 · Cited by 668
UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction
Oechsle, M., Peng, S., & Geiger, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00554 · Cited by 622
DeepVoxels: Learning Persistent 3D Feature Embeddings
Sitzmann, V., Thies, J., Heide, F., Nießner, M., Wetzstein, G., & Zollhöfer, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00254 · Cited by 558
Self-citationFastNeRF: High-Fidelity Neural Rendering at 200FPS
Garbin, S. J., Kowalski, M., Johnson, M., Shotton, J., & Valentin, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01408 · Cited by 541
Local Implicit Grid Representations for 3D Scenes
Jiang, C., Sud, A., Makadia, A., Huang, J., NieBner, M., & Funkhouser, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00604 · Cited by 526
NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis
Srinivasan, P. P., Deng, B., Zhang, X., Tancik, M., Mildenhall, B., & Barron, J. T. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00741 · Cited by 515
Neural scene representation and rendering
Eslami, S. M. A., Jimenez Rezende, D., Besse, F., Viola, F., Morcos, A. S., Garnelo, M., Ruderman, A., Rusu, A. A., Danihelka, I., Gregor, K., et al. (2018) · Science · doi:10.1126/science.aar6170 · Cited by 495
NeRF++: Analyzing and Improving Neural Radiance Fields
Zhang, K., Riegler, G., Snavely, N., & Koltun, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.07492 · Cited by 487
Baking Neural Radiance Fields for Real-Time View Synthesis
Hedman, P., Srinivasan, P. P., Mildenhall, B., Barron, J. T., & Debevec, P. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00582 · Cited by 451
Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes
Takikawa, T., Litalien, J., Yin, K., Kreis, K., Loop, C., Nowrouzezahrai, D., Jacobson, A., McGuire, M., & Fidler, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01120 · Cited by 449
SAL: Sign Agnostic Learning of Shapes From Raw Data
Atzmon, M., & Lipman, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00264 · Cited by 433
GANs Trained by a Two Time-Scale Update Rule Converge to a Nash EquilibriumPreprint
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Klambauer, G., & Hochreiter, S. (2017) · arXiv (Cornell University) · Cited by 407
Pivotal Tuning for Latent-based Editing of Real Images
Roich, D., Mokady, R., Bermano, A. H., & Cohen-Or, D. (2023) · ACM Transactions on Graphics · doi:10.1145/3544777 · Cited by 407
Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction
Chabra, R., Lenssen, J. E., Ilg, E., Schmidt, T., Straub, J., Lovegrove, S., & Newcombe, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58526-6_36 · Cited by 380
HoloGAN: Unsupervised Learning of 3D Representations From Natural Images
Nguyen-Phuoc, T., Li, C., Theis, L., Richardt, C., & Yang, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00768 · Cited by 340
GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis
Schwarz, K., Liao, Y., Niemeyer, M., & Geiger, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02442 · Cited by 314
3D Shape Induction from 2D Views of Multiple Objects
Gadelha, M., Maji, S., & Wang, R. (2017) · 2017 International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00053 · Cited by 260
DONeRF: Towards Real‐Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks
Neff, T., Stadlbauer, P., Parger, M., Kurz, A., Mueller, J. H., Chaitanya, C. R. A., Kaplanyan, A., & Steinberger, M. (2021) · Computer Graphics Forum · doi:10.1111/cgf.14340 · Cited by 256
DIST: Rendering Deep Implicit Signed Distance Function With Differentiable Sphere Tracing
Liu, S., Zhang, Y., Peng, S., Shi, B., Pollefeys, M., & Cui, Z. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00209 · Cited by 250
Implicit Surface Representations As Layers in Neural Networks
Michalkiewicz, M., Pontes, J. K., Jack, D., Baktashmotlagh, M., & Eriksson, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00484 · Cited by 227
AutoInt: Automatic Integration for Fast Neural Volume Rendering
Lindell, D. B., Martel, J. N. P., & Wetzstein, G. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01432 · Cited by 213
Self-citationSDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization
Jiang, Y., Ji, D., Han, Z., & Zwicker, M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00133 · Cited by 208
Escaping Plato’s Cave: 3D Shape From Adversarial Rendering
Henzler, P., Mitra, N., & Ritschel, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.01008 · Cited by 194
StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
Gu, J., Liu, L., Wang, P., & Theobalt, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08985 · Cited by 178
Visual Object Networks: Image Generation with Disentangled 3D RepresentationPreprint
Zhu, J.-Y., Zhang, Z., Zhang, C., Wu, J., Torralba, A. B., Tenenbaum, J., & Freeman, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.02725 · Cited by 167
Acorn: adaptive coordinate networks for neural scene representation
Martel, J. N. P., Lindell, D. B., Lin, C. Z., Chan, E. R., Monteiro, M., & Wetzstein, G. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459785 · Cited by 165
Self-citationCIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel SynthesisPreprint
Zhou, P., Xie, L., Ni, B., & Tian, Q. C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.09788 · Cited by 148
Kellnhofer, P., Jebe, L., Jones, A., Spicer, R. P., Pulli, K., & Wetzstein, G. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00427 · Cited by 133
Self-citationBlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled ImagesPreprint
Nguyen-Phuoc, T., Richardt, C., Mai, L., Yang, Y., & Mitra, N. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.08988 · Cited by 119
Unconstrained Scene Generation with Locally Conditioned Radiance Fields
DeVries, T., Bautista, M. Á., Srivastava, N., Taylor, G. W., & Susskind, J. M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01404 · Cited by 119
Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis
Liao, Y., Schwarz, K., Mescheder, L., & Geiger, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00591 · Cited by 115
GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds
Hao, Z., Mallya, A., Belongie, S., & Liu, M.-Y. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01381 · Cited by 99
Unsupervised Generative 3D Shape Learning from Natural ImagesPreprint
Szabó, A., Meishvili, G., & Favaro, P. (2019) · arXiv.org · doi:10.48550/arxiv.1910.00287 · Cited by 75
Lifting 2D StyleGAN for 3D-Aware Face Generation
Shi, Y., Aggarwal, D., & Jain, A. K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00619 · Cited by 70
Cited by
TensoRF: Tensorial Radiance Fields
Chen, A., Xu, Z., Geiger, A., Yu, J., & Su, H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19824-3_20 · Cited by 1,184
A review of convolutional neural networks in computer vision
Zhao, X., Wang, L., Zhang, Y., Han, X., Deveci, M., & Parmar, M. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10721-6 · Cited by 1,148
Data augmentation: A comprehensive survey of modern approaches
Mumuni, A., & Mumuni, F. (2022) · Array · doi:10.1016/j.array.2022.100258 · Cited by 891
Magic3D: High-Resolution Text-to-3D Content Creation
Lin, C.-H., Gao, J., Tang, L., Takikawa, T., Zeng, X., Huang, X., Kreis, K., Fidler, S., Liu, M.-Y., & Lin, T.-Y. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00037 · Cited by 857
Self-citationK-Planes: Explicit Radiance Fields in Space, Time, and Appearance
Fridovich-Keil, S., Meanti, G., Warburg, F. R., Recht, B., & Kanazawa, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01201 · Cited by 560
Metadata
- Authors
- Chan, E. R., Lin, C. Z., Chan, M. A., Nagano, K., Pan, B., de Mello, S., Gallo, O., Guibas, L., Tremblay, J., Khamis, S., Karras, T., & Wetzstein, G.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 16102-16112
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4312453532
- Cited by
- 1,093
- References
- 62
- FWCI
- 95.13
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.