DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis
Open accessThe DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
We present DiffuScene for indoor 3D scene synthesis based on a novel scene configuration denoising diffusion model. It generates 3D instance properties stored in an unordered object set and retrieves the most similar geometry for each object configuration, which is characterized as a concatenation of different attributes, including location, size, orientation, semantics, and geometry features. We introduce a diffusion network to synthesize a collection of 3D indoor objects by denoising a set of unordered object attributes. Unordered parametrization simplifies and eases the joint distribution approximation. The shape feature diffusion facilitates natural object placements, including symmetries. Our method enables many downstream applications, including scene completion, scene arrangement, and text-conditioned scene synthesis. Experiments on the 3D-FRONT dataset show that our method can synthesize more physically plausible and diverse indoor scenes than state-of-the-art methods. Extensive ablation studies verify the effectiveness of our design choice in scene diffusion models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Generative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020) · Communications of the ACM · doi:10.1145/3422622 · Cited by 18,384
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365
NeRF: representing scenes as neural radiance fields for view synthesis
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2022) · Communications of the ACM · doi:10.1145/3503250 · Cited by 6,852
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362
Generating Sequences With Recurrent Neural Networks
Graves, A. (2013) · arXiv · doi:10.48550/arxiv.1308.0850 · Cited by 2,894
Neural Discrete Representation Learning
Oord, A. V. D., Vinyals, O., & Kavukcuoglu, K. (2017) · arXiv · doi:10.48550/arxiv.1711.00937 · Cited by 2,729
Deep Unsupervised Learning using Nonequilibrium Thermodynamics
Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., & Ganguli, S. (2015) · arXiv · doi:10.48550/arxiv.1503.03585 · Cited by 2,637
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2020) · arXiv · doi:10.48550/arxiv.2012.09841 · Cited by 2,514
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01268 · Cited by 2,514 · cited 1×
“Diffusion models offer a compelling balance between diversity and realism and are relatively easier to train compared to other generative models [6, 13, 20, 21, 31, 49, 50, 64, 65].”
BackgroundHierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511 · cited 1×
“The performances degrade when we use the transformer in DALLE-2 [48].”
MethodDenoising Diffusion Implicit Models
Song, J., Meng, C., & Ermon, S. (2020) · arXiv · doi:10.48550/arxiv.2010.02502 · Cited by 2,507
Neural ordinary differential equations
Chen, R. T. Q., Rubanova, Y., Bettencourt, J., & Duvenaud, D. (2018) · arXiv · doi:10.48550/arxiv.1806.07366 · Cited by 2,097
Score-Based Generative Modeling through Stochastic Differential Equations
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2020) · arXiv · doi:10.48550/arxiv.2011.13456 · Cited by 1,953 · cited 1×
“To this end, we design a denoising diffusion model [24, 25, 59] to estimate object attributes to determine the placements and types of 3D instances and then perform shape retrieval to obtain final surface geometries.”
MethodImage Super-Resolution Via Iterative Refinement
Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D. J., & Norouzi, M. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3204461 · Cited by 1,829 · cited 1×
“Recently, diffusion models [25, 55–58] have shown impressive visual quality in generative tasks, especially in various applications of 2D image synthesis [1, 9, 12, 25–27, 30, 34, 37, 41, 52, 53] and 3D shape generation [3, 28, 35, 45, 60, 62, 63, 81, 83, 84, 86].”
BackgroundImage Super-Resolution via Iterative Refinement
Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D. J., & Norouzi, M. (2021) · arXiv · doi:10.48550/arxiv.2104.07636 · Cited by 1,829
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., & Van Gool, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01117 · Cited by 1,778 · cited 1×
“Recently, diffusion models [25, 55–58] have shown impressive visual quality in generative tasks, especially in various applications of 2D image synthesis [1, 9, 12, 25–27, 30, 34, 37, 41, 52, 53] and 3D shape generation [3, 28, 35, 45, 60, 62, 63, 81, 83, 84, 86].”
BackgroundGenerative Modeling by Estimating Gradients of the Data Distribution
Song, Y., & Ermon, S. (2019) · arXiv · doi:10.48550/arxiv.1907.05600 · Cited by 1,453 · cited 1×
“In terms of implementation, our approach is based on a denoising diffusion model [25], while LEGO-Net uses a Langevin Dynamics scheme based on a score-based method [57].”
MethodLearning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
Wu, J., Zhang, C., Xue, T., Freeman, W. T., & Tenenbaum, J. B. (2016) · arXiv · doi:10.48550/arxiv.1610.07584 · Cited by 1,393 · cited 1×
“Compared to other scene representations like multi-view images [10, 22, 32], voxel grids [8, 71], and neural fields [7, 39, 40, 43, 61], our representation is more compact and lightweight, making it suitable for learning through diffusion models.”
BackgroundGLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., & Chen, M. (2021) · arXiv · doi:10.48550/arxiv.2112.10741 · Cited by 1,357 · cited 1×
“Recently, diffusion models [25, 55–58] have shown impressive visual quality in generative tasks, especially in various applications of 2D image synthesis [1, 9, 12, 25–27, 30, 34, 37, 41, 52, 53] and 3D shape generation [3, 28, 35, 45, 60, 62, 63, 81, 83, 84, 86].”
BackgroundClassifier-Free Diffusion Guidance
Ho, J., & Salimans, T. (2022) · arXiv · doi:10.48550/arxiv.2207.12598 · Cited by 1,126 · cited 1×
“To this end, we design a denoising diffusion model [24, 25, 59] to estimate object attributes to determine the placements and types of 3D instances and then perform shape retrieval to obtain final surface geometries.”
MethodHigh-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737
Generating diverse high-fidelity images with vq-vae-2
Razavi, A., Oord, A. V. D., & Vinyals, O. (2019) · arXiv · doi:10.48550/arxiv.1906.00446 · Cited by 709 · cited 1×
“Diffusion models offer a compelling balance between diversity and realism and are relatively easier to train compared to other generative models [6, 13, 20, 21, 31, 49, 50, 64, 65].”
BackgroundDiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation
Kim, G., Kwon, T., & Ye, J. C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00246 · Cited by 570 · cited 1×
“Recently, diffusion models [25, 55–58] have shown impressive visual quality in generative tasks, especially in various applications of 2D image synthesis [1, 9, 12, 25–27, 30, 34, 37, 41, 52, 53] and 3D shape generation [3, 28, 35, 45, 60, 62, 63, 81, 83, 84, 86].”
BackgroundCascaded Diffusion Models for High Fidelity Image Generation
Ho, J., Saharia, C., Chan, W., Fleet, D. J., Norouzi, M., & Salimans, T. (2021) · arXiv · doi:10.48550/arxiv.2106.15282 · Cited by 514
Improved Techniques for Training Score-Based Generative Models
Song, Y., & Ermon, S. (2020) · arXiv · doi:10.48550/arxiv.2006.09011 · Cited by 482 · cited 2×
“For simplicity, we pre-define the constants of Σ ϕ ( x t ) := σ t := 1 − ¯ α t − 1 1 − ¯ α t β t , although Song et al. have shown that learnable covariances can increase generation quality in DDIM [58].”
BackgroundImagen Video: High Definition Video Generation with Diffusion Models
Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D. P., Poole, B., Norouzi, M., Fleet, D. J., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.02303 · Cited by 399
Improved Precision and Recall Metric for Assessing Generative Models
Kynkäänniemi, T., Karras, T., Laine, S., Lehtinen, J., & Aila, T. (2019) · arXiv · doi:10.48550/arxiv.1904.06991 · Cited by 379 · cited 1×
“For each room type, we use 80% of rooms as the training sets, while the remaining are for testing.”
3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics
Fu, H., Cai, B., Gao, L., Zhang, L.-X., Wang, J., Li, C., Zeng, Q., Sun, C., Jia, R., Zhao, B., et al. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01075 · Cited by 267
LEGO-Net: Learning Regular Rearrangements of Objects in Rooms
Wei, Q. A., Ding, S., Park, J. J., Sajnani, R., Poulenard, A., Sridhar, S., & Guibas, L. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01825 · Cited by 60
Cited by
Holodeck: Language Guided Generation of 3D Embodied AI Environments
Yang, Y., Sun, F.-Y., Weihs, L., Vanderbilt, E., Herrasti, A., Han, W., Wu, J., Haber, N., Krishna, R., Liu, L., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01536 · Cited by 101
Zhao, Y., Zhao, Z., Li, J., Dong, S., & Gao, S. (2024) · 2024 International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00132 · Cited by 10
ArtiLatent: Realistic Articulated 3D Object Generation via Structured Latents
Chen, H., Lan, Y., Chen, Y., & Pan, X. (2025) · Proceedings of the SIGGRAPH Asia 2025 Conference Papers · doi:10.1145/3757377.3763922 · Cited by 4
Metadata
- Authors
- Tang, J., Nie, Y., Markhasin, L., Dai, A., Thies, J., & Nießner, M.
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 20507-20518
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402753875
- Cited by
- 106
- References
- 32
- FWCI
- 29.87
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.