SceneFormer: Indoor Scene Generation with Transformers
The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
We address the task of indoor scene generation by generating a sequence of objects, along with their locations and orientations conditioned on a room layout. Large-scale indoor scene datasets allow us to extract patterns from user-designed indoor scenes, and generate new scenes based on these patterns. Existing methods rely on the 2D or 3D appearance of these scenes in addition to object positions, and make assumptions about the possible relations between objects. In contrast, we do not use any appearance information, and implicitly learn object relations using the self-attention mechanism of transformers. We show that our model design leads to faster scene generation with similar or improved levels of realism compared to previous methods. Our method is also flexible, as it can be conditioned not only on the room layout but also on text descriptions of the room, using only the cross-attention mechanism of transformers. Our user study shows that our generated scenes are preferred to the state-of-the-art FastSynth scenes 53.9% and 56.7% of the time for bedroom and living room scenes, respectively. At the same time, we generate a scene in 1.48 seconds on average, 20% faster than FastSynth.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
ShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2015) · arXiv · doi:10.48550/arxiv.1512.03012 · Cited by 2,330
Generating Long Sequences with Sparse Transformers
Child, R., Gray, S., Radford, A., & Sutskever, I. (2019) · arXiv · doi:10.48550/arxiv.1904.10509 · Cited by 1,063
CTRL: A Conditional Transformer Language Model for Controllable Generation
Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.05858 · Cited by 751
Language-driven synthesis of 3D scenes from scene databases
Ma, R., Patil, A. G., Fisher, M., Li, M., Pirk, S., Hua, B.-S., Yeung, S.-K., Tong, X., Guibas, L., & Zhang, H. (2018) · ACM Transactions on Graphics · doi:10.1145/3272127.3275035 · Cited by 100
Research works printed but not linked: 28
- 10.1145/3197517.3201362 · DOI · Paper page
- wang, 2019, Planit: Planning and instantiating indoor scenes with relation graph and spatial prior networks, ACM Transactions on Graphics (TOG), 38, 1
- vaswani, 2017, Attention is all you need, Advances in neural information processing systems, 5998
- 10.1109/CVPR.2019.00687 · DOI · Paper page
- 10.1109/ICCV.2019.00748 · DOI · Paper page
- 10.1145/3381866 · DOI
- wu, 2015, 3d shapenets: A deep representation for volumetric shapes, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1912
- 10.1145/2366145.2366154 · DOI · Paper page
- henderson, 2017, Automatic generation of constrained furniture layouts, arXiv preprint arXiv 1711 11585
- 10.1111/cgf.12976 · DOI
- kipf, 2017, Semi-supervised classification with graph convolutional networks, International Conference on Learning Representations (ICLR)
- 10.1145/3303766 · DOI · Paper page
- 10.1007/978-3-319-69487-0_10 · DOI
- savva, 2017, Scenesuggest: Context-driven 3d scene design, arXiv preprint arXiv 1703 00410
- 10.3115/v1/W14-3102 · DOI
- 10.1109/CVPR.2019.00634 · DOI · Paper page
- 10.3115/v1/P15-1006 · DOI
- 10.3115/v1/W14-2404 · DOI
- 10.1109/CVPR.2017.28 · DOI · Paper page
- 10.3115/v1/D14-1217 · DOI · Paper page
- chang, 2017, Sceneseer: 3d scene design with natural language, arXiv preprint arXiv 1703 00410
- 10.1109/CVPR42600.2020.01264 · DOI
- akbik, 2018, Contextual string embeddings for sequence labeling, COLING 2018 27th International Conference on Computational Linguistics, 1638
- 10.1109/CVPR42600.2020.00381 · DOI
- 10.1145/2010324.1964982 · DOI · Paper page
- pennington, 2014, Glove: Global vectors for word representation, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1532, 10.3115/v1/D14-1162 · DOI · Paper page
- nash, 2020, Polygen: An autoregressive generative model of 3d meshes, arXiv preprint arXiv 2002 10381
- purkait, 2019, Learning to generate new indoor scenes, arXiv preprint arXiv 1912 04554
Cited by
Han (2023) · doi:10.1109/tpami.2022.3152247 · Cited by 4,682
Neural Fields in Visual Computing and Beyond
Xie, Y., Takikawa, T., Saito, S., Litany, O., Yan, S., Khan, N., Tombari, F., Tompkin, J., sitzmann, V., & Sridhar, S. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14505 · Cited by 484
Holodeck: Language Guided Generation of 3D Embodied AI Environments
Yang, Y., Sun, F.-Y., Weihs, L., Vanderbilt, E., Herrasti, A., Han, W., Wu, J., Haber, N., Krishna, R., Liu, L., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01536 · Cited by 101
“Many works on scene design either learn spatial knowledge priors from existing 3D scene databases [4, 31, 49–51, 54, 59] or leverage user input and refine the 3D scene iteratively [3, 5].”
BackgroundLEGO-Net: Learning Regular Rearrangements of Objects in Rooms
Wei, Q. A., Ding, S., Park, J. J., Sajnani, R., Poulenard, A., Sridhar, S., & Guibas, L. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01825 · Cited by 60
Zhao, Y., Zhao, Z., Li, J., Dong, S., & Gao, S. (2024) · 2024 International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00132 · Cited by 10
“These work explore different modeling such as VAEs [26, 37, 44, 45, 55], GANs [1, 50, 54], Autoregressive [34, 35, 48] and Diffusion model for scene synthesis.”
BackgroundMethodInfluential
Metadata
- Authors
- Wang, X., Yeshwanth, C., & Niesner, M.
- Venue
- 2021 International Conference on 3D Vision (3DV)
- Publisher
- IEEE
- Published
- 2021-12
- Pages
- 106-115
- OpenAlex
- W4205481600
- Cited by
- 114
- References
- 11
- FWCI
- 12.03
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: SceneFormer: Indoor Scene Generation with Transformers. https://app.cochat.ai/papers/10.1109/3dv53792.2021.00021
- The paper
- Wang, X., Yeshwanth, C., & Niesner, M. (2021). SceneFormer: Indoor Scene Generation with Transformers. 2021 International Conference on 3D Vision (3DV). https://doi.org/10.1109/3dv53792.2021.00021
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.