The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Language-driven synthesis of 3D scenes from scene databases
Open accessAbstract
We introduce a novel framework for using natural language to generate and edit 3D indoor scenes, harnessing scene semantics and text-scene grounding knowledge learned from large annotated 3D scene databases. The advantage of natural language editing interfaces is strongest when performing semantic operations at the sub-scene level, acting on groups of objects. We learn how to manipulate these sub-scenes by analyzing existing 3D scenes. We perform edits by first parsing a natural language command from the user and transforming it into a semantic scene graph that is used to retrieve corresponding sub-scenes from the databases that match the command. We then augment this retrieved sub-scene by incorporating other objects that may be implied by the scene context. Finally, a new 3D scene is synthesized by aligning the augmented sub-scene with the user's current scene, where new objects are spliced into the environment, possibly triggering appropriate adjustments to the existing scene arrangement. A suggestive modeling interface with multiple interpretations of user commands is used to alleviate ambiguities in natural language. We conduct studies comparing our approach against both prior text-to-scene work and artist-made scenes and find that our method significantly outperforms prior work and is comparable to handmade scenes even when complex and varied natural sentences are used.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Existing Embodied AI environments are typically crafted through manual design [6, 13, 26, 27], 3D scanning [8, 43, 45], or procedurally generated with hard-coded rules [7].”
From Holodeck: Language Guided Generation of 3D Embodied AI Environments (2024) · cites this paper as [27]
Citations
Cites
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., & McClosky, D. (2014) · Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.3115/v1/p14-5010 · Cited by 7,037
Indoor segmentation and support inference from rgbd images
Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33715-4_54 · Cited by 5,760
Approximate is Better than “Exact” for Interval Estimation of Binomial Proportions
Agresti, A., & Coull, B. A. (1998) · The American Statistician · doi:10.1080/00031305.1998.10480550 · Cited by 3,223
A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457
SUN RGB-D: A RGB-D scene understanding benchmark suite
Song, S., Lichtenberg, S. P., & Xiao, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298655 · Cited by 1,943 · cited 1×
“Recent advances in computational design, VR/AR, and robotics are placing an increasing demand for rich 3D content, especially those in indoor environments [Hua et al. 2016; Silberman et al. 2012; Song et al. 2015, 2017].”
BackgroundShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T. A., Guibas, L. J., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2023) · doi:10.5281/zenodo.10089705 · Cited by 1,623
Self-citationSemantic Scene Completion from a Single Depth Image
Song, S., Yu, F., Zeng, A., Chang, A. X., Savva, M., & Funkhouser, T. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.28 · Cited by 1,209
Approximate Is Better than "Exact" for Interval Estimation of Binomial Proportions
Agresti, A., & Coull, B. A. (1998) · The American Statistician · doi:10.2307/2685469 · Cited by 972
KnowRob: A knowledge processing infrastructure for cognition-enabled robots
Tenorth, M., & Beetz, M. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913481635 · Cited by 388
WordsEye: an automatic text-to-scene conversion system
Coyne, B., & Sproat, R. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383316 · Cited by 377
Research works printed but not linked: 2
- Angel X. Chang , Mihail Eric , Manolis Savva , and Christopher D . Manning . 2017 . SceneSeer: 3D Scene Design with Natural Language. CoRR abs/1703.00050 (2017). http://arxiv.org/abs/1703.00050 Angel X. Chang, Mihail Eric, Manolis Savva, and Christopher D. Manning. 2017. SceneSeer: 3D Scene Design with Natural Language. CoRR abs/1703.00050 (2017). http://arxiv.org/abs/1703.00050 · arXiv
- 10.5555/2386438.2386446 · doi:10.5555/2386438.2386446
Cited by
Synthetic Data for Deep Learning
Nikolenko, S. I. (2021) · Springer Optimization and Its Applications · doi:10.1007/978-3-030-75178-4 · Cited by 423
Learning 3D Semantic Scene Graphs From 3D Indoor Reconstructions
Wald, J., Dhamo, H., Navab, N., & Tombari, F. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00402 · Cited by 244
GRAINS: Generative Recursive Autoencoders for INdoor Scenes
Li, M., Patil, A. G., Xu, K., Chaudhuri, S., Khan, O., Shamir, A., Tu, C., Chen, B., Cohen-Or, D., & Zhang, H. (2019) · ACM Transactions on Graphics · doi:10.1145/3303766 · Cited by 184
Self-citationPlanIT: planning and instantiating indoor scenes with relation graph and spatial prior networks
Wang, K., Lin, Y.-A., Weissmann, B., Savva, M., Chang, A. X., & Ritchie, D. (2019) · ACM Transactions on Graphics · doi:10.1145/3306346.3322941 · Cited by 195
SceneFormer: Indoor Scene Generation with Transformers
Wang, X., Yeshwanth, C., & Nießner, M. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00021 · Cited by 258
Metadata
- Authors
- Ma, R., Patil, A. G., Fisher, M., Li, M., Pirk, S., Hua, B.-S., Yeung, S.-K., Tong, X., Guibas, L., & Zhang, H.
- Affiliations
- Simon Fraser University (CA)
- Adobe Systems (United States) (US)
- Shandong University (CN)
- Stanford University (US)
- The University of Tokyo (JP)
- Hong Kong University of Science and Technology (HK)
- Microsoft Research Asia (China) (CN)
- ORCID
- Venue
- ACM Transactions on Graphics
- Published
- 2018-12-04
- Volume
- 37
- Issue
- 6
- Pages
- 1-16
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2902539442
- Cited by
- 100
- References
- 42
- Citations by year
- 2026: 7 · 2025: 15 · 2024: 19 · 2023: 17 · 2022: 6 · 2021: 16 · 2020: 12 · 2019: 7 · 2018: 1
- FWCI
- 6.63
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Transactions on Graphics (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- Natural Sciences and Engineering Research Council of Canada (611370)
- National Science Foundation (IIS-1528025)
- Ministry of Education - Singapore (MOE2016-T2-2-154)
- Amazon Web Services
- Adobe Systems
- Hong Kong University of Science and Technology (R9429)
- Stanford AI Lab-Toyota Center for Artificial Intelligence Research
- Topics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Keywords
- 3D scene synthesis, indoor scenes, scene semantics, scene graph, natural language ambiguity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Language-driven synthesis of 3D scenes from scene databases. https://app.cochat.ai/papers/10.1145/3272127.3275035
- The paper
- Ma, R., Patil, A. G., Fisher, M., Li, M., Pirk, S., Hua, B.-S., Yeung, S.-K., Tong, X., Guibas, L., & Zhang, H. (2018). Language-driven synthesis of 3D scenes from scene databases. ACM Transactions on Graphics. https://doi.org/10.1145/3272127.3275035
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.