The DOI matches its DataCite record.
Sceneseer:3d scene design with natural language.
Open accessAbstract
Designing 3D scenes is currently a creative task that requires significant expertise and effort in using complex 3D design interfaces. This effortful design process starts in stark contrast to the easiness with which people can use language to describe real and imaginary environments. We present SceneSeer: an interactive text to 3D scene generation system that allows a user to design 3D scenes using natural language. A user provides input text from which we extract explicit constraints on the objects that should appear in the scene. Given these explicit constraints, the system then uses a spatial knowledge base learned from an existing database of 3D scenes and 3D object models to infer an arrangement of the objects forming a natural scene matching the input description. Using textual commands the user can then iteratively refine the created scene by adding, removing, replacing, and manipulating objects. We evaluate the quality of 3D scenes generated by SceneSeer in a perceptual evaluation experiment where we compare against manually designed scenes and simpler baselines for 3D scene generation. We demonstrate how the generated scenes can be iteratively refined through simple natural language commands.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Many works on scene design learn spatial knowledge priors from existing 3D scene databases or leverage user input and refine the 3D scene iteratively.
“Many works on scene design either learn spatial knowledge priors from existing 3D scene databases [4, 31, 49-51, 54, 59] or leverage user input and refine the 3D scene iteratively [3, 5].”
From Holodeck: Language Guided Generation of 3D Embodied AI Environments (2024) · cites this paper as [3]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 15
- [1]1. Baayen, R. H., Davidson, D. J., and Bates, D. M. Mixed-effects modeling with crossed random effects for subjects and items. Journal of memory and language (2008).
- [2]2. Chang, A. X., Savva, M., and Manning, C. D. Interactive learning of spatial knowledge for text to 3D scene generation. In Proceedings of the ACL 2014 Workshop on Interactive Language Learning, Visualization, and Interfaces (ACL-ILLVI) (2014).
- [3]3. Chang, A. X., Savva, M., and Manning, C. D. Learning spatial knowledge for text to 3D scene generation. In EMNLP (2014).
- [4]4. Chaudhuri, S., Kalogerakis, E., Giguere, S., and Funkhouser, T. AttribIt: Content Creation with Semantic Attributes. In UIST (2013).
- [5]5. Cheong, H., Li, W., Shu, L., Tessier, A., Bradner, E., and Iorio, F. Natural language problem definition for computer-aided mechanical design. In CHI-DSLI Workshop 2014 Conference Proceedings (2014).
- [7]7. Conway, M., Audia, S., Burnette, T., Cosgrove, D., and Christiansen, K. Alice: lessons learned from building a 3D system for novices. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems , ACM (2000), 486-493.
- [8]8. Coyne, B., and Sproat, R. WordsEye: an automatic text-to-scene conversion system. In SIGGRAPH (2001).
- [10]10. Fisher, M., Ritchie, D., Savva, M., Funkhouser, T., and Hanrahan, P. Example-based synthesis of 3D object arrangements. SIGGRAPH Asia (2012).
- [11]11. Jiang, Y., Lim, M., and Saxena, A. Learning object arrangements in 3D scenes using human context. In ICML (2012).
- [13]13. Kelleher, C., and Pausch, R. Lessons learned from designing a programming system to support middle school girls creating animated stories. In Visual Languages and Human-Centric Computing, 2006. VL/HCC 2006. IEEE Symposium on , IEEE (2006), 165-172.
- [14]14. Khan, A., Mordatch, I., Fitzmaurice, G., Matejka, J., and Kurtenbach, G. Viewcube: a 3D orientation indicator and controller. In Proceedings of the 2008 symposium on Interactive 3D graphics and games , ACM (2008), 17-25.
- [15]15. McCrae, J., Mordatch, I., Glueck, M., and Khan, A. Multiscale 3D navigation. In Proceedings of the 2009 symposium on Interactive 3D graphics and games , ACM (2009), 7-14.
- [16]16. Merrell, P., Schkufza, E., Li, Z., Agrawala, M., and Koltun, V. Interactive furniture layout using interior design guidelines. In SIGGRAPH (2011).
- [17]17. Owen, R., Kurtenbach, G., Fitzmaurice, G., Baudel, T., and Buxton, B. When it gets more difficult, use both hands: exploring bimanual curve manipulation. In Proceedings of Graphics Interface 2005 , Canadian Human-Computer Communications Society (2005), 17-24.
- [19]19. Seversky, L. M., and Yin, L. Real-time automatic 3D scene generation from natural language voice and text descriptions. In Proceedings of the 14th annual ACM international conference on Multimedia (2006).
Cited by
Sorted by citations
Holodeck: Language Guided Generation of 3D Embodied AI Environments
Yang, Y., Sun, F.-Y., Weihs, L., Vanderbilt, E., Herrasti, A., Han, W., Wu, J., Haber, N., Krishna, R., Liu, L., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.01536 · Cited by 101
Holodeck: Language Guided Generation of 3D Embodied AI Environments
Yang, Y., Sun, F.-Y., Weihs, L., VanderBilt, E., Herrasti, A., Han, W., Wu, J., Haber, N., Krishna, R., Liu, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.09067 · Cited by 101
Metadata
- Authors
- Chang, A. X., Eric, M., Savva, M., & Manning, C. D.
- Affiliations
- Stanford University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1703.00050
- OpenAlex
- W2594830175
- Cited by
- 38
- References
- 0
- Citations by year
- 2025: 1 · 2024: 7 · 2023: 5 · 2022: 4 · 2021: 5 · 2020: 9 · 2019: 3 · 2018: 2 · 2017: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Keywords
- 3D scene design, text-to-3D generation, natural language interfaces, 3D object models, natural language commands, perceptual evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
