DreamSketch: Early Stage 3D Design Explorations with Sketching and Generative Design
Abstract
We present DreamSketch, a novel 3D design interface that combines the free-form and expressive qualities of sketching with the computational power of generative design algorithms. In DreamSketch, a user coarsely defines the problem by sketching the design context. Then, a generative design algorithm produces multiple solutions that are augmented as 3D objects in the sketched context. The user can interact with the scene to navigate through the generated solutions. The combination of sketching and generative algorithms enables designers to explore multiple ideas and make better informed design decisions during the early stages of design. Design study sessions with designers and mechanical engineers demonstrate the expressive nature and creative possibilities of DreamSketch.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ML can potentially reduce global emissions by helping to streamline supply chains, improve production quality, predict machine breakdowns, optimize heating and cooling systems, and prioritize the use of clean electricity over fossil fuels.
“In particular, ML can potentially reduce global emissions (Figure 4) by helping to streamline supply chains, improve production quality, predict machine breakdowns, optimize heating and cooling systems, and prioritize the use of clean electricity over fossil fuels [74,227,416,880].”
From Tackling climate change with machine learning (2023) · cites this paper as [416]
Researchers have combined ML with generative design to develop structural products that require less raw material, thus reducing the resulting GHG emissions.
“To reduce the use of cement and steel, researchers have combined ML with generative design to develop structural products that require less raw material, thus reducing the resulting GHG emissions [416].”
From Tackling climate change with machine learning (2023) · cites this paper as [416]
Cites
Optimal shape design as a material distribution problem
Bendsøe, M. (1989) · Structural and Multidisciplinary Optimization · doi:10.1007/bf01650949 · Cited by 4,453 · see citing works
A level set method for structural topology optimization
Wang, M. Y., Wang, X., & Guo, D. (2002) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/s0045-7825(02)00559-5 · Cited by 3,267 · see citing works
Bendsøe, M. P., & Sigmund, O. (2004) · Springer eBooks · doi:10.1007/978-3-662-05086-6 · Cited by 1,790 · see citing works
(2007) · International Linear Collider · Cited by 1,382 · see citing works
Goel, V. (1995) · The MIT Press eBooks · doi:10.7551/mitpress/6270.001.0001 · Cited by 924 · see citing works
Parish, Y. I. H., & Müller, P. (2001) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/383259.383292 · Cited by 735 · see citing works
Müller, P. (2006) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1185657.1185716 · Cited by 663 · see citing works
A level-set method for shape optimization
Allaire, G., Jouve, F., & Toader, A.-M. (2002) · Comptes Rendus Mathématique · doi:10.1016/s1631-073x(02)02412-3 · Cited by 654 · see citing works
Sketch-based modeling: A survey
Olsen, L., Samavati, F. F., Sousa, M. C., et al. (2008) · Computers & Graphics · doi:10.1016/j.cag.2008.09.013 · Cited by 387 · see citing works
Bae, S.-H., Balakrishnan, R., & Singh, K. S. (2008) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/1449715.1449740 · Cited by 298 · see citing works
Gross, M. D., & Do, E. Y.-L. (1996) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/237091.237119 · Cited by 257 · see citing works
Xu, B., Chang, W., Sheffer, A., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2601097.2601128 · Cited by 184 · see citing works
Guided exploration of physically valid shapes for furniture design
Umetani, N., Igarashi, T., & Mitra, N. J. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185582 · Cited by 173 · see citing works
Interactive sketching of urban procedural models
Nishida, G., Garcia‐Dorado, I., Aliaga, D. G., et al. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925951 · Cited by 163 · see citing works
A suggestive interface for 3D drawing
Igarashi, Y., & Hughes, J. F. (2001) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/502348.502379 · Cited by 157 · see citing works
Lee, Y. J., Zitnick, C. L., & Cohen, M. F. (2011) · ACM SIGGRAPH Conference Papers (SIGGRAPH) · doi:10.1145/1964921.1964922 · Cited by 152 · see citing works
Lee, Y. J., Zitnick, C. L., & Cohen, M. F. (2011) · ACM Transactions on Graphics · doi:10.1145/2010324.1964922 · Cited by 149 · see citing works
What architects see in their sketches
Suwa, M., & Tversky, B. (1996) · doi:10.1145/257089.257255 · Cited by 119 · see citing works
Kazi, R. H., Chevalier, F., Grossman, T., & Fitzmaurice, G. W. (2014) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2642918.2647375 · Cited by 111 · see citing works
Self-citationBae, S.-H., Balakrishnan, R., & Singh, K. S. (2009) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/1622176.1622189 · Cited by 85 · see citing works
Shao, C., Bousseau, A., Sheffer, A., et al. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185541 · Cited by 76 · see citing works
Inverse Kinematics – Cyclic Coordinate Descent (CCD)
Kenwright, B. (2012) · Journal of Graphics Tools · doi:10.1080/2165347x.2013.823362 · Cited by 75 · see citing works
Structure and appearance optimization for controllable shape design
Martínez, J., Dumas, J., Lefèbvre, S., et al. (2015) · ACM Transactions on Graphics · doi:10.1145/2816795.2818101 · Cited by 74 · see citing works
Computational multicopter design
Du, T., Schulz, A., Zhu, B., et al. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2982427 · Cited by 71 · see citing works
On expert performance in 3D curve-drawing tasks
Schmidt, R., Khan, A., Kurtenbach, G., et al. (2009) · doi:10.1145/1572741.1572765 · Cited by 67 · see citing works
Kim, Y., & Bae, S.-H. (2016) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2984511.2984567 · Cited by 60 · see citing works
<i>GEM-NI</i>: A System for Creating and Managing Alternatives In Generative Design
Zaman, L., Stuerzlinger, W., Neugebauer, C., Woodbury, R., Elkhaldi, M., Shireen, N., & Terry, M. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702398 · Cited by 59 · see citing works
Design sketches and sketch design tools
Yi–Luen, E. (2005) · Knowledge-Based Systems · doi:10.1016/j.knosys.2005.07.001 · Cited by 51 · see citing works
Line Drawing as a Dynamic Process
House, D. H., & Singh, M. (2007) · doi:10.1109/pg.2007.62 · Cited by 47 · see citing works
Shao, T., Li, W., Zhou, K., et al. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2462003 · Cited by 40 · see citing works
3D6B Editor: Projective 3D Sketching with Line-Based Rendering
Kallio, K. (2005) · Eurographics · doi:10.2312/sbm/sbm05/073-079 · Cited by 32 · see citing works
FEAsy: A Sketch-Based Interface Integrating Structural Analysis in Early Design
Murugappan, S., & Ramani, K. (2009) · doi:10.1115/detc2009-87727 · Cited by 32 · see citing works
3D reconstruction of curved objects from single 2D line drawings
Wang, Y., Chen, Y., Liu, J., et al. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206841 · Cited by 32 · see citing works
Ergonomics-Inspired Reshaping and Exploration of Collections of Models
Zheng, Y., Liu, H., Dorsey, J. O., et al. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2448084 · Cited by 31 · see citing works
Paczkowski, P., Kim, M. H., Morvan, Y., Dorsey, J. O., Rushmeier, H., & O’Sullivan, C. (2011) · ACM Transactions on Graphics · doi:10.1145/2070781.2024216 · Cited by 31 · see citing works
Single-view sketch based modeling
André, A., & Saito, S. (2011) · doi:10.1145/2021164.2021189 · Cited by 29 · see citing works
PHYSICS-BASED GENERATIVE DESIGN
Attar, R., Aish, R., Stam, J., et al. (2009) · Cumulative Index of Computer Aided Architectural Design · Cited by 26 · see citing works
Paczkowski, P., Kim, M. H., Morvan, Y., Dorsey, J. O., Rushmeier, H., & O’Sullivan, C. (2011) · ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia) · doi:10.1145/2024156.2024216 · Cited by 26 · see citing works
SketchSoup: Exploratory Ideation Using Design Sketches
Arora, R., Darolia, I., Namboodiri, V. P., Singh, K., & Bousseau, A. (2017) · Computer Graphics Forum · doi:10.1111/cgf.13081 · Cited by 26 · see citing works
Kazi, R. H., Grossman, T., Mogk, C., Schmidt, R., & Fitzmaurice, G. W. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858138 · Cited by 18 · see citing works
Self-citationDMS2015-33: Generative interface structure design for supporting existing objects
Ulu, N. G., & Kara, L. B. (2015) · Journal of Visual Languages & Computing · doi:10.1016/j.jvlc.2015.10.016 · Cited by 9 · see citing works
Dynamic Design: Cognitive Processes in Design Sketching
Dillon, M. R. (2010) · Cited by 5 · see citing works
Generative Interface Structure Design for Supporting Existing Objects
Ulu, N. G., & Kara, L. B. (2015) · Proceedings · doi:10.18293/vlss2015-033 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Tackling climate change with machine learning
Rolnick, D., Donti, P. L., Kaack, L. H., Kochanski, K., Lacoste, A., Sankaran, K., Ross, A. S., Milojevic-Dupont, N., Jaques, N., Waldman-Brown, A., et al. (2023) · ACM Computing Surveys · doi:10.1145/3485128 · Cited by 336 · see citing works
Gmeiner, F., Yang, H., Yao, L., Holstein, K., & Martelaro, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580999 · Cited by 130 · see citing works
It Is Your Turn: Collaborative Ideation With a Co-Creative Robot through Sketch
Lin, Y., Guo, J., Chen, Y., Yao, C., & Ying, F. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376258 · Cited by 125 · see citing works
PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement
Wang, Z., Huang, Y., Song, D., Ma, L., & Zhang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642803 · Cited by 115 · see citing works
Rethinking designer agency: A case study of co-creation between designers and AI
Guo, X., Xiao, Y., Wang, J., & Ji, T. (2023) · IASDR 2023: Life-Changing Design · doi:10.21606/iasdr.2023.478 · Cited by 47 · see citing works
Ceci n’est pas une chaise: Emerging practices in designer-AI collaboration
van der Burg, V., Akdag Salah, A., Chandrasegaran, S., & Lloyd, P. (2022) · Proceedings of DRS · doi:10.21606/drs.2022.653 · Cited by 10 · see citing works
- Authors
- Kazi, R. H., Grossman, T., Cheong, H., Hashemi, A., & Fitzmaurice, G.
- Affiliations
- Autodesk Research, Toronto, ON, Canada
- Venue
- Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology
- Publisher
- ACM
- Published
- 2017-10-20
- Pages
- 401-414
- OpenAlex
- W2766572115
- Cited by
- 142
- References
- 43
- Citations by year
- 2026: 15 · 2025: 15 · 2024: 18 · 2023: 17 · 2022: 15 · 2021: 18 · 2020: 15 · 2019: 21 · 2018: 8
- FWCI
- 69.11
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Symposium on User Interface Software and Technology (UIST) (publishedVersion)
- Topics
- Physical Sciences › Engineering › Architecture › Architecture and Computational Design
- Physical Sciences › Engineering › Mechanical Engineering › Design Education and Practice
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Keywords
- generative design, sketch-based design, design exploration, preliminary design, design decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
