The DOI matches its Crossref record.
Sketchforme: Composing Sketched Scenes from Text Descriptions for Interactive Applications
Open accessAbstract
Sketching is an effective communication medium that augments and enhances what can be communicated in text. We introduce Sketchforme, the first neural-network-based system that can generate complex sketches based on text descriptions specified by users. Sketchforme's key contribution is to factor complex sketch rendering into layout and rendering subtasks using neural networks. The sketches composed by Sketchforme are expressive and realistic: we show in our user study that these sketches convey descriptions better than human-generated sketches in several cases, and 36.5% of those sketches were identified as human-generated. We develop some interactive applications using these generated sketches, and show that Sketchforme can significantly improve language learning applications and support intelligent language-based sketching assistants.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
GenAI leverages large data to provide a natural language interface for idea generation, promising to be a solution for designers to overcome issues of lack of originality.
“GenAI leverages large data to provide a natural language interface for idea generation, promising to be a solution for designers to overcome issues of lack of originality [40], applied in domains like fashion [34] and design sketching [5, 29].”
From Sketchar: Supporting Character Design and Illustration Prototyping Using Generative AI (2024) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 27
- Jamie Combs and Brenda Hoddinott. 2011. Drawing for dummies. Wiley.
- Richard C. Davis, Brien Colwell, and James A. Landay. 2008. K-sketch: A 'Kinetic' Sketch Pad for Novice Animators. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 413--422. http://dx.doi.org/10.1145/1357054.1357122 · DOI
- 10.1145/2047196.2047245 · DOI
- Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. 2015. A Neural Algorithm of Artistic Style. CoRR abs/1508.06576 (2015). http://arxiv.org/abs/1508.06576 · arXiv
- David Ha, Andrew M. Dai, and Quoc V. Le. 2016. HyperNetworks. CoRR abs/1609.09106 (2016). http://arxiv.org/abs/1609.09106 · arXiv
- David Ha and Douglas Eck. 2018. A Neural Representation of Sketch Drawings. In ICLR 2018. https://openreview.net/pdf?id=Hy6GHpkCW 2018.
- 10.1145/3023368.3023371 · DOI
- 10.1109/CVPR.2018.00833 · DOI
- 10.1145/2501988.2501997 · DOI
- Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017).
- Jonas Jongejan, Henry Rowley, Takashi Kawashima, Jongmin Kim, and Nick Fox-Gieg. 2016. The Quick, Draw! - A.I. Experiment. (2016). https://quickdraw.withgoogle.com/
- 10.1007/s11263-016-0981--7 · DOI
- James A. Landay. 1996. SILK: Sketching Interfaces Like Krazy. In Conference Companion on Human Factors in Computing Systems (CHI '96). ACM, New York, NY, USA, 398--399. http://dx.doi.org/10.1145/257089.257396 · DOI
- Walter S. Lasecki, Juho Kim, Nick Rafter, Onkur Sen, Jeffrey P. Bigham, and Michael S. Bernstein. 2015. Apparition: Crowdsourced User Interfaces That Come to Life As You Sketch Them. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1925--1934. http://dx.doi.org/10.1145/2702123.2702565 · DOI
- 10.1145/2010324.1964922 · DOI
- 10.1145/1015706.1015739 · DOI
- 10.1145/2461912.2462016 · DOI
- Scott E. Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. 2016. Generative Adversarial Text to Image Synthesis. CoRR abs/1605.05396 (2016). http://arxiv.org/abs/1605.05396 · arXiv
- 10.1145/2601097.2601202 · DOI
- Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez Łukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998--6008.
- 10.1080/17470218.2015.1094494 · DOI
- Lidan Wang, Vishwanath A. Sindagi, and Vishal M. Patel. 2017. High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks. CoRR abs/1710.10182 (2017). http://arxiv.org/abs/1710.10182 · arXiv
- 10.1145/2642918.2647399 · DOI
- 10.1109/ICCV.2017.629 · DOI · Paper page
- Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. CoRR abs/1703.10593 (2017). http://arxiv.org/abs/1703.10593 · arXiv
- C Lawrence Zitnick and Devi Parikh. 2013. Bringing semantics into focus using visual abstraction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3009--3016.
- 10.1007/978-3-030-01267-0_26 · DOI
Cited by
Sorted by citations
Sketchar: Supporting Character Design and Illustration Prototyping Using Generative AI
Ling, L., Chen, X., Wen, R., Li, T. J.-J., & LC, R. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3677102 · Cited by 52
Metadata
- Authors
- Huang, F., & Canny, J. F.
- Affiliations
- University of California, Berkeley (US)
- Venue
- Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology
- Publisher
- ACM
- Published
- 2019-10-17
- Pages
- 209-220
- OpenAlex
- W2981123187
- Cited by
- 23
- References
- 0
- Citations by year
- 2026: 3 · 2024: 1 · 2023: 5 · 2022: 5 · 2021: 7 · 2020: 2
- FWCI
- 1.14
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Symposium on User Interface Software and Technology (UIST) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
- Keywords
- sketch generation, neural rendering, language learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
