Prompting for Discovery: Flexible Sense-Making for AI Art-Making with Dreamsheets
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Design space exploration (DSE) for Text-to-Image (TTI) models entails navigating a vast, opaque space of possible image outputs, through a commensurately vast input space of hyperparameters and prompt text. Perceptually small movements in prompt-space can surface unexpectedly disparate images. How can interfaces support end-users in reliably steering prompt-space explorations towards interesting results? Our design probe, DreamSheets, supports user-composed exploration strategies with LLM-assisted prompt construction and large-scale simultaneous display of generated results, hosted in a spreadsheet interface. Two studies, a preliminary lab study and an extended two-week study where five expert artists developed custom TTI sheet-systems, reveal various strategies for targeted TTI design space exploration—such as using templated text generation to define and layer semantic “axes” for exploration. We identified patterns in exploratory structures across our participants’ sheet-systems: configurable exploration “units” that we distill into a UI mockup, and generalizable UI components to guide future interfaces.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Adding Conditional Control to Text-to-Image Diffusion Models
Zhang, L., Rao, A., & Agrawala, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00355 · Cited by 4,547
Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581388 · Cited by 1,274 · cited 3×
“Building such an understanding is neither trivial nor straightforward [48, 49]: the spaces of possible inputs and outputs are massive , and the mapping of one to the other is highly opaque .”
BackgroundSelf-citationHigh-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737
Design Guidelines for Prompt Engineering Text-to-Image Generative Models
Liu, V., & Chilton, L. B. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501825 · Cited by 639 · cited 3×
“Choosing the right language to achieve desirable visual results in these prompt-based interactions can be difficult, presenting challenges for users seeking to reliably harness TTI systems towards creative goals[49]; this has motivated online user communities [8, 32] and researchers to develop and investigate new prom…”
BackgroundMethodDesign Guidelines for Prompt Engineering Text-to-Image Generative Models
Liu, V., & Chilton, L. B. (2021) · arXiv · doi:10.48550/arxiv.2109.06977 · Cited by 635
Multitask Prompted Training Enables Zero-Shot Task Generalization
Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.08207 · Cited by 612 · cited 1×
“How a prompt directly impacts model outputs is an active area of research [20, 40].”
BackgroundMapping the Landscape of Creativity Support Tools in HCI
Frich, J., MacDonald Vermeulen, L., Remy, C., Biskjaer, M. M., & Dalsgaard, P. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300619 · Cited by 410 · cited 1×
“A more recent body of research explores how CST design can aid users’ creative processes and productivity [5, 10].”
BackgroundShneiderman, B. (2002) · Communications of the ACM · doi:10.1145/570907.570945 · Cited by 280
Designing with interactive example galleries
Lee, B., Srivastava, S., Kumar, R., Brafman, R., & Klemmer, S. R. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753667 · Cited by 195 · cited 1×
“Our work here builds on prior research showing that considering many alternatives in parallel can effectively aid design space exploration [14, 34], such as through gallery interfaces [26], tracking exploration history [15, 18], offering suggestions for possible input shifts [26], and effective organization [25, 36, 4…”
Background
Research works printed but not linked: 41
- 10.1145/3586183.3606725 · DOI · Paper page
- Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan. 2023. Muse: Text-To-Image Generation via Masked Generative Transformers. arxiv:2301.00704 [cs.CV] · arXiv
- Minsuk Chang Stefania Druga Alexander J. Fiannaca Pedro Vergani Chinmay Kulkarni Carrie J Cai and Michael Terry. 2023. The Prompt Artists. In Creativity and Cognition. ACM Virtual Event USA 75–87. https://doi.org/10.1145/3591196.3593515 · DOI · Paper page
- 10.1145/263407.263513 · doi:10.1145/263407.263513
- 10.1145/3461778.3462050 · DOI · Paper page
- 10.1145/133160.133214 · DOI · Paper page
- 10.1145/3059454.3059478 · DOI · Paper page
- 10.1145/3526113.3545638 · DOI · Paper page
- Karl Friston. 2009. The free-energy principle: a rough guide to the brain?Trends in cognitive sciences 13, 7 (2009), 293–301.
- Pol Baladas Gerard Serra, Oriol Domingo. 2022. A programmable interface for creative exploration. Machine Learning for Creativity and Design Workshop at the 36th Conference on Neural Information Processing Systems (NeurIPS 2022) (December 2022). https://neuripscreativityworkshop.github.io/2022/papers/ml4cd2022_paper19.pdf
- 10.1145/3173574.3174154 · DOI · Paper page
- 10.1145/1449715.1449732 · DOI · Paper page
- Jeffrey Heer, Jock Mackinlay, Chris Stolte, and Maneesh Agrawala. 2008. Graphical histories for visualization: Supporting analysis, communication, and evaluation. IEEE transactions on visualization and computer graphics 14, 6 (2008), 1189–1196.
- 10.1145/3392850 · DOI · Paper page
- 10.1145/3357236.3395494 · DOI · Paper page
- Chinmay Kulkarni Stefania Druga Minsuk Chang Alex Fiannaca Carrie Cai and Michael Terry. 2023. A Word is Worth a Thousand Pictures: Prompts as AI Design Material. https://arxiv.org/abs/2303.12647v1 · arXiv
- 10.1145/192161.192190 · doi:10.1145/192161.192190
- 10.1145/3586183.3606831 · DOI · Paper page
- Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi and Graham Neubig. 2021. Pre-train Prompt and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. arxiv:2107.13586 [cs.CL] · arXiv
- 10.1145/3563657.3596098 · DOI · Paper page
- Bryan Loh and Tom White. 2018. SpaceSheets: interactive latent space exploration through a spreadsheet interface. (2018). https://doi.org/10.26686/wgtn.17134886.v1 · DOI
- 10.1145/1314683.1314685 · DOI
- Grace Luo, Trevor Darrell, Oliver Wang, Dan B Goldman, and Aleksander Holynski. 2023. Readout Guidance: Learning Control from Diffusion Features. arxiv:2312.02150 [cs.CV] · arXiv
- 10.1145/258734.258887 · DOI · Paper page
- Aditi Mishra Utkarsh Soni Anjana Arunkumar Jinbin Huang Bum Chul Kwon and Chris Bryan. 2023. PromptAid: Prompt Exploration Perturbation Testing and Iteration using Visual Analytics for Large Language Models. arxiv:2304.01964 [cs.HC] · arXiv
- 10.1145/3569219.3569352 · DOI · Paper page
- Jonas Oppenlaender. 2023. The Cultivated Practices of Text-to-Image Generation. arxiv:2306.11393 [cs.CY] · DOI
- 10.1080/0144929x.2023.2286532 · DOI · Paper page
- Jonas Oppenlaender Rhema Linder and Johanna Silvennoinen. 2023. Prompting AI Art: An Investigation into the Creative Skill of Prompt Engineering. arxiv:2303.13534 [cs.HC] · arXiv
- Peter Pirolli and Stuart Card. 1999. Information foraging.Psychological review 106, 4 (1999), 643.
- Peter Pirolli and Stuart Card. 2005. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of international conference on intelligence analysis, Vol. 5. McLean, VA, USA, 2–4.
- Aditya Ramesh Prafulla Dhariwal Alex Nichol Casey Chu and Mark Chen. 2022. Hierarchical Text-Conditional Image Generation with CLIP Latents. arxiv:2204.06125 [cs.CV] · arXiv
- Eric Rawn, Jingyi Li, Eric Paulos, and Sarah Chasins. 2023. Understanding Version Control as Material Interaction with Quickpose. In Proceedings of the SIGCHI conference on human factors in computing systems.
- 10.1184/R1 · doi:10.1184/r1
- 10.1145/2047196.2047216 · DOI · Paper page
- 10.1145/169059.169209 · DOI · Paper page
- 10.1145/1661412.1618513 · DOI
- Edward R Tufte. 1993. Envisioning information. Vol. 199. Graphics Press Cheshire, CT.
- 10.1017/S0890060406060136 · DOI
- 10.1145/2702123.2702398 · DOI · Paper page
- JD Zamfirescu-Pereira Heather Wei Amy Xiao Kitty Gu Grace Jung Matthew G Lee Bjoern Hartmann and Qian Yang. 2023. Herding AI Cats: Lessons from Designing a Chatbot by Prompting GPT-3. (2023). · DOI · Paper page
Cited by
AIdeation: Designing a Human-AI Collaborative Ideation System for Concept Designers
Wang, W.-F., Lu, C.-T., Ponsa i Campanyà, N., Chen, B.-Y., & Chen, M. Y. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714148 · Cited by 50
Zhang, S., Wang, H., & Yi, X. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757594 · Cited by 36
“textual interaction [6, 9, 10, 15, 29, 40, 42, 45, 55, 55, 59, 59, 62, 70, 74, 77, 78, 82, 85, 92, 94, 108, 114, 122, 128, 130, 132, 146, 148, 148, 149, 154, 163, 165, 187, 191, 191, 193, 197] visual interaction [12, 16, 35, 44, 48, 66, 67, 88, 89, 96, 102, 112, 115, 121, 135, 150, 153, 158, 178,…”
InfluentialStable Diffusion Models Reveal a Persisting Human–AI Gap in Visual Creativity
Rondini, S., Alvarez‐Martin, C., Angermair‐Barkai, P., Penacchio, O., Paz, M., Pelowski, M., Dediu, D., Rodriguez‐Fornells, A., & Cerda‐Company, X. (2026) · Advanced Science · doi:10.1002/advs.202524142 · Cited by 2
Metadata
- Authors
- Almeda, S. G., Zamfirescu-Pereira, J. D., Kim, K. W., Mani Rathnam, P., & Hartmann, B.
- Venue
- Proceedings of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2024-05-11
- Pages
- 1-17
- OpenAlex
- W4396831984
- Cited by
- 46
- References
- 13
- FWCI
- 18.37
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Prompting for Discovery: Flexible Sense-Making for AI Art-Making with Dreamsheets. https://app.cochat.ai/papers/10.1145/3613904.3642858
- The paper
- Almeda, S. G., Zamfirescu-Pereira, J. D., Kim, K. W., Mani Rathnam, P., & Hartmann, B. (2024). Prompting for Discovery: Flexible Sense-Making for AI Art-Making with Dreamsheets. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3642858
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.