CoChat

Prompting for Discovery: Flexible Sense-Making for AI Art-Making with Dreamsheets

Open access
Almeda, Shm Garanganao, Zamfirescu-Pereira, J.D., Kim, Kyu Won, Mani Rathnam, Pradeep, Hartmann, Bjoern · 2024 · Proceedings of the CHI Conference on Human Factors in Computing Systems
Cited by 46

The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.

Abstract

Design space exploration (DSE) for Text-to-Image (TTI) models entails navigating a vast, opaque space of possible image outputs, through a commensurately vast input space of hyperparameters and prompt text. Perceptually small movements in prompt-space can surface unexpectedly disparate images. How can interfaces support end-users in reliably steering prompt-space explorations towards interesting results? Our design probe, DreamSheets, supports user-composed exploration strategies with LLM-assisted prompt construction and large-scale simultaneous display of generated results, hosted in a spreadsheet interface. Two studies, a preliminary lab study and an extended two-week study where five expert artists developed custom TTI sheet-systems, reveal various strategies for targeted TTI design space exploration—such as using templated text generation to define and layer semantic “axes” for exploration. We identified patterns in exploratory structures across our participants’ sheet-systems: configurable exploration “units” that we distill into a UI mockup, and generalizable UI components to guide future interfaces.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3613904.3642858

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 13 references

Research works printed but not linked: 41

  1. 10.1145/3586183.3606725 · DOI · Paper page
  2. Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T. Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan. 2023. Muse: Text-To-Image Generation via Masked Generative Transformers. arxiv:2301.00704 [cs.CV] · arXiv
  3. Minsuk Chang Stefania Druga Alexander J. Fiannaca Pedro Vergani Chinmay Kulkarni Carrie J Cai and Michael Terry. 2023. The Prompt Artists. In Creativity and Cognition. ACM Virtual Event USA 75–87. https://doi.org/10.1145/3591196.3593515 · DOI · Paper page
  4. 10.1145/263407.263513 · doi:10.1145/263407.263513
  5. 10.1145/3461778.3462050 · DOI · Paper page
  6. 10.1145/133160.133214 · DOI · Paper page
  7. 10.1145/3059454.3059478 · DOI · Paper page
  8. 10.1145/3526113.3545638 · DOI · Paper page
  9. Karl Friston. 2009. The free-energy principle: a rough guide to the brain?Trends in cognitive sciences 13, 7 (2009), 293–301.
  10. Pol Baladas Gerard Serra, Oriol Domingo. 2022. A programmable interface for creative exploration. Machine Learning for Creativity and Design Workshop at the 36th Conference on Neural Information Processing Systems (NeurIPS 2022) (December 2022). https://neuripscreativityworkshop.github.io/2022/papers/ml4cd2022_paper19.pdf
  11. 10.1145/3173574.3174154 · DOI · Paper page
  12. 10.1145/1449715.1449732 · DOI · Paper page
  13. Jeffrey Heer, Jock Mackinlay, Chris Stolte, and Maneesh Agrawala. 2008. Graphical histories for visualization: Supporting analysis, communication, and evaluation. IEEE transactions on visualization and computer graphics 14, 6 (2008), 1189–1196.
  14. 10.1145/3392850 · DOI · Paper page
  15. 10.1145/3357236.3395494 · DOI · Paper page
  16. Chinmay Kulkarni Stefania Druga Minsuk Chang Alex Fiannaca Carrie Cai and Michael Terry. 2023. A Word is Worth a Thousand Pictures: Prompts as AI Design Material. https://arxiv.org/abs/2303.12647v1 · arXiv
  17. 10.1145/192161.192190 · doi:10.1145/192161.192190
  18. 10.1145/3586183.3606831 · DOI · Paper page
  19. Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi and Graham Neubig. 2021. Pre-train Prompt and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. arxiv:2107.13586 [cs.CL] · arXiv
  20. 10.1145/3563657.3596098 · DOI · Paper page
  21. Bryan Loh and Tom White. 2018. SpaceSheets: interactive latent space exploration through a spreadsheet interface. (2018). https://doi.org/10.26686/wgtn.17134886.v1 · DOI
  22. 10.1145/1314683.1314685 · DOI
  23. Grace Luo, Trevor Darrell, Oliver Wang, Dan B Goldman, and Aleksander Holynski. 2023. Readout Guidance: Learning Control from Diffusion Features. arxiv:2312.02150 [cs.CV] · arXiv
  24. 10.1145/258734.258887 · DOI · Paper page
  25. Aditi Mishra Utkarsh Soni Anjana Arunkumar Jinbin Huang Bum Chul Kwon and Chris Bryan. 2023. PromptAid: Prompt Exploration Perturbation Testing and Iteration using Visual Analytics for Large Language Models. arxiv:2304.01964 [cs.HC] · arXiv
  26. 10.1145/3569219.3569352 · DOI · Paper page
  27. Jonas Oppenlaender. 2023. The Cultivated Practices of Text-to-Image Generation. arxiv:2306.11393 [cs.CY] · DOI
  28. 10.1080/0144929x.2023.2286532 · DOI · Paper page
  29. Jonas Oppenlaender Rhema Linder and Johanna Silvennoinen. 2023. Prompting AI Art: An Investigation into the Creative Skill of Prompt Engineering. arxiv:2303.13534 [cs.HC] · arXiv
  30. Peter Pirolli and Stuart Card. 1999. Information foraging.Psychological review 106, 4 (1999), 643.
  31. Peter Pirolli and Stuart Card. 2005. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of international conference on intelligence analysis, Vol. 5. McLean, VA, USA, 2–4.
  32. Aditya Ramesh Prafulla Dhariwal Alex Nichol Casey Chu and Mark Chen. 2022. Hierarchical Text-Conditional Image Generation with CLIP Latents. arxiv:2204.06125 [cs.CV] · arXiv
  33. Eric Rawn, Jingyi Li, Eric Paulos, and Sarah Chasins. 2023. Understanding Version Control as Material Interaction with Quickpose. In Proceedings of the SIGCHI conference on human factors in computing systems.
  34. 10.1184/R1 · doi:10.1184/r1
  35. 10.1145/2047196.2047216 · DOI · Paper page
  36. 10.1145/169059.169209 · DOI · Paper page
  37. 10.1145/1661412.1618513 · DOI
  38. Edward R Tufte. 1993. Envisioning information. Vol. 199. Graphics Press Cheshire, CT.
  39. 10.1017/S0890060406060136 · DOI
  40. 10.1145/2702123.2702398 · DOI · Paper page
  41. JD Zamfirescu-Pereira Heather Wei Amy Xiao Kitty Gu Grace Jung Matthew G Lee Bjoern Hartmann and Qian Yang. 2023. Herding AI Cats: Lessons from Designing a Chatbot by Prompting GPT-3. (2023). · DOI · Paper page

Cited by

See all 46

Metadata

Authors
Almeda, S. G., Zamfirescu-Pereira, J. D., Kim, K. W., Mani Rathnam, P., & Hartmann, B.
Venue
Proceedings of the CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2024-05-11
Pages
1-17
OpenAlex
W4396831984
Cited by
46
References
13
FWCI
18.37
Type
conference paper
Language
en
Open access
hybrid

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Prompting for Discovery: Flexible Sense-Making for AI Art-Making with Dreamsheets. https://app.cochat.ai/papers/10.1145/3613904.3642858
The paper
Almeda, S. G., Zamfirescu-Pereira, J. D., Kim, K. W., Mani Rathnam, P., & Hartmann, B. (2024). Prompting for Discovery: Flexible Sense-Making for AI Art-Making with Dreamsheets. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3642858

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.