Is It AI or Is It Me? Understanding Users’ Prompt Journey with Text-to-Image Generative AI Tools
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Generative Artificial Intelligence (AI) has witnessed unprecedented growth in text-to-image AI tools. Yet, much remains unknown about users’ prompt journey with such tools in the wild. In this paper, we posit that designing human-centered text-to-image AI tools requires a clear understanding of how individuals intuitively approach crafting prompts, and what challenges they may encounter. To address this, we conducted semi-structured interviews with 19 existing users of a text-to-image AI tool. Our findings (1) offer insights into users’ prompt journey including structures and processes for writing, evaluating, and refining prompts in text-to-image AI tools and (2) indicate that users must overcome barriers to aligning AI to their intents, and mastering prompt crafting knowledge. From the findings, we discuss the prompt journey as an individual yet a social experience and highlight opportunities for aligning text-to-image AI tools and users’ intents.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., et al. (2009) · Communications of the ACM · doi:10.1145/1592761.1592779 · Cited by 3,333
What is AI Literacy? Competencies and Design Considerations
Long, D., & Magerko, B. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376727 · Cited by 3,241 · cited 1×
“Following established human-AI interaction guidelines [5, 34, 45], text-to-image tools should explicitly communicate both strengths and limitations, emphasizing the inherent imperfection and unpredictability of results [56, 59, 63].”
Self-citationGuidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 2×
“Following established human-AI interaction guidelines [5, 34, 45], text-to-image tools should explicitly communicate both strengths and limitations, emphasizing the inherent imperfection and unpredictability of results [56, 59, 63].”
Newborn infants imitate adult facial gestures
Meltzoff, A. N., & Moore, M. K. (1983) · Child Development · doi:10.2307/1130058 · Cited by 1,307
Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581388 · Cited by 1,274 · cited 2×
“A noteworthy example is the work by Zamfirescu-Pereira et al. [68], which explores intuitive prompting behaviors with large language models with a conversation agent.”
Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm
Reynolds, L., & McDonell, K. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3451760 · Cited by 952 · cited 3×
“As a result, users of text-to-image AI tools and researchers have developed guidelines and documentation in the form of handbooks [1, 3, 19, 53] and tutorials [38, 65] to address the ambiguity of crafting prompts within these tools, a process commonly referred to as ‘prompt engineering’ [47].”
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2021) · arXiv · doi:10.48550/arxiv.2104.08786 · Cited by 615 · cited 1×
“Prior research on prompt-based interactions has examined factors influencing the effectiveness of prompts in large language models, such as the negative impact of lengthy instructions [42] or the role of word order in achieving accurate model responses [36].”
Multitask Prompted Training Enables Zero-Shot Task Generalization
Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.08207 · Cited by 612 · cited 1×
“It has since evolved to denote the meticulous process of selecting the right prompt to produce intended outputs with large language models [51].”
Denny, P., Kumar, V., & Giacaman, N. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569823 · Cited by 287
Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models
Louie, R., Coenen, A., Huang, C. Z., Terry, M., & Cai, C. J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376739 · Cited by 271 · cited 1×
“Challenges of content generation with AI include the open-ended nature of generative outputs, lack of user agency, misalignment with users’ creative goals, and information overload [10, 35, 48].”
Research works printed but not linked: 32
- Ash Amin and Joanne Roberts. 2008. Knowing in action: Beyond communities of practice. Research policy 37, 2 (2008), 353–369.
- Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
- John Seely Brown and Paul Duguid. 1991. Organizational learning and communities-of-practice: Toward a unified view of working, learning, and innovation. Organization science 2, 1 (1991), 40–57.
- 10.1177/1478077120934126 · DOI
- 10.1145/3591196.3593515 · DOI
- 10.1145/3563657.3596001 · DOI · Paper page
- 10.1145/3586183.3606777 · DOI · Paper page
- 10.1016/j.ijhcs.2023.103029 · DOI
- Nassim Dehouche and Kullathida Dehouche. 2023. What’s in a text-to-image prompt? The potential of stable diffusion in visual arts education. Heliyon (2023).
- Daniel Dewey. 2014. Reinforcement learning and the reward engineering principle. In 2014 AAAI Spring Symposium Series.
- 10.1145/3544549.3585737 · DOI
- Matthew Guzdial and Mark Riedl. 2019. An interaction framework for studying co-creative ai. arXiv preprint arXiv:1903.09709 (2019). · arXiv
- Rowan T Hughes Liming Zhu and Tomasz Bednarz. 2021. Generative Adversarial Networks–Enabled Human–Artificial Intelligence Collaborative Applications for Creative and Design Industries: A Systematic Review of Current Approaches and Trends. Frontiers in artificial intelligence 4 (2021) 604234. · DOI · Paper page
- Ellen Jiang, Kristen Olson, Edwin Toh, Alejandra Molina, Aaron Donsbach, Michael Terry, and Carrie J Cai. 2022. PromptMaker: Prompt-based Prototyping with Large Language Models. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–8.
- Ellen Jiang, Edwin Toh, Alejandra Molina, Kristen Olson, Claire Kayacik, Aaron Donsbach, Carrie J Cai, and Michael Terry. 2022. Discovering the Syntax and Strategies of Natural Language Programming with Generative Language Models. In CHI Conference on Human Factors in Computing Systems. 1–19.
- Vivian Liu and Lydia B Chilton. 2022. Design Guidelines for Prompt Engineering Text-to-Image Generative Models. In CHI Conference on Human Factors in Computing Systems. 1–23.
- 10.18653/v1 · doi:10.18653/v1
- 10.19173/irrodl.v14i5.1579 · DOI
- Joe O’Connor and Jacob Andreas. 2021. What Context Features Can Transformer Language Models Use?arXiv preprint arXiv:2106.08367 (2021). · arXiv
- 10.1080/0144929X.2023.2286532 · DOI · Paper page
- 10.1145/3472538.3472573 · DOI
- Jeba Rezwana and Mary Lou Maher. 2022. Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systems. ACM Transactions on Computer-Human Interaction (2022).
- 10.1145/3591196.3593051 · DOI · Paper page
- 10.1145/3531146.3534639 · DOI · Paper page
- Josh Urban Davis Fraser Anderson Merten Stroetzel Tovi Grossman and George Fitzmaurice. 2021. Designing co-creative ai for virtual environments. In Creativity and Cognition. 1–11. · DOI · Paper page
- 10.1007/s10676-017-9440-6 · DOI
- Veera Vimpari, Annakaisa Kultima, Perttu Hämäläinen, and Christian Guckelsberger. 2023. " An Adapt-or-Die Type of Situation": Perception, Adoption, and Use of Text-To-Image-Generation AI by Game Industry Professionals. arXiv preprint arXiv:2302.12601 (2023). · arXiv
- Justin D Weisz, Michael Muller, Jessica He, and Stephanie Houde. 2023. Toward general design principles for generative AI applications. arXiv preprint arXiv:2301.05578 (2023). · arXiv
- 10.1145/3397481.3450656 · DOI · Paper page
- Sam Witteveen and Martin Andrews. 2022. Investigating Prompt Engineering in Diffusion Models. arXiv preprint arXiv:2211.15462 (2022). · arXiv
- Xiao Xie, Xiwen Cai, Junpei Zhou, Nan Cao, and Yingcai Wu. 2018. A semantic-based method for visualizing large image collections. IEEE transactions on visualization and computer graphics 25, 7 (2018), 2362–2377.
- Daijin Yang, Yanpeng Zhou, Zhiyuan Zhang, Toby Jia-Jun Li, and Ray LC. 2022. AI as an Active Writer: Interaction strategies with generated text in human-AI collaborative fiction writing. In Joint Proceedings of the ACM IUI Workshops 2022, Vol. 10.
Cited by
AIdeation: Designing a Human-AI Collaborative Ideation System for Concept Designers
Wang, W.-F., Lu, C.-T., Ponsa i Campanyà, N., Chen, B.-Y., & Chen, M. Y. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714148 · Cited by 50
Zhang, S., Wang, H., & Yi, X. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757594 · Cited by 36
Bennett, C. L., Kane, S. K., & Harrington, C. N. (2025) · Proceedings of the 5th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3757887.3763012 · Cited by 3
Alharbi, R., Cheong, A. D., Herskovitz, J., Brewer, R. N., & Schoenebeck, S. (2025) · Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663547.3746376 · Cited by 3
“Generally, there is a wealth of HCI and AI scholarship dedicated to understanding how sighted users recognize and resolve errors (e.g., [32, 44, 61, 67, 75]).”
Raees, M., Iwamoto, Y., Papangelis, K., Heard, J., & Tigwell, G. W. (2026) · arXiv · doi:10.48550/arxiv.2602.08937 · Cited by 2
“For instance, research examines accessibility guidelines [35, 81], the difficulty in writing alt text [45, 79, 73], and the emerging role of AI in supporting authors [46, 66].”
Metadata
- Authors
- Mahdavi Goloujeh, A., Sullivan, A., & Magerko, B.
- Venue
- Proceedings of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2024-05-11
- Pages
- 1-13
- OpenAlex
- W4396833147
- Cited by
- 89
- References
- 16
- FWCI
- 29.45
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Is It AI or Is It Me? Understanding Users’ Prompt Journey with Text-to-Image Generative AI Tools. https://app.cochat.ai/papers/10.1145/3613904.3642861
- The paper
- Mahdavi Goloujeh, A., Sullivan, A., & Magerko, B. (2024). Is It AI or Is It Me? Understanding Users’ Prompt Journey with Text-to-Image Generative AI Tools. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3642861
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.