The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
The Creativity of Text-to-Image Generation
Open accessAbstract
Text-guided synthesis of images has made a giant leap towards becoming a mainstream phenomenon. With text-to-image generation systems, anybody can create digital images and artworks. This provokes the question of whether text-to-image generation is creative. This paper expounds on the nature of human creativity involved in text-to-image art (so-called “AI art”) with a specific focus on the practice of prompt engineering. The paper argues that the current product-centered view of creativity falls short in the context of text-to-image generation. A case exemplifying this shortcoming is provided and the importance of online communities for the creative ecosystem of text-to-image art is highlighted. The paper provides a high-level summary of this online ecosystem drawing on Rhodes’ conceptual four P model of creativity. Challenges for evaluating the creativity of text-to-image generation and opportunities for research on text-to-image generation in the field of Human-Computer Interaction (HCI) are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Searle, J. R. (1980) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00005756 · Cited by 6,954
The standard definition of creativity
Runco, M. A., & Jaeger, G. J. (2012) · Creativity Research Journal · doi:10.1080/10400419.2012.650092 · Cited by 3,895
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Beyond Big and Little: The Four C Model of Creativity
Kaufman, J. C., & Beghetto, R. A. (2009) · Review of General Psychology · doi:10.1037/a0013688 · Cited by 2,669
Plucker, J. A., Beghetto, R. A., & Dow, G. T. (2004) · Educational Psychologist · doi:10.1207/s15326985ep3902_1 · Cited by 1,794
Jansson, D. G., & Smith, S. M. (1991) · Design Studies · doi:10.1016/0142-694x(91)90003-f · Cited by 1,087
AUGMENTING HUMAN INTELLECT: A CONCEPTUAL FRAMEWORK
ENGELBART, D. C. (1962) · doi:10.21236/ad0289565 · Cited by 1,082
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737
Design Guidelines for Prompt Engineering Text-to-Image Generative Models
Liu, V., & Chilton, L. B. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501825 · Cited by 639 · cited 5×
“One such skill is writing effective prompts or the practice of “prompt engineering” [18, 28, 35].”
BackgroundInfluentialDesign Guidelines for Prompt Engineering Text-to-Image Generative Models
Liu, V., & Chilton, L. B. (2021) · arXiv · doi:10.48550/arxiv.2109.06977 · Cited by 635
AI-synthesized faces are indistinguishable from real faces and more trustworthy
Nightingale, S. J., & Farid, H. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2120481119 · Cited by 397
Rezwana, J., & Maher, M. L. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3519026 · Cited by 342
VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance
Crowson, K., Biderman, S., Kornis, D., Stander, D., Hallahan, E., Castricato, L., & Raff, E. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19836-6_6 · Cited by 279
Creativity Support Tools: Report From a U.S. National Science Foundation Sponsored Workshop
Shneiderman, B., Fischer, G., Czerwinski, M., Resnick, M., Myers, B., Candy, L., Edmonds, E., Eisenberg, M., Giaccardi, E., Hewett, T., et al. (2006) · International Journal of Human-Computer Interaction · doi:10.1207/s15327590ijhc2002_1 · Cited by 274
Hertzmann, A. (2018) · Arts · doi:10.3390/arts7020018 · Cited by 265
A taxonomy of prompt modifiers for text-to-image generation
Oppenlaender, J. (2024) · Behaviour & Information Technology · doi:10.1080/0144929x.2023.2286532 · Cited by 225
Self-citationBoden, M. A., & Edmonds, E. A. (2009) · Digital Creativity · doi:10.1080/14626260902867915 · Cited by 222
Curiosity made the cat more creative: Specific curiosity as a driver of creativity
Hagtvedt, L. P., Dossinger, K., Harrison, S. H., & Huang, L. Y. (2018) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.10.007 · Cited by 183
Non-fungible tokens and the future of art
Kugler, L. (2021) · Communications of the ACM · doi:10.1145/3474355 · Cited by 172
Differentiable Prompt Makes Pre-trained Language Models Better Few-shot LearnersPreprint
Zhang, N., Li, L., Chen, X., Deng, S., Bi, Z., Tan, C., Huang, F., & Chen, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.13161 · Cited by 159
Frich, J., Biskjær, M. M., & Dalsgaard, P. (2018) · Designing Interactive Systems Conference · doi:10.1145/3196709.3196732 · Cited by 150
RICHARDS, R. L. (1990) · Creativity Research Journal · doi:10.1080/10400419009534363 · Cited by 149
Proceedings of the 20th European Conference on Artificial Intelligence
Raedt, L. D., Bessière, C., Didier Dubois, A., Doherty, P., & Frasconi, P. (2012) · Cited by 121
The Intersection of Users, Roles, Interactions, and Technologies in Creativity Support Tools
Chung, J. J. Y., He, S., & Adar, E. (2021) · Designing Interactive Systems Conference · doi:10.1145/3461778.3462050 · Cited by 117
Using Metrics of Curation to Evaluate Information-Based Ideation
Kerne, A., Webb, A. M., Smith, S. M., Linder, R., Lupfer, N., Qu, Y., Moeller, J., & Damaraju, S. (2014) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2591677 · Cited by 106
Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion ModelsPreprint
Rombach, R., Blattmann, A., & Ommer, B. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.13038 · Cited by 93
Strategies of Free-Form Web Curation
Kerne, A., Lupfer, N., Linder, R., Qu, Y., Valdez, A., Jain, A., Keith, K., Carrasco, M., Vanegas, J. A., & Billingsley, A. (2017) · Creativity and Cognition · doi:10.1145/3059454.3059471 · Cited by 30
DC 2 S 2: Designing Crowd-powered Creativity Support Systems
Oppenlaender, J., Mackeprang, M., Khiat, A., Vuković, M., Gonçalves, J., & Hosio, S. J. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3299027 · Cited by 15
Self-citationCrowdsourcing Creative WorkPreprint
Oppenlaender, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.16495 · Cited by 5
Self-citationCrowd-powered creativity support systems
Oppenlaender, J. (2020) · ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS) · doi:10.1145/3393672.3398646 · Cited by 3
Self-citation
Research works printed but not linked: 21
- Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell Sandhini Agarwal Ariel Herbert-Voss Gretchen Krueger Tom Henighan Rewon Child Aditya Ramesh Daniel M. Ziegler Jeffrey Wu Clemens Winter Christopher Hesse Mark Chen Eric Sigler Mateusz Litwin Scott Gray Benjamin Chess Jack Clark Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever and Dario Amodei. 2020. Language Models are Few-Shot Learners. https://doi.org/10.48550/ARXIV.2005.14165 · DOI · Paper page
- Simon Colton , Amy Smith , Sebastian Berns , Ryan Murdock , and Michael Cook . 2021 . Generative Search Engines: Initial Experiments . In Proceedings of the 12th International Conference on Computational Creativity(ICCC ’21) . Association for Computational Creativity, 237–246. Simon Colton, Amy Smith, Sebastian Berns, Ryan Murdock, and Michael Cook. 2021. Generative Search Engines: Initial Experiments. In Proceedings of the 12th International Conference on Computational Creativity(ICCC ’21). Association for Computational Creativity, 237–246.
- Katherine Crowson Stella Biderman Daniel Kornis Dashiell Stander Eric Hallahan Louis Castricato and Edward Raff. 2022. VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance. https://doi.org/10.48550/ARXIV.2204.08583 · DOI · Paper page
- Mihaly Csikszentmihalyi . 1996 . Creativity: Flow and the Psychology of Discovery and Invention . HarperCollins Publishers , New York, NY, USA . Mihaly Csikszentmihalyi. 1996. Creativity: Flow and the Psychology of Discovery and Invention. HarperCollins Publishers, New York, NY, USA.
- Prafulla Dhariwal and Alexander Nichol . 2021. Diffusion Models Beat GANs on Image Synthesis . In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (Eds.). Vol. 34. Curran Associates , Inc ., 8780–8794. https://proceedings.neurips.cc/paper/ 2021 /file/49ad23d1ec9fa4bd8d77d02681df5cfa-Paper.pdf Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion Models Beat GANs on Image Synthesis. In Advances in Neural Information Processing Systems, M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (Eds.). Vol. 34. Curran Associates, Inc., 8780–8794. https://proceedings.neurips.cc/paper/2021/file/49ad23d1ec9fa4bd8d77d02681df5cfa-Paper.pdf
- Igor N. Dubina and Suzanna J. Ramos . 2016. Creativity Through a Cultural Lens: The Dichotomy of “The West ” and “The East”. In Creativity, Innovation, and Entrepreneurship Across Cultures: Theory and Practices, Igor N. Dubina and Elias G. Carayannis (Eds.). Springer New York , New York, NY, 29–34. https://doi.org/10.1007/978-1-4939-3261-0_2 10.1007/978-1-4939-3261-0_2 Igor N. Dubina and Suzanna J. Ramos. 2016. Creativity Through a Cultural Lens: The Dichotomy of “The West” and “The East”. In Creativity, Innovation, and Entrepreneurship Across Cultures: Theory and Practices, Igor N. Dubina and Elias G. Carayannis (Eds.). Springer New York, New York, NY, 29–34. https://doi.org/10.1007/978-1-4939-3261-0_2 · doi:10.1007/978-1-4939-3261-0_2
- Philip Galanter . 2016. Generative Art Theory . John Wiley & Sons, Ltd , Chichester, West Sussex, UK, Chapter 5, 146–180. https://doi.org/10.1002/97811 1847 5249.ch5 10.1002/9781118475249.ch5 Philip Galanter. 2016. Generative Art Theory. John Wiley & Sons, Ltd, Chichester, West Sussex, UK, Chapter 5, 146–180. https://doi.org/10.1002/9781118475249.ch5 · doi:10.1002/97811
- Ian Goodfellow , Jean Pouget-Abadie , Mehdi Mirza , Bing Xu , David Warde-Farley , Sherjil Ozair , Aaron Courville , and Yoshua Bengio . 2014. Generative Adversarial Nets . In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q . Weinberger (Eds.). Vol. 27. Curran Associates, Inc . Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger (Eds.). Vol. 27. Curran Associates, Inc.
- 10.1145/2447976.2447987 · doi:10.1145/2447976.2447987
- Anna Kantosalo , Prashanth Thattai Ravikumar , Kazjon Grace , and Tapio Takala . 2020 . Modalities, Styles and Strategies: An Interaction Framework for Human-Computer Co-Creativity . In Proceedings of the Eleventh International Conference on Computational Creativity(ICCC 2020). Association for Computational Creativity, 57–64. Anna Kantosalo, Prashanth Thattai Ravikumar, Kazjon Grace, and Tapio Takala. 2020. Modalities, Styles and Strategies: An Interaction Framework for Human-Computer Co-Creativity. In Proceedings of the Eleventh International Conference on Computational Creativity(ICCC 2020). Association for Computational Creativity, 57–64.
- Robert V. Kozinets . 2010. Netnography. Doing Ethnographic Research Online . Sage , London, UK . Robert V. Kozinets. 2010. Netnography. Doing Ethnographic Research Online. Sage, London, UK.
- Alexander Mordvintsev , Christopher Olah , and Mike Tyka . 2015 . Inceptionism: Going Deeper into Neural Networks. https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html Alexander Mordvintsev, Christopher Olah, and Mike Tyka. 2015. Inceptionism: Going Deeper into Neural Networks. https://ai.googleblog.com/2015/06/inceptionism-going-deeper-into-neural.html
- Jonas Oppenlaender. 2022. A Taxonomy of Prompt Modifiers for Text-to-Image Generation. https://doi.org/10.48550/ARXIV.2204.13988 · DOI
- Alec Radford , Jong Wook Kim , Chris Hallacy , Aditya Ramesh , Gabriel Goh , Sandhini Agarwal , Girish Sastry , Amanda Askell , Pamela Mishkin , Jack Clark , Gretchen Krueger , and Ilya Sutskever . 2021 . Learning Transferable Visual Models From Natural Language Supervision . In Proceedings of the 38th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 139) , Marina Meila and Tong Zhang (Eds.). PMLR, 8748–8763. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 8748–8763.
- Laria Reynolds and Kyle McDonell . 2021 . Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery , New York, NY, USA, Article 314, 7 pages. Laria Reynolds and Kyle McDonell. 2021. Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, Article 314, 7 pages.
- Jeba Rezwana and Mary Lou Maher. 2022. Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systems. ACM Trans. Comput.-Hum. Interact.(2022) 27 pages. https://doi.org/10.1145/3519026 · DOI · Paper page
- Mel Rhodes . 1961 . An Analysis of Creativity . The Phi Delta Kappan 42 , 7 (1961), 305 – 310 . Mel Rhodes. 1961. An Analysis of Creativity. The Phi Delta Kappan 42, 7 (1961), 305–310.
- Robin Rombach Andreas Blattmann and Björn Ommer. 2022. Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models. https://doi.org/10.48550/ARXIV.2207.13038 · DOI
- Jordan G. Teicher. 2016. When Photography Wasn’t Art. https://daily.jstor.org/when-photography-was-not-art/ Jordan G. Teicher. 2016. When Photography Wasn’t Art. https://daily.jstor.org/when-photography-was-not-art/
- Xintao Wang , Yu Li , Honglun Zhang , and Ying Shan . 2021 . Towards Real-World Blind Face Restoration with Generative Facial Prior. In The IEEE Conference on Computer Vision and Pattern Recognition(CVPR ’21) . Xintao Wang, Yu Li, Honglun Zhang, and Ying Shan. 2021. Towards Real-World Blind Face Restoration with Generative Facial Prior. In The IEEE Conference on Computer Vision and Pattern Recognition(CVPR ’21).
- Ningyu Zhang Luoqiu Li Xiang Chen Shumin Deng Zhen Bi Chuanqi Tan Fei Huang and Huajun Chen. 2021. Differentiable Prompt Makes Pre-trained Language Models Better Few-shot Learners. https://doi.org/10.48550/ARXIV.2108.13161 · DOI
Cited by
Generative artificial intelligence
Banh, L., & Strobel, G. (2023) · Electronic Markets · doi:10.1007/s12525-023-00680-1 · Cited by 579
Prompt Engineering in Large Language Models
Marvin, G., Hellen, N., Jjingo, D., & Nakatumba-Nabende, J. (2024) · Algorithms for Intelligent Systems · doi:10.1007/978-981-99-7962-2_30 · Cited by 403
Using artificial intelligence in craft education: crafting with text-to-image generative models
Vartiainen, H., & Tedre, M. (2023) · Digital Creativity · doi:10.1080/14626268.2023.2174557 · Cited by 263
Brinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132
Typology of Risks of Generative Text-to-Image Models
Bird, C., Ungless, E., & Kasirzadeh, A. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604722 · Cited by 121
Metadata
- Authors
- Oppenlaender, J.
- Venue
- Proceedings of the 25th International Academic Mindtrek Conference
- Publisher
- ACM
- Published
- 2022-11-16
- Pages
- 192-202
- OpenAlex
- W4308623137
- Cited by
- 290
- References
- 33
- FWCI
- 67.63
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.