Cited by 190Digital Media and Visual ArtGenerative Adversarial Networks and Image SynthesisArt, Technology, and Culture

Large-scale Text-to-Image Generation Models for Visual Artists’ Creative Works

Open access
Ko, Hyung-Kwon, Park, Gwanmo, Jeon, Hyeon, Jo, Jaemin, Kim, Juho, Seo, Jinwook·2023·Proceedings of the 28th International Conference on Intelligent User Interfaces

Abstract

Large-scale Text-to-image Generation Models (LTGMs) (e.g., DALL-E), self-supervised deep learning models trained on a huge dataset, have demonstrated the capacity for generating high-quality open-domain images from multi-modal input. Although they can even produce anthropomorphized versions of objects and animals, combine irrelevant concepts in reasonable ways, and give variation to any user-provided images, we witnessed such rapid technological advancement left many visual artists disoriented in leveraging LTGMs more actively in their creative works. Our goal in this work is to understand how visual artists would adopt LTGMs to support their creative works. To this end, we conducted an interview study as well as a systematic literature review of 72 system/application papers for a thorough examination. A total of 28 visual artists covering 35 distinct visual art domains acknowledged LTGMs’ versatile roles with high usability to support creative works in automating the creation process (i.e., automation), expanding their ideas (i.e., exploration), and facilitating or arbitrating in communication (i.e., mediation). We conclude by providing four design guidelines that future researchers can refer to in making intelligent user interfaces using LTGMs.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3581641.3584078

Cites

80 of 80 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • People with art skills seem to have more concerns about AI-generated images.

    “People with art skills seem to have more concerns about AI-generated images [15, 36].”

    From Sketchar: Supporting Character Design and Illustration Prototyping Using Generative AI (2024) · cites this paper as [36]

    The Introduction says rapid improvements in text-to-image models left many visual artists perplexed, and §7 reports that some feared losing their jobs. These passages support concerns among people with art skills, but the paper does not establish that they have more concerns than people without those skills. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • Privacy Preservation focuses on maintaining data confidentiality and integrity, employing strategies like anonymization and federated learning to minimize direct data exposure, especially as generative AI poses risks of user profiling.

    “This domain focuses on maintaining data confidentiality and integrity, employing strategies like anonymization and federated learning to minimize direct data exposure, especially when the rise of generative AI poses risks of user profiling [238], [239].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 20

  1. Toshiki Aoki, Rintaro Chujo, Katsufumi Matsui, Saemi Choi, and Ari Hautasaari. 2022. EmoBalloon-Conveying Emotional Arousal in Text Chats with Speech Balloons. In CHI Conference on Human Factors in Computing Systems. 1-16.
  2. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77-101.
  3. Judith E Fan, Monica Dinculescu, and David Ha. 2019. Collabdraw: an environ- ment for collaborative sketching with an artificial agent. In Proceedings of the 2019 on Creativity and Cognition. 556-561.
  4. Dilrukshi Gamage, Piyush Ghasiya, Vamshi Bonagiri, Mark E Whiting, and Kazutoshi Sasahara. 2022. Are Deepfakes Concerning? Analyzing Conversations of Deepfakes on Reddit and Exploring Societal Implications. In CHI Conference on Human Factors in Computing Systems. 1-19.
  5. Ishani Janveja, Akshay Nambi, Shruthi Bannur, Sanchit Gupta, and Venkat Padmanabhan. 2020. Insight: monitoring the state of the driver in low-light using smartphones. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1-29.
  6. Ellen Jiang, Edwin Toh, Alejandra Molina, Kristen Olson, Claire Kayacik, Aaron Donsbach, Carrie J Cai, and Michael Terry. 2022. Discovering the Syntax and Strategies of Natural Language Programming with Generative Language Models. In CHI Conference on Human Factors in Computing Systems. 1-19.
  7. Pegah Karimi, Jeba Rezwana, Safat Siddiqui, Mary Lou Maher, and Nasrin Dehbozorgi. 2020. Creative sketching partner: an analysis of human-AI co- creativity. In Proceedings of the 25th International Conference on Intelligent User Interfaces. 221-230.
  8. Philippe Laban, Elicia Ye, Srujay Korlakunta, John Canny, and Marti Hearst. 2022. NewsPod: Automatic and Interactive News Podcasts. In 27th International Conference on Intelligent User Interfaces. 691-706.
  9. Toby Jia-Jun Li, Lindsay Popowski, Tom Mitchell, and Brad A Myers. 2021. Screen2vec: Semantic embedding of gui screens and gui components. In Pro- ceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-15.
  10. Xinyi Li, Liqiong Chang, Fangfang Song, Ju Wang, Xiaojiang Chen, Zhanyong Tang, and Zheng Wang. 2021. Crossgr: accurate and low-cost cross-target gesture recognition using Wi-Fi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 1 (2021), 1-23.
  11. Vivian Liu and Lydia B Chilton. 2022. Design Guidelines for Prompt Engineer- ing Text-to-Image Generative Models. In CHI Conference on Human Factors in Computing Systems. 1-23.
  12. Vivian Liu, Han Qiao, and Lydia Chilton. 2022. Opal: Multimodal Image Genera- tion for News Illustration. In The 35th Annual ACM Symposium on User Interface Software and Technology. 1-17.
  13. Yihe Liu, Anushk Mittal, Diyi Yang, and Amy Bruckman. 2022. Will AI Console Me when I Lose my Pet? Understanding Perceptions of AI-Mediated Email Writing. In CHI Conference on Human Factors in Computing Systems. 1-13.
  14. Mohammad Amin Mozaffari, Xinyuan Zhang, Jinghui Cheng, and Jin LC Guo. 2022. GANSpiration: Balancing Targeted and Serendipitous Inspiration in User Interface Design with Style-Based Generative Adversarial Network. In CHI Conference on Human Factors in Computing Systems. 1-15.
  15. Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Rad- ford, Mark Chen, and Ilya Sutskever. 2021. Zero-shot text-to-image generation. In International Conference on Machine Learning. PMLR, 8821-8831.
  16. Margarete Sandelowski. 1995. Sample size in qualitative research. Research in nursing & health 18, 2 (1995), 179-183.
  17. Ben Shneiderman and Catherine Plaisant. 2010. Designing the user interface: Strategies for effective human-computer interaction. Pearson Education India.
  18. Fenna Van Nes, Tineke Abma, Hans Jonsson, and Dorly Deeg. 2010. Language differences in qualitative research: is meaning lost in translation? European journal of ageing 7, 4 (2010), 313-316.
  19. Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. 2022. Wordcraft: Story Writing With Large Language Models. In 27th International Conference on Intelligent User Interfaces. 841-852.
  20. Chao Zhang, Cheng Yao, Jiayi Wu, Weijia Lin, Lijuan Liu, Ge Yan, and Fangtian Ying. 2022. StoryDrawer: A Child-AI Collaborative Drawing System to Support Children's Creative Visual Storytelling. In CHI Conference on Human Factors in Computing Systems. 1-15.

Cited by

Sorted by citations

Authors
Ko, H.-K., Park, G., Jeon, H., Jo, J., Kim, J., & Seo, J.
Affiliations
  • Korea Advanced Institute of Science and Technology (KR)
  • Seoul National University (KR)
  • Sungkyunkwan University (KR)
Venue
Proceedings of the 28th International Conference on Intelligent User Interfaces
Publisher
ACM
Published
2023-03-27
Pages
919-933
OpenAlex
W4321275178
Cited by
189
References
71
Citations by year
2026: 43 · 2025: 70 · 2024: 61 · 2023: 15
FWCI
55.32
Type
conference paper
Language
en
Open access
Locations
  • ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Research Foundation of Korea (NRF-2019R1A2C2089062,NRF-2019R1A2C1088900)
  • Information & communications Technology Planning & Evaluation(IITP) (2019-0-00075,2019-0-00421)
Topics
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Social Sciences › Arts and Humanities › Visual Arts and Performing Arts › Art, Technology, and Culture
Keywords
text-to-image generation, visual artists, intelligent user interfaces, design guidelines, DALL-E

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review