Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 121 of the 121 references it cites were found in a registry or an index; 8 of the 91 sources its claims cite were checked for support.

Design Principles for Generative AI Applications

Open access
Weisz, Justin D., He, Jessica, Muller, Michael, Hoefer, Gabriela, Miles, Rachel, Geyer, Werner · 2024 · Proceedings of the CHI Conference on Human Factors in Computing Systems

Abstract

Generative AI applications present unique design challenges. As generative AI technologies are increasingly being incorporated into mainstream applications, there is an urgent need for guidance on how to design user experiences that foster effective and safe use. We present six principles for the design of generative AI applications that address unique characteristics of generative AI UX and offer new interpretations and extensions of known issues in the design of AI applications. Each principle is coupled with a set of design strategies for implementing that principle via UX capabilities or through the design process. The principles and strategies were developed through an iterative process involving literature review, feedback from design practitioners, validation against real-world generative AI applications, and incorporation into the design process of two generative AI applications. We anticipate the principles to usefully inform the design of generative AI applications by driving actionable design recommendations.

Overview

CoChat Summary

TL;DR

The paper presents six design principles and 24 strategies for generative AI applications, developed through literature review, practitioner feedback, heuristic evaluation, and application to two product design teams. Three principles reinterpret known AI design issues, while three address generative-AI-specific concerns: generative variability, co-creation, and imperfection.

Key findings

  • The final framework contains six principles: Design Responsibly, Design for Mental Models, Design for Appropriate Trust & Reliance, Design for Generative Variability, Design for Co-Creation, and Design for Imperfection, each with four strategies (24 total).
  • In a modified heuristic evaluation, 18 design practitioners evaluated 9 commercial generative AI applications and identified 286 total examples across all principles, with a collective average of 11.9 examples per strategy and at least one example for every strategy.
  • In two workshops applying the principles, participants generated 46 design ideas (Workshop 1) and 56 design ideas (Workshop 2), with groups generating between 5 and 14 ideas per principle.
  • A follow-up clarity survey with 14 of 18 evaluators found a small but significant improvement in clarity ratings from M = 4.29 (SD = 0.90) to M = 4.58 (SD = 0.58), F(1, 184) = 5.88, p = .02, η²p = .03.

Methods

  • The authors used a four-iteration process: literature review to craft initial principles; feedback from an IUI workshop and an internal design guide viewed by over 1,000 practitioners; a modified heuristic evaluation; and application workshops with two generative AI product teams.
  • The modified heuristic evaluation involved 18 design practitioners who each evaluated one of 9 commercial generative AI applications (e.g., ChatGPT, Google Bard, DALL-E, DreamStudio, Midjourney, Adobe Firefly Generative Fill, IBM watsonx.ai Prompt Lab, GitHub Copilot, AIVA) using a Mural template.
  • The application workshops involved 4 practitioners on IBM watsonx.ai Prompt Lab and 10 practitioners on an internal LLM-based conversational tool; analysis involved reviewing canvases, ratings, comments, design ideas, and transcripts.

Limitations

  • The paper states that generative AI is rapidly changing, so new challenges may require new guidelines, tools, best practices, design patterns, and evaluative methods.
  • The paper states that some evaluators misunderstood the modified heuristic evaluation’s goal of evaluating the principles rather than the applications, and that the evaluation was restricted to commercially available applications, excluding experimental applications and interactions such as narrative, lyric/poetic forms, and movement.
  • The paper states that the principles focus only on UX design, not the full AI development lifecycle (e.g., model selection, tuning, prompt engineering, deployment/monitoring, and policy), and the text makes evident that the evaluation and workshop samples were drawn largely from the authors’ organization, limiting generalizability.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3613904.3642466

Cites

121 of 121 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

3 Valid · 2 Not valid · 31 Not checked · 80 Own results and interpretations

38 more claims

Create a free account to see all 116 claims and to verify the 83 references still open.

See all claims — free account
  • It may state two separate claims as one. 40
  • Probably not a substantial claim. 48

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 100 references

Research works printed but not linked: 60

  1. A Alsswey, IN Umar, and H Al-Samarraie. 2018. Towards mobile design guidelines-based cultural values for elderly Arabic users. Journal of Fundamental and Applied Sciences 10, 2S (2018), 964–977.
  2. Apple Computer Inc. 1987. Apple human interface guidelines: The Apple desktop interface. Addison Wesley Publishing Company.
  3. Nigel Bevan. 2005. Guidelines and standards for web usability. In Proceedings of HCI International, Vol. 2005. Citeseer, 10.
  4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.). Vol. 33. Curran Associates, Inc., 1877–1901. https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
  5. Jiaao Chen, Aston Zhang, Xingjian Shi, Mu Li, Alex Smola, and Diyi Yang. 2023. Parameter-Efficient Fine-Tuning Design Spaces. arXiv preprint arXiv:2301.01821 (2023). · arXiv
  6. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021). · arXiv
  7. John Joon Young Chung, Minsuk Chang, and Eytan Adar. 2021. Gestural Inputs as Control Interaction for Generative Human-AI Co-Creation. In Workshops at the International Conference on Intelligent User Interfaces (IUI).
  8. Fabio Cuzzolin, Alice Morelli, Bogdan Cirstea, and Barbara J Sahakian. 2020. Knowing me, knowing you: theory of mind in AI. Psychological medicine 50, 7 (2020), 1057–1061.
  9. Batya Friedman. 1996. Value-sensitive design. interactions 3, 6 (1996), 16–23.
  10. Helena Anna Frijns and Christina Schmidbauer. 2021. Design Guidelines for Collaborative Industrial Robot User Interfaces. In Human-Computer Interaction–INTERACT 2021: 18th IFIP TC 13 International Conference, Bari, Italy, August 30–September 3, 2021, Proceedings, Part III 18. Springer, 407–427.
  11. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Advances in neural information processing systems 27 (2014).
  12. Imke Grabe, Miguel González-Duque, Sebastian Risi, and Jichen Zhu. 2022. Towards a Framework for Human-AI Interaction Patterns in Co-Creative GAN Applications. Joint Proceedings of the ACM IUI Workshops 2022, March 2022, Helsinki, Finland (2022).
  13. Stephanie Houde, Vera Liao, Jacquelyn Martino, Michael Muller, David Piorkowski, John Richards, Justin Weisz, and Yunfeng Zhang. 2020. Business (mis) use cases of generative ai. arXiv preprint arXiv:2003.07679 (2020). · arXiv
  14. Mikhail Jacob, Alexander Zook, and Brian Magerko. 2013. Viewpoints AI: Procedurally Representing and Reasoning about Gestures.. In DiGRA conference.
  15. Jiaming Ji, Mickel Liu, Juntao Dai, Xuehai Pan, Chi Zhang, Ce Bian, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. 2023. BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset. arXiv preprint arXiv:2307.04657 (2023). · arXiv
  16. Anna Jobin, Marcello Ienca, and Effy Vayena. 2019. The global landscape of AI ethics guidelines. Nature machine intelligence 1, 9 (2019), 389–399.
  17. Nick Jones and Matthew Moffitt. 2016. Ethical guidelines for mobile app development within health and mental health fields.Professional Psychology: Research and Practice 47, 2 (2016), 155.
  18. Max Kreminski, Isaac Karth, Michael Mateas, and Noah Wardrip-Fruin. 2022. Evaluating Mixed-Initiative Creative Interfaces via Expressive Range Coverage Analysis.. In IUI Workshops. 34–45.
  19. Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. 2020. Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006.16668 (2020). · arXiv
  20. Barbara Leporini and Fabio Paternò. 2008. Applying web usability criteria for vision-impaired users: does it really improve task performance?Intl. Journal of Human–Computer Interaction 24, 1 (2008), 17–47.
  21. Tianyi Li, Mihaela Vorvoreanu, Derek DeBellis, and Saleema Amershi. 2022. Assessing Human-AI Interaction Early through Factorial Surveys: A Study on the Guidelines for Human-AI Interaction. ACM Transactions on Computer-Human Interaction (2022).
  22. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, 2022. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110 (2022). · arXiv
  23. Joseph Carl Robnett Licklider and Welden E Clark. 1962. On-line man-computer communication. In Proceedings of the May 1-3, 1962, spring joint computer conference. 113–128.
  24. Vivian Liu and Lydia B Chilton. 2021. Neurosymbolic Generation of 3D Animal Shapes through Semantic Controls.. In IUI Workshops.
  25. Nicola Lucchi. 2023. ChatGPT: A Case Study on Copyright Challenges for Generative Artificial Intelligence Systems. European Journal of Risk Regulation (2023), 1–23.
  26. Alexandra Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. 2023. Stable bias: Analyzing societal representations in diffusion models. arXiv preprint arXiv:2303.11408 (2023). · arXiv
  27. I Lupanda and JT Janse van Rensburg. 2021. Design Guidelines for Mobile Applications. In International Conference Interfaces & Human Computer Interaction. 92–99.
  28. Céline Mariage, Jean Vanderdonckt, and Costin Pribeanu. 2011. State of the Art of Web Usability Guidelines. In Handbook of human factors in Web design, Kim-Phuong L Vu and Robert W Proctor (Eds.). Crc Press.
  29. Fadel M Megahed, Ying-Ju Chen, Joshua A Ferris, Sven Knoth, and L Allison Jones-Farmer. 2023. How generative ai models such as chatgpt can be (mis) used in spc practice, education, and research? an exploratory study. Quality Engineering (2023), 1–29.
  30. Michael Muller, Plamen Agelov, Hal Daume, Q Vera Liao, Nuria Oliver, David Piorkowski, 2022. HCAI@NeurIPS 2022, Human Centered AI. In Annual Conference on Neural Information Processing Systems.
  31. Michael Muller, Heloisa Candello, and Justin Weisz. 2023. Interactional Co-Creativity of Human and AI in Analogy-Based Design. In International Conference on Computational Creativity.
  32. Michael Muller, Lydia B Chilton, Anna Kantosalo, Charles Patrick Martin, and Greg Walsh. 2022. GenAICHI: Generative AI and HCI. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–7.
  33. Michael Muller, Justin D. Weisz, and Werner Geyer. 2020. Mixed initiative generative AI interfaces: An analytic framework for generative AI applications. ICCC 2020 Workshop, The Future of Co-Creative Systems. https://computationalcreativity.net/workshops/cocreative-iccc20/papers/Future_of_co-creative_systems_185.pdf
  34. Jakob Nielsen. 2023. AI: First New UI Paradigm in 60 Years. Nielsen Norman Group (18 06 2023). https://www.nngroup.com/articles/ai-paradigm/
  35. Jonas Oppenlaender. 2022. A taxonomy of prompt modifiers for text-to-image generation. arXiv preprint arXiv:2204.13988 2 (2022). · arXiv
  36. Kyudong Park, Taedong Goh, and Hyo-Jeong So. 2014. Toward accessible mobile application design: developing mobile application accessibility guidelines for people with visual impairment. HCI Korea (2014), 31–38.
  37. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
  38. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 (2022). · arXiv
  39. Steven I Ross, Fernando Martinez, Stephanie Houde, Michael Muller, and Justin D Weisz. 2023. The Programmer’s Assistant: Conversational Interaction with a Large Language Model for Software Development. In 28th International Conference on Intelligent User Interfaces.
  40. Mattias Rost and Sebastian Andreasson. 2023. Stable Walk: An interactive environment for exploring Stable Diffusion outputs. 3124 (2023). http://ceur-ws.org/Vol-3359/paper14.pdf
  41. Vildan Salikutluk, Dorothea Koert, and Frank Jäkel. 2023. Interacting with Large Language Models: A Case Study on AI-Aided Brainstorming for Guesstimation Problems. In HHAI 2023: Augmenting Human Intellect. IOS Press, 153–167.
  42. Jose Ma Santiago III, Richard Lance Parayno, Jordan Aiko Deja, and Briane Paul V Samson. 2023. Rolling the Dice: Imagining Generative AI as a Dungeons & Dragons Storytelling Companion. arXiv preprint arXiv:2304.01860 (2023). · arXiv
  43. Yashraj Shashikant Sharma. 2023. Generating Wildfire Risk Maps for Critical Infrastructure Systems Using Integrated Generative AI and Simulation Techniques Under Information Uncertainty. Ph. D. Dissertation. State University of New York at Buffalo.
  44. Maria Shitkova, Justus Holler, Tobias Heide, Nico Clever, and Jörg Becker. 2015. Towards usability guidelines for mobile websites and applications. Wirtschaftsinformatik Proceedings (2015). https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1106&context=wi2015
  45. Ben Shneiderman Catherine Plaisant Maxine S Cohen Steven Jacobs Niklas Elmqvist and Nicholas Diakopoulos. 2016. Designing the user interface: strategies for effective human-computer interaction. Pearson.
  46. RC Sidorsky, RN Parrish, JL Gates, SJ Munger, and SYNECTICS CORP FAIRFAX VA. 1984. Design guidelines for user transactions with battlefield automated systems: Prototype for a handbook. ARI Researoh Product (1984), 84–08.
  47. Mannat Singh, Quentin Duval, Kalyan Vasudev Alwala, Haoqi Fan, Vaibhav Aggarwal, Aaron Adcock, Armand Joulin, Piotr Dollár, Christoph Feichtenhofer, Ross Girshick, 2023. The effectiveness of MAE pre-pretraining for billion-scale pretraining. arXiv preprint arXiv:2303.13496 (2023). · arXiv
  48. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615 (2022). · arXiv
  49. Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. arXiv preprint arXiv:1906.08976 (2019). · arXiv
  50. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
  51. Pranav Narayanan Venkit, Sanjana Gautam, Ruchi Panchanadikar, Shomir Wilson, 2023. Nationality Bias in Text Generation. arXiv preprint arXiv:2302.02463 (2023). · arXiv
  52. Mathias Peter Verheijden and Mathias Funk. 2023. Collaborative Diffusion: Boosting Designerly Co-Creation with Generative AI. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–8.
  53. Qian Wan, Siying Hu, Yu Zhang, Piaohong Wang, Bo Wen, and Zhicong Lu. 2023. "It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Models. arXiv preprint arXiv:2307.10811 (2023). · arXiv
  54. Zhen Wang, Rameswar Panda, Leonid Karlinsky, Rogerio Feris, Huan Sun, and Yoon Kim. 2023. Multitask prompt tuning enables parameter-efficient transfer learning. arXiv preprint arXiv:2303.02861 (2023). · arXiv
  55. Justin D Weisz, Michael Muller, Jessica He, and Stephanie Houde. 2023. Toward general design principles for generative AI applications. In Joint Proceedings of the IUI 2023 Workshops: HAI-GEN, ITAH, MILC, SHAI, SketchRec, SOCIALIZE co-located with the ACM International Conference on Intelligent User Interfaces, Vol. 3124. CEUR. http://ceur-ws.org/Vol-3359/paper14.pdf
  56. Justin D Weisz, Michael Muller, Stephanie Houde, John Richards, Steven I Ross, Fernando Martinez, Mayank Agarwal, and Kartik Talamadupula. 2021. Perfection Not Required? Human-AI Partnerships in Code Translation. In 26th International Conference on Intelligent User Interfaces. 402–412.
  57. Austin P Wright, Zijie J Wang, Haekyu Park, Grace Guo, Fabian Sperrle, Mennatallah El-Assady, Alex Endert, Daniel Keim, and Duen Horng Chau. 2020. A comparative analysis of industry human-AI interaction guidelines. arXiv preprint arXiv:2010.11761 (2020). · arXiv
  58. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. 2023. Large Language Models as Optimizers. arXiv preprint arXiv:2309.03409 (2023). · arXiv
  59. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, 2023. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. arXiv preprint arXiv:2306.05685 (2023). · arXiv
  60. Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593 (2019). · arXiv

Cited by

See all 233

Metadata

Authors
Weisz, J. D., He, J., Muller, M., Hoefer, G., Miles, R., & Geyer, W.
Affiliations
IBM (United States) (US)
Venue
Proceedings of the CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2024-05-11
Pages
1-22
OpenAlex
W4391334978
Cited by
232
References
100
Citations by year
2026: 87 · 2025: 119 · 2024: 25
FWCI
74.26
Type
conference paper
Language
en
Open access
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
Keywords
generative AI applications, user experience design, design principles, design strategies

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Design Principles for Generative AI Applications. https://app.cochat.ai/papers/10.1145/3613904.3642466
The paper
Weisz, J. D., He, J., Muller, M., Hoefer, G., Miles, R., & Geyer, W. (2024). Design Principles for Generative AI Applications. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3642466

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.