The DOI matches its Crossref record; 121 of the 121 references it cites were found in a registry or an index; 8 of the 91 sources its claims cite were checked for support.
Design Principles for Generative AI Applications
Open accessAbstract
Generative AI applications present unique design challenges. As generative AI technologies are increasingly being incorporated into mainstream applications, there is an urgent need for guidance on how to design user experiences that foster effective and safe use. We present six principles for the design of generative AI applications that address unique characteristics of generative AI UX and offer new interpretations and extensions of known issues in the design of AI applications. Each principle is coupled with a set of design strategies for implementing that principle via UX capabilities or through the design process. The principles and strategies were developed through an iterative process involving literature review, feedback from design practitioners, validation against real-world generative AI applications, and incorporation into the design process of two generative AI applications. We anticipate the principles to usefully inform the design of generative AI applications by driving actionable design recommendations.
Overview
CoChat Summary
TL;DR
The paper presents six design principles and 24 strategies for generative AI applications, developed through literature review, practitioner feedback, heuristic evaluation, and application to two product design teams. Three principles reinterpret known AI design issues, while three address generative-AI-specific concerns: generative variability, co-creation, and imperfection.
Key findings
- The final framework contains six principles: Design Responsibly, Design for Mental Models, Design for Appropriate Trust & Reliance, Design for Generative Variability, Design for Co-Creation, and Design for Imperfection, each with four strategies (24 total).
- In a modified heuristic evaluation, 18 design practitioners evaluated 9 commercial generative AI applications and identified 286 total examples across all principles, with a collective average of 11.9 examples per strategy and at least one example for every strategy.
- In two workshops applying the principles, participants generated 46 design ideas (Workshop 1) and 56 design ideas (Workshop 2), with groups generating between 5 and 14 ideas per principle.
- A follow-up clarity survey with 14 of 18 evaluators found a small but significant improvement in clarity ratings from M = 4.29 (SD = 0.90) to M = 4.58 (SD = 0.58), F(1, 184) = 5.88, p = .02, η²p = .03.
Methods
- The authors used a four-iteration process: literature review to craft initial principles; feedback from an IUI workshop and an internal design guide viewed by over 1,000 practitioners; a modified heuristic evaluation; and application workshops with two generative AI product teams.
- The modified heuristic evaluation involved 18 design practitioners who each evaluated one of 9 commercial generative AI applications (e.g., ChatGPT, Google Bard, DALL-E, DreamStudio, Midjourney, Adobe Firefly Generative Fill, IBM watsonx.ai Prompt Lab, GitHub Copilot, AIVA) using a Mural template.
- The application workshops involved 4 practitioners on IBM watsonx.ai Prompt Lab and 10 practitioners on an internal LLM-based conversational tool; analysis involved reviewing canvases, ratings, comments, design ideas, and transcripts.
Limitations
- The paper states that generative AI is rapidly changing, so new challenges may require new guidelines, tools, best practices, design patterns, and evaluative methods.
- The paper states that some evaluators misunderstood the modified heuristic evaluation’s goal of evaluating the principles rather than the applications, and that the evaluation was restricted to commercially available applications, excluding experimental applications and interactions such as narrative, lyric/poetic forms, and movement.
- The paper states that the principles focus only on UX design, not the full AI development lifecycle (e.g., model selection, tuning, prompt engineering, deployment/monitoring, and policy), and the text makes evident that the evaluation and workshop samples were drawn largely from the authors’ organization, limiting generalizability.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
3 Valid · 2 Not valid · 31 Not checked · 80 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Although recent work has begun to probe at design considerations for generative AI, this work has been limited to specific application domains or technologies. For example, guidelines of various maturity levels exist for GAN-based interfaces [57, 195], image creation [102, 104, 173], prompt engineering [102, 104], virtual reality [169], collaborative storytelling [146], and workflows with co-crea”
What the sources say
- Not checked yetcites [57]
- Not checked yetcites [195]
- Supportedcites [102] source
Section 2.1, “Prompting” (page unavailable), says the author’s research synthesized design guidelines specifically for prompt engineering text-to-image generations. This supports citing the paper as an example of design work focused on a particular generative-AI technology; it does not need to establish the scope of all recent work. (LLM, conf=0.90)
- Supportedcites [104] source
The citation identifies this paper as an example of domain- or technology-specific design guidelines. The Conclusion (p. 20) says it synthesizes guidelines for text-to-image interaction, and Limitations and Future Work (p. 20) specifies its focus on prompting a text-to-image framework. (LLM, conf=0.93)
- Not checked yetcites [173] source
- Not checked yetcites [169]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“But, as noted by both Soni et al. [160] and Stark et al. [164], our community struggles with bridging the gap between the development of scientifically-grounded guidelines and real-world design practice.”
What the sources say
- Not checked yetcites [160]
- Could not verifycites [164] sourceExperimental — accuracy not yet measured
The Goals and Tasks section (p. 3) discusses difficulties in prompt engineering, and Design Iterations and Rationale (pp. 6–7) describes choices made for PromptIDE. Neither addresses an HCI-wide gap between scientifically grounded guidelines and real-world design practice. Searches for that gap and for Soni and Stark found no passage in our copy that speaks to the claim. (LLM, conf=0.74)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Recognized uniqueness of Generative Vari- ability, Co-Creation, and Imperfection to generative AI; re-categorized Exploration and Optimization separately as user goals”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“We also identified how generative AI could enable people to explore a space of possibilities [88] as a byproduct of the generative process.”
What the sources say
- Not checked yetcites [88]
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“At the end of Iteration 2, we had a set of 8 high-level principles implemented by 29 specific strategies.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
38 more claims
Create a free account to see all 116 claims and to verify the 83 references still open.
- It may state two separate claims as one. 40
- Probably not a substantial claim. 48
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“LLM models support problem-solving [137] and programming [155] by completing or correcting code, and describing runtime errors [110].”
From The Rise of Agentic AI: A Review of Definitions, Frameworks, Architectures, Applications, Evaluation Metrics, and Challenges (2025) · cites this paper as [155]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Design Responsibly · p. 7
“- Expose or limit emergent behaviors . Determine whether generative capabilities beyond the intended use case should be surfaced to the user or restricted.”
§ A.1 Design responsibly · p. 29
“A.1.3 Expose or limit emergent behaviors*.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 10 DISCUSSION · p. 17
“We then collected feedback from 12 design practitioners on two design teams who applied them to both the formative and evaluative stages of product design.”
§ 9.1 Method · p. 15
“Workshop 1 was held with an internal team comprised of four design practitioners working on the IBM watsonx.ai Prompt Lab 21 , a prompt testing environment for large language models. Workshop 2 involved a separate internal team of ten design practitioners in the early, formative stages of designing an internal LLM-based conversational tool that provides UX research support.”
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 1×
“Licklider and Clark provide an early example of guidelines for the design of console interfaces in, “On-line man-computer communica-tion” [99] (refecting the gendered language of their time).”
BackgroundLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 2×
“One emerging skill revolves around crafting efective natural language prompts, known as in-context learning [20, 40, 189] or prompt engineering [185, 193].”
BackgroundHigh-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · cited 2×
“We provide a brief historical sketch of human-computer interaction design guidelines and their evolution as a result of new paradigms in user interface technologies.”
BackgroundChain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · cited 1×
“New guidelines were needed for the new visual metaphors of windows, icons, menus, and pointers.”
BackgroundKung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
Research works printed but not linked: 60
- A Alsswey, IN Umar, and H Al-Samarraie. 2018. Towards mobile design guidelines-based cultural values for elderly Arabic users. Journal of Fundamental and Applied Sciences 10, 2S (2018), 964–977.
- Apple Computer Inc. 1987. Apple human interface guidelines: The Apple desktop interface. Addison Wesley Publishing Company.
- Nigel Bevan. 2005. Guidelines and standards for web usability. In Proceedings of HCI International, Vol. 2005. Citeseer, 10.
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.). Vol. 33. Curran Associates, Inc., 1877–1901. https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
- Jiaao Chen, Aston Zhang, Xingjian Shi, Mu Li, Alex Smola, and Diyi Yang. 2023. Parameter-Efficient Fine-Tuning Design Spaces. arXiv preprint arXiv:2301.01821 (2023). · arXiv
- Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021). · arXiv
- John Joon Young Chung, Minsuk Chang, and Eytan Adar. 2021. Gestural Inputs as Control Interaction for Generative Human-AI Co-Creation. In Workshops at the International Conference on Intelligent User Interfaces (IUI).
- Fabio Cuzzolin, Alice Morelli, Bogdan Cirstea, and Barbara J Sahakian. 2020. Knowing me, knowing you: theory of mind in AI. Psychological medicine 50, 7 (2020), 1057–1061.
- Batya Friedman. 1996. Value-sensitive design. interactions 3, 6 (1996), 16–23.
- Helena Anna Frijns and Christina Schmidbauer. 2021. Design Guidelines for Collaborative Industrial Robot User Interfaces. In Human-Computer Interaction–INTERACT 2021: 18th IFIP TC 13 International Conference, Bari, Italy, August 30–September 3, 2021, Proceedings, Part III 18. Springer, 407–427.
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Advances in neural information processing systems 27 (2014).
- Imke Grabe, Miguel González-Duque, Sebastian Risi, and Jichen Zhu. 2022. Towards a Framework for Human-AI Interaction Patterns in Co-Creative GAN Applications. Joint Proceedings of the ACM IUI Workshops 2022, March 2022, Helsinki, Finland (2022).
- Stephanie Houde, Vera Liao, Jacquelyn Martino, Michael Muller, David Piorkowski, John Richards, Justin Weisz, and Yunfeng Zhang. 2020. Business (mis) use cases of generative ai. arXiv preprint arXiv:2003.07679 (2020). · arXiv
- Mikhail Jacob, Alexander Zook, and Brian Magerko. 2013. Viewpoints AI: Procedurally Representing and Reasoning about Gestures.. In DiGRA conference.
- Jiaming Ji, Mickel Liu, Juntao Dai, Xuehai Pan, Chi Zhang, Ce Bian, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. 2023. BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset. arXiv preprint arXiv:2307.04657 (2023). · arXiv
- Anna Jobin, Marcello Ienca, and Effy Vayena. 2019. The global landscape of AI ethics guidelines. Nature machine intelligence 1, 9 (2019), 389–399.
- Nick Jones and Matthew Moffitt. 2016. Ethical guidelines for mobile app development within health and mental health fields.Professional Psychology: Research and Practice 47, 2 (2016), 155.
- Max Kreminski, Isaac Karth, Michael Mateas, and Noah Wardrip-Fruin. 2022. Evaluating Mixed-Initiative Creative Interfaces via Expressive Range Coverage Analysis.. In IUI Workshops. 34–45.
- Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. 2020. Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006.16668 (2020). · arXiv
- Barbara Leporini and Fabio Paternò. 2008. Applying web usability criteria for vision-impaired users: does it really improve task performance?Intl. Journal of Human–Computer Interaction 24, 1 (2008), 17–47.
- Tianyi Li, Mihaela Vorvoreanu, Derek DeBellis, and Saleema Amershi. 2022. Assessing Human-AI Interaction Early through Factorial Surveys: A Study on the Guidelines for Human-AI Interaction. ACM Transactions on Computer-Human Interaction (2022).
- Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, 2022. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110 (2022). · arXiv
- Joseph Carl Robnett Licklider and Welden E Clark. 1962. On-line man-computer communication. In Proceedings of the May 1-3, 1962, spring joint computer conference. 113–128.
- Vivian Liu and Lydia B Chilton. 2021. Neurosymbolic Generation of 3D Animal Shapes through Semantic Controls.. In IUI Workshops.
- Nicola Lucchi. 2023. ChatGPT: A Case Study on Copyright Challenges for Generative Artificial Intelligence Systems. European Journal of Risk Regulation (2023), 1–23.
- Alexandra Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. 2023. Stable bias: Analyzing societal representations in diffusion models. arXiv preprint arXiv:2303.11408 (2023). · arXiv
- I Lupanda and JT Janse van Rensburg. 2021. Design Guidelines for Mobile Applications. In International Conference Interfaces & Human Computer Interaction. 92–99.
- Céline Mariage, Jean Vanderdonckt, and Costin Pribeanu. 2011. State of the Art of Web Usability Guidelines. In Handbook of human factors in Web design, Kim-Phuong L Vu and Robert W Proctor (Eds.). Crc Press.
- Fadel M Megahed, Ying-Ju Chen, Joshua A Ferris, Sven Knoth, and L Allison Jones-Farmer. 2023. How generative ai models such as chatgpt can be (mis) used in spc practice, education, and research? an exploratory study. Quality Engineering (2023), 1–29.
- Michael Muller, Plamen Agelov, Hal Daume, Q Vera Liao, Nuria Oliver, David Piorkowski, 2022. HCAI@NeurIPS 2022, Human Centered AI. In Annual Conference on Neural Information Processing Systems.
- Michael Muller, Heloisa Candello, and Justin Weisz. 2023. Interactional Co-Creativity of Human and AI in Analogy-Based Design. In International Conference on Computational Creativity.
- Michael Muller, Lydia B Chilton, Anna Kantosalo, Charles Patrick Martin, and Greg Walsh. 2022. GenAICHI: Generative AI and HCI. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–7.
- Michael Muller, Justin D. Weisz, and Werner Geyer. 2020. Mixed initiative generative AI interfaces: An analytic framework for generative AI applications. ICCC 2020 Workshop, The Future of Co-Creative Systems. https://computationalcreativity.net/workshops/cocreative-iccc20/papers/Future_of_co-creative_systems_185.pdf
- Jakob Nielsen. 2023. AI: First New UI Paradigm in 60 Years. Nielsen Norman Group (18 06 2023). https://www.nngroup.com/articles/ai-paradigm/
- Jonas Oppenlaender. 2022. A taxonomy of prompt modifiers for text-to-image generation. arXiv preprint arXiv:2204.13988 2 (2022). · arXiv
- Kyudong Park, Taedong Goh, and Hyo-Jeong So. 2014. Toward accessible mobile application design: developing mobile application accessibility guidelines for people with visual impairment. HCI Korea (2014), 31–38.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
- Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 (2022). · arXiv
- Steven I Ross, Fernando Martinez, Stephanie Houde, Michael Muller, and Justin D Weisz. 2023. The Programmer’s Assistant: Conversational Interaction with a Large Language Model for Software Development. In 28th International Conference on Intelligent User Interfaces.
- Mattias Rost and Sebastian Andreasson. 2023. Stable Walk: An interactive environment for exploring Stable Diffusion outputs. 3124 (2023). http://ceur-ws.org/Vol-3359/paper14.pdf
- Vildan Salikutluk, Dorothea Koert, and Frank Jäkel. 2023. Interacting with Large Language Models: A Case Study on AI-Aided Brainstorming for Guesstimation Problems. In HHAI 2023: Augmenting Human Intellect. IOS Press, 153–167.
- Jose Ma Santiago III, Richard Lance Parayno, Jordan Aiko Deja, and Briane Paul V Samson. 2023. Rolling the Dice: Imagining Generative AI as a Dungeons & Dragons Storytelling Companion. arXiv preprint arXiv:2304.01860 (2023). · arXiv
- Yashraj Shashikant Sharma. 2023. Generating Wildfire Risk Maps for Critical Infrastructure Systems Using Integrated Generative AI and Simulation Techniques Under Information Uncertainty. Ph. D. Dissertation. State University of New York at Buffalo.
- Maria Shitkova, Justus Holler, Tobias Heide, Nico Clever, and Jörg Becker. 2015. Towards usability guidelines for mobile websites and applications. Wirtschaftsinformatik Proceedings (2015). https://aisel.aisnet.org/cgi/viewcontent.cgi?article=1106&context=wi2015
- Ben Shneiderman Catherine Plaisant Maxine S Cohen Steven Jacobs Niklas Elmqvist and Nicholas Diakopoulos. 2016. Designing the user interface: strategies for effective human-computer interaction. Pearson.
- RC Sidorsky, RN Parrish, JL Gates, SJ Munger, and SYNECTICS CORP FAIRFAX VA. 1984. Design guidelines for user transactions with battlefield automated systems: Prototype for a handbook. ARI Researoh Product (1984), 84–08.
- Mannat Singh, Quentin Duval, Kalyan Vasudev Alwala, Haoqi Fan, Vaibhav Aggarwal, Aaron Adcock, Armand Joulin, Piotr Dollár, Christoph Feichtenhofer, Ross Girshick, 2023. The effectiveness of MAE pre-pretraining for billion-scale pretraining. arXiv preprint arXiv:2303.13496 (2023). · arXiv
- Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615 (2022). · arXiv
- Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. arXiv preprint arXiv:1906.08976 (2019). · arXiv
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
- Pranav Narayanan Venkit, Sanjana Gautam, Ruchi Panchanadikar, Shomir Wilson, 2023. Nationality Bias in Text Generation. arXiv preprint arXiv:2302.02463 (2023). · arXiv
- Mathias Peter Verheijden and Mathias Funk. 2023. Collaborative Diffusion: Boosting Designerly Co-Creation with Generative AI. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–8.
- Qian Wan, Siying Hu, Yu Zhang, Piaohong Wang, Bo Wen, and Zhicong Lu. 2023. "It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Models. arXiv preprint arXiv:2307.10811 (2023). · arXiv
- Zhen Wang, Rameswar Panda, Leonid Karlinsky, Rogerio Feris, Huan Sun, and Yoon Kim. 2023. Multitask prompt tuning enables parameter-efficient transfer learning. arXiv preprint arXiv:2303.02861 (2023). · arXiv
- Justin D Weisz, Michael Muller, Jessica He, and Stephanie Houde. 2023. Toward general design principles for generative AI applications. In Joint Proceedings of the IUI 2023 Workshops: HAI-GEN, ITAH, MILC, SHAI, SketchRec, SOCIALIZE co-located with the ACM International Conference on Intelligent User Interfaces, Vol. 3124. CEUR. http://ceur-ws.org/Vol-3359/paper14.pdf
- Justin D Weisz, Michael Muller, Stephanie Houde, John Richards, Steven I Ross, Fernando Martinez, Mayank Agarwal, and Kartik Talamadupula. 2021. Perfection Not Required? Human-AI Partnerships in Code Translation. In 26th International Conference on Intelligent User Interfaces. 402–412.
- Austin P Wright, Zijie J Wang, Haekyu Park, Grace Guo, Fabian Sperrle, Mennatallah El-Assady, Alex Endert, Daniel Keim, and Duen Horng Chau. 2020. A comparative analysis of industry human-AI interaction guidelines. arXiv preprint arXiv:2010.11761 (2020). · arXiv
- Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. 2023. Large Language Models as Optimizers. arXiv preprint arXiv:2309.03409 (2023). · arXiv
- Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, 2023. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. arXiv preprint arXiv:2306.05685 (2023). · arXiv
- Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593 (2019). · arXiv
Cited by
Bandi, A., Kongari, B., Naguru, R., Pasnoor, S., & Vilipala, S. V. (2025) · Future Internet · doi:10.3390/fi17090404 · Cited by 207
Smart product platforming powered by AI and generative AI: Personalization for the circular economy
Akhtar, P., Ghouri, A. M., Ashraf, A., Lim, J. J., Khan, N. R., & Ma, S. (2024) · International Journal of Production Economics · doi:10.1016/j.ijpe.2024.109283 · Cited by 158
“This helps to explore the ideas and then optimize them (Weisz et al., 2024).”
BackgroundWu, J., Cai, Y., Sun, T., Ma, K., & Lu, C. (2024) · Journal of Engineering Design · doi:10.1080/09544828.2024.2362587 · Cited by 134
“…although it violates the common principle of Human–Computer Interaction (HCI) that requires a system to respond consistently to the user’s inputs (Weisz et al. 2024), it is also to a certain extent compatible with design as a structurally ill-posed problem, which realises the exploration of…”
ID.8: Co-Creating Visual Stories with Generative AI
Antony, V. N., & Huang, C.-M. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3672277 · Cited by 104
“Critical issues like the loss of user autonomy and the generation of potentially dangerous or unsafe creative outputs can manifest in co-creative systems, often due to AI decisions taking precedence over human input [9].”
Al-kfairy, M. (2025) · IEEE Engineering Management Review · doi:10.1109/emr.2025.3534034 · Cited by 93
“…– Integration with legacy systems – Lack of best practices and standards – Continuous evolution and need for regular updates [3], [4], [28], [29], [30], [31] Data Privacy and Security – Managing sensitive data – Ensuring robust data privacy and security – Addressing biases in data used for AI –”
Metadata
- Authors
- Weisz, J. D., He, J., Muller, M., Hoefer, G., Miles, R., & Geyer, W.
- Affiliations
- IBM (United States) (US)
- Venue
- Proceedings of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2024-05-11
- Pages
- 1-22
- OpenAlex
- W4391334978
- Cited by
- 232
- References
- 100
- Citations by year
- 2026: 87 · 2025: 119 · 2024: 25
- FWCI
- 74.26
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
- Keywords
- generative AI applications, user experience design, design principles, design strategies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Design Principles for Generative AI Applications. https://app.cochat.ai/papers/10.1145/3613904.3642466
- The paper
- Weisz, J. D., He, J., Muller, M., Hoefer, G., Miles, R., & Geyer, W. (2024). Design Principles for Generative AI Applications. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3642466
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.