The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
How Experienced Designers of Enterprise Applications Engage AI as a Design Material
Open accessAbstract
HCI research has explored AI as a design material, suggesting that designers can envision AI’s design opportunities to improve UX. Recent research claimed that enterprise applications offer an opportunity for AI innovation at the user experience level. We conducted design workshops to explore the practices of experienced designers who work on cross-functional AI teams in the enterprise. We discussed how designers successfully work with and struggle with AI. Our findings revealed that designers can innovate at the system and service levels. We also discovered that making a case for an AI feature’s return on investment is a barrier for designers when they propose AI concepts and ideas. Our discussions produced novel insights on designers’ role on AI teams, and the boundary objects they used for collaborating with data scientists. We discuss the implications of these findings as opportunities for future research aiming to empower designers in working with data and AI.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This process is typically informal and relies on trial-and-error [104, 131, 165, 190].”
From Design Principles for Generative AI Applications (2024) · cites this paper as [190]
In the citing paper
“The recommendation to, 'Make clear what the system can do' may be difficult to implement in light of the emergent and unanticipated behaviors of generative foundation models [19], as well as the trial-and-error methods by which users iterate toward a desired outcome [104, 131, 165, 190].”
From Design Principles for Generative AI Applications (2024) · cites this paper as [190]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
What is AI Literacy? Competencies and Design Considerations
Long, D., & Magerko, B. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376727 · Cited by 3,241 · cited 1×
“However, there is no consensus on what constitutes a “good enough” understanding of AI [55, 68].”
BackgroundGuidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 2×
“In response, researchers have developed practitioner-facing AI tools, methods, guidelines, and design patterns to aid designers in accounting for AI systems’ UX breakdowns [2, 3, 31, 51, 61, 65], such as planning for AI inference errors [38] or setting user expectations [45].”
MethodResearch through design as a method for interaction design research in HCI
Zimmerman, J., Forlizzi, J., & Evenson, S. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240704 · Cited by 2,304
Self-citationLee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · cited 4×
“Recent HCI research has highlighted the use of boundary objects to scafold collaboration across diferent roles and disciplines in industry AI teams [14, 37, 43, 91].”
BackgroundInfluentialAlgorithmic injustice: A relational ethics approach
Birhane, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100205 · Cited by 707 · cited 2×
“The HCI community studied the issues around fairness, accountability, transparency, and ethics as they relate to AI systems [6, 25, 49, 75, 78, 85].”
Background'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691 · cited 1×
“For example, researchers studied users’ perceptions related to fairness in AI systems [5, 48, 89], and practitioners’ challenges and needs around designing responsible AI [37, 84].”
BackgroundRe-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design
Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376301 · Cited by 622 · cited 9×
“Several studies reported challenges in prototyping the user experience of AI systems [21, 93].”
BackgroundInfluentialSelf-citationFilho, S. L., & Kane, B. T. (2016) · National Conference on Artificial Intelligence · Cited by 576
Research works printed but not linked: 14
- Accenture. 2020 . Accenture Logistics Platform: Win the last mile . Retrieved Mar 24, 2022 from https://www.accenture.com/gb-en/services/public-service/accenture-logistics-platform Accenture. 2020. Accenture Logistics Platform: Win the last mile. Retrieved Mar 24, 2022 from https://www.accenture.com/gb-en/services/public-service/accenture-logistics-platform
- Design Council . 2005. The ‘double diamond’design process model . Design Council ( 2005 ). Design Council. 2005. The ‘double diamond’design process model. Design Council (2005).
- Hugh Dubberly and Paul Pangaro . 2019. Cybernetics and design: Conversations for action . In Design Cybernetics . Springer , 85–99. Hugh Dubberly and Paul Pangaro. 2019. Cybernetics and design: Conversations for action. In Design Cybernetics. Springer, 85–99.
- Jodi Forlizzi . 2018. Moving beyond user-centered design. interactions 25, 5 ( 2018 ), 22–23. Jodi Forlizzi. 2018. Moving beyond user-centered design. interactions 25, 5 (2018), 22–23.
- Andrea Gasparini , Ahmed Abdi Mohammed , and Gabriele Oropallo . 2018 . Service design for artificial intelligence . In Linköping Electronic Conference Proceedings . Linköping University Electronic Press, 1064–1073. Andrea Gasparini, Ahmed Abdi Mohammed, and Gabriele Oropallo. 2018. Service design for artificial intelligence. In Linköping Electronic Conference Proceedings. Linköping University Electronic Press, 1064–1073.
- Nielsen Norman Group . [n.d.]. Using Prioritization Matrices to Inform UX Decisions . Retrieved Mar 24, 2022 from https://www.nngroup.com/articles/prioritization-matrices/ Nielsen Norman Group. [n.d.]. Using Prioritization Matrices to Inform UX Decisions. Retrieved Mar 24, 2022 from https://www.nngroup.com/articles/prioritization-matrices/
- Titta Jylkäs , M Aijälä , Tytti Vuorikari , and Vésaal Rajab . 2018 . AI Assistants as Non-human Actors in Service Design . In In Proc of DMI Acad Des Manag Conf. 1–2. Titta Jylkäs, M Aijälä, Tytti Vuorikari, and Vésaal Rajab. 2018. AI Assistants as Non-human Actors in Service Design. In In Proc of DMI Acad Des Manag Conf. 1–2.
- R. King , E.F. Churchill , and C. Tan . 2017 . Designing with Data: Improving the User Experience with A/B Testing. O’Reilly Media. https://books.google.com/books?id=eM-PDgAAQBAJ R. King, E.F. Churchill, and C. Tan. 2017. Designing with Data: Improving the User Experience with A/B Testing. O’Reilly Media. https://books.google.com/books?id=eM-PDgAAQBAJ
- Ezio Manzini . 2009. New design knowledge. Design studies 30, 1 ( 2009 ), 4–12. Ezio Manzini. 2009. New design knowledge. Design studies 30, 1 (2009), 4–12.
- Virpi Roto , Jung-Joo Lee , Effie Lai-Chong Law , and John Zimmerman . 2021 . The Overlaps and Boundaries Between Service Design and User Experience Design. In Designing Interactive Systems Conference 2021. 1915 –1926. Virpi Roto, Jung-Joo Lee, Effie Lai-Chong Law, and John Zimmerman. 2021. The Overlaps and Boundaries Between Service Design and User Experience Design. In Designing Interactive Systems Conference 2021. 1915–1926.
- M Sloane , E Moss , O Awomolo , and L Forlano . 2020. Participation is not a Design Fix for Machine Learning. Participatory Approaches to Machine Learning ( 2020 ). M Sloane, E Moss, O Awomolo, and L Forlano. 2020. Participation is not a Design Fix for Machine Learning. Participatory Approaches to Machine Learning (2020).
- Aaron Springer , Jean Garcia-Gathright , and Henriette Cramer . 2018 . Assessing and Addressing Algorithmic Bias-But Before We Get There .... In 2018 AAAI Spring Symposium Series. Aaron Springer, Jean Garcia-Gathright, and Henriette Cramer. 2018. Assessing and Addressing Algorithmic Bias-But Before We Get There.... In 2018 AAAI Spring Symposium Series.
- Susan Leigh Star and James R Griesemer . 1989. Institutional ecology,translations ’ and boundary objects: Amateurs and professionals in Berkeley’s Museum of Vertebrate Zoology, 1907 -39. Social studies of science 19, 3 (1989), 387–420. Susan Leigh Star and James R Griesemer. 1989. Institutional ecology,translations’ and boundary objects: Amateurs and professionals in Berkeley’s Museum of Vertebrate Zoology, 1907-39. Social studies of science 19, 3 (1989), 387–420.
- S Stumpf , L Strappelli , S Ahmed , Y Nakao , A Naseer , GD Gamba , and D Regoli . 2021 . Design Methods for Artificial Intelligence Fairness and Transparency. In CEUR Workshop Proceedings, Vol. 2903 . S Stumpf, L Strappelli, S Ahmed, Y Nakao, A Naseer, GD Gamba, and D Regoli. 2021. Design Methods for Artificial Intelligence Fairness and Transparency. In CEUR Workshop Proceedings, Vol. 2903.
Cited by
Design Principles for Generative AI Applications
Weisz, J. D., He, J., Muller, M., Hoefer, G., Miles, R., & Geyer, W. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642466 · Cited by 232
“Within the past few years, the emergence of AI as a design material [41, 46, 62, 191] has necessitated guidelines that inform its use.”
BackgroundPosition: Towards Bidirectional Human-AI Alignment
Shen, H., Knearem, T., Ghosh, R., Alkiek, K., Krishna, K., Yachuan, L., Petridis, S., Peng, Y.-H., Qiwei, L., Si, C., et al. (2025) · neural information processing systems · doi:10.52202/085713-4186 · Cited by 209
Shi, Y., Gao, T., Jiao, X., & Cao, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610217 · Cited by 170
Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook
Yildirim, N., Pushkarna, M., Goyal, N., WATTENBERG, M. P., & Viégas, F. B. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580900 · Cited by 159
Self-citationLiao, Q. V., Subramonyam, H., Wang, J., & Vaughan, J. W. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580652 · Cited by 154
Metadata
- Authors
- Yildirim, N., Kass, A., Tung, T., Upton, C., Costello, D., Giusti, R., Lacin, S., Lovic, S., O'Neill, J. M., Meehan, R. O., Ó Loideáin, E., Pini, A., Corcoran, M., Hayes, J., Cahalane, D. J., Shivhare, G., Castoro, L., Caruso, G., Oh, C., McCann, J., Forlizzi, J., & Zimmerman, J.
- Affiliations
- Carnegie Mellon University (US)
- Accenture (United States) (US)
- Accenture (Ireland) (IE)
- Accenture The Dock, Ireland
- Accenture (Italy) (IT)
- Boston College (US)
- ORCID
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-13
- OpenAlex
- W4224985411
- Cited by
- 71
- References
- 69
- Citations by year
- 2026: 7 · 2025: 24 · 2024: 19 · 2023: 21
- FWCI
- 12.80
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by-nd)
- Funders
- NSF (National Science Foundation) (2112633)
- Accenture
- National Science Foundation (2007501)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Engineering › Mechanical Engineering › Design Education and Practice
- Keywords
- enterprise software, user experience design, boundary objects, return on investment, design workshop
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How Experienced Designers of Enterprise Applications Engage AI as a Design Material. https://app.cochat.ai/papers/10.1145/3491102.3517491
- The paper
- Yildirim, N., Kass, A., Tung, T., Upton, C., Costello, D., Giusti, R., Lacin, S., Lovic, S., O'Neill, J. M., Meehan, R. O., Ó Loideáin, E., Pini, A., Corcoran, M., Hayes, J., Cahalane, D. J., Shivhare, G., Castoro, L., Caruso, G., Oh, C., . . . Zimmerman, J. (2022). How Experienced Designers of Enterprise Applications Engage AI as a Design Material. CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517491
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.