Solving Separation-of-Concerns Problems in Collaborative Design of Human-AI Systems through Leaky Abstractions
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
In conventional software development, user experience (UX) designers and engineers collaborate through separation of concerns (SoC): designers create human interface specifications, and engineers build to those specifications. However, we argue that Human-AI systems thwart SoC because human needs must shape the design of the AI interface, the underlying AI sub-components, and training data. How do designers and engineers currently collaborate on AI and UX design? To find out, we interviewed 21 industry professionals (UX researchers, AI engineers, data scientists, and managers) across 14 organizations about their collaborative work practices and associated challenges. We find that hidden information encapsulated by SoC challenges collaboration across design and engineering concerns. Practitioners describe inventing ad-hoc representations exposing low-level design and implementation details (which we characterize as leaky abstractions) to “puncture” SoC and share information across expertise boundaries. We identify how leaky abstractions are employed to collaborate at the AI-UX boundary and formalize a process of creating and using leaky abstractions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 5×
“Our primary sources include Microsoft’s “Guidelines for Human-AI Interactions” [4, 7], Google’s “People + AI Guidebook” [39], and Apple’s “Human Interface Guidelines” [51].”
BackgroundInfluentialHuman-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743 · cited 1×
“We contribute to the current understanding of challenges faced by UX roles [32, 100, 102] and AI roles [19, 73, 104] in developing AI systems that align with human needs, values and are useful and usable by people [37, 67, 71, 79, 86, 97].”
BackgroundFairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · cited 1×
“For instance, SoC abstractionsmay lose key information about complex socio-technical contexts necessary for fairness [1, 85].”
BackgroundMemoing in qualitative research: Probing data and processes
Birks, M., Chapman, Y., & Francis, K. (2008) · Journal of Research in Nursing · doi:10.1177/1744987107081254 · Cited by 1,519
McDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence
Mohamed, S., Png, M.-T., & Isaac, W. (2020) · Philosophy & Technology · doi:10.1007/s13347-020-00405-8 · Cited by 912
Artificial Intelligence, Values, and Alignment
Gabriel, I. (2020) · Minds and Machines · doi:10.1007/s11023-020-09539-2 · Cited by 896 · cited 1×
“We contribute to the current understanding of challenges faced by UX roles [32, 100, 102] and AI roles [19, 73, 104] in developing AI systems that align with human needs, values and are useful and usable by people [37, 67, 71, 79, 86, 97].”
BackgroundImproving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · cited 1×
“Despite increasing awareness, organizational goals and resource constraints continue to pose challenges for building responsible AI guidelines into current design practices [49, 64].”
BackgroundResearch Priorities for Robust and Beneficial Artificial Intelligence
Russell, S., Dewey, D., & Tegmark, M. (2015) · AI Magazine · doi:10.1609/aimag.v36i4.2577 · Cited by 637 · cited 1×
“Several aspects of HAI design need to be incorporated throughout AI workflow, including identifyingmodel requirements, data collection and labeling, features engineering, and model training [5, 42, 80].”
BackgroundRe-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design
Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376301 · Cited by 622 · cited 6×
“We contribute to the current understanding of challenges faced by UX roles [32, 100, 102] and AI roles [19, 73, 104] in developing AI systems that align with human needs, values and are useful and usable by people [37, 67, 71, 79, 86, 97].”
BackgroundInfluentialKnowledge Practices in Design: The Role of Visual Representations as `Epistemic Objects'
Ewenstein, B., & Whyte, J. (2009) · Organization Studies · doi:10.1177/0170840608083014 · Cited by 584
The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems
Dellermann, D., Calma, A., Lipusch, N., Weber, T., Weigel, S., & Ebel, P. (2021) · arXiv · doi:10.48550/arxiv.2105.03354 · Cited by 456
Madaio, M. A., Stark, L., Wortman Vaughan, J., & Wallach, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376445 · Cited by 436 · cited 2×
“Despite increasing awareness, organizational goals and resource constraints continue to pose challenges for building responsible AI guidelines into current design practices [49, 64].”
BackgroundLee, C. P. (2007) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-007-9044-5 · Cited by 413
UX Design Innovation: Challenges for Working with Machine Learning as a Design Material
Dove, G., Halskov, K., Forlizzi, J., & Zimmerman, J. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025739 · Cited by 391 · cited 5×
“Further, by focusing on creating the algorithm, engineers often fail to consider the AI experience as a whole and their involvement in UX design tapers [32, 38].”
BackgroundInfluentialThe Future of Human-AI Collaboration: A Taxonomy of Design Knowledge for Hybrid Intelligence Systems
Dellermann, D., Calma, A., Lipusch, N., Weber, T., Weigel, S., & Ebel, P. (2019) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2019.034 · Cited by 279
Kocielnik, R., Amershi, S., & Bennett, P. N. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300641 · Cited by 277
Lwakatare, L. E., Raj, A., Bosch, J., Olsson, H. H., & Crnkovic, I. (2019) · Lecture Notes in Business Information Processing · doi:10.1007/978-3-030-19034-7_14 · Cited by 214 · cited 2×
“2 Challenges for Engineers: Similarly, engineers focused on algorithms and techniques fail to consider human perspectives during initial experimentation and AI prototyping processes [47, 61].”
BackgroundAssessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support
Madaio, M., Egede, L., Subramonyam, H., Wortman Vaughan, J., & Wallach, H. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512899 · Cited by 156
Self-citationToward human-centered algorithm design
Baumer, E. P. (2017) · Big Data & Society · doi:10.1177/2053951717718854 · Cited by 125 · cited 5×
“In complex domains such as HAI, teams would ideally address knowledge differences or “symmetry of ignorance” between HCI and AI professionals through collaboration and social creativity [35].”
BackgroundInfluentialSketching NLP: A Case Study of Exploring the Right Things To Design with Language Intelligence
Yang, Q., Cranshaw, J., Amershi, S., Iqbal, S. T., & Teevan, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300415 · Cited by 90 · cited 6×
“We contribute to the current understanding of challenges faced by UX roles [32, 100, 102] and AI roles [19, 73, 104] in developing AI systems that align with human needs, values and are useful and usable by people [37, 67, 71, 79, 86, 97].”
BackgroundInfluentialDesigning Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs
Barocas, S., Guo, A., Kamar, E., Krones, J., Morris, M. R., Vaughan, J. W., Wadsworth, W. D., & Wallach, H. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462610 · Cited by 72 · cited 1×
“For instance, to design AI systems with fairness in mind, teams need to collaboratively define fair performance by considering diverse stakeholders, contexts of use, and assessment criteria (i.e., disaggregated evaluation [14]).”
Background
Cited by
AI-Powered Innovation in Digital Transformation: Key Pillars and Industry Impact
Aldoseri, A., Al-Khalifa, K. N., & Hamouda, A. M. (2024) · Sustainability · doi:10.3390/su16051790 · Cited by 311
Widder, D. G., & Nafus, D. (2023) · Big Data & Society · doi:10.1177/20539517231177620 · Cited by 119
Yildirim, N., Richardson, H., Wetscherek, M. T., Bajwa, J., Jacob, J., Pinnock, M. A., Harris, S., Coelho De Castro, D., Bannur, S., Hyland, S., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642013 · Cited by 82
Ayobi, A., Hughes, J., Duckworth, C. J., Dylag, J. J., Gordon James, S., Marshall, P., Guy, M., Kumaran, A., Chapman, A., Boniface, M., et al. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581424 · Cited by 26
Tan, M., Lee, H., Wang, D., & Subramonyam, H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637358 · Cited by 14
“Furthermore, traditional engineering processes limit engagement with domain experts and end-users, separating the work of technical development and understanding end-user requirements [85], and prioritizing technical affordances over the problems of practitioners and real-world contexts [8, 38].”
BackgroundSelf-citation
Metadata
- Authors
- Subramonyam, H., Im, J., Seifert, C., & Adar, E.
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-21
- OpenAlex
- W4224991249
- Cited by
- 54
- References
- 23
- FWCI
- 9.47
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.