How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
Open accessThe DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Abstract
The development of AI applications is a multidisciplinary effort, involving multiple roles collaborating with the AI developers, an umbrella term we use to include data scientists and other AI-adjacent roles on the same team. During these collaborations, there is a knowledge mismatch between AI developers, who are skilled in data science, and external stakeholders who are typically not. This difference leads to communication gaps, and the onus falls on AI developers to explain data science concepts to their collaborators. In this paper, we report on a study including analyses of both interviews with AI developers and artifacts they produced for communication. Using the analytic lens of shared mental models, we report on the types of communication gaps that AI developers face, how AI developers communicate across disciplinary and organizational boundaries, and how they simultaneously manage issues regarding trust and expectations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“AI documentation is advocated to increase the transparency and facilitate understanding of AI [35,36,40,75,76,82], and is embodied in recently proposed features of AI Factsheets [8,81] and Model Cards [64].”
From Investigating Explainability of Generative AI for Code through Scenario-based Design (2022) · cites this paper as [76]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“In addition, AI -based translation tools can help bridge the communication gap between employees with different language backgrounds and help them communicate effectively (Piorkowski et al., 2021).”
From The Impact of Artificial Intelligence on Workers' Skills: Upskilling and Reskilling in Organisations (2023) · cites this paper as Piorkowski et al., 2021
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Clark, H. H., & Brennan, S. E. (1991) · Perspectives on socially shared cognition. · doi:10.1037/10096-006 · Cited by 4,555
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“Existing efforts around documenting aspects of the AI lifecycle in dataset documentation [15] and model service documentation [4, 34] are fairly static and do not consider questions of how to dynamically vary the content that is displayed for the role in question.”
BackgroundMcDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206
Software Engineering for Machine Learning: A Case Study
Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., & Zimmermann, T. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00042 · Cited by 1,135 · cited 6×
“Echoing findings from previous work, one of the common practices is to hold AI education sessions for software engineers to overcome the knowledge barrier [2].”
BackgroundInfluentialMetaphor No More: A 15-Year Review of the Team Mental Model Construct
Mohammed, S., Ferzandi, L., & Hamilton, K. (2010) · Journal of Management · doi:10.1177/0149206309356804 · Cited by 832 · cited 1×
“This allows them to more readily anticipate each others’ needs and work together more efficiently [35].”
BackgroundAustin, J. R. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.5.866 · Cited by 656 · cited 1×
“[69] summarized four stages for developing an SMM within agile software development teams from prior literature: knowing meta-knowledge of the team [26, 32], learning and building the team’s transactive memory system [6], understanding and reaching consensus [53], and executing team goals.”
MethodA New Chatbot for Customer Service on Social Media
Xu, A., Liu, Z., Guo, Y., Sinha, V., & Akkiraju, R. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025496 · Cited by 550 · cited 1×
“AI has been widely applied to many domains such as helping diagnose patients [14, 60], building chatbots to improve customer service [29, 66], and enhancing the education experience with intelligent tutoring systems [67].”
BackgroundSurvey on user interface programming
Myers, B. A., & Rosson, M. B. (1992) · Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '92 · doi:10.1145/142750.142789 · Cited by 445 · cited 1×
“Software development teams tend to spend extensive time on planning and brainstorming [39].”
BackgroundFactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434 · cited 1×
“Existing efforts around documenting aspects of the AI lifecycle in dataset documentation [15] and model service documentation [4, 34] are fairly static and do not consider questions of how to dynamically vary the content that is displayed for the role in question.”
BackgroundSelf-citationTeam learning: building shared mental models
Van den Bossche, P., Gijselaers, W., Segers, M., Woltjer, G., & Kirschner, P. (2011) · Instructional Science · doi:10.1007/s11251-010-9128-3 · Cited by 379 · cited 2×
“In this approach, an SMM provides one means to establish common ground [53].”
BackgroundMethod
Research works printed but not linked: 19
- Arunima Chaudhary et almbox. 2021. AutoText: An End-to-End AutoAI Framework for Text. In AAAI. ACM.
- Herbert H Clark. 2006. Context and common ground. Concise Encyclopedia of Philosophy of Language and Linguistics (2006), 85--87.
- Sharolyn Converse, JA Cannon-Bowers, and E Salas. 1993. Shared mental models in expert team decision making. Individual and group decision making: Current issues, Vol. 221 (1993), 221--46.
- Paul Dourish. 2014. Reading and interpreting ethnography. In Ways of Knowing in HCI. Springer, 1--23.
- Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018. Datasheets for datasets. arXiv preprint arXiv:1803.09010 (2018). · arXiv
- Penny Lacey. 2002. 9 Multidisciplinary Work Challenges and Possibilities. Special Education Reformed: Inclusion-Beyond Rhetoric? (2002), 157.
- Charlotte P Lee and Drew Paine. 2015. From The Matrix to a Model of Coordinated Action (MoCA) A Conceptual Framework of and for CSCW. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing. 179--194.
- M Lee, Tristan Johnson, and Myung H Jin. 2012. Toward understanding the dynamic relationship between team and task shared mental models as determinants of team and individual performances. International journal of information technology and business management, Vol. 8, 1 (2012), 1--14.
- Sara A McComb. 2007. Mental model convergence: The shift from being an individual to being a team member. Research in multi-level issues, Vol. 6 (2007), 95--147.
- Michael Muller, Melanie Feinberg, Timothy George, Steven J Jackson, Bonnie E John, Mary Beth Kery, and Samir Passi. 2019 a. Human-Centered Study of Data Science Work Practices. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, W15.
- DJ Patil. 2011. Building data science teams." O'Reilly Media, Inc.".
- Gerry Stahl. 2013. Theories of collaborative cognition: Foundations for CSCL and CSCW together. In Computer-supported collaborative learning at the workplace. Springer, 43--63.
- Christoph Treude Margaret-anne Storey and Jens Weber. 2009. Empirical studies on collaboration in software development: A systematic literature review. (2009).
- Wil MP Van der Aalst. 2014. Data scientist: The engineer of the future. In Enterprise interoperability VI. Springer, 13--26.
- April Yi Wang, Anant Mittal, Christopher Brooks, and Steve Oney. 2019 a. How Data Scientists Use Computational Notebooks for Real-Time Collaboration. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--30.
- Dakuo Wang, Liuping Wang, Zhan Zhang, Ding Wang, Haiyi Zhu, Yvonne Gao, Xiangmin Fan, and Feng Tian. 2021 c. Brilliant AI Doctor in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment. In Proceedings of the CHI 2021.
- Bryan Wilder, Eric Horvitz, and Ece Kamar. 2020. Learning to Complement Humans. arXiv preprint arXiv:2005.00582 (2020). · arXiv
- Ying Xu, Dakuo Wang, Penelope Collins, Hyelim Lee, and Mark Warschauer. [n.d.]. Same benefits, different communication patterns: Comparing Children's reading with a conversational agent vs. a human partner. Computers & Education, Vol. 161 ([n.,d.]), 104059.
- Xiaodan Yu and Stacie Petter. 2014. Understanding agile software development practices using shared mental models theory. Information and software technology, Vol. 56, 8 (2014), 911--921.
Cited by
The Impact of Artificial Intelligence on Workers' Skills: Upskilling and Reskilling in Organisations
Morandini, S., Fraboni, F., De Angelis, M., Puzzo, G., Giusino, D., & Pietrantoni, L. (2023) · Informing Science: The International Journal of an Emerging Transdiscipline · doi:10.28945/5078 · Cited by 439
“In addition, AI - based translation tools can help bridge the communication gap between employees with different language backgrounds and help them communicate effectively (Piorkowski et al., 2021).”
BackgroundInvestigating Explainability of Generative AI for Code through Scenario-based Design
Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219
“AI documentation is advocated to increase the transparency and facilitate understanding of AI [35, 36, 40, 75, 76, 82], and is embodied in recently proposed features of AI Factsheets [8, 81] and Model Cards [64].”
BackgroundSelf-citationCembrowska-Lech, D., Krzemińska, A., Miller, T., Nowakowska, A., Adamski, C., Radaczyńska, M., Mikiciuk, G., & Mikiciuk, M. (2023) · Biology · doi:10.3390/biology12101298 · Cited by 175
Li, B., Lowell, V. L., Wang, C., & Li, X. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100266 · Cited by 120
Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook
Yildirim, N., Pushkarna, M., Goyal, N., WATTENBERG, M. P., & Viégas, F. B. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580900 · Cited by 159
Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges
Wang, Q., Madaio, M., Kane, S., Kapania, S., Terry, M., & Wilcox, L. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581278 · Cited by 90
“However, this collaboration also poses challenges in that UX practitioners and engineers don’t always share a common perspective, language, or workfow [82, 97, 119], leading UX practitioners to take on extra work to bridge disciplinary boundaries through sharing user stories and raw user feedback from”
BackgroundWang, A. Y., Wang, D., Drozdal, J., Muller, M. J., Park, S., Weisz, J. D., Liu, X., Wu, L., & Dugan, C. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3489465 · Cited by 121
“1 INTRODUCTION Documenting the story behind code and results is critical for data scientists to collabrate effectively with others, as well as their future selves [27, 31, 43, 51].”
BackgroundSelf-citationYildirim, N., Richardson, H., Wetscherek, M. T., Bajwa, J., Jacob, J., Pinnock, M. A., Harris, S., Coelho De Castro, D., Bannur, S., Hyland, S., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642013 · Cited by 82
“HCI research highlights the difficulty of eliciting input from domain stakeholders in AI design and development, especially in early ideation and problem formulation phases to inform what is the right thing to design [22, 39, 69, 103].”
BackgroundSoleimani, M., Intezari, A. H., & Pauleen, D. J. (2021) · International Journal of Knowledge Management · doi:10.4018/ijkm.290022 · Cited by 131
“For example, a recent study by Piorkowski et al. (2021) shows that there are communication and collaboration problems between AI developers and external stakeholders such as domain experts and business experts that cause challenges in developing AI.”
BackgroundAutoDS: Towards Human-Centered Automation of Data Science
Wang, D., Andres, J., Weisz, J. D., Oduor, E., & Dugan, C. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445526 · Cited by 75
“, domain experts and data scientists) [29, 52, 70].”
Self-citationCreating Design Resources to Scaffold the Ideation of AI Concepts
Yildirim, N., Oh, C., Sayar, D., Brand, K., Challa, S., Turri, V., Crosby Walton, N., Wong, A. E., Forlizzi, J., McCann, J., et al. (2023) · Proceedings of the 2023 ACM Designing Interactive Systems Conference · doi:10.1145/3563657.3596058 · Cited by 76
Zając, H. D., Li, D., Dai, X., Carlsen, J. F., Kensing, F., & Andersen, T. O. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3582430 · Cited by 96
“Piorkowski et al. [97] highlight three communication challenges in interdisciplinary environments.”
BackgroundHow Experienced Designers of Enterprise Applications Engage AI as a Design Material
Yildirim, N., Kass, A., Tung, T., Upton, C., Costello, D., Giusti, R., Lacin, S., Lovic, S., O'Neill, J. M., Meehan, R. O., et al. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517491 · Cited by 71
“There has been a growing interest in the HCI community around the cross-disciplinary collaboration throughout the AI lifecycle [37, 59, 66, 69, 95].”
BackgroundFacilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models
Park, S., Wang, A. Y., Kawas, B., Liao, Q. V., Piorkowski, D., & Danilevsky, M. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450637 · Cited by 88
“Currently most ML projects still rely on data scientists to write code and set up the pipelines [61].”
BackgroundSelf-citationDeng, W. H., Yildirim, N., Chang, M., Eslami, M., Holstein, K., & Madaio, M. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594037 · Cited by 104
Zheng, C., Wang, D., Wang, A. Y., & Ma, X. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517615 · Cited by 52
Self-citationNahar, N., Zhang, H., Lewis, G. A., Zhou, S., & Kästner, C. (2023) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain58948.2023.00034 · Cited by 85
Guidelines for Integrating Value Sensitive Design in Responsible AI Toolkits
Sadek, M., Constantinides, M., Quercia, D., & Mougenot, C. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642810 · Cited by 84
A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities
Tocchetti, A., Corti, L., Balayn, A., Yurrita, M., Lippmann, P., Brambilla, M., & Yang, J. (2024) · ACM Computing Surveys · doi:10.1145/3665926 · Cited by 98
Bach, A. K. P., Nørgaard, T. M., Brok, J. C., & van Berkel, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581513 · Cited by 47
What Did My AI Learn? How Data Scientists Make Sense of Model Behavior
Cabrera, Á. A., Ribeiro, M. T., Lee, B., DeLine, R. A., Perer, A., & Drucker, S. M. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3542921 · Cited by 62
“Some of the most common schemas are classic methods for evaluating AI systems, such as the confusion matrix for classification problems [62, 75] and residual plots for regressions.”
MethodHuman-Centered AI Design in Reality: A Study of Developer Companies’ Practices
Hartikainen, M., Väänänen, K., Lehtiö, A., Ala-Luopa, S., & Olsson, T. (2022) · Nordic Human-Computer Interaction Conference · doi:10.1145/3546155.3546677 · Cited by 48
How Do Manufacturing Firms Manage Artificial Intelligence to Drive Iterative Product Innovation?
Jiang, X., Jiang, X., Sun, W., & Fan, W. (2023) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2023.3259396 · Cited by 47
“For example, Omohundro [41] argues that AI applications that developers use to communicate with other stakeholders could disrupt how stakeholders interact with each other and influence stakeholders’ trust in such interactions.”
BackgroundBalayn, A., Yurrita, M., Yang, J., & Gadiraju, U. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604674 · Cited by 63
A study of UX practitioners roles in designing real-world, enterprise ML systems
Zdanowska, S., & Taylor, A. S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517607 · Cited by 46
“Other research focuses solely on data scientists [57] or natural language practitioners [37].”
BackgroundNotable: On-the-fly Assistant for Data Storytelling in Computational Notebooks
Li, H., Ying, L., Zhang, H., Wu, Y., Qu, H., & Wang, Y. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580965 · Cited by 38
AI in Context: Harnessing Domain Knowledge for Smarter Machine Learning
Miller, T. I., Durlik, I., Łobodzińska, A., Dorobczyński, L., & Jasionowski, R. (2024) · Applied Sciences · doi:10.3390/app142411612 · Cited by 56
Slide4N: Creating Presentation Slides from Computational Notebooks with Human-AI Collaboration
Wang, F., Liu, X., Liu, O., Neshati, A., Ma, T., Zhu, M., & Zhao, J. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580753 · Cited by 42
“Much tedious and repetitive work needs to be done to transfer contents between the computational notebooks and the slides [52], since they are fundamentally two diferent mediums.”
BackgroundInfluential(2022) · doi:10.1145/3491102.3501868 · Cited by 36
“HCI researchers have examined the role of collaboration between data science workers for data work [6, 51, 62, 64, 73, 74, 78, 103].”
BackgroundSketching AI Concepts with Capabilities and Examples: AI Innovation in the Intensive Care Unit
Yildirim, N., Zlotnikov, S., Sayar, D., Kahn, J. M., Bukowski, L. A., Amin, S. S., Riman, K. A., Davis, B. S., Minturn, J. S., King, A. J., et al. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641896 · Cited by 55
On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub
Watanabe, M., Li, H., Kashiwa, Y., Reid, B., Iida, H., & Hassan, A. E. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3798166 · Cited by 34
Rädler, S., Berardinelli, L., Winter, K., Rahimi, A., & Rinderle‐Ma, S. (2024) · Software & Systems Modeling · doi:10.1007/s10270-024-01211-y · Cited by 59
D2S: Document-to-Slide Generation Via Query-Based Text Summarization
Sun, E., Hou, Y., Wang, D., Zhang, Y., & Wang, N. X. R. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.111 · Cited by 85
Self-citationConstantinides, M., Bogucka, E. P., Quercia, D., Kallio, S., & Tahaei, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686927 · Cited by 59
“Building on these themes, previous works have proposed a set of guidelines involving specific groups of AI practitioners.”
Beyond Anthropomorphism: Social Presence in Human– AI Collaboration Processes
Siemon, D., Elshan, E., de Vreede, T., Ebel, P., & de Vreede, G. (2025) · Journal of Management Studies · doi:10.1111/joms.70000 · Cited by 49
“This open and interactive communication can foster a shared understanding among team members and enhance their willingness to depend on one another’s expertise and insights, including those provided by AI (La Torre et al., 2021; Piorkowski et al., 2021).”
Dealing With Data Challenges When Delivering Data-Intensive Software Solutions
Graetsch, U. M., Khalajzadeh, H., Shahin, M., Hoda, R., & Grundy, J. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3291003 · Cited by 35
“Similarly, the IBM study focusing on AI Developer communication in multi-disciplinary teams [5] identified ’knowledge gaps’ as a challenge where domain expert stakeholders spent a lot of time communicating with domain expertise with data experts, which aligns with the understanding data challenge…”
BackgroundInfluentialHohman, F., Kery, M. B., Ren, D., & Moritz, D. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642109 · Cited by 50
“Researchers have studied how AI/ML teams collaborate across diverse technical roles [68, 70, 118], or adopt new technology like AutoML [99–101].”
BackgroundAyobi, A., Hughes, J., Duckworth, C. J., Dylag, J. J., Gordon James, S., Marshall, P., Guy, M., Kumaran, A., Chapman, A., Boniface, M., et al. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581424 · Cited by 26
Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML
Narkar, S., Zhang, Y., Liao, Q. V., Wang, D., & Weisz, J. D. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450658 · Cited by 29
“However, research on model analytics and comparison of manually-built models has shown that none of the stakeholders are satisfied to only see performance metrics [21, 23, 28, 29, 35].”
BackgroundSelf-citationModel Sketching: Centering Concepts in Early-Stage Machine Learning Model Design
Lam, M. S., Ma, Z., Li, A., Freitas, I., Wang, D., Landay, J. A., & Bernstein, M. S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581290 · Cited by 33
“Recent literature frames the data science and ML project workflow as a multi-stage collaborative lifecycle where domain experts focus on high-level problem definition and results interpretation, while ML practitioners place their cognitive focus on low-level implementation details [50, 62, 68].”
BackgroundSelf-citationHow Domain Experts Work with Data: Situating Data Science in the Practices and Settings of Craftwork
Jung, J. Y., Steinberger, T., King, J. L., & Ackerman, M. S. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512905 · Cited by 35
“, data scientists professionally trained in statistical analytics techniques — make decisions [44], perceive and understand automated tools and their results [92, 35, 42, 23, 94], and collaborate with other technical data scientists [91, 74, 95].”
BackgroundYoo, D. W., Woo, H., Pendse, S. R., Lu, N. Y., Birnbaum, M. L., Abowd, G. D., & Choudhury, M. D. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637372 · Cited by 38
Gerakari, M., Katsileros, A., Kleftogianni, K., Tani, E., Bebeli, P. J., & Papasotiropoulos, V. (2025) · Agronomy · doi:10.3390/agronomy15030757 · Cited by 25
The Value-Sensitive Conversational Agent Co-Design Framework
Sadek, M., Calvo, R. A., & Mougenot, C. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2426737 · Cited by 32
“Currently, the majority of CA co-design (Sadek et al., 2023a), and collaboration with non-technical stakeholders in AI in general (Piorkowski et al., 2021), occurs during the early stages of the design process for scoping and ideation, or during the final stages for testing and verification.”
Lancaster, C., Schulenberg, K., Flathmann, C., McNeese, N. J., & Freeman, G. (2023) · ACM Journal on Responsible Computing · doi:10.1145/3632121 · Cited by 38
“While responsible AI is often situated at this high level, research suggests that AI governance may fail to address the societal perception of fairness from these systems [53, 90].”
BackgroundWang, F., Lin, Y., Yang, L., Li, H., Gu, M., Zhu, M., & Qu, H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642865 · Cited by 20
“It’s difcult for other collaborators to understand, which in turn hinders communication [15, 19, 46].”
InfluentialThe Power of AI-Enabled Chatbots as an Organizational Social Listening Tool
Zhou, A. Y., Men, L. R., & Tsai, W. S. (2023) · Routledge eBooks · doi:10.4324/9781003273851-6 · Cited by 29
Managerial insights for AI/ML implementation: a playbook for successful organizational integration
Abonamah, A. A., & Abdelhamid, N. (2024) · Discover Artificial Intelligence · doi:10.1007/s44163-023-00100-5 · Cited by 19
Challenges in Value-Sensitive AI Design: Insights from AI Practitioner Interviews
Sadek, M., & Mougenot, C. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2439021 · Cited by 29
“Practitioners in previous studies also reported struggling with stakeholder expectations (Kross, 2022; Piorkowski et al., 2021; Vogelsang & Borg, 2019; Zdanowska & Taylor, 2022).”
Balayn, A., Yurrita, M., Rancourt, F., Casati, F., & Gadiraju, U. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713787 · Cited by 50
“In the context of AI, researchers have investigated AI lifecy-cles [93, 101] and AI actors involved in trustworthy AI questions [36, 52, 62, 103, 118].”
Li, H., Wang, Y., Liao, Q. V., & Qu, H. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3552017 · Cited by 25
Schulenberg, K., Hauptman, A. I., Schlesener, E. A., Watkins, H., & Freeman, G. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.022 · Cited by 22
Tan, M., Lee, H., Wang, D., & Subramonyam, H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637358 · Cited by 14
“This finding extends the existing understanding of ML team collaboration where technical members lack domain contexts, in which parties assume a diverse but equal contribution [51, 57, 64].”
BackgroundSelf-citationMadaio, M., Chen, J., Wallach, H., & Vaughan, J. W. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653705 · Cited by 30
Towards more sustainable and trustworthy reporting in machine learning
Fischer, R., Liebig, T., & Morik, K. J. (2024) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-024-01020-3 · Cited by 19
“Reporting plays a vital role for bridging communication gaps between ML experts and less informed practitioners (Piorkowski et al. 2021).”
Kalogiannidis, S., Patitsa, C. D., & Chalaris, M. (2024) · WSEAS TRANSACTIONS ON BUSINESS AND ECONOMICS · doi:10.37394/23207.2024.21.157 · Cited by 19
“Speech Recognition System (SRS) also referred to as Automatic audio Recognition (ASR) or computer speech recognition, SRS is a computer software that uses an algorithm to translate an audio signal into a string of words, [5], [25].”
InfluentialXu, Y., Shao, Y., Dong, J., Shi, S., Jiang, C., & Li, Q. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713772 · Cited by 16
Distribution of Forward-Looking Responsibility in the EU Process on AI Regulation
Hedlund, M. (2022) · Frontiers in Human Dynamics · doi:10.3389/fhumd.2022.703510 · Cited by 12
Simbolon, Y. (2023) · Veritas et Justitia · doi:10.25123/vej.v9i1.6037 · Cited by 40
An interdisciplinary perspective of the built-environment microbiome
McAlister, J. S., Blum, M. J., Bromberg, Y., Fefferman, N. H., He, Q., Lofgren, E., Miller, D. L., Schreiner, C. L., Candan, K. S., Szabo‐Rogers, H. L., et al. (2024) · FEMS Microbiology Ecology · doi:10.1093/femsec/fiae166 · Cited by 11
Xie, S., Zimmerman, J., & Eslami, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713841 · Cited by 24
Merkelbach, S., von Enzberg, S., Kühn, A., & Dumitrescu, R. (2022) · IEEE International Conference on Industrial Cyber-Physical Systems (ICPS) · doi:10.1109/icps51978.2022.9816871 · Cited by 20
Balayn, A., Rikalo, N., Yang, J., & Bozzon, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581555 · Cited by 18
Train clinical AI to reason like a team of doctors
Banerji, C. R. S., Chakraborti, T., Ismail, A. A., Ostmann, F., & MacArthur, B. D. (2025) · Nature · doi:10.1038/d41586-025-00618-x · Cited by 9
Staub, L., van Giffen, B., Hehn, J., & Sturm, S. G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-48057-7_16 · Cited by 13
Deng, W. H., Barocas, S., & Vaughan, J. W. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711076 · Cited by 23
“…has been a significant push to enhance our understanding of industry practitioners’ current practices, challenges, and needs around responsible technology design, and to develop tools and guidelines to support them [40, 41, 62, 64, 84, 105, 106, 108, 111, 117, 123, 124, 145, 150, 152, 156, 157].”
Security Patterns for Machine Learning: The Data-Oriented Stages
Zhang, X., & Jaskolka, J. (2022) · European Conference on Pattern Languages of Programs · doi:10.1145/3551902.3565070 · Cited by 20
Unpacking human systems in data science innovations: Key innovator perspectives
Li, K., & Griffin, M. A. (2023) · Technovation · doi:10.1016/j.technovation.2023.102869 · Cited by 13
AIMEE: An Exploratory Study of How Rules Support AI Developers to Explain and Edit Models
Piorkowski, D., Vejsbjerg, I., Cornec, O., Daly, E., & Alkan, Ö. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610046 · Cited by 16
“Better knowledge alignment has downstream effects such as easing communication in teams [71] and improved understanding of how a model works [81].”
BackgroundInfluentialSelf-citationA Systematic Review of Improving Knowledge Management with Generative AI and Large Language Models
Nguyen, L. T. K., Connolly, J., & Nguyen, H. N. (2025) · Journal of Advances in Information Technology · doi:10.12720/jait.16.4.594-612 · Cited by 13
“Accuracy and credibility [1, 2, 13, 18, 21, 39] Ensuring accuracy and contextual relevance in AI-generated information Privacy concerns [1, 26] Supporting various organizational tasks with privacy, developing responsible ML systems Organizational knowledge and ML Work design characteristics [18]…”
InfluentialAnalyzing Collaborative Challenges and Needs of UX Practitioners when Designing with AI/ML
Muralikumar, M. D., & McDonald, D. W. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686986 · Cited by 12
“While some prior HCI and CSCW research studies seek to study end-users and inform the design of a particular type of AI/ML application [15], other studies have examined work practices of professionals who develop AI/ML systems to understand more generic and systemic challenges [19, 35, 43, 53, 56].”
An Emerging Design Space of How Tools Support Collaborations in AI Design and Development
Muralikumar, M. D., & McDonald, D. W. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3701181 · Cited by 11
“One challenge from the perspective of AI practitioners is the time and effort it takes to create presentations using Powerpoint or Google Slides and maintain them over time, especially since they are removed from where analyses happen (computational notebooks) [8, 75].”
Wang, J. Q., Liu, G., & Lei, Y. (2025) · International Communication of Chinese Culture · doi:10.1007/s40636-025-00334-1 · Cited by 13
“Studies exploring AI developers’ ethical challenges in multidisciplinary contexts propose adaptive multimodal technologies to enhance human-AI communication (Girju 2021; Piorkowski et al. 2021).”
Retrospector: Rapid collaborative reflection to improve collaborative practices
Park, S., & Kulkarni, C. E. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610084 · Cited by 11
Self-citationAINeedsPlanner: A Workbook to Support Effective Collaboration Between AI Experts and Clients
Kim, D. H., Shin, H., Yadgarova, S., Son, J., Subramonyam, H., & Kim, J. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661577 · Cited by 11
“Based on these existing challenges and the widespread adoption of documentation during the planning stage [48], we introduce the AINeedsPlanner workbook as a tool for facilitating effective collaboration betweenAI expertsand theirclients withdeeper client involvement (Figure 1).”
BackgroundMethodOn the Current State of Industrial Data Science: Challenges, Best Practices, and Future Directions
von Enzberg, S., Weller, J., Brock, J., Merkelbach, S., Panzner, M., Lick, J., Kühn, A., & Dumitrescu, R. (2024) · Procedia CIRP · doi:10.1016/j.procir.2024.10.266 · Cited by 11
“• Collaboration between domain experts and data scientists can be challenging due to different prior knowledge and experiences [34].”
A Case Study on AI Engineering Practices: Developing an Autonomous Stock Trading System
Grote, M., & Bogner, J. (2023) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain58948.2023.00032 · Cited by 8
“[40], knowledge in a domain as complex as AI cannot be transferred during a single meeting, as new questions will continuously arise during development.”
BackgroundArtificial intelligence adoption in nursing students' academic writing: A qualitative study
Jembu, J. P., & Balang, R. V. (2025) · Teaching and learning in nursing · doi:10.1016/j.teln.2025.04.015 · Cited by 7
Annunziata, G., Lambiase, S., Tamburri, D. A., van den Heuvel, W., Palomba, F., Catolino, G., Ferrucci, F., & Lucia, A. D. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3712198 · Cited by 10
“Finally, Section 9 discusses the implications of the results, the potential threats to the study, and the measures taken to mitigate them.”
The Secret to Better AI and Better Software (Is Requirements Engineering)
Bencomo, N., Guo, J., Harrison, R., Heyn, H.-M., & Menzies, T. (2021) · IEEE Software · doi:10.1109/ms.2021.3118099 · Cited by 7
“A common vocabulary and understanding are needed for these communities, to be able to exploit their strengths [24].”
BackgroundHow Do Data Science Workers Communicate Intermediate Results?
Pang, R. Y., Wang, R., Nelson, J., & Battle, L. (2022) · IEEE Visualization in Data Science (VDS) · doi:10.1109/vds57266.2022.00010 · Cited by 6
“Previous work has also presented in-depth cases study of practices under the contexts of civic data hackathons [19] and collaboration between domain experts and data scientists [26, 34].”
BackgroundThe Product Beyond the Model - An Empirical Study of Repositories of Open-Source ML Products
Nahar, N., Zhang, H., Lewis, G. A., Zhou, S., & Kästner, C. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00006 · Cited by 24
“[1], [66] • RQ #2 (Architecture): How are open-source ML products architected to incorporate models?”
Chung, J. G., Hong, S., Choi, J., & Oh, C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713532 · Cited by 11
“Similar issues were reported in the previously mentioned European broadcasting study on AI adoption [41], and additionally, another study from developers’ perspective identifed organizational communication challenges due to knowledge mismatches [116].”
Towards a Responsible AI Organizational Maturity Model
Heger, A., Passi, S., Dhanorkar, S., Kahn, Z., Wang, R., & Vorvoreanu, M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757514 · Cited by 17
Reinmund, T., Kunze, L., & Jirotka, M. D. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642247 · Cited by 9
Prospective Sensemaking in the Front End of Innovation of AI Projects
Särner, E., Yström, A., Lakemond, N., & Holmberg, G. (2024) · Research-Technology Management · doi:10.1080/08956308.2024.2350407 · Cited by 16
“Previous research has suggested that actors need to overcome such differences in understandings, intensified by AI’s inherent characteristics, by bridging gaps in knowledge, trust, and expectations in their communication during the development (Piorkowski et al. 2021).”
BackgroundHeo, J., & Lee, U. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610189 · Cited by 7
“[56] analyzed how AI developers overcame communication gaps within a multidisciplinary team with respect to the lack of a shared mental model and provided information to stakeholders who collaborated using documentation tools such as Microsoft PowerPoint, despite the high cost involved.”
MethodKeelawat, P. (2023) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3584931.3607020 · Cited by 10
Ren, W., & Barrett, S. (2023) · Computer Applications in Engineering Education · doi:10.1002/cae.22675 · Cited by 7
“Communication is also a central tenet of all collaboration practices and processes, as it contributes to building trust among team members, establishing a shared understanding of task responsibilities [51, 66], and facilitating the sharing of knowledge and information among team members [26].”
BackgroundKim, D. H., Jeong, D., Yadgarova, S., Shin, H., Son, J., Subramonyam, H., & Kim, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714044 · Cited by 8
Governance of Collaborative AI Development Strategies
Wiesmüller, S., & Bauer, M. (2023) · CSR, sustainability, ethics & governance · doi:10.1007/978-3-031-09245-9_4 · Cited by 5
Multidisciplinary Hackathons: Towards Developing Practical Software Engineering Skills
Moshirpour, M., Shoura, T. A., Duffett‐Leger, L., & Moshirpour, M. (2023) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie58773.2023.10343214 · Cited by 4
“Hackathons are intensive project-based events that require participants to obtain results in a short time frame [13].”
BackgroundPreparing Engineers for the Future: Project Management for Developing Generative AI
Aggrawal, S., & Dittman, K. C. (2024) · doi:10.18260/1-2--45127 · Cited by 4
Developing and evaluating a design method for positive artificial intelligence
van der Maden, W., Lomas, D., & Hekkert, P. (2024) · Artificial intelligence for engineering design analysis and manufacturing · doi:10.1017/s0890060424000155 · Cited by 11
“Taken together, these factors demonstrate the need for human-centered design approaches like the proposed Positive AI method to bridge the gap between aspirations and design practices when developing AI to actively support human flourishing.”
AI-Based Chatbots as Civil Servants: A Utopia or a Dystopia?
TAMER, H. Y. (2025) · Advances in public policy and administration (APPA) book series · doi:10.4018/979-8-3693-6547-2.ch006 · Cited by 6
Singh, P., Budhiraja, R., Jalote, P., Kumar, M. J., & Singh, P. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711093 · Cited by 6
A Data-Centric Approach to Evaluate Requirements Engineering in Multidisciplinary Projects
Salmani, A., Imani, A., Bahrehvar, M., Duffett‐Leger, L., & Moshirpour, M. (2022) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53654.2022.9945270 · Cited by 4
“Ineffective communication is the first and most important challenge observed in the management of such teams [3], [4].”
BackgroundDesign Indirections: How Designers Find Their Ways in Shaping Algorithmic Systems
Poiroux, J., Maudet, N., Pineau, K., Brulé, É., & Tabard, A. (2023) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-022-09459-y · Cited by 3
“Piorkowski et al. (2021) similarly observed that an AI team had to interact with many other professions (domain and business experts, other software developers) to bring a product to life.”
A Systemic Approach to Machine Learning Project Management
Uysal, M. P., & Akturk, E. (2024) · IEEE Engineering Management Review · doi:10.1109/emr.2024.3503677 · Cited by 10
“In addition, team members often come from diverse technical backgrounds,which can result in skill gaps that affect the project’s progress and overall success [12].”
Towards Actionable Data Science: Domain Experts as End-Users of Data Science Systems
Jung, J. Y., Steinberger, T., & So, C. (2023) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-023-09475-6 · Cited by 6
“…include facilitating collaboration between domain experts and data scientists by building common ground or shared mental models (Mao et al., 2019; Piorkowski et al., 2021), structuring the stages of collaboration in a data science workflow (Zhang et al., 2020), or capturing or incorporating…”
Metadata
- Authors
- Piorkowski, D., Park, S., Wang, A. Y., Wang, D., Muller, M., & Portnoy, F.
- Affiliations
- IBM (United States) (US)
- Massachusetts Institute of Technology (US)
- University of Michigan (US)
- IBM Research - Cambridge (United States) (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2021-04-13
- Volume
- 5
- Issue
- CSCW1
- Pages
- 1-25
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W3121784008
- Cited by
- 151
- References
- 46
- Citations by year
- 2026: 11 · 2025: 27 · 2024: 44 · 2023: 38 · 2022: 20 · 2021: 11
- FWCI
- 25.55
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Interdisciplinary Research and Collaboration
- Social Sciences › Psychology › Social Psychology › Team Dynamics and Performance
- Keywords
- AI developers, multidisciplinary team, communication gap, shared mental models, stakeholder communication, trust, expectation management, interdisciplinary collaboration, organizational boundaries
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.