How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
Open accessThe DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Abstract
The development of AI applications is a multidisciplinary effort, involving multiple roles collaborating with the AI developers, an umbrella term we use to include data scientists and other AI-adjacent roles on the same team. During these collaborations, there is a knowledge mismatch between AI developers, who are skilled in data science, and external stakeholders who are typically not. This difference leads to communication gaps, and the onus falls on AI developers to explain data science concepts to their collaborators. In this paper, we report on a study including analyses of both interviews with AI developers and artifacts they produced for communication. Using the analytic lens of shared mental models, we report on the types of communication gaps that AI developers face, how AI developers communicate across disciplinary and organizational boundaries, and how they simultaneously manage issues regarding trust and expectations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“AI documentation is advocated to increase the transparency and facilitate understanding of AI [35,36,40,75,76,82], and is embodied in recently proposed features of AI Factsheets [8,81] and Model Cards [64].”
From Investigating Explainability of Generative AI for Code through Scenario-based Design (2022) · cites this paper as [76]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“In addition, AI -based translation tools can help bridge the communication gap between employees with different language backgrounds and help them communicate effectively (Piorkowski et al., 2021).”
From The Impact of Artificial Intelligence on Workers' Skills: Upskilling and Reskilling in Organisations (2023) · cites this paper as Piorkowski et al., 2021
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Clark, H. H., & Brennan, S. E. (1991) · Perspectives on socially shared cognition. · doi:10.1037/10096-006 · Cited by 4,555
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“Existing efforts around documenting aspects of the AI lifecycle in dataset documentation [15] and model service documentation [4, 34] are fairly static and do not consider questions of how to dynamically vary the content that is displayed for the role in question.”
BackgroundMcDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206
Software Engineering for Machine Learning: A Case Study
Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., & Zimmermann, T. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00042 · Cited by 1,135 · cited 6×
“Echoing findings from previous work, one of the common practices is to hold AI education sessions for software engineers to overcome the knowledge barrier [2].”
BackgroundInfluentialMetaphor No More: A 15-Year Review of the Team Mental Model Construct
Mohammed, S., Ferzandi, L., & Hamilton, K. (2010) · Journal of Management · doi:10.1177/0149206309356804 · Cited by 832 · cited 1×
“This allows them to more readily anticipate each others’ needs and work together more efficiently [35].”
BackgroundAustin, J. R. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.5.866 · Cited by 656 · cited 1×
“[69] summarized four stages for developing an SMM within agile software development teams from prior literature: knowing meta-knowledge of the team [26, 32], learning and building the team’s transactive memory system [6], understanding and reaching consensus [53], and executing team goals.”
MethodA New Chatbot for Customer Service on Social Media
Xu, A., Liu, Z., Guo, Y., Sinha, V., & Akkiraju, R. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025496 · Cited by 550 · cited 1×
“AI has been widely applied to many domains such as helping diagnose patients [14, 60], building chatbots to improve customer service [29, 66], and enhancing the education experience with intelligent tutoring systems [67].”
BackgroundSurvey on user interface programming
Myers, B. A., & Rosson, M. B. (1992) · Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '92 · doi:10.1145/142750.142789 · Cited by 445 · cited 1×
“Software development teams tend to spend extensive time on planning and brainstorming [39].”
BackgroundFactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434 · cited 1×
“Existing efforts around documenting aspects of the AI lifecycle in dataset documentation [15] and model service documentation [4, 34] are fairly static and do not consider questions of how to dynamically vary the content that is displayed for the role in question.”
BackgroundSelf-citationTeam learning: building shared mental models
Van den Bossche, P., Gijselaers, W., Segers, M., Woltjer, G., & Kirschner, P. (2011) · Instructional Science · doi:10.1007/s11251-010-9128-3 · Cited by 379 · cited 2×
“In this approach, an SMM provides one means to establish common ground [53].”
BackgroundMethodHuman-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI
Wang, D., Weisz, J. D., Muller, M., Ram, P., Geyer, W., Dugan, C., Tausczik, Y., Samulowitz, H., & Gray, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359313 · Cited by 342 · cited 2×
“Recent work in AutoML and machine teaching makes promising progress [9, 58, 61, 64].”
BackgroundMethodSelf-citationExploration and Explanation in Computational Notebooks
Rule, A., Tabard, A., & Hollan, J. D. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173606 · Cited by 335 · cited 3×
“Prior work on data scientists’ collaborative practices have provided high-level accounts on how data scientists work, and suggested that data science teams face intra-role communication gaps in code reading, code reuse, and code documentation activities [21, 46, 70].”
BackgroundDiscussion of shared and unshared information in decision-making groups.
Larson, J. R., Foster-Fishman, P. G., & Keys, C. B. (1994) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.67.3.446 · Cited by 319 · cited 1×
“[69] summarized four stages for developing an SMM within agile software development teams from prior literature: knowing meta-knowledge of the team [26, 32], learning and building the team’s transactive memory system [6], understanding and reaching consensus [53], and executing team goals.”
MethodHow do Data Science Workers Collaborate? Roles, Workflows, and Tools
Zhang, A. X., Muller, M., & Wang, D. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392826 · Cited by 268 · cited 9×
“Furthermore, we acknowledge that communication in an AI project is bi-directional, where both AI developers and stakeholders may drive and participate different parts of communication in an AI project [70].”
BackgroundInfluentialSelf-citationHow Data Science Workers Work with Data: Discovery, Capture, Curation, Design, Creation
Muller, M., Lange, I., Wang, D., Piorkowski, D., Tsay, J., Liao, Q. V., Dugan, C., & Erickson, T. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300356 · Cited by 261 · cited 1×
“While some studies in HCI and CSCW investigated the workflow from a data-centric perspective [24, 37, 40], others discussed the process of integrating models and building AI products in detail [2, 5, 55, 63], Amershi et al.”
Self-citationYang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247 · cited 1×
“, interactive machine learning tools [68]), hoping to adopt machine learning into their own practices [8].”
BackgroundThe emerging role of data scientists on software development teams
Kim, M., Zimmermann, T., DeLine, R., & Begel, A. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884783 · Cited by 231 · cited 1×
“Studies have investigated collaboration models among different team settings, such as professional data scientists [56, 57], civic data scientists [18], domain experts [31], and software-oriented data analysts [22].”
BackgroundPassi, S., & Jackson, S. J. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274405 · Cited by 225
Storey, M.-A., Zagalsky, A., Filho, F. F., Singer, L., & German, D. M. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2584053 · Cited by 221 · cited 1×
“found that engineer users on GitHub have a wide variety of “socially enabled communication channels” and social media to exchange information along with their coding project, but they still faced communication challenges such as miscommunication, language barriers, fear of intimidation and poor attitudes [50].”
BackgroundThe Story in the Notebook: Exploratory Data Science using a Literate Programming Tool
Kery, M. B., Radensky, M., Arya, M., John, B. E., & Myers, B. A. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173748 · Cited by 214 · cited 1×
“Prior work on data scientists’ collaborative practices have provided high-level accounts on how data scientists work, and suggested that data science teams face intra-role communication gaps in code reading, code reuse, and code documentation activities [21, 46, 70].”
BackgroundLwakatare, L. E., Raj, A., Bosch, J., Olsson, H. H., & Crnkovic, I. (2019) · Lecture Notes in Business Information Processing · doi:10.1007/978-3-030-19034-7_14 · Cited by 214 · cited 1×
“Prior research investigated the technical challenges of software engineering for AI systems, such as difficulties in problem formulation and specifying desired outcome, lack of critical analysis of training data, and lack of evaluation of models with business-centric measures [30, 62].”
BackgroundSoftware Engineering Challenges of Deep Learning
Arpteg, A., Brinne, B., Crnkovic-Friis, L., & Bosch, J. (2018) · 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa.2018.00018 · Cited by 201 · cited 2×
“While some studies in HCI and CSCW investigated the workflow from a data-centric perspective [24, 37, 40], others discussed the process of integrating models and building AI products in detail [2, 5, 55, 63], Amershi et al.”
BackgroundThe Politics of Measurement and Action
Pine, K. H., & Liboiron, M. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702298 · Cited by 194 · cited 2×
“Thus, the AI development team must embrace collaboration between different roles and leverage expertise from each other—despite the relatively low usage of documentation for this purpose [42, 46, 70].”
BackgroundUtilization of Self-Diagnosis Health Chatbots in Real-World Settings: Case Study
Fan, X., Chao, D., Zhang, Z., Wang, D., Li, X., & Tian, F. (2020) · Journal of Medical Internet Research · doi:10.2196/19928 · Cited by 192
Self-citationHow does Machine Learning Change Software Development Practices?
Wan, Z., Xia, X., Lo, D., & Murphy, G. C. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2937083 · Cited by 181
DataHub: Collaborative Data Science & Dataset Version Management at ScalePreprint
Bhardwaj, A., Bhattacherjee, S., Chavan, A., Deshpande, A., Elmore, A. J., Madden, S. R., & Parameswaran, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.0798 · Cited by 138 · cited 1×
“To remediate these communication challenges in an AI or software development team, prior works in HCI have proposed novel collaboration systems to aid the teamwork [7, 49, 52, 57].”
MethodMao, Y., Wang, D., Muller, M., Varshney, K. R., Baldini, I., Dugan, C., & Mojsilović, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3361118 · Cited by 124
Self-citationA Framework for Developing and Using Shared Mental Models in Human-Agent Teams
Scheutz, M. J., DeLoach, S. A., & Adams, J. A. (2017) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343416682891 · Cited by 123 · cited 4×
“A summary of the principles of how humans realize shared mental models [49].”
BackgroundMethodInfluentialLiao, Q. V., Mas-ud Hussain, M., Chandar, P., Davis, M., Khazaeni, Y., Crasso, M. P., Wang, D., Muller, M., Shami, N. S., & Geyer, W. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173577 · Cited by 116
Self-citationShared waypoints and social tagging to support collaboration in software development
Storey, M., Cheng, L.-T., Bull, I., & Rigby, P. C. (2006) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1180875.1180906 · Cited by 108 · cited 2×
“To remediate these communication challenges in an AI or software development team, prior works in HCI have proposed novel collaboration systems to aid the teamwork [7, 49, 52, 57].”
BackgroundMethodTrials and tribulations of developers of intelligent systems: A field study
Hill, C., Bellamy, R., Erickson, T., & Burnett, M. (2016) · 2016 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2016.7739680 · Cited by 108 · cited 3×
“Prior research has shown the kinds of problems faced in developing AI software [2, 16] and the tools used in this space [70], but to our knowledge, there is little known beyond survey results [70] about the contextualized specifics of how AI developers communicate with other roles or the motivations behind that commun…”
BackgroundStudying Software Engineering Patterns for Designing Machine Learning Systems
Washizaki, H., Uchida, H., Khomh, F., & Guéhéneuc, Y. (2019) · International Workshop on Empirical Software Engineering in Practice (IWESEP) · doi:10.1109/iwesep49350.2019.00017 · Cited by 96 · cited 1×
“Prior research investigated the technical challenges of software engineering for AI systems, such as difficulties in problem formulation and specifying desired outcome, lack of critical analysis of training data, and lack of evaluation of models with business-centric measures [30, 62].”
BackgroundMultidisciplinary team working: Collaboration and conflict
Jones, A. (2006) · International Journal of Mental Health Nursing · doi:10.1111/j.1447-0349.2006.00400.x · Cited by 92 · cited 2×
“Studies revealed that in multidisciplinary collaboration, professionals are unwilling to accept overlapping duties over roles [19] and rely on collaborative work tools to establish common ground [1].”
BackgroundFacilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models
Park, S., Wang, A. Y., Kawas, B., Liao, Q. V., Piorkowski, D., & Danilevsky, M. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450637 · Cited by 88
Self-citationAutoDS: Towards Human-Centered Automation of Data Science
Wang, D., Andres, J., Weisz, J. D., Oduor, E., & Dugan, C. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445526 · Cited by 75 · cited 1×
“Recent work in AutoML and machine teaching makes promising progress [9, 58, 61, 64].”
BackgroundSelf-citationHou, Y., & Wang, D. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134688 · Cited by 67
Self-citationMental model metrics and team adaptability: A multi-facet multi-method examination.
Resick, C. J., Murase, T., Bedwell, W. L., Sanz, E. J., Jiménez-Rodríguez, M., & DeChurch, L. A. (2010) · Group Dynamics Theory Research and Practice · doi:10.1037/a0018822 · Cited by 54
Callisto: Capturing the "Why" by Connecting Conversations with Computational Narratives
Wang, A. Y., Wu, Z., Brooks, C. A., & Oney, S. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376740 · Cited by 54
Self-citationDeveloping a Research Agenda for Human-Centered Data Science
Aragon, C., Hutto, C., Echenique, A., Fiore-Gartland, B., Huang, Y., Kim, J., Neff, G., Xing, W., & Bayer, J. (2016) · Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion · doi:10.1145/2818052.2855518 · Cited by 53
Supporting multidisciplinary collaboration: requirements from novel HCI education
Adamczyk, P. D., & Twidale, M. B. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240787 · Cited by 50 · cited 2×
“Studies revealed that in multidisciplinary collaboration, professionals are unwilling to accept overlapping duties over roles [19] and rely on collaborative work tools to establish common ground [1].”
BackgroundSoftware Developers Learning Machine Learning: Motivations, Hurdles, and Desires
Cai, C. J., & Guo, P. J. (2019) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2019.8818751 · Cited by 50 · cited 2×
“ML and AI involve several domain specific concepts such as model algorithms, evaluation metrics and advanced statistical measures that make it difficult for software engineers to learn, let alone business stakeholders [8].”
BackgroundThe many faces of data-centric workflow optimization: a survey
Kougka, G., Gounaris, A., & Simitsis, A. (2018) · International Journal of Data Science and Analytics · doi:10.1007/s41060-018-0107-0 · Cited by 45 · cited 1×
“While some studies in HCI and CSCW investigated the workflow from a data-centric perspective [24, 37, 40], others discussed the process of integrating models and building AI products in detail [2, 5, 55, 63], Amershi et al.”
Experiences with Improving the Transparency of AI Models and Services
Hind, M., Houde, S., Martino, J., Mojsilovic, A., Piorkowski, D., Richards, J., & Varshney, K. R. (2020) · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3334480.3383051 · Cited by 44 · cited 1×
“’s formative work on difficulties users have creating such documentation, which also advocated for user-centric output [17].”
BackgroundSelf-citationMapping Out Human-Centered Data Science: Methods, Approaches, and Best Practices
Kogan, M., Halfaker, A., Guha, S., Aragon, C., Muller, M., & Geiger, S. (2020) · Companion of the 2020 ACM International Conference on Supporting Group Work · doi:10.1145/3323994.3369898 · Cited by 40
Self-citationAn interactive human centered data science approach towards crime pattern analysis
Qazi, N., & Wong, B. L. W. (2019) · Information Processing & Management · doi:10.1016/j.ipm.2019.102066 · Cited by 31
Translation between software designers and users
Williams, M. G., & Begg, V. (1993) · Communications of the ACM · doi:10.1145/153571.214831 · Cited by 18 · cited 1×
“Thus, it may need a third party to be the bridge or the “broker role” [65] to do the translation for the communication.”
BackgroundPreliminary Systematic Literature Review of Machine Learning System Development Process
Watanabe, Y., Washizaki, H., Sakamoto, K., Saito, D., Honda, K., Tsuda, N., Fukazawa, Y., & Yoshioka, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05528 · Cited by 18 · cited 2×
“While some studies in HCI and CSCW investigated the workflow from a data-centric perspective [24, 37, 40], others discussed the process of integrating models and building AI products in detail [2, 5, 55, 63], Amershi et al.”
BackgroundInformation Curators in an Enterprise File-Sharing Service
Muller, M. J., Milien, D. R., & Feinberg, J. (2009) · Springer eBooks · doi:10.1007/978-1-84882-854-4_23 · Cited by 17 · cited 1×
“” The consolidation of information into a shared resources can be viewed as a way to reduce the cost of interruption by making the questions publicly available to others (e.g., [38]).”
BackgroundSelf-citationAL: autogenerating supervised learning programs
Cambronero, J., & Rinard, M. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3360601 · Cited by 15 · cited 1×
“Recent work in AutoML and machine teaching makes promising progress [9, 58, 61, 64].”
BackgroundPinhanez, C. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.08931 · Cited by 13 · cited 1×
“, additional mediators have to be involved to incorporate their domain knowledge [20, 43].”
BackgroundModelLens: An Interactive System to Support the Model Improvement Practices of Data Science Teams
Katsis, Y., & Wolf, C. T. (2019) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3311957.3359512 · Cited by 6 · cited 1×
“, additional mediators have to be involved to incorporate their domain knowledge [20, 43].”
Background
Research works printed but not linked: 19
- Arunima Chaudhary et almbox. 2021. AutoText: An End-to-End AutoAI Framework for Text. In AAAI. ACM.
- Herbert H Clark. 2006. Context and common ground. Concise Encyclopedia of Philosophy of Language and Linguistics (2006), 85--87.
- Sharolyn Converse, JA Cannon-Bowers, and E Salas. 1993. Shared mental models in expert team decision making. Individual and group decision making: Current issues, Vol. 221 (1993), 221--46.
- Paul Dourish. 2014. Reading and interpreting ethnography. In Ways of Knowing in HCI. Springer, 1--23.
- Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018. Datasheets for datasets. arXiv preprint arXiv:1803.09010 (2018). · arXiv
- Penny Lacey. 2002. 9 Multidisciplinary Work Challenges and Possibilities. Special Education Reformed: Inclusion-Beyond Rhetoric? (2002), 157.
- Charlotte P Lee and Drew Paine. 2015. From The Matrix to a Model of Coordinated Action (MoCA) A Conceptual Framework of and for CSCW. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing. 179--194.
- M Lee, Tristan Johnson, and Myung H Jin. 2012. Toward understanding the dynamic relationship between team and task shared mental models as determinants of team and individual performances. International journal of information technology and business management, Vol. 8, 1 (2012), 1--14.
- Sara A McComb. 2007. Mental model convergence: The shift from being an individual to being a team member. Research in multi-level issues, Vol. 6 (2007), 95--147.
- Michael Muller, Melanie Feinberg, Timothy George, Steven J Jackson, Bonnie E John, Mary Beth Kery, and Samir Passi. 2019 a. Human-Centered Study of Data Science Work Practices. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, W15.
- DJ Patil. 2011. Building data science teams." O'Reilly Media, Inc.".
- Gerry Stahl. 2013. Theories of collaborative cognition: Foundations for CSCL and CSCW together. In Computer-supported collaborative learning at the workplace. Springer, 43--63.
- Christoph Treude Margaret-anne Storey and Jens Weber. 2009. Empirical studies on collaboration in software development: A systematic literature review. (2009).
- Wil MP Van der Aalst. 2014. Data scientist: The engineer of the future. In Enterprise interoperability VI. Springer, 13--26.
- April Yi Wang, Anant Mittal, Christopher Brooks, and Steve Oney. 2019 a. How Data Scientists Use Computational Notebooks for Real-Time Collaboration. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--30.
- Dakuo Wang, Liuping Wang, Zhan Zhang, Ding Wang, Haiyi Zhu, Yvonne Gao, Xiangmin Fan, and Feng Tian. 2021 c. Brilliant AI Doctor in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment. In Proceedings of the CHI 2021.
- Bryan Wilder, Eric Horvitz, and Ece Kamar. 2020. Learning to Complement Humans. arXiv preprint arXiv:2005.00582 (2020). · arXiv
- Ying Xu, Dakuo Wang, Penelope Collins, Hyelim Lee, and Mark Warschauer. [n.d.]. Same benefits, different communication patterns: Comparing Children's reading with a conversational agent vs. a human partner. Computers & Education, Vol. 161 ([n.,d.]), 104059.
- Xiaodan Yu and Stacie Petter. 2014. Understanding agile software development practices using shared mental models theory. Information and software technology, Vol. 56, 8 (2014), 911--921.
Cited by
The Impact of Artificial Intelligence on Workers' Skills: Upskilling and Reskilling in Organisations
Morandini, S., Fraboni, F., De Angelis, M., Puzzo, G., Giusino, D., & Pietrantoni, L. (2023) · Informing Science: The International Journal of an Emerging Transdiscipline · doi:10.28945/5078 · Cited by 439
“In addition, AI - based translation tools can help bridge the communication gap between employees with different language backgrounds and help them communicate effectively (Piorkowski et al., 2021).”
BackgroundInvestigating Explainability of Generative AI for Code through Scenario-based Design
Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219
“AI documentation is advocated to increase the transparency and facilitate understanding of AI [35, 36, 40, 75, 76, 82], and is embodied in recently proposed features of AI Factsheets [8, 81] and Model Cards [64].”
BackgroundSelf-citationCembrowska-Lech, D., Krzemińska, A., Miller, T., Nowakowska, A., Adamski, C., Radaczyńska, M., Mikiciuk, G., & Mikiciuk, M. (2023) · Biology · doi:10.3390/biology12101298 · Cited by 175
Li, B., Lowell, V. L., Wang, C., & Li, X. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100266 · Cited by 120
Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook
Yildirim, N., Pushkarna, M., Goyal, N., WATTENBERG, M. P., & Viégas, F. B. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580900 · Cited by 159
Metadata
- Authors
- Piorkowski, D., Park, S., Wang, A. Y., Wang, D., Muller, M., & Portnoy, F.
- Affiliations
- IBM (United States) (US)
- Massachusetts Institute of Technology (US)
- University of Michigan (US)
- IBM Research - Cambridge (United States) (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2021-04-13
- Volume
- 5
- Issue
- CSCW1
- Pages
- 1-25
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W3121784008
- Cited by
- 151
- References
- 46
- Citations by year
- 2026: 11 · 2025: 27 · 2024: 44 · 2023: 38 · 2022: 20 · 2021: 11
- FWCI
- 25.55
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Interdisciplinary Research and Collaboration
- Social Sciences › Psychology › Social Psychology › Team Dynamics and Performance
- Keywords
- AI developers, multidisciplinary team, communication gap, shared mental models, stakeholder communication, trust, expectation management, interdisciplinary collaboration, organizational boundaries
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.