Onboarding Materials as Cross-functional Boundary Objects for Developing AI Assistants
The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Deep neural networks (DNNs) routinely achieve state-of-the-art performance in a wide range of tasks, but it can often be challenging for them to meet end-user needs in practice. This case study reports on the development of human-AI onboarding materials (i.e., training materials for users prior to using an AI) for a DNN-based medical AI Assistant to aid in the grading of prostate cancer. Specifically, we describe how the process of developing these materials changed the team’s understanding of end-user requirements, contributing to modifications in the development and assessment of the underlying machine learning model. Importantly, we discovered that onboarding materials served as a useful boundary object for cross-functional teams, uncovering a new way to assess the ML model and specify its end-user requirements. We also present evidence of the utility of the onboarding materials by describing how it affected user strategies and decision-making with AI in a study deployment to pathologists.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“Thus, model transparency toolkits (e.g. Model Cards [8]) may also benefit from transforming detailed model metrics into explicit, human-actionable strategies .”
BackgroundZhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
Beede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., & Vardoulakis, L. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376718 · Cited by 614 · cited 1×
“Although deep neural networks (DNNs) routinely achieve state-of-the-art performance in a wide range of tasks, it can often be challenging for these AI-powered systems to meet end-user needs in practice [3, 14].”
BackgroundSelf-citationCai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594 · cited 4×
“In our early work [4], we conducted a series of interviews with pathologists to determine what information they thought should be included in training materials to effectively use the AI assistant during the diagnostic process.”
BackgroundMethodInfluentialSelf-citationNagpal, K., Foote, D., Liu, Y., Chen, P.-H. C., Wulczyn, E., Tan, F., Olson, N., Smith, J. L., Mohtashamian, A., Wren, J. H., et al. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0112-2 · Cited by 495
Lee, C. P. (2007) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-007-9044-5 · Cited by 413
Yang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247
How do visual explanations foster end users' appropriate trust in machine learning?
Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377480 · Cited by 200 · cited 1×
“This information may also help engender appropriate levels of trust in the AI [13, 15] by illuminating in which situations they should pay special attention to the AI’s predictions.”
Background
Cited by
You Complete Me: Human-AI Teams and Complementary Expertise
Zhang, Q., Lee, M. L., & Carter, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517791 · Cited by 85
Prabhudesai, S., Yang, L., Asthana, S., Huan, X., Liao, Q. V., & Banovic, N. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584033 · Cited by 82
“Such mechanisms [21] to match visualization researchers’ expectations and decision-makers’ behavioural strategies, and to facilitate a streamlined meaning-making process during first-time use [39].”
BackgroundYildirim, N., Richardson, H., Wetscherek, M. T., Bajwa, J., Jacob, J., Pinnock, M. A., Harris, S., Coelho De Castro, D., Bannur, S., Hyland, S., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642013 · Cited by 82
How Experienced Designers of Enterprise Applications Engage AI as a Design Material
Yildirim, N., Kass, A., Tung, T., Upton, C., Costello, D., Giusti, R., Lacin, S., Lovic, S., O'Neill, J. M., Meehan, R. O., et al. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517491 · Cited by 71
Kim, S. S. Y., Vaughan, J. W., Liao, Q. V., Lombrozo, T., & Russakovsky, O. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714020 · Cited by 58
“…accuracy [43, 128, 129] and (un)certainty [6, 14, 17, 41, 132], explanations of outputs [6, 14, 17, 40, 41, 69, 132], and onboarding materials [18, 68] impact user reliance, as well as the roles played by human intuition [20], task complexity [101, 102], and other human, AI, and…”
Metadata
- Authors
- Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M.
- Venue
- Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-08
- Pages
- 1-7
- OpenAlex
- W3160122248
- Cited by
- 48
- References
- 8
- FWCI
- 3.83
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Onboarding Materials as Cross-functional Boundary Objects for Developing AI Assistants. https://app.cochat.ai/papers/10.1145/3411763.3443435
- The paper
- Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2021). Onboarding Materials as Cross-functional Boundary Objects for Developing AI Assistants. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3411763.3443435
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.