Designing Interactive Transfer Learning Tools for ML Non-Experts
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Interactive machine learning (iML) tools help to make ML accessible to users with limited ML expertise. However, gathering necessary training data and expertise for model-building remains challenging. Transfer learning, a process where learned representations from a model trained on potentially terabytes of data can be transferred to a new, related task, offers the possibility of providing ”building blocks” for non-expert users to quickly and effectively apply ML in their work. However, transfer learning largely remains an expert tool due to its high complexity. In this paper, we design a prototype to understand non-expert user behavior in an interactive environment that supports transfer learning. Our findings reveal a series of data- and perception-driven decision-making strategies non-expert users employ, to (in)effectively transfer elements using their domain expertise. Finally, we synthesize design implications which might inform future interactive transfer learning environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
A Comprehensive Survey on Transfer Learning
Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., & He, Q. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3004555 · Cited by 6,678
Can we open the black box of AI?
Castelvecchi, D. (2016) · Nature · doi:10.1038/538020a · Cited by 1,773 · cited 1×
“This reliance on intuition may also be attributed to the black-boxed nature of CNN models [11], which has received considerable attention in research community [23].”
BackgroundPower to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
Fails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
Hohman, F., Kahng, M., Pienta, R., & Chau, D. H. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2843369 · Cited by 582
UX Design Innovation: Challenges for Working with Machine Learning as a Design Material
Dove, G., Halskov, K., Forlizzi, J., & Zimmerman, J. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025739 · Cited by 391 · cited 3×
“instance, [14] identify that UX designers, even after working with ML professionals, have difculty in grasping the algorithmic part of ML.”
BackgroundParallel prototyping leads to better design results, more divergence, and increased self-efficacy
Dow, S. P., Glassco, A., Kass, J., Schwarz, M., Schwartz, D. L., & Klemmer, S. R. (2010) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1879831.1879836 · Cited by 367 · cited 1×
“Prior work in the design community has emphasized how early stages of a prototyping process can cause individuals to fxate on singular, potentially disadvantageous paths, risking poisoning the entire workfow [15].”
BackgroundTell me more?: the effects of mental model soundness on personalizing an intelligent agent
Kulesza, T., Stumpf, S., Burnett, M., & Kwan, I. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207678 · Cited by 225
Human factors in model interpretability: Industry practices, challenges, and needs
Hong, S. R., Hullman, J., & Bertini, E. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392878 · Cited by 219 · cited 1×
“One approach towards striking a balance between too few and too many afordances is to adopt a humancentered design methodology in designing iML tools [25, 56, 57].”
Human model evaluation in interactive supervised learning
Fiebrink, R., Cook, P. R., & Trueman, D. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1978965 · Cited by 207
Interacting meaningfully with machine learning systems: Three experiments
Stumpf, S., Rajaram, V., Li, L., Wong, W.-K., Burnett, M., Dietterich, T., Sullivan, E., & Herlocker, J. (2009) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.03.004 · Cited by 184
Evaluating saliency map explanations for convolutional neural networks: a user study
Alqaraawi, A., Schuessler, M., Weiß, P., Costanza, E., & Berthouze, N. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377519 · Cited by 163
Grounding Interactive Machine Learning Tool Design in How Non-Experts Actually Build Models
Yang, Q., Suh, J., Chen, N.-C., & Ramos, G. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196729 · Cited by 156
FairSight: Visual Analytics for Fairness in Decision Making
Ahn, Y., & Lin, Y.-R. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934262 · Cited by 126
An Evaluation of the Human-Interpretability of Explanation
Lage, I., Chen, E., He, J., Narayanan, M., Kim, B., Gershman, S., & Doshi-Velez, F. (2019) · arXiv · doi:10.48550/arxiv.1902.00006 · Cited by 125
Understanding and Visualizing Data Iteration in Machine Learning
Hohman, F., Wongsuphasawat, K., Kery, M. B., & Patel, K. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376177 · Cited by 110
Wu, T., Weld, D. S., & Heer, J. (2019) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3319616 · Cited by 39 · cited 1×
“Several user studies have been conducted to investigate how individuals and teams build ML models [57], select features [55], iterate over data [24], interpret results and explanations [3, 36], and integrate their domain expertise [8, 10].”
Background
Cited by
What is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
“The extent to which explanations and recommendations depend on the dataset is typically not discussed in detail and might beneft : [190, 259, 278] Iterative design process: [7, 87, Interview study: Wizard of Oz: [297] 119, 132, 196, 203, 273, 318, 326] [75, 78, 211, 312] Design workbook: [36]…”
BackgroundMethodFrom explainable to interactive AI: A literature review on current trends in human-AI interaction
Raees, M., Meijerink, I., Lykourentzou, I., Khan, V.-J., & Papangelis, K. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103301 · Cited by 180
Interactive Model Cards: A Human-Centered Approach to Model Documentation
Crisan, A., Drouhard, M., Vig, J., & Rajani, N. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533108 · Cited by 101
“Our considerations around trust and productive skepticism have been informed by prior work identifying domain expertise and predispositions for trust as risk factors in bringing models into practice [17, 36, 43, 51, 75].”
BackgroundZhang, S., Yu, J., Xu, X., Yin, C., Lu, Y., Yao, B., Tory, M., Padilla, L. M., Caterino, J., Zhang, P., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642343 · Cited by 99
“In healthcare, there are a number of decision-making tasks that have similar properties as sepsis diagnosis: highly uncertain, high-stakes, and time-sensitive [41].”
BackgroundRetroLens: A Human-AI Collaborative System for Multi-step Retrosynthetic Route Planning
Shi, C., Hu, Y., Wang, S., Ma, S., Zheng, C., Ma, X., & Luo, Q. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581469 · Cited by 19
Metadata
- Authors
- Mishra, S., & Rzeszotarski, J. M.
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-15
- OpenAlex
- W3160872964
- Cited by
- 32
- References
- 19
- FWCI
- 2.17
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.