Grounding Interactive Machine Learning Tool Design in How Non-Experts Actually Build Models
The DOI matches its Crossref record; 26 of the 28 references it cites were found in a registry or an index.
Abstract
Machine learning (ML) promises data-driven insights and solutions for people from all walks of life, but the skill of crafting these solutions is possessed by only a few. Emerging research addresses this issue by creating ML tools that are easy and accessible to people who are not formally trained in ML (non-experts). This work investigated how non-experts build ML solutions for themselves in real life. Our interviews and surveys revealed unique potentials of non-expert ML, as well several pitfalls that non-experts are susceptible to. For example, many perceived percentage accuracy as a sole measure of performance, thus problematic models proceeded to deployment. These observations suggested that, while challenging, making ML easy and robust should both be important goals of designing novice-facing ML tools. To advance on this insight, we discuss design implications and created a sensitizing concept to demonstrate how designers might guide non-experts to easily build robust solutions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Early HCI research tells us that program-authoring interactions that do not require programming are not necessarily accessible to non-programmers; drag-and-drop and interactive machine learning (iML) tools cannot necessarily enable non-ML experts to build models [50], and prompting can be viewed as a programming or iML task.”
From Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts (2023) · cites this paper as [50]
In the citing paper
“In this”
From Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts (2023) · cites this paper as [50]
In the citing paper
“Deep learning models for natural language processing (NLP) are increasingly adopted and deployed by analysts without formal training in NLP or machine learning (ML).”
From Interactive Model Cards: A Human-Centered Approach to Model Documentation (2022) · cites this paper as [75]
Citations
Cites
The Reflective Practitioner: How Professionals Think in Action
Checkoway, B., & Schon, D. A. (1985) · Journal of Policy Analysis and Management · doi:10.2307/3324262 · Cited by 21,268
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
The reflective practitioner: How professionals think in action
Bogumil, R. J. (1985) · Proceedings of the IEEE · doi:10.1109/proc.1985.13210 · Cited by 2,323
(2015) · neural information processing systems · Cited by 1,305
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
Beyer, H. R., & Holtzblatt, K. (1997) · Cited by 704
Fails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633
Principles of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576
An analysis and critique of Research through Design: towards a formalization of a research approach
Zimmerman, J., Stolterman, E., & Forlizzi, J. (2010) · Proceedings of the 8th ACM Conference on Designing Interactive Systems · doi:10.1145/1858171.1858228 · Cited by 491
Test-driven development concepts, taxonomy, and future direction
Janzen, D., & Saiedian, H. (2005) · Computer · doi:10.1109/mc.2005.314 · Cited by 445
Research works printed but not linked: 5
- Bonsai. 2017 . BONSAI WHITEPAPER: A fundamentally different approach for building intelligent industrial systems. Technical Report. https://bons.ai/resources. Bonsai. 2017. BONSAI WHITEPAPER: A fundamentally different approach for building intelligent industrial systems. Technical Report. https://bons.ai/resources.
- Justin Bozonier . 2015. Test-Driven Machine Learning . Packt Publishing Ltd . Justin Bozonier. 2015. Test-Driven Machine Learning. Packt Publishing Ltd.
- TensorFlow : Smarter machine learning, for everyone 2016 . (2016). https://www.google.com/intl/en/about/main/tensorflow/. TensorFlow: Smarter machine learning, for everyone 2016. (2016). https://www.google.com/intl/en/about/main/tensorflow/.
- Udemy Online Course : Applied machine learning for Everyone 2017 . (2017). https://www.udemy.com/ applied-machine-learning-for-everyone/. Udemy Online Course: Applied machine learning for Everyone 2017. (2017). https://www.udemy.com/ applied-machine-learning-for-everyone/.
- 10.5555/2888116.2888288 · doi:10.5555/2888116.2888288
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts
Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., & Yang, Q. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581388 · Cited by 1,274
“We want to dive deep into what those struggles reveal about people’s assumptions and understanding of prompt-based systems in order to inform how end-user-facing prompt design tools might best support their users.”
BackgroundInfluentialSelf-citationHuman-in-the-loop machine learning: a state of the art
Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J., & Fernández-Leal, Á. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-022-10246-w · Cited by 1,025
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777
“[110]) or case studies from other teams who have worked in specific application domains.”
BackgroundRe-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design
Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376301 · Cited by 622
“The first project focused on novice AI product designers [56].”
BackgroundSelf-citationEmpowering Educators to Be AI-Ready
Luckin, R., Cukurova, M., Kent, C., & du Boulay, B. (2022) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2022.100076 · Cited by 422
“The need for a less technical approach to educating people about AI has been highlighted, for example by researchers concerned with crowdsourcing and citizen science (see for example, Wang et al., 2019; Smith et al., 2020;Yang et al., 2018) and explainable AI (Liao et al., 2020).”
BackgroundHow Data Science Workers Work with Data: Discovery, Capture, Curation, Design, Creation
Muller, M., Lange, I., Wang, D., Piorkowski, D., Tsay, J., Liao, Q. V., Dugan, C., & Erickson, T. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300356 · Cited by 261
Nahar, N., Zhou, S., Lewis, G., & Kästner, C. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510209 · Cited by 131
What is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
“in terms of AI and ML education, and democratization [247, 316] to empower people in the design and use of AI systems.”
BackgroundGamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models
Hohman, F., Head, A., Caruana, R., DeLine, R., & Drucker, S. M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300809 · Cited by 225
Human factors in model interpretability: Industry practices, challenges, and needs
Hong, S. R., Hullman, J., & Bertini, E. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392878 · Cited by 219
“’s, survey on how laypeople use ML [74]; and Kandel et al.”
BackgroundInteractive machine teaching: a human-centered approach to building machine-learned models
Ramos, G., Meek, C., Simard, P., Suh, J., & Ghorashi, S. (2020) · Human–Computer Interaction · doi:10.1080/07370024.2020.1734931 · Cited by 119
Self-citationExplainability scenarios: towards scenario-based XAI design
Wolf, C. T. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302317 · Cited by 172
A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning
Kaluarachchi, T., Reis, A., & Nanayakkara, S. (2021) · Sensors · doi:10.3390/s21072514 · Cited by 124
A Survey on Interactive Reinforcement Learning: Design Principles and Open Challenges
Cruz, C. A., & Igarashi, T. (2020) · Designing Interactive Systems Conference · doi:10.1145/3357236.3395525 · Cited by 131
Machine Learning Uncertainty as a Design Material: A Post-Phenomenological Inquiry
Benjamin, J. J., Berger, A., Merrill, N., & Pierce, J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445481 · Cited by 130
Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows
Xin, D., Wu, E. Y., Lee, D. J.-L., Salehi, N., & Parameswaran, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445306 · Cited by 110
“r at a high-level and the system performs some form of automated search to generate an appropriate ML pipeline or model [19, 28]. This work is related to a larger body of studies on ML work practices [4, 22, 23, 25, 48] of specific groups of practitioners, including software engineers using ML [4], nonexpert ML users…”
BackgroundBrown, N., Garland, A., Fadel, G., & Li, G. (2022) · Materials & Design · doi:10.1016/j.matdes.2022.110672 · Cited by 110
PromptAid: Visual Prompt Exploration, Perturbation, Testing and Iteration for Large Language Models
Mishra, A., Danzy, B., Soni, U., Arunkumar, A., Huang, J., Kwon, B. C., & Bryan, C. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3535332 · Cited by 106
Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels
Görtler, J., Hohman, F., Moritz, D., Wongsuphasawat, K., Ren, D., Nair, R. S., Kirchner, M., & Patel, K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501823 · Cited by 104
Shen, H., Deng, W. H., Chattopadhyay, A., Wu, Z. S., Wang, X., & Zhu, H. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445971 · Cited by 71
Interactive Model Cards: A Human-Centered Approach to Model Documentation
Crisan, A., Drouhard, M., Vig, J., & Rajani, N. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533108 · Cited by 101
Human-in-the-loop machine learning: Reconceptualizing the role of the user in interactive approaches
Gómez-Carmona, O., Casado-Mansilla, D., López-de-Ipiña, D., & García-Zubia, J. (2024) · Internet of Things · doi:10.1016/j.iot.2023.101048 · Cited by 68
Why is Developing Machine Learning Applications Challenging? A Study on Stack Overflow Posts
Alshangiti, M., Sapkota, H., Murukannaiah, P. K., Liu, X., & Yu, Q. (2019) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2019.8870187 · Cited by 100
mage: Fluid Moves Between Code and Graphical Work in Computational Notebooks
Kery, M. B., Ren, D., Hohman, F., Moritz, D., Wongsuphasawat, K., & Patel, K. (2020) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3379337.3415842 · Cited by 96
A Human-in-the-loop Perspective on AutoML: Milestones and the Road Ahead.
Lee, D. J. L., Macke, S., Xin, D., Lee, A. H.-C., Huang, S., & Parameswaran, A. (2019) · IEEE Data(base) Engineering Bulletin · Cited by 85
Wizard of Oz Prototyping for Machine Learning Experiences
Browne, J. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3312877 · Cited by 80
Weidele, D. K. I., Weisz, J. D., Oduor, E., Muller, M., Andres, J., Gray, A., & Wang, D. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377538 · Cited by 57
A multifaceted vision of the human-AI collaboration: a comprehensive review
Puerta-Beldarrain, M., Gómez-Carmona, O., Sánchez-Corcuera, R., Casado-Mansilla, D., López-de-Ipiña, D., & Chen, L. (2025) · IEEE Access · doi:10.1109/access.2025.3536095 · Cited by 73
“Empirical findings from Yang et al. suggest that incorporating interactive processes can lead to greater confidence among non-experts in learning outcomes, particularly when explainability is prioritized [56].”
InfluentialZhang, A., Walker, O., Nguyen, K., Dai, J., Chen, A., & Lee, M. K. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579601 · Cited by 72
Exploring Machine Teaching with Children
Dwivedi, U., Gandhi, J., Parikh, R., Coenraad, M., Bonsignore, E., & Kacorri, H. (2021) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc51201.2021.9576171 · Cited by 73
Towards an AI‐driven business development framework: A multi‐case study
John, M. M., Olsson, H. H., & Bosch, J. (2022) · Journal of Software Evolution and Process · doi:10.1002/smr.2432 · Cited by 71
PromptInfuser: How Tightly Coupling AI and UI Design Impacts Designers’ Workflows
Petridis, S., Terry, M. A., & Cai, C. J. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661613 · Cited by 63
What Did My AI Learn? How Data Scientists Make Sense of Model Behavior
Cabrera, Á. A., Ribeiro, M. T., Lee, B., DeLine, R. A., Perer, A., & Drucker, S. M. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3542921 · Cited by 62
Feng, K. J. K., Coppock, M. J., & McDonald, D. W. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563657.3596101 · Cited by 58
Human-Centered Study of Data Science Work Practices
Muller, M., Feinberg, M., George, T., Jackson, S. J., John, B. E., Kery, M. B., & Passi, S. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3299018 · Cited by 47
Towards Design Principles for User-Centric Explainable AI in Fraud Detection
Cirqueira, D., Helfert, M., & Bezbradica, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77772-2_2 · Cited by 56
Prototyping Machine Learning Through Diffractive Art Practice
Scurto, H., Caramiaux, B., & Bevilacqua, F. (2021) · Designing Interactive Systems Conference 2021 · doi:10.1145/3461778.3462163 · Cited by 31
Developing ML/DL Models: A Design Framework
John, M. M., Olsson, H. H., & Bosch, J. (2020) · International Conference on Software and System Process · doi:10.1145/3379177.3388892 · Cited by 55
Planning for Natural Language Failures with the AI Playbook
Hong, M. K., Fourney, A., DeBellis, D., & Amershi, S. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445735 · Cited by 54
“While prior work has articulated several challenges of prototyping AI experiences [7, 30, 31], our research sheds new light to this discussion, suggesting that practitioners often skip early-stage interaction prototyping.”
BackgroundA Survey of Domain Knowledge Elicitation in Applied Machine Learning
Kerrigan, D., Hullman, J. R., & Bertini, E. (2021) · Multimodal Technologies and Interaction · doi:10.3390/mti5120073 · Cited by 47
Khuat, T. T., Kedziora, D. J., & Gabryś, B. (2023) · Foundations and Trends® in Human–Computer Interaction · doi:10.1561/1100000091 · Cited by 47
BEAMES: Interactive Multimodel Steering, Selection, and Inspection for Regression Tasks
Das, S., Cashman, D., Chang, R., & Endert, A. (2019) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2019.2922592 · Cited by 46
ML Lifecycle Canvas: Designing Machine Learning-Empowered UX with Material Lifecycle Thinking
Zhou, Z., Sun, L., Zhang, Y., Liu, X., & Gong, Q. S. (2020) · Human-Computer Interaction · doi:10.1080/07370024.2020.1736075 · Cited by 46
Designing Interactive Transfer Learning Tools for ML Non-Experts
Mishra, S., & Rzeszotarski, J. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445096 · Cited by 32
‘It Is Not Always Discovery Time’: Four Pragmatic Approaches in Designing AI Systems
Windl, M., Feger, S. S., Zijlstra, L., Schmidt, A., & Woźniak, P. W. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501943 · Cited by 41
Co-ML: Collaborative Machine Learning Model Building for Developing Dataset Design Practices
Tseng, T., Davidson, M. J., Morales‐Navarro, L., Chen, J. K., Delaney, V., Leibowitz, M., Beason, J., & Shapiro, R. B. (2024) · ACM Transactions on Computing Education · doi:10.1145/3641552 · Cited by 41
Crowdsourcing the Perception of Machine Teaching
Hong, J., Lee, K., Xu, J., & Kacorri, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376428 · Cited by 30
Using Expert Patterns in Assisted Interactive Machine Learning: A Study in Machine Teaching
Wall, E., Ghorashi, S., & Ramos, G. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29387-1_34 · Cited by 37
Self-citationXiao, Q., Fan, X., Simon, F. M., Zhang, B., & Eslami, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714090 · Cited by 35
Toward User-Driven Sound Recognizer Personalization with People Who Are d/Deaf or Hard of Hearing
Goodman, S. M., Liu, P., Jain, D., McDonnell, E. J., Froehlich, J. E., & Findlater, L. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463501 · Cited by 33
Feng, K. J. K., & McDonald, D. W. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584064 · Cited by 31
Rahman, S., & Kandogan, E. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502068 · Cited by 29
Brand, N., Odom, W., & Barnett, S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581336 · Cited by 27
A Survey on Deep Learning for Design and Generation of Virtual Architecture
Wang, A., Dong, J., Lee, L., Shen, J., & Hui, P. (2024) · ACM Computing Surveys · doi:10.1145/3688569 · Cited by 27
How do People Train a Machine? Strategies and (Mis)Understandings
Sanchez, T., Caramiaux, B., Françoise, J., Bevilacqua, F. P., & Mackay, W. E. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449236 · Cited by 27
Position: A Call to Action for a Human-Centered AutoML ParadigmPreprint
Lindauer, M., Karl, F., Klier, A., Moosbauer, J., Tornede, A., Mueller, A., Hutter, F., Feurer, M., & Bischl, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.03348 · Cited by 27
Kellogg, K. C., Lifshitz‐Assaf, H., Randazzo, S., Mollick, E., Dell’Acqua, F., McFowland, E., Candelon, F., & Lakhani, K. R. (2025) · Information and Organization · doi:10.1016/j.infoandorg.2025.100559 · Cited by 27
Mix & Match Machine Learning: An Ideation Toolkit to Design Machine Learning-Enabled Solutions
Jansen, A., & Colombo, S. (2023) · International Conference on Tangible, Embedded, and Embodied Interaction (TEI) · doi:10.1145/3569009.3572739 · Cited by 26
Integrating Iterative Machine Teaching and Active Learning into the Machine Learning Loop
Mosqueira-Rey, E., Alonso-Ríos, D., & Baamonde-Lozano, A. (2021) · Procedia Computer Science · doi:10.1016/j.procs.2021.08.057 · Cited by 26
Impact of Machine Learning on the Productivity of Employees in Workplace
Wardhani, R. S., Kant, K., Sreeram, A., Gupta, M., Erwandy, E., & Bora, P. K. (2022) · International Conference on Inventive Research in Computing Applications (ICIRCA) · doi:10.1109/icirca54612.2022.9985471 · Cited by 25
Wang, L., Yurechko, K., Dani, P., Chen, Q. Z., & Zhang, A. X. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713691 · Cited by 25
TADA : Task Agnostic Dialect Adapters for English
Held, W., Ziems, C., & Yang, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.51 · Cited by 25
Elwahsh, S., Stern, N., Singh, A., & Ayobi, A. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3736615 · Cited by 24
Exploring AI Problem Formulation with Children via Teachable Machines
Dwivedi, U., Elsayed-Ali, S., Bonsignore, E., & Kacorri, H. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642692 · Cited by 23
A Survey on Interactive Reinforcement Learning: Design Principles and Open ChallengesPreprint
Cruz, C. A., & Igarashi, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.12949 · Cited by 23
Benjamin, J. J., Kinkeldey, C., Müller-Birn, C., Korjakow, T., & Herbst, E.-M. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3492858 · Cited by 19
Hamidi, A., Antoniol, G., Khomh, F., Penta, M. D., & Hamidi, M. (2021) · IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM) · doi:10.1109/scam52516.2021.00016 · Cited by 22
Opportunities and Risks of LLMs for Scalable Deliberation with Polis
Small, C. T., Vendrov, I., Durmus, E., Homaei, H., Barry, E., Cornebise, J., Suzman, T., Ganguli, D., & Megill, C. (2023) · arXiv · doi:10.48550/arxiv.2306.11932 · Cited by 20
“Prior work has found this type of participatory approach, where the users and stakeholders of an AI system are involved in evaluating and improving its outputs, can help identify errors, biases, and misrepresentations (Lee et al., 2022; Yang et al., 2018).”
BackgroundCollaborative Machine Learning Model Building with Families Using Co-ML
Tseng, T., Chen, J. K., Abdelrahman, M., Kery, M. B., Hohman, F., Hilliard, A., & Shapiro, R. B. (2023) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3585088.3589356 · Cited by 20
Balayn, A., Rikalo, N., Yang, J., & Bozzon, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581555 · Cited by 18
Almahmoud, J., DeLine, R. A., & Drucker, S. M. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3463934 · Cited by 18
Mental models of AI-based systems: User predictions and explanations of image classification results
Bos, N., Glasgow, K., Gersh, J. R., Harbison, I., & Paul, C. L. (2019) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181319631392 · Cited by 18
Use of Machine Learning by Non-Expert DHH People: Technological Understanding and Sound Perception
Nakao, Y., & Sugano, Y. (2020) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3419249.3420157 · Cited by 17
Kim, H., & Lim, Y.-K. (2021) · Designing Interactive Systems Conference · doi:10.1145/3461778.3462141 · Cited by 17
Human-AI Co-Learning for Data-Driven AIPreprint
Huang, Y., Cheng, Y.-T., Chen, L., & Hsu, J. Y.-J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12544 · Cited by 18
Revealing Neural Network Bias to Non-Experts Through Interactive Counterfactual ExamplesPreprint
Myers, C. M., Freed, E., Pardo, L. L., Furqan, A., Risi, S., & Zhu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.02271 · Cited by 17
Towards a Non-Ideal Methodological Framework for Responsible ML
Mothilal, R. K., Guha, S., & Ahmed, S. I. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642501 · Cited by 16
Sun, T., Gao, Y., Khaladkar, S., Liu, S., Zhao, L., Kim, Y., & Hong, S.-S. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610187 · Cited by 16
AIMEE: An Exploratory Study of How Rules Support AI Developers to Explain and Edit Models
Piorkowski, D., Vejsbjerg, I., Cornec, O., Daly, E., & Alkan, Ö. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610046 · Cited by 16
Visualizing Examples of Deep Neural Networks at Scale
Yan, L., Glassman, E. L., & Zhang, T. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445654 · Cited by 16
Teaching Information Literacy and Critical Thinking Skills in Chat GPT Time
Bušelić, V. (2023) · International Conference on Computing, Networking, Telecommunications & Engineering Sciences Applications (CoNTESA) · doi:10.1109/contesa61248.2023.10384921 · Cited by 15
In Situ AI Prototyping: Infusing Multimodal Prompts into Mobile Settings with MobileMaker
Petridis, S., Liu, M. X., Fiannaca, A. J., Tsai, V., Terry, M. A., & Cai, C. J. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00023 · Cited by 14
Wikipedia ORES Explorer: Visualizing Trade-offs For Designing Applications With Machine Learning API
Ye, Z., Yuan, X., Gaur, S., Halfaker, A., Forlizzi, J., & Zhu, H. (2021) · Designing Interactive Systems Conference · doi:10.1145/3461778.3462099 · Cited by 15
Evaluating Non-AI Experts' Interaction with AI: A Case Study In Library Context
Zheng, Q., Chen, M., Park, H., Zhong-wei, X., & Huang, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714219 · Cited by 13
Winter, M., & Jackson, P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-49760-6_12 · Cited by 13
Eardley, R., Mackinnon, S., Tonkin, E. L., Soubutts, E., Ayobi, A., Linington, J., Tourte, G. J. L., Gross, Z. B., Bailey, D. J., Knights, R., et al. (2022) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3534625 · Cited by 13
From Discovery to Adoption: Understanding the ML Practitioners’ Interpretability Journey
Ashtari, N., Mullins, R. S., Qian, C., Wexler, J., Tenney, I., & Pushkarna, M. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563657.3596046 · Cited by 13
Enabling Collaborative Data Science Development with the Ballet Framework
Smith, M. J., Cito, J., Lu, K., & Veeramachaneni, K. K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479575 · Cited by 13
Demystifying a Dark Art: Understanding Real-World Machine Learning Model DevelopmentPreprint
Lee, A. H.-C., Xin, D., Lee, D., & Parameswaran, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.01520 · Cited by 13
Optimizing AI-Assisted Code GenerationPreprint
Torka, S., & Albayrak, Ş. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.10953 · Cited by 14
UX Challenges in Implementing an Interactive B2B Customer Segmentation Tool
Raees, M., Khan, V.-J., & Papangelis, K. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3706687 · Cited by 12
Goodman, S. M., McDonnell, E. J., Froehlich, J. E., & Findlater, L. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713294 · Cited by 12
Bridging the Communication Gap: Evaluating AI Labeling Practices for Trustworthy AI Development
Fischer, R., Wischnewski, M., van der Staay, A., Poitz, K., Janiesch, C., & Liebig, T. (2025) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v8i1.36601 · Cited by 12
Coordination Mechanisms in AI Development: Practitioner Experiences on Integrating UX Activities
Bruun, A., van Berkel, N., Raptis, D., & Law, E. L. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713200 · Cited by 11
A Design Space for Live Music Agents
Kim, Y., Brade, S., Wang, A., Zhou, D., Kim, H., Wang, B., Lee, S.-J., García, H. F., Huang, A., & Donahue, C. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791291 · Cited by 11
Wang, C., & An, P. (2021) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3447527.3474873 · Cited by 11
Democratizing AI: non-expert design of prediction tasks
Bagrow, J. P. (2020) · PeerJ Computer Science · doi:10.7717/peerj-cs.296 · Cited by 11
ODEN: Live Programming for Neural Network Architecture Editing
Zhao, C., Shen, I., Fukusato, T., Kato, J., & Igarashi, T. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511120 · Cited by 10
Sun, L., Li, Z., Zhou, Z., Lou, S., Li, W., & Zhang, Y. (2023) · Artificial intelligence for engineering design analysis and manufacturing · doi:10.1017/s0890060422000270 · Cited by 10
Metadata
- Authors
- Yang, Q., Suh, J., Chen, N.-C., & Ramos, G.
- Affiliations
- Carnegie Mellon University (US)
- Microsoft (United States) (US)
- University of Washington (US)
- Venue
- Proceedings of the 2018 Designing Interactive Systems Conference
- Publisher
- ACM
- Published
- 2018-06-08
- Pages
- 573-584
- OpenAlex
- W2807910285
- Semantic Scholar
- 8d4b89081ba250f2e35585af1c5eb8d764b11ab9
- Cited by
- 156
- References
- 28
- Citations by year
- 2026: 7 · 2025: 23 · 2024: 16 · 2023: 21 · 2022: 17 · 2021: 31 · 2020: 24 · 2019: 15 · 2018: 2
- FWCI
- 12.20
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Designing Interactive Systems Conference (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Software › Spreadsheets and End-User Computing
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- interactive machine learning, model robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.