Gestalt: integrated support for implementation and analysis in machine learning
Abstract
We present Gestalt, a development environment designed to support the process of applying machine learning. While traditional programming environments focus on source code, we explicitly support both code and data. Gestalt allows developers to implement a classification pipeline, analyze data as it moves through that pipeline, and easily transition between implementation and analysis. An experiment shows this significantly improves the ability of developers to find and fix bugs in machine learning systems. Our discussion of Gestalt and our experimental observations provide new insight into general-purpose support for the machine learning process.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Interfaces that facilitate seamless transitions between iML activities result in fewer errors and outputs that better match users' expectations.
“Interfaces that facilitate seamless transitions between these activities result in fewer errors and outputs that better match users' expectations [38, 46].”
From Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences (2024) · cites this paper as [38]
Cites
Data Mining: Practical Machine Learning Tools and Techniques
Witten, I. H., Frank, E., & Hall, M. A. (2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,493 · see citing works
Witten, I. H. (2008) · Cited by 9,115 · see citing works
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024 · see citing works
Readings in Information Visualization: Using Vision to Think
Card, S. K., Mackinlay, J. D., & Shneiderman, B. (1999) · Cited by 3,959 · see citing works
Consolvo, S., McDonald, D. W., Toscos, T., Chen, M. Y., Froehlich, J. E., Harrison, B. L., Klasnja, P., LaMarca, A., LeGrand, L., Libby, R., et al. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357335 · Cited by 1,133 · see citing works
Self-citationApplications of machine learning in cancer prediction and prognosis.
Cruz, J. A., & Wishart, D. S. (2007) · PubMed Central · Cited by 879 · see citing works
Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes
Wobbrock, J. O., Wilson, A. D., & Li, Y. (2007) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/1294211.1294238 · Cited by 823 · see citing works
Specifying gestures by example
Rubine, D. (1991) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/122718.122753 · Cited by 780 · see citing works
Fails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633 · see citing works
Data mining and machine learning in astronomy
BALL, N. M., & BRUNNER, R. J. (2010) · International Journal of Modern Physics D · doi:10.1142/s0218271810017160 · Cited by 383 · see citing works
Juergens, E., Deissenboeck, F., Hummel, B., & Wagner, S. (2009) · Cited by 354 · see citing works
Juergens, E., Deissenboeck, F., Hummel, B., & Wagner, S. (2009) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2009.5070547 · Cited by 318 · see citing works
Path-based faliure and evolution management
Chen, M. Y., Accardi, A., Kıcıman, E., Lloyd, J., Patterson, D., Fox, A., & Brewer, E. (2004) · Networked Systems Design and Implementation · Cited by 308 · see citing works
Wings: Intelligent Workflow-Based Design of Computational Experiments
Gil, Y., Ratnakar, V., Kim, J., Gonzalez-Calero, P. A., Groth, P., Moody, J., & Deelman, E. (2010) · IEEE Intelligent Systems · doi:10.1109/mis.2010.9 · Cited by 201 · see citing works
Fogarty, J. A., Tan, D. S., Kapoor, A., & Winder, S. A. J. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357061 · Cited by 178 · see citing works
Self-citationSimon, I., Morris, D., & Basu, S. (2008) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/1357054.1357169 · Cited by 154 · see citing works
Hartmann, B., Abdulla, L., Mittal, M., & Klemmer, S. (2007) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240646 · Cited by 154 · see citing works
Recursive restartability: turning the reboot sledgehammer into a scalpel
Candea, G., & Fox, A. (2005) · Workshop on Hot Topics in Operating Systems · doi:10.1109/hotos.2001.990072 · Cited by 139 · see citing works
Investigating statistical machine learning as a tool for software development
Patel, K., Fogarty, J. A., Landay, J. A., & Harrison, B. L. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357160 · Cited by 131 · see citing works
Self-citationMaynes-Aminzade, D., Winograd, T. A., & Igarashi, T. (2007) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/1294211.1294219 · Cited by 53 · see citing works
An 'object-use fingerprint': the use of electronic sensors for human identification
Hodges, M. R., & Pollack, M. E. (2007) · doi:10.5555/1771592.1771609 · Cited by 39 · see citing works
Blume, P. A. (2007) · Cited by 36 · see citing works
Stolte, C. (2010) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/1807167.1807287 · Cited by 9 · see citing works
Flagged first, then by citations
1 of the 24 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any ClassifierPreprint
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04938 · Cited by 587 · see citing works
Deep learning in the construction industry: A review of present status and future innovations
Akinosho, T. D., Oyedele, L. O., Bilal, M., Ajayi, A., Delgado, J. M. D., Akinadé, O. O., & Ahmed, A. A. (2020) · Journal of Building Engineering · doi:10.1016/j.jobe.2020.101827 · Cited by 544 · see citing works
Interacting with Predictions: Visual Inspection of Black-box Machine Learning Models
Krause, J., Perer, A., & Ng, K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858529 · Cited by 390 · see citing works
ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models
Kahng, M., Andrews, P., Kalro, A., & Chau, D. H. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2744718 · Cited by 349 · see citing works
Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., & Suh, J. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702509 · Cited by 263 · see citing works
Self-citationSquares: Supporting Interactive Performance Analysis for Multiclass Classifiers
Ren, D., Amershi, S., Lee, B., Suh, J., & Williams, J. D. (2016) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2598828 · Cited by 200 · see citing works
Do Convolutional Neural Networks Learn Class Hierarchy?
Alsallakh, B., Jourabloo, A., Ye, M., Liu, X., & Ren, L. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2744683 · Cited by 199 · see citing works
Scaling big data mining infrastructure
Lin, J., & Ryaboy, D. V. (2013) · ACM SIGKDD Explorations Newsletter · doi:10.1145/2481244.2481247 · Cited by 164 · see citing works
Grounding Interactive Machine Learning Tool Design in How Non-Experts Actually Build Models
Yang, Q., Suh, J., Chen, N.-C., & Ramos, G. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196729 · Cited by 156 · see citing works
Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences
Shankar, S., Zamfirescu-Pereira, J. D., Hartmann, B., Parameswaran, A., & Arawjo, I. (2024) · Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3654777.3676450 · Cited by 151 · see citing works
Human-XAI Interaction: A Review and Design Principles for Explanation User Interfaces
Chromik, M., & Butz, A. M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85616-8_36 · Cited by 128 · see citing works
Interactive machine teaching: a human-centered approach to building machine-learned models
Ramos, G., Meek, C., Simard, P., Suh, J., & Ghorashi, S. (2020) · Human-Computer Interaction · doi:10.1080/07370024.2020.1734931 · Cited by 120 · see citing works
Cheng, J., & Bernstein, M. S. (2015) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2675133.2675214 · Cited by 112 · see citing works
Fast, E., Chen, B., Mendelsohn, J., Bassen, J., & Bernstein, M. S. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174047 · Cited by 110 · see citing works
Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows
Xin, D., Wu, E. Y., Lee, D. J.-L., Salehi, N., & Parameswaran, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445306 · Cited by 110 · see citing works
Visual Diagnosis of Tree Boosting Methods
Liu, S., Xiao, J., Liu, J., Wang, X., Wu, J., & Zhu, J. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2744378 · Cited by 97 · see citing works
Machine Learning Education for Artists, Musicians, and Other Creative Practitioners
Fiebrink, R. A. (2019) · ACM Transactions on Computing Education · doi:10.1145/3294008 · Cited by 95 · see citing works
Interactive and interpretable machine learning models for human machine collaboration
Kim, B. (2015) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 83 · see citing works
Visual exploration of machine learning results using data cube analysis
Kahng, M., Fang, D., & Chau, D. H. (2016) · Workshop on Human-In-the-Loop Data Analytics (HILDA) · doi:10.1145/2939502.2939503 · Cited by 78 · see citing works
The Gesture Recognition Toolkit
Gillian, N. E., & Paradiso, J. A. (2017) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-57021-1_17 · Cited by 77 · see citing works
mage: Fluid Moves Between Code and Graphical Work in Computational Notebooks
Kery, M. B., Ren, D., Hohman, F., Moritz, D., Wongsuphasawat, K., & Patel, K. (2020) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3379337.3415842 · Cited by 71 · see citing works
Self-citationNeo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels
Görtler, J., Hohman, F., Moritz, D., Wongsuphasawat, K., Ren, D., Nair, R. S., Kirchner, M., & Patel, K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501823 · Cited by 69 · see citing works
Self-citationOn Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems
Nushi, B., Kamar, E. S., Horvitz, E., & Kossmann, D. (2017) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v31i1.10633 · Cited by 65 · see citing works
Heger, A., Marquis, L. B., Vorvoreanu, M., Wallach, H., & Vaughan, J. W. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555760 · Cited by 60 · see citing works
Understanding the Role of Alternatives in Data Analysis Practices
Liu, J., Boukhelifa, N., & Eagan, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934593 · Cited by 58 · see citing works
Leiva, L. A., Martín-Albo, D., & Plamondon, R. (2015) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2799648 · Cited by 56 · see citing works
Understanding the Role of Interactive Machine Learning in Movement Interaction Design
Gillies, M. F. P. (2019) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3287307 · Cited by 53 · see citing works
UMLAUT: Debugging Deep Learning Programs using Program Structure and Model Behavior
Schoop, E., Huang, F., & Hartmann, B. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445538 · Cited by 52 · see citing works
DeepCompare: Visual and Interactive Comparison of Deep Learning Model Performance
Murugesan, S., Malik, S., Du, F., Koh, E., & Lai, T. (2019) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2019.2919033 · Cited by 49 · see citing works
Xin, D., Macke, S., Ma, L., Liu, J., Song, S., & Parameswaran, A. (2018) · Proceedings of the VLDB Endowment · doi:10.14778/3297753.3297763 · Cited by 48 · see citing works
Symphony: Composing Interactive Interfaces for Machine Learning
Bäuerle, A., Cabrera, Á. A., Hohman, F., Maher, M., Koski, D., Suau, X., Barik, T., & Moritz, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502102 · Cited by 50 · see citing works
Researching AI Legibility through Design
Lindley, J., Akmal, H. A., Pilling, F. L., & Coulton, P. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376792 · Cited by 45 · see citing works
A User‐based Visual Analytics Workflow for Exploratory Model Analysis
Cashman, D., Humayoun, S. R., Heimerl, F., Park, K., Das, S., Thompson, J., Saket, B., Mosca, A., Stasko, J., Endert, A., et al. (2019) · Computer Graphics Forum · doi:10.1111/cgf.13681 · Cited by 41 · see citing works
Kato, J., McDirmid, S., & Cao, X. G. (2012) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2380116.2380142 · Cited by 38 · see citing works
Software Developers Learning Machine Learning: Motivations, Hurdles, and Desires
Cai, C. J., & Guo, P. J. (2019) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2019.8818751 · Cited by 37 · see citing works
Interactive Debugging and Steering of Multi-Agent AI Systems
Epperson, W., Bansal, G., Dibia, V. C., Fourney, A., Gerrits, J., Zhu, E. (., & Amershi, S. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713581 · Cited by 35 · see citing works
Evolving the Ecosystem of Personal Behavioral Data
Wiese, J., Das, S., Hong, J., & Zimmerman, J. (2017) · Human-Computer Interaction · doi:10.1080/07370024.2017.1295857 · Cited by 35 · see citing works
Motion-Sound Mapping through Interaction
Françoise, J., & Bevilacqua, F. P. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3211826 · Cited by 35 · see citing works
(2015) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 34 · see citing works
Lü, H., Fogarty, J. A., & Li, Y. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557263 · Cited by 33 · see citing works
Self-citationUsing Multiple Models to Understand Data
Patel, K., Drucker, S. M., Fogarty, J. A., Kapoor, A., & Tan, D. S. (2012) · Cited by 32 · see citing works
Self-citationSun, Y., Lank, E., & Terry, M. A. (2017) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3025171.3025208 · Cited by 31 · see citing works
SparseIMU: Computational Design of Sparse IMU Layouts for Sensing Fine-grained Finger Microgestures
Sharma, A., Salchow-Hömmen, C., Mollyn, V., Nittala, A. S., Hedderich, M. A., Koelle, M., Seel, T., & Steimle, J. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3569894 · Cited by 29 · see citing works
Using Expert Patterns in Assisted Interactive Machine Learning: A Study in Machine Teaching
Wall, E., Ghorashi, S., & Ramos, G. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29387-1_34 · Cited by 28 · see citing works
Model Sketching: Centering Concepts in Early-Stage Machine Learning Model Design
Lam, M. S., Ma, Z., Li, A., Freitas, I., Wang, D., Landay, J. A., & Bernstein, M. S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581290 · Cited by 25 · see citing works
Self-citationAmini, S., & Li, Y. (2013) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2501988.2502029 · Cited by 24 · see citing works
ActiVis: Visual Exploration of Industry-Scale Deep Neural Network ModelsPreprint
Kahng, M., Andrews, P., Kalro, A., & Chau, D. H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01942 · Cited by 21 · see citing works
Rahman, S., & Kandogan, E. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502068 · Cited by 20 · see citing works
Interactive Machine Learning for End-User Innovation
Bernardo, F., Zbyszyński, M., Fiebrink, R. A., & Grierson, M. (2016) · Goldsmiths (University of London) · Cited by 20 · see citing works
Making data analysis expertise broadly accessible through workflows
Hauder, M., Gil, Y., Sethi, R. J., Liu, Y., & Jo, H. (2011) · Workshop on Workflows in Support of Large-Scale Science (WORKS) · doi:10.1145/2110497.2110507 · Cited by 19 · see citing works
Making machine learning useable
Zhou, J. J., & Chen, F. (2015) · International Journal of Intelligent Systems Technologies and Applications · doi:10.1504/ijista.2015.074069 · Cited by 19 · see citing works
Skyline: Interactive In-Editor Computational Performance Profiling for Deep Neural Network Training
Yu, G. X., Grossman, T., & Pekhimenko, G. (2020) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3379337.3415890 · Cited by 19 · see citing works
A study on interpretability of decision of machine learning
Shirataki, S., & Yamaguchi, S. (2017) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2017.8258557 · Cited by 19 · see citing works
Lowering the barrier to applying machine learning
Patel, K. (2010) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/1866218.1866222 · Cited by 18 · see citing works
Self-citationCrowdsourcing – A Step Towards Advanced Machine Learning
Abhigna, B. S., Soni, N., & Dixit, S. (2018) · Procedia Computer Science · doi:10.1016/j.procs.2018.05.062 · Cited by 18 · see citing works
Iris: A Conversational Agent for Complex TasksPreprint
Fast, E., Chen, B., Mendelsohn, J., Bassen, J., & Bernstein, M. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05015 · Cited by 16 · see citing works
Making Machine-Learning Applications for Time-Series Sensor Data Graphical and Interactive
Kim, S., Tasse, D., & Dey, A. K. (2017) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/2983924 · Cited by 15 · see citing works
FAIRVIS: Visual Analytics for Discovering Intersectional Bias in Machine Learning
Cabrera, Á. A., Epperson, W., Hohman, F., Kahng, M., Morgenstern, J., & Chau, D. H. (2019) · 2019 IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast47406.2019.8986948 · Cited by 14 · see citing works
Programming with Examples to Develop Data-Intensive User Interfaces
Kato, J., Igarashi, T., & Goto, M. (2016) · Computer · doi:10.1109/mc.2016.217 · Cited by 13 · see citing works
Supporting Data Workers To Perform Exploratory Programming
Subramanian, K., Zubarev, I., Völker, S., & Borchers, J. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3313027 · Cited by 12 · see citing works
Kato, J., Sakamoto, D., & Igarashi, T. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466422 · Cited by 12 · see citing works
Mellis, D. A., Zhang, B., Leung, A. S., & Hartmann, B. (2017) · Designing Interactive Systems Conference · doi:10.1145/3064663.3064735 · Cited by 12 · see citing works
Agile Modeling: From Concept to Classifier in Minutes
Stretcu, O., Vendrow, E. B., Hata, K., Viswanathan, K., Ferrari, V., Tavakkol, S., Zhou, W., Avinash, A., Luo, E., Alldrin, N., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02040 · Cited by 11 · see citing works
User-Centred Design Actions for Lightweight Evaluation of an Interactive Machine Learning Toolkit
Bernardo, F., Grierson, M., & Fiebrink, R. A. (2018) · Journal of Science and Technology of the Arts · doi:10.7559/citarj.v10i2.509 · Cited by 11 · see citing works
Balayn, A., Rikalo, N., Yang, J., & Bozzon, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581555 · Cited by 10 · see citing works
Morrison, K., Jain, M., Hammer, J., & Perer, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610064 · Cited by 10 · see citing works
Evaluating the Effect of Feedback from Different Computer Vision Processing Stages
Kittley-Davies, J., Alqaraawi, A., Yang, R., Costanza, E., Rogers, A., & Stein, S. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300273 · Cited by 10 · see citing works
Fourney, A., & Terry, M. A. (2012) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2380116.2380188 · Cited by 8 · see citing works
Yang, H., Wickham, D. P., Chiticariu, L., Li, Y., Nguyen, B., & Carreno-Fuentes, A. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466212 · Cited by 7 · see citing works
VisionSketch: Integrated Support for Example-Centric Programming of Image Processing Applications
Kato, J., & Igarashi, T. (2020) · A K Peters/CRC Press eBooks · doi:10.1201/9781003059325-15 · Cited by 7 · see citing works
ODEN: Live Programming for Neural Network Architecture Editing
Zhao, C., Shen, I., Fukusato, T., Kato, J., & Igarashi, T. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511120 · Cited by 7 · see citing works
QUESTO: Interactive Construction of Objective Functions for Classification Tasks
Das, S., Xu, S., Gleicher, M. L., Chang, R., & Endert, A. (2020) · Computer Graphics Forum · doi:10.1111/cgf.13970 · Cited by 7 · see citing works
Helix: Holistic Optimization for Accelerating Iterative Machine LearningPreprint
Xin, D., Macke, S., Ma, L., Liu, J., Song, S., & Parameswaran, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.05762 · Cited by 7 · see citing works
EvalAssist: Insights on Task-Specific Evaluations and AI-Assisted Judgment Strategy Preferences
Ashktorab, Z., Desmond, M. J., Pan, Q., Johnson, J. M., Cooper, M. S., Daly, E., Nair, R., Pedapati, T., Do, H. J., & Geyer, W. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747740 · Cited by 6 · see citing works
OpenVL: A task-based abstraction for developer-friendly computer vision
Miller, G., & Fels, S. (2013) · Proceedings · doi:10.1109/wacv.2013.6475031 · Cited by 6 · see citing works
Lowering the barrier to applying machine learning
Patel, K. (2010) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1753846.1753882 · Cited by 6 · see citing works
Self-citationSCRAM: Simple Checks for Realtime Analysis of Model Training for Non-Expert ML Programmers
Schoop, E., Huang, F., & Hartmann, B. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3382879 · Cited by 6 · see citing works
Mallard: Turn the Web into a Contextualized Prototyping Environment for Machine Learning
Zhang, X., & Guo, P. J. (2019) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3332165.3347936 · Cited by 5 · see citing works
SparseEMG: Computational Design of Sparse EMG Layouts for Sensing Gestures
Kumar, A., Irudayaraj, A. A. R., Chandra, I., Sharma, A., & Nittala, A. S. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747614 · Cited by 4 · see citing works
Antar, A. D., Molaei, S., Chen, Y.-Y., Lee, M. L., & Banovic, N. (2024) · ACM Symposium on User Interface Software and Technology · doi:10.1145/3654777.3676323 · Cited by 8 · see citing works
Developer-centred interface design for computer vision
Miller, G., & Fels, S. (2011) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2011.6130274 · Cited by 3 · see citing works
TextAlive Online: Live Programming of Kinetic Typography Videos with Online Music
Kato, J., Nakano, T., & Goto, M. (2015) · doi:10.5281/zenodo.19355 · Cited by 3 · see citing works
Integrated visual representations for programming with real-world input and output
Kato, J. (2013) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2508468.2508476 · Cited by 3 · see citing works
Subramanian, K., Maas, J., Ellers, M., Wacharamanotham, C., Voelker, S., & Borchers, J. (2018) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3170427.3188528 · Cited by 3 · see citing works
ML Tools for the Web: A Way for Rapid Prototyping and HCI Research
Li, N., Mayes, J., & Yu, P. (2021) · Human-computer interaction series · doi:10.1007/978-3-030-82681-9_10 · Cited by 3 · see citing works
Novice-Centric Visualizations for Machine Learning
Sun, Y. (2016) · UWSpace (University of Waterloo) · Cited by 3 · see citing works
Garcia-Chimeno, Y., & García-Zapirain, B. (2015) · Bio-Medical Materials and Engineering · doi:10.3233/bme-151484 · Cited by 2 · see citing works
Developer toolchains for large-scale analytics: Two case studies
Rosenthal, S. L., McMillan, S., & Gaston, M. E. (2015) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2015.7363887 · Cited by 2 · see citing works
The Role of Expertise on Insight Generation from Visualization Sequences
Rosenthal, S. L., & Chung, T. R. (2022) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc53370.2022.9833097 · Cited by 2 · see citing works
DSWorkFlow: A Framework for Capturing Data Scientists’ Workflows
Mash, M., Rosenthal, S. L., & Simmons, R. G. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451683 · Cited by 2 · see citing works
On the Feasibility of Automatically Describing n-dimensional Objects
Duboue, P. A. (2013) · Cited by 1 · see citing works
Kim, Y., Lee, K., & Oh, U. (2020) · International journal of advanced smart convergence · doi:10.7236/ijasc.2020.9.2.90 · Cited by 1 · see citing works
Explainable Machine Learning and Visual Knowledge Discovery
Kovalerchuk, B. (2023) · Springer eBooks · doi:10.1007/978-3-031-24628-9_39 · Cited by 1 · see citing works
Slide to Explore 'What If': An Analysis of Explainable Interfaces
Meyer, L. S., & Zhu, J. (2024) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3677045.3685416 · Cited by 1 · see citing works
Weakly Supervised Learning by a Confusion Matrix of Contexts
Wu, W. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-26142-9_6 · Cited by 1 · see citing works
Do Machine Teachers Dream of Algorithms
Ramos, G., Ng, F., Sultanum, N., Meek, C. E., Suh, J., & Ghorashi, S. (2019) · Cited by 1 · see citing works
Accelerating Entrepreneurial Decision-Making Through Hybrid IntelligencePreprint
Dellermann, D. (2020) · arXiv (Cornell University) · doi:10.17170/kobra-202004301196 · Cited by 2 · see citing works
Towards Evaluating Exploratory Model Building Process with AutoML SystemsPreprint
Hong, S.-S., Castelo, S., D'Orazio, V. J., Benthune, C., Santos, A. S. R., Langevin, S., Jonker, D., Bertini, E., & Freire, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.00449 · Cited by 1 · see citing works
- Authors
- Patel, K., Bancroft, N., Drucker, S. M., Fogarty, J., Ko, A. J., & Landay, J.
- Affiliations
- University of Washington (US)
- Microsoft (United States) (US)
- Venue
- Proceedings of the 23nd annual ACM symposium on User interface software and technology
- Publisher
- ACM
- Published
- 2010-10-03
- Pages
- 37-46
- OpenAlex
- W2165254944
- Cited by
- 112
- References
- 24
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 3 · 2023: 5 · 2022: 7 · 2021: 7 · 2020: 12 · 2019: 14 · 2018: 11 · 2017: 12 · 2016: 8 · 2015: 11 · 2014: 1 · 2013: 9 · 2012: 3
- FWCI
- 3.59
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Symposium on User Interface Software and Technology (UIST) (publishedVersion)
- Funders
- National Science Foundation
- National Defense Science and Engineering Graduate
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- Gestalt, machine learning pipelines, classification pipeline, bug fixes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review