Fixing the program my computer learned: barriers for end users, challenges for the machine
The DOI matches its Crossref record; 29 of the 30 references it cites were found in a registry or an index.
Abstract
The results of a machine learning from user behavior can be thought of as a program, and like all programs, it may need to be debugged. Providing ways for the user to debug it matters, because without the ability to fix errors users may find that the learned program's errors are too damaging for them to be able to trust such programs. We present a new approach to enable end users to debug a learned program. We then use an early prototype of our new approach to conduct a formative study to determine where and when debugging issues arise, both in general and also separately for males and females. The results suggest opportunities to make machine-learned programs more effective tools.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It can also promote user learning and insight from complex data, as humans increasingly work with complex inferential systems for analytic purposes [44,45,73].”
From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [45]
In the citing paper
“There are many other ways to illustrate how an algorithm operates including verbal explanations, concrete user exploration, and so forth [44,45,51,73,88].”
From Progressive Disclosure: When, Why, and How Do Users Want Algorithmic Transparency Information? (2020) · cites this paper as [45]
In the citing paper
“There are many other ways to depict how an algorithm operates including verbal explanations, concrete user exploration, and so forth [42,44,50,71].”
From Progressive disclosure (2019) · cites this paper as [44]
Citations
Cites
Artificial intelligence: a modern approach
(1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 21,974
Self-efficacy: Toward a unifying theory of behavioral change
Bandura, A. (1978) · Advances in Behaviour Research and Therapy · doi:10.1016/0146-6402(78)90002-4 · Cited by 10,586
Information Systems Research: Relevant Theory and Informed Practice
(2004) · IFIP International Federation for Information Processing · doi:10.1007/b115738 · Cited by 3,122
Application of Social Cognitive Theory to Training for Computer Skills
Compeau, D. R., & Higgins, C. A. (1995) · Information Systems Research · doi:10.1287/isre.6.2.118 · Cited by 1,501 · cited 2×
“1 for males (via a self-efficacy question set [7]).”
BackgroundInternational Conference on Software Engineering
(2017) · IEEE Software · doi:10.1109/ms.2017.3571566 · Cited by 622
Information visualization in data mining and knowledge discovery
Fayyad, U. M. (2002) · Cited by 565
Six Learning Barriers in End-User Programming Systems
Ko, A. J., Myers, B. A., & Aung, H. H. (2004) · 2004 IEEE Symposium on Visual Languages - Human Centric Computing · doi:10.1109/vlhcc.2004.47 · Cited by 549 · cited 2×
“To select from these, we then drew from earlier work on learning barriers for novice programmers [12]; we chose the questions that could help a user overcome one or more of these barriers.”
BackgroundSelf-citationYour wish is my command: programming by example
(2001) · Choice Reviews Online · doi:10.5860/choice.39-0357 · Cited by 491
Designing the whyline: a debugging interface for asking questions about program behavior
Ko, A. J., & Myers, B. A. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985712 · Cited by 361 · cited 3×
“The Whyline [10] is a debugging tool aimed at event-driven programs and has recently been extended to help users debug the document and application state in word processors [18].”
MethodInfluentialSelf-citationINDUCTIVE AND BAYESIAN LEARNING IN MEDICAL DIAGNOSIS
KONONENKO, I. (1993) · Applied Artificial Intelligence · doi:10.1080/08839519308949993 · Cited by 302 · cited 1×
“Much of the work in explaining probabilistic machine learning algorithms has focused on the naïve Bayes classifier [2, 13] and, more generally, on linear additive classifiers [21] because explanation is more straightforward.”
Background
Research works printed but not linked: 4
- 10.5555/383784.383809 · doi:10.5555/383784.383809
- 10.5555/1622810.1622818 · doi:10.5555/1622810.1622818
- Davies , S. P. Display-based problem solving strategies in computer programming , Proc. Wkshp. Empirical Studies of Programmers , Ablex , ( 1996 ), 59--76. Davies, S. P. Display-based problem solving strategies in computer programming, Proc. Wkshp. Empirical Studies of Programmers, Ablex, (1996), 59--76.
- Meyers --Levy, J. Gender differences in information processing: A selectivity interpretation. P. Cafferata & ; A. Tybout, (Eds) Cognitive and Affective Responses to Advertising, Lexington Books ( 1989 ). Meyers--Levy, J. Gender differences in information processing: A selectivity interpretation. P. Cafferata & A. Tybout, (Eds) Cognitive and Affective Responses to Advertising, Lexington Books (1989).
Cited by
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
“Another relevant stream of work in the intelligent user interfaces community explored how end-users make sense of and control machine-learned programs, working towards intelligible and debuggable machine-learned programs [100, 101].”
BackgroundInfluentialLearning from a learning thermostat: lessons for intelligent systems for the home
Yang, R., & Newman, M. W. (2013) · Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing · doi:10.1145/2493432.2493489 · Cited by 268
“, [11,24]) or on eliciting more and higher quality input from the user (e.”
Assessing demand for intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2009) · Proceedings of the 11th international conference on Ubiquitous computing · doi:10.1145/1620545.1620576 · Cited by 246
“This approach has been employed in several domains including decision making [11], recommender systems [17], end-user debugging [18, 19], and systems that model user profiles [7], with the goal of increasing user trust and acceptance of these systems.”
MethodTell me more?: the effects of mental model soundness on personalizing an intelligent agent
Kulesza, T., Stumpf, S., Burnett, M., & Kwan, I. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207678 · Cited by 225
“There has been recent interest in supporting the debugging of intelligent agents’ reasoning [1,11,13,14,16,25], but the mental models users build while attempting this task have received little attention.”
BackgroundSelf-citationEvaluating saliency map explanations for convolutional neural networks: a user study
Alqaraawi, A., Schuessler, M., Weiß, P., Costanza, E., & Berthouze, N. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377519 · Cited by 163
“[34], participants improved the performance of a Naive Bayes e-mail classier.”
Background
Metadata
- Authors
- Kulesza, T., Wong, W.-K., Stumpf, S., Perona, S., White, R., Burnett, M. M., Oberst, I., & Ko, A. J.
- Affiliations
- Oregon State University (US)
- University of Washington (US)
- Venue
- Proceedings of the 14th international conference on Intelligent user interfaces
- Publisher
- ACM
- Published
- 2009-02-08
- Pages
- 187-196
- OpenAlex
- W2134874052
- Cited by
- 83
- References
- 30
- Citations by year
- 2025: 2 · 2024: 6 · 2023: 3 · 2022: 4 · 2021: 6 · 2020: 6 · 2019: 3 · 2018: 4 · 2017: 1 · 2016: 2 · 2015: 6 · 2014: 3 · 2013: 5 · 2012: 9
- FWCI
- 6.73
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- City Research Online (City University London) (acceptedVersion)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- program repair, formative research, gender differences
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Fixing the program my computer learned: barriers for end users, challenges for the machine. https://app.cochat.ai/papers/10.1145/1502650.1502678
- The paper
- Kulesza, T., Wong, W.-K., Stumpf, S., Perona, S., White, R., Burnett, M. M., Oberst, I., & Ko, A. J. (2009). Fixing the program my computer learned: barriers for end users, challenges for the machine. Proceedings of the 14th international conference on Intelligent user interfaces. https://doi.org/10.1145/1502650.1502678
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.