Overview based example selection in end user interactive concept learning
The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
Interaction with large unstructured datasets is difficult because existing approaches, such as keyword search, are not always suited to describing concepts corresponding to the distinctions people want to make within datasets. One possible solution is to allow end users to train machine learning systems to identify desired concepts, a strategy known as interactive concept learning. A fundamental challenge is to design systems that preserve end user flexibility and control while also guiding them to provide examples that allow the machine learning system to effectively learn the desired concept. This paper presents our design and evaluation of four new overview based approaches to guiding example selection. We situate our explorations within CueFlik, a system examining end user interactive concept learning in Web image search. Our evaluation shows our approaches not only guide end users to select better training examples than the best performing previous design for this application, but also reduce the impact of not knowing when to stop training the system. We discuss challenges for end user interactive concept learning systems and identify opportunities for future research on the effective design of such systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Principles of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490
Fails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633
Cited by
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
Self-citationHuman model evaluation in interactive supervised learning
Fiebrink, R., Cook, P. R., & Trueman, D. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1978965 · Cited by 207
Chung, J., Kamar, E., & Amershi, S. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.34 · Cited by 72
“…human interventions more interactive in model training pipelines, by closing gaps between model training and data curation (Fogarty et al., 2008; Amershi et al., 2009, 2012; Levonian et al., 2022), humans extracting features (Branson et al., 2010; Cheng and Bern-stein, 2015), interactively…”
BackgroundSelf-citationLessons Learned from Designing an AI-Enabled Diagnosis Tool for Pathologists
Gu, H., Huang, J., Hung, L., & Chen, X. '. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449084 · Cited by 56
“ves as input for the next iteration to re…ect the user’s intent [12]. Amershi et al. propose a system that gives the user …exibility to provide be−er training examples in interactive concept learning [1]. „e system also allows users to control the learning process and helps them decide when to stop training to avoid o…”
BackgroundWu, T., Weld, D. S., & Heer, J. (2019) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3319616 · Cited by 39
“Broadly speaking, most IML systems seek to either (1) enable non-programmers to build ML systems [4, 18, 19], T. Wu et al. (2) help end users understand their model failures [1, 3], and (3) improve their models [25, 30, 61].”
Background
Metadata
- Authors
- Amershi, S., Fogarty, J., Kapoor, A., & Tan, D.
- Venue
- Proceedings of the 22nd annual ACM symposium on User interface software and technology
- Publisher
- ACM
- Published
- 2009-10-04
- Pages
- 247-256
- OpenAlex
- W2148378803
- Cited by
- 41
- References
- 2
- FWCI
- 2.45
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Overview based example selection in end user interactive concept learning. https://app.cochat.ai/papers/10.1145/1622176.1622222
- The paper
- Amershi, S., Fogarty, J., Kapoor, A., & Tan, D. (2009). Overview based example selection in end user interactive concept learning. Proceedings of the 22nd annual ACM symposium on User interface software and technology. https://doi.org/10.1145/1622176.1622222
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.