The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index; 3 of the 14 sources its claims cite were checked for support.
Using Interactive Machine Learning to Support Interface Development Through Workshops with Disabled People
Abstract
We have applied interactive machine learning (IML) to the creation and customisation of gesturally controlled musical interfaces in six workshops with people with learning and physical disabilities. Our observations and discussions with participants demonstrate the utility of IML as a tool for participatory design of accessible interfaces. This work has also led to a better understanding of challenges in end-user training of learning models, of how people develop personalised interaction strategies with different types of pre-trained interfaces, and of how properties of control spaces and input devices influence people's customisation strategies and engagement with instruments. This work has also uncovered similarities between the musical goals and practices of disabled people and those of expert musicians.
Overview
CoChat Summary
TL;DR
This paper reports six workshops with people with learning and physical disabilities using interactive machine learning (IML) to create and customise gesturally controlled musical interfaces. It finds IML useful for participatory design of accessible interfaces, while identifying challenges in end-user training and benefits of flexible continuous or large discrete control spaces.
Key findings
- IML was useful as a design tool: the team used it between workshops to quickly translate observations into new prototypes, and participants could develop effective customisation strategies with many pre-trained instruments.
- Continuous-control instruments let people explore a “sound space,” then settle on personal gestural vocabularies; this allowed experimentation with a wider variety of gestures and gradual adaptation without interruptions for model training.
- Discrete gesture-recognition instruments became frustrating when using too many gesture classes, typically more than 5, because vocabularies were hard to remember and classifiers less accurate; larger discrete spaces (e.g., selecting among 10–12 percussive samples) could be engaging when used exploratorily.
- The paper reports similarities between disabled participants’ musical goals/practices and those of expert musicians, and no clear differences between disabled people from the community centre and members of the public in how they engaged with the instruments.
Methods
- Over two months, the authors conducted six music-making workshops with Heart n Soul: four two-hour sessions with two to four disabled participants, and two drop-in public sessions lasting around five hours each with approximately 20 attendees.
- They built 12 prototype digital musical instruments, each pairing an input sensor with musical output; sensors included Microsoft Kinect, Leap Motion, video face tracking, Gametrak, and squeezable tangibles, with Wekinator used for IML and algorithms including neural networks, AdaBoost, k-nearest neighbour, decision trees, and support vector machines.
- Data came from observations of engagement, return visits, control finesse, and musical output, plus discussions with participants when possible and session video logs; the team discussed observations between workshops to prepare new prototypes.
Limitations
- The paper does not include a dedicated limitations section; it states challenges of on-the-fly end-user customisation, including participants having trouble choosing non-ambiguous gestures, remembering custom gesture vocabularies, performing gestures consistently, and understanding how training built the instrument.
- Evident from the text: the workshop sample was small and not formally characterised—four sessions had only two to four disabled participants, the centre did not collect information about people’s specific disabilities, and the drop-in sessions involved approximately 20 attendees.
- Evident from the text: findings are based on observations, discussions, and video logs rather than controlled or quantitative evaluation, and the authors report no clear differences between disabled participants and public attendees.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 1 Not valid · 6 Not checked · 29 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“This prior work focused primarily on highly skilled computer musicians creating custom gestural interfaces for experimental music.”
What the sources say
- Disputedcites [2] 2. Fiebrink, R., P. R. Cook, and D. Trueman. 2011. Human model evaluation in interactive supervised learning. Proc. CHI , 147-56. sourceExperimental — accuracy not yet measured
Our copy characterises prior IML work quite differently: the Interactive Machine Learning section (p. 2) credits the approach to Crayons (image classification), CueTip (handwriting), CueFlik (web image classification), Baker et al. (document analysis), Exemplar (sensor-based interaction design), ManiMatrix and EnsembleMatrix — not to computer musicians, and the Motivation (p. 3) states that 'interactive machine learning of this type has not been widely studied in gesture analysis or in real-world applications.' Application Domain and Software (p. 3) is explicitly contrary about the music side: 'despite widespread application of supervised learning to gesture modeling in music, previous work does not incorporate human interaction in this way.' The musician-building-custom-instruments picture in our copy is the authors' own present studies (Study A composers, p. 4, using webcams, HID devices and custom-built sensor arrays), whose users are described as 'machine learning novices' and undergraduates (pp. 4, 7), not as prior work by highly skilled musicians. (LLM, conf=0.75)
Checked by a reasoning model — may be wrong.
- Not checked yetcites [3] 3. Fiebrink, R., D. Trueman, C. Britt et al. 2010. Toward understanding human-computer interactions in composing the instrument. Int'l Comuter Music Conf.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we observed that many pre-trained instruments still enabled people to develop effective customisation strategies.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“People would then play the instrument by switching back and forth among these poses or motions, almost as if the instrument had been trained using a discrete gesture classifier.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that such instruments were easily built by training a regression model and discretising its output, as opposed to providing arbitrarily selected training examples for every discrete 'class.'”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These strategies were strongly influenced by the types of control afforded by an instrument, especially whether control was discrete (using gesture recognition to choose among a set of musical events) or continuous”
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
18 more claims
Create a free account to see all 38 claims and to verify the 11 references still open.
- It may state two separate claims as one. 11
- Probably not a substantial claim. 7
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“this software has been used in a recent project aimed at building customized musical rehabilitation devices for children with severe motor impairments [145].”
From Hands-Free Accessible Digital Musical Instruments: Conceptual Framework, Challenges, and Perspectives (2020) · cites this paper as [145]
Experimental — accuracy not yet measured
Citations
Cites
Past, present, and future of user interface software tools
Myers, B., Hudson, S. E., & Pausch, R. (2000) · ACM Transactions on Computer-Human Interaction · doi:10.1145/344949.344959 · Cited by 808
Fails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633
Automatically generating personalized user interfaces with Supple
Gajos, K. Z., Weld, D. S., & Wobbrock, J. O. (2010) · Artificial Intelligence · doi:10.1016/j.artint.2010.05.005 · Cited by 317
Human model evaluation in interactive supervised learning
Fiebrink, R., Cook, P. R., & Trueman, D. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1978965 · Cited by 207
Self-citationTailor: creating custom user interfaces based on gesture
Pausch, R., & Williams, R. D. (1990) · Proceedings of the 3rd annual ACM SIGGRAPH symposium on User interface software and technology - UIST '90 · doi:10.1145/97924.97939 · Cited by 20
Xu, Z., Fiebrink, R., & Matsuoka, Y. (2012) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio.2012.6491206 · Cited by 3
Self-citation
Research works printed but not linked: 2
- Fiebrink R., 2010, Britt et al
- Xu Z., Proc. IEEE Robotics & Biometrics (ROBIO).
Cited by
A Review of User Interface Design for Interactive Machine Learning
Dudley, J. J., & Kristensson, P. O. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3185517 · Cited by 343
Teachable Machine: Approachable Web-Based Tool for Exploring Machine Learning Classification
Carney, M., Webster, B., Alvarado, I., Phillips, K., Howell, N., Griffith, J., Jongejan, J., Pitaru, A., & Chen, A. (2020) · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3334480.3382839 · Cited by 313
The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice
Delgado, F., Yang, S., Madaio, M., & Yang, Q. (2023) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3617694.3623261 · Cited by 256
Accessible Digital Musical Instruments—A Review of Musical Interfaces in Inclusive Music Practice
Frid, E. (2019) · Multimodal Technologies and Interaction · doi:10.3390/mti3030057 · Cited by 177
Considerations for AI fairness for people with disabilities
Trewin, S., Basson, S., Muller, M., Branham, S., Treviranus, J., Gruen, D., Hebert, D., Lyckowski, N., & Manser, E. (2019) · AI Matters · doi:10.1145/3362077.3362086 · Cited by 143
Metadata
- Authors
- Katan, S., Grierson, M., & Fiebrink, R.
- Affiliations
- Goldsmiths University of London (GB)
- Venue
- Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2015-04-18
- Pages
- 251-254
- OpenAlex
- W1984342914
- Cited by
- 50
- References
- 6
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 2 · 2023: 7 · 2022: 2 · 2021: 5 · 2020: 8 · 2019: 9 · 2018: 3 · 2017: 3 · 2016: 9
- FWCI
- 2.05
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Goldsmiths Research Online (acceptedVersion)
- Goldsmiths (University of London) (acceptedVersion)
- University of the Arts London Research Online (University of the Arts London) (acceptedVersion)
- Funders
- UK NESTA/AHRC/ACE Digital R&D (N/A)
- Topics
- Social Sciences › Arts and Humanities › Music › Music Technology and Sound Studies
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- interactive machine learning, accessible interfaces, participatory design, learning disabilities, physical disability, input devices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Using Interactive Machine Learning to Support Interface Development Through Workshops with Disabled People. https://app.cochat.ai/papers/10.1145/2702123.2702474
- The paper
- Katan, S., Grierson, M., & Fiebrink, R. (2015). Using Interactive Machine Learning to Support Interface Development Through Workshops with Disabled People. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. https://doi.org/10.1145/2702123.2702474
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.