Human-Centred Machine Learning
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Machine learning is one of the most important and successful techniques in contemporary computer science. It involves the statistical inference of models (such as classifiers) from data. It is often conceived in a very impersonal way, with algorithms working autonomously on passively collected data. However, this viewpoint hides considerable human work of tuning the algorithms, gathering the data, and even deciding what should be modeled in the first place. Examining machine learning from a human-centered perspective includes explicitly recognising this human work, as well as reframing machine learning workflows based on situated human working practices, and exploring the co-adaptation of humans and systems. A human-centered understanding of machine learning in human context can lead not only to more usable machine learning tools, but to new ways of framing learning computationally. This workshop will bring together researchers to discuss these issues and suggest future research questions aimed at creating a human-centered approach to machine learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084 · cited 1×
“It will end with a general discussion period (45 minutes) (The last panel will be shorter to ensure time for discussion.)”
BackgroundSelf-citationFails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633
Parallel prototyping leads to better design results, more divergence, and increased self-efficacy
Dow, S. P., Glassco, A., Kass, J., Schwarz, M., Schwartz, D. L., & Klemmer, S. R. (2010) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1879831.1879836 · Cited by 367
Cited by
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design
Yang, Q., Steinfeld, A., Rosé, C., & Zimmerman, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376301 · Cited by 622
“Interestingly, the research shows something else, that HCI designers largely struggle to envision and prototype AI systems [2, 3, 13, 18, 22, 26, 49].”
BackgroundSharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5285 · Cited by 466
“While confidence estimation and reporting is informative for decision-making, research in human-centered machine learning (Gillies et al. 2016) and HCI shows that people have difficulties with correctly interpreting probabilistic statements (Handmer and Proudley 2007) and even system accuracy…”
BackgroundUX Design Innovation: Challenges for Working with Machine Learning as a Design Material
Dove, G., Halskov, K., Forlizzi, J., & Zimmerman, J. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025739 · Cited by 391
A Review of User Interface Design for Interactive Machine Learning
Dudley, J. J., & Kristensson, P. O. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3185517 · Cited by 343
Teachable Machine: Approachable Web-Based Tool for Exploring Machine Learning Classification
Carney, M., Webster, B., Alvarado, I., Phillips, K., Howell, N., Griffith, J., Jongejan, J., Pitaru, A., & Chen, A. (2020) · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3334480.3382839 · Cited by 313
Google Earth Engine and Artificial Intelligence (AI): A Comprehensive Review
Yang, L., Driscol, J., Sarigai, S., Wu, Q., Chen, H., & Lippitt, C. D. (2022) · Remote Sensing · doi:10.3390/rs14143253 · Cited by 264
Human-Centred Learning Analytics
Buckingham Shum, S., Ferguson, R., & Martinez-Maldonado, R. (2019) · Journal of Learning Analytics · doi:10.18608/jla.2019.62.1 · Cited by 258
What is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
“The initial CHI workshop on ‘Human-Centered Machine Learning’ intended to gather the community around the topic in 2016, identifying key research questions in the application of human-centered approaches of machine learning [110].”
BackgroundGrounding Interactive Machine Learning Tool Design in How Non-Experts Actually Build Models
Yang, Q., Suh, J., Chen, N.-C., & Ramos, G. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196729 · Cited by 156
"Explorers of Unknown Planets": Practices and Politics of Artificial Intelligence in Visual Arts
Caramiaux, B., & Fdili Alaoui, S. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555578 · Cited by 114
“This approach is what HCI researchers called Interactive Machine Learning (IML) [2, 23, 32].”
BackgroundSelf-citationSensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory
Kaur, H., Adar, E., Gilbert, E., & Lampe, C. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533135 · Cited by 92
A multifaceted vision of the human-AI collaboration: a comprehensive review
Puerta-Beldarrain, M., Gómez-Carmona, O., Sánchez-Corcuera, R., Casado-Mansilla, D., López-de-Ipiña, D., & Chen, L. (2025) · IEEE Access · doi:10.1109/access.2025.3536095 · Cited by 73
“Taking as guidelines the previous statements, IML solutions should also be a driver to promote this human-AI collaboration in different phases of the model design, creating long-lasting interactions to make ML algorithms more useful, accessible and usable for every particular individual [65].”
Zytek, A., Liu, D., Vaithianathan, R., & Veeramachaneni, K. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3114864 · Cited by 62
“Past literature has advocated for a humancentered perspective to ML [4] — one that considers machines and algorithms as part of collaborative systems alongside humans.”
BackgroundFeature-Based Explanations Don't Help People Detect Misclassifications of Online Toxicity
Carton, S., Mei, Q., & Resnick, P. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7282 · Cited by 58
“(Abdul et al. 2018) and (Gillies et al. 2016) suggest that robust human experimentation can bridge this gap, while (Dove et al. 2017) calls generally for work that explores “the interplay between ML statistical intelligence and human common sense intelligence”.”
BackgroundBehavioral Use Licensing for Responsible AI
Contractor, D., McDuff, D., Haines, J. K., Lee, J., Hines, C., Hecht, B., Vincent, N., & Li, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533143 · Cited by 54
“The context in which a model is applied can be far removed from that which the developers had intended, a major point of concern from the perspective of human-centered machine learning [31].”
BackgroundModel Positionality and Computational Reflexivity: Promoting Reflexivity in Data Science
Cambo, S. A., & Gergle, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501998 · Cited by 49
“Data Science and related fields like Artificial Intelligence, Machine Learning, and Statistics provide indispensable methods for extracting and understanding a variety of phenomena from large datasets.”
MethodThe design of reciprocal learning between human and artificial intelligence
Zagalsky, A., Te'eni, D., Yahav, I., Schwartz, D. G., Silverman, G., Cohen, D., Mann, Y., & Lewinsky, D. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479587 · Cited by 35
“Inspired by other researchers looking into how human-centered approaches affect machine learning [25, 59], we re-iterate the following two questions: (1) What role do humans play in HML configurations? and (2) Do HML configurations change the way in which machine learning is being done, if so, how? We believe that our…”
BackgroundInvestigations of Performance and Bias in Human-AI Teamwork in Hiring
Peng, A., Nushi, B., Kiciman, E., Inkpen, K., & Kamar, E. (2022) · arXiv · doi:10.48550/arxiv.2202.11812 · Cited by 34
Ayobi, A., Hughes, J., Duckworth, C. J., Dylag, J. J., Gordon James, S., Marshall, P., Guy, M., Kumaran, A., Chapman, A., Boniface, M., et al. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581424 · Cited by 26
“A promising research direction is to intertwine participatory design [44, 52, 70] with human-centred ML approaches [7, 38, 50] to support inclusive experienceswhere participants fromnon-data science backgrounds can feel and act as equal co-designers throughout the”
User-centred design for machine learning in health care: a case study from care management
Seneviratne, M. G., Li, R. C., Schreier, M., Lopez-Martinez, D., Patel, B. S., Yakubovich, A., Kemp, J. B., Loreaux, E., Gamble, P., El-Khoury, K., et al. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2022-100656 · Cited by 23
Towards a Human-Centered Approach for VRET Systems: Case Study for Acrophobia
Bălan, O., Cristea, Ș., Moldoveanu, A., Moise, G., Leordeanu, M., & Moldoveanu, F. (2020) · Lecture Notes in Information Systems and Organisation · doi:10.1007/978-3-030-49644-9_11 · Cited by 3
Taka, E., Nakao, Y., Sonoda, R., Yokota, T., Luo, L., & Stumpf, S. (2023) · arXiv · doi:10.48550/arxiv.2312.08064
Metadata
- Authors
- Gillies, M., Fiebrink, R., Tanaka, A., Garcia, J., Bevilacqua, F., Heloir, A., Nunnari, F., Mackay, W., Amershi, S., Lee, B., d'Alessandro, N., Tilmanne, J., Kulesza, T., & Caramiaux, B.
- Venue
- Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2016-05-07
- Pages
- 3558-3565
- OpenAlex
- W2345597971
- Cited by
- 134
- References
- 3
- FWCI
- 7.52
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.