Cited by 26Explainable Artificial Intelligence (XAI)Artificial Intelligence in HealthcareArtificial Intelligence in Healthcare and Education

Computational Notebooks as Co-Design Tools: Engaging Young Adults Living with Diabetes, Family Carers, and Clinicians with Machine Learning Models

Open access
Ayobi, Amid, Hughes, Jacob, Duckworth, Christopher J, Dylag, Jakub J, Gordon James, Sam, Marshall, Paul, Guy, Matthew, Kumaran, Anitha +3 more·2023·Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Abstract

Engaging end user groups with machine learning (ML) models can help align the design of predictive systems with people's needs and expectations. We present a co-design study investigating the benefits and challenges of using computational notebooks to inform ML models with end user groups. We used a computational notebook to engage young adults, carers, and clinicians with an example ML model that predicted health risk in diabetes care. Through co-design workshops and retrospective interviews, we found that participants particularly valued using the interactive data visualisations of the computational notebook to scaffold multidisciplinary learning, anticipate benefits and harms of the example ML model, and create fictional feature importance plots to highlight care needs. Participants also reported challenges, from running code cells to managing information asymmetries and power imbalances. We discuss the potential of leveraging computational notebooks as interactive co-design tools to meet end user needs early in ML model lifecycles.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3544548.3581424

Cites

94 of 94 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. [4]4. Amid Ayobi, Katarzyna Stawarz, Dmitri Katz, Paul Marshall, Taku Yamagata, Raúl Santos-Rodríguez, Peter Flach, and Aisling Ann O'Kane. 2021. Machine Learning Explanations as Boundary Objects: How AI Researchers Explain and Non-Experts Perceive Machine Learning. Joint Proceedings of the ACM IUI 2021 Workshops, April 13-17, 2021, College Station, USA. DOI:http://ceur-ws.org/Vol- 2903/IUI21WS-TExSS-3.pdf
  2. [14]14. Mirian Calvo, Leon Cruickshank, and Madeleine Sclater. 2022. Exploring mutual learning in co-design. DISCERN: International Journal of Design for Social Change, Sustainable Innovation and Entrepreneurship 3, 2 (November 2022), 79-96.
  3. [24]24. A. Crabtree, M. Rouncefield, and P. Tolmie. 2012. Doing Design Ethnogra- phy. Springer London. Retrieved from https://books.google.co.uk/books?id$= $Irm2KKegDjQC
  4. [34]34. Neil Fleming and David Baume. 2006. Learning Styles Again: VARKing up the right tree! Educational developments 7, 4 (2006), 4.
  5. [35]35. Neil D. Fleming. 1995. I'm different; not dumb. Modes of presentation (VARK) in the tertiary classroom. In Research and development in higher education, Pro- ceedings of the 1995 Annual Conference of the Higher Education and Research Development Society of Australasia (HERDSA), HERDSA, 308-313.
  6. [39]39. Fabien Girardin and Neal Lathia. 2017. When User Experience Designers Partner with Data Scientists. In AAAI Spring Symposia.
  7. [73]73. Aisling Ann O'Kane, Yvonne Rogers, and Ann E. Blandford. 2015. Concealing or Revealing Mobile Medical Devices?: Designing for Onstage and Offstage Presentation. In Proceedings of the 33rd annual ACM conference on human factors in computing systems, ACM, 1689-1698.

Cited by

Sorted by citations

Authors
Ayobi, A., Hughes, J., Duckworth, C. J., Dylag, J. J., Gordon James, S., Marshall, P., Guy, M., Kumaran, A., Chapman, A., Boniface, M., & O'Kane, A. A.
Affiliations
  • University College London (GB)
  • University of Bristol (GB)
  • University of Southampton (GB)
  • University of Bristol, United Kingdom
  • University Hospital Southampton NHS Foundation Trust (GB)
Venue
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2023-04-19
Pages
1-20
OpenAlex
W4366590502
Cited by
26
References
94
Citations by year
2026: 4 · 2025: 13 · 2024: 8 · 2023: 1
FWCI
4.61
Type
conference paper
Language
en
Open access
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion)
  • ePrints Soton (University of Southampton) (acceptedVersion)
  • UCL Discovery (University College London) (submittedVersion)
  • Bristol Research (University of Bristol) (submittedVersion, other-oa)
  • Explore Bristol Research (submittedVersion, other-oa)
Funders
UK Research and Innovation (RITM0372366)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
computational notebooks, co-design, machine learning models, diabetes management, young adults with diabetes, family caregivers, clinicians, end-user involvement, health risk prediction, interactive data visualization, feature importance, multidisciplinary learning, power imbalance, information asymmetry

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review