CoChat
Sign Up
Cited by 38Digital Accessibility for DisabilitiesAssistive Technology in Communication and MobilityUsability and User Interface Design

Understanding Personalized Accessibility through Teachable AI: Designing and Evaluating Find My Things for People who are Blind or Low Vision

Open access
Morrison, Cecily, Grayson, Martin, Marques, Rita Faia, Massiceti, Daniela, Longden, Camilla, Wen, Linda, Cutrell, Edward·2023·The 25th International ACM SIGACCESS Conference on Computers and Accessibility
DOI 10.1145/3597638.3608395
Open this paper in CoChat — free accountOpen PDF

Abstract

The opportunity for artificial intelligence, or AI, to enable accessibility is rapidly growing, but widely impactful applications can be challenging to build given the diversity of user need within and across disability communities. Teachable AI systems give users with disabilities a way to leverage the power of AI to personalize applications for their own specific needs, as long as the effort of providing examples is balanced with the benefit of the personalization received. As an example, this paper presents the design and evaluation of Find My Things, an end-to-end application that can be taught by people who are blind or low vision to find their personal things. Through synthesis of the design process, this paper offers design considerations for the teaching loop that is so critical to realizing the power of teachable AI for accessibility.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3597638.3608395

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Academic scholars and industry corporations have emphasized the pressing need for more disability-centered datasets.

    “Although academic scholars and industry corporations have emphasized the pressing need for more disability-centered datasets [20, 38, 55, 79, 97, 111, 124]”

    From "It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models (2026) · cites this paper as [97]

    Section 2.3 (Few-Shot Learning; page unavailable) says research is shifting toward real-world data through the collection of new datasets and identifies ORBIT, recorded by people who are blind or low vision, as an example. This supports interest in disability-relevant datasets, but the paper does not state a pressing need for more disability-centered datasets or attribute that call to both academic scholars and industry corporations. (LLM, conf=0.81)

    Experimental — accuracy not yet measured

  • BLV people want key product information and visual details of the product and nearby objects in image captions.

    “which BLV people want in captions (shown by our survey and prior work [67, 97])”

    From "It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models (2026) · cites this paper as [97]

    The Conclusion mentions personalized audio descriptions and captions only as a possible future application. The paper’s Dynamic Support and Findings sections concern teaching an app to recognize and locate personal objects; they do not report that BLV people want product information, visual details, or nearby objects in image captions. Searched for captions, product information, and nearby-object descriptions. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 29

  1. Dragan Ahmetovic, Daisuke Sato, Uran Oh, Tatsuya Ishihara, Kris Kitani, and Chieko Asakawa. 2020. Recog: Supporting blind people in recognizing personal objects. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1-12.
  2. Saleema Amershi, James Fogarty, Ashish Kapoor, and Desney Tan. 2009. Design- ing for End-User Interactive Concept Learning in CueFlik. Neural Information Processing Systems (NIPS) Workshop on Analysis and Design of Algorithms for Interactive Machine Learning.
  3. C. Andrews. 2014. Accessible Participatory Design: Engaging and Including Visually Impaired Participants. In Inclusive Designing. Springer International Publishing, 201-210.
  4. Harshadha Balasubramanian, Cecily Morrison, Martin Grayson, Zhanat Makhataeva, Rita Marques, Thomas Gable, Dalya Perez, and Edward Cutrell. 2023. Enable Blind Users' Experience in 3D Virtual Environments: The Scene Weaver Prototype. In In 2023 CHI EA Conference on Human Factors in Computing Systems, 1-4.
  5. Michelle Carney, Barron Webster, Irene Alvarado, Kyle Phillips, Noura Howell, Jordan Grifth, Jonas Jongejan, Amit Pitaru, and Alexander Chen. 2020. Teach- able machine: Approachable Web-based tool for exploring machine learning classifcation. In The 2020 CHI EA Conference on Human Factors in Computing Systems, 1-8.
  6. Da Chen, Yuefeng Chen, Yuhong Li, Feng Mao, Yuan He, and Hui Xue. 2021. Self-Supervised Learning for Few-Shot Image Classifcation. In ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1745-1749.
  7. Rebecca Fiebrink, Perry R. Cook, and Dan Trueman. 2011. Human model evalua- tion in interactive supervised learning. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 147-156.
  8. C. Finn, P. Abbeel, and S. Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 2017 ICML International Con- ference on Machine Learning, 1126-1135.
  9. Steven M. Goodman, Ping Liu, Dhruv Jain, Emma J. McDonnell, Jon E. Froehlich, and Leah Findlater. 2021. Toward User-Driven Sound Recognizer Personalization with People Who Are d/Deaf or Hard of Hearing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 2: 1-23.
  10. Jonggi Hong, Kyungjun Lee, June Xu, and Hernisa Kacorri. 2020. Crowdsourcing the Perception of Machine Teaching. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1-14.
  11. Hernisa Kacorri. 2017. Teachable machines for accessibility. ACM SIGACCESS Accessibility and Computing 119: 10-18.
  12. Hernisa Kacorri, Kris M. Kitani, Jefrey P. Bigham, and Chieko Asakawa. 2017. People with visual impairment training personal object recognizers: Feasibility and challenges. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 5839-5849.
  13. Rie, Kamikubo, Kyungjun Lee, and Hernisa Kacorri. 2023. Contributing to Accessi- bility Datasets: Refections on Sharing Study Data by Blind People . In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 1-18.
  14. Simon Katan, Mick Grierson, and Rebecca Fiebrink. 2015. Using Interactive Machine Learning to Support Interface Development Through Workshops with Disabled People. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 251-254.
  15. Todd Kulesza, Weng-Keen Wong, Simone Stumpf, Stephen Perona, Rachel White, Margaret M. Burnett, Ian Oberst, and Amy J. Ko. 2009. Fixing the program my computer learned. In Proceedings of the 14th international conference on Intelligent user interfaces, 187-196.
  16. Kyungjun Lee, Abhinav Shrivastava, and Hernisa Kacorri. 2020. Hand-Priming in Object Localization for Assistive Egocentric Vision. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 3422-3432.
  17. Kelly Mack, Emma McDonnell, Dhruv Jain, Lucy Lu Wang, Jon E. Froehlich, and Leah Findlater. 2021. What Do We Mean by "Accessibility Research"? In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1-18.
  18. Yuri Nakao and Yusuke Sugano. 2020. Use of Machine Learning by Non-Expert DHH People: Technological Understanding and Sound Perception. In Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society, 1-12.
  19. Eunbyung Park and Junier B Oliva. 2019. Meta-Curvature. In Advances in Neural Information Processing Systems 32, 1-11.
  20. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision 115, 211-252.
  21. Téo Sanchez, Baptiste Caramiaux, Jules Françoise, Frédéric Bevilacqua, and Wendy E. Mackay. 2021. How do People Train a Machine? Proceedings of the ACM on Human-Computer Interaction 5, CSCW1: 1-26.
  22. Dan Shapiro. 2005. Participatory design: the will to succeed. In Proceedings of the CC Conference on Critical Computing, 29-38.
  23. Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical Networks for Few-shot Learning. In Advances in Neural Information Processing Systems 30, 1 - 11.
  24. Mingxing Tan and Quoc Le. 2019. EfcientNet: Rethinking Model Scaling for Con- volutional Neural Networks. In Proceedings of the ICML International Conference on Machine Learning, 6105-6114.
  25. Anja Thieme, Cynthia L. Bennett, Cecily Morrison, Edward Cutrell, and Alex S. Taylor. 2018. "I can do everything but see!-How People with Vision Impair- ments Negotiate their Abilities in Social Contexts. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 203.
  26. Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. 2016. Matching Networks for One Shot Learning. In Advances in Neural Information Processing Systems, 29, 1-9.
  27. Michele A Williams, Amy Hurst, and Shaun K Kane. 2013. "Pray Before You Step out": Describing Personal and Situational Blind Navigation Behaviors. In Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility: 1-8.
  28. Yuhang Zhao, Edward Cutrell, Christian Holz, Meredith Ringel Morris, Eyal Ofek, and Andrew D. Wilson. 2019. SeeingVR. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1-14.
  29. Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon White- son. 2019. Fast Context Adaptation via Meta-Learning. In Proceedings of the ICML Conference on Machine Learning, 7693-7702.

Cited by

Sorted by citations

Authors
Morrison, C., Grayson, M., Marques, R. F., Massiceti, D., Longden, C., Wen, L., & Cutrell, E.
Affiliations
  • Microsoft Research (United Kingdom) (GB)
  • Microsoft Research, United Kingdom
  • Microsoft (United States) (US)
ORCID
Venue
The 25th International ACM SIGACCESS Conference on Computers and Accessibility
Publisher
ACM
Published
2023-10-22
Pages
1-12
DOI
10.1145/3597638.3608395
OpenAlex
W4387793598
Cited by
38
References
1
Citations by year
2026: 10 · 2025: 15 · 2024: 13
FWCI
13.65
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) (publishedVersion)
Funders
CNIB
Topics
  • Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
  • Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
Keywords
blind and visually impaired users, object finding, assistive technology design, accessibility evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review