CoChat
Sign Up
Cited by 42Digital Accessibility for DisabilitiesAssistive Technology in Communication and MobilityExplainable Artificial Intelligence (XAI)

Misfitting With AI: How Blind People Verify and Contest AI Errors

Open access
Alharbi, Rahaf, Lor, Pa, Herskovitz, Jaylin, Schoenebeck, Sarita, Brewer, Robin N.·2024·The 26th International ACM SIGACCESS Conference on Computers and Accessibility
DOI 10.1145/3663548.3675659
Open this paper in CoChat — free accountOpen PDF

Abstract

Blind people use artificial intelligence-enabled visual assistance technologies (AI VAT) to gain visual access in their everyday lives, but these technologies are embedded with errors that may be difficult to verify non-visually. Previous studies have primarily explored sighted users’ understanding of AI output and created vision-dependent explainable AI (XAI) features. We extend this body of literature by conducting an in-depth qualitative study with 26 blind people to understand their verification experiences and preferences. We begin by describing errors blind people encounter, highlighting how AI VAT fails to support complex document layouts, diverse languages, and cultural artifacts. We then illuminate how blind people make sense of AI through experimenting with AI VAT, employing non-visual skills, strategically including sighted people, and cross-referencing with other devices. Participants provided detailed opportunities for designing accessible XAI, such as affordances to support contestation. Informed by disability studies framework of misfitting and fitting, we unpacked harmful assumptions with AI VAT, underscoring the importance of celebrating disabled ways of knowing. Lastly, we offer practical takeaways for Responsible AI practice to push the field of accessible XAI forward.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3663548.3675659

Cites

87 of 87 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In research involving people with disabilities, participants mostly collaborate on the design or evaluation of access-supporting generative AI systems.

    “Similarly, in research involving PwD, participants mostly collaborate on the design or evaluation of accesssupporting generative AI systems [2, 3, 22, 60, 61, 73, 98, 109].”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [3]

    The Method and Introduction describe interviews with 26 blind users about errors in access-supporting AI visual assistance technologies and features they would like improved. That supports participant input on existing systems, but the paper does not describe participants collaborating on system design or a formal evaluation, nor establish that such collaboration characterizes most disability research. The technologies studied are also not described as exclusively generative AI. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • VLMs are increasingly used by blind and low-vision (BLV) people to identify and understand products in their everyday lives.

    “AI tools and their underlying VLMs are becoming more integral to how BLV people perform a variety of everyday tasks, including grocery shopping, cooking, cleaning, and personal care [9, 120, 132].”

    From "It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models (2026) · cites this paper as [9]

    Section 2.2 says AI visual-assistance tools use object recognition to identify products, and Section 4 describes participants using Seeing AI’s Product mode to check product identities. The paper supports everyday use of AI tools for this purpose, but these passages do not establish that VLM use specifically is increasing. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Most platforms provide little support for helping BLV users understand errors, such as confidence rates or multiple likely image descriptions.

    “Moreover, most platforms provide little support for helping BLV users understand errors, such as confidence rates or multiple likely image descriptions [3, 9].”

    From "It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models (2026) · cites this paper as [9]

    Section 5.2 describes insufficient support for blind users’ existing verification efforts in AI visual-assistance applications, and section 4.3.2 discusses participants’ interest in confidence ratings for assessing errors. The paper does not establish that most platforms lack such features or discuss providing multiple likely image descriptions as an error-understanding feature. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Cross-cultural bias is a significant limitation of VLMs perceived by BLV users.

    “Previous research has identified cross-cultural bias as a significant limitation of VLMs perceived by BLV users [9].”

    From "It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models (2026) · cites this paper as [9]

    Section 4.1 identifies cross-cultural bias as an error reported by blind participants using AI visual-assistance technologies, including failures with non-English text and cultural artifacts. But the paper does not establish this as a limitation of VLMs specifically; its examples concern a broader range of technologies, including OCR and scene-description tools. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Prior work indicates BLV people expect error-prone output from AI tools.

    “Although prior work indicates BLV people expect error-prone output from AI tools [3, 9, 120]”

    From "It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models (2026) · cites this paper as [9]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 88 references couldn’t be matched to a known paper.

Research works printed but not linked: 34

  1. Y Alginahi. 2017. Document processing and Arabic optical character recognition: A user perspective. International Journal of Computer Science and Information Security 15, 12 (2017), 86-97.
  2. Rawan Alharbi, Mariam Tolba, Lucia C Petito, Josiah Hester, and Nabil Alshurafa. 2019. To mask or not to mask? balancing privacy with visual confrmation utility in activity-oriented wearable cameras. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 3, 3 (2019), 1-29.
  3. Chris Bell. 2010. Is disability studies actually white disability studies. The disability studies reader 5 (2010), 402-410.
  4. Cynthia L Bennett and Os Keyes. 2020. What is the point of fairness? Disability, AI and the complexity of justice. ACM SIGACCESS Accessibility and Computing 125 (2020), 1-1.
  5. Patricia Berne, Aurora Levins Morales, David Langstaf, and Sins Invalid. 2018. Ten principles of disability justice. WSQ: Women's Studies Quarterly 46, 1 (2018), 227-230.
  6. Jefrey P Bigham and Patrick Carrington. 2018. Learning from the front: People with disabilities as early adopters of AI. Proceedings of the 2018 HCIC Human- Computer Interaction Consortium (2018).
  7. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77-101.
  8. Virginia Braun and Victoria Clarke. 2019. Refecting on refexive thematic analysis. Qualitative research in sport, exercise and health 11, 4 (2019), 589-597.
  9. Terrance De Vries, Ishan Misra, Changhan Wang, and Laurens Van der Maaten. 2019. Does object recognition work for everyone?. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 52-59.
  10. Upol Ehsan, Samir Passi, Q Vera Liao, Larry Chan, I Lee, Michael Muller, Mark O Riedl, et al. 2021. The who in explainable ai: How ai background shapes percep- tions of ai explanations. arXiv preprint arXiv:2107.13509 (2021). · arXiv
  11. Philip M Ferguson and Emily Nusbaum. 2012. Disability studies: What is it and what diference does it make? Research and Practice for Persons with Severe Disabilities 37, 2 (2012), 70-80.
  12. Ricardo Gonzalez, Jazmin Collins, Shiri Azenkot, and Cynthia Bennett. 2024. Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision People. arXiv preprint arXiv:2403.15604 (2024). · arXiv
  13. Anhong Guo, Ece Kamar, Jennifer Wortman Vaughan, Hanna Wallach, and Meredith Ringel Morris. 2020. Toward fairness in AI for people with disabilities SBG@ a research roadmap. ACM SIGACCESS accessibility and computing 125 (2020), 1-1.
  14. Aimi Hamraie and Kelly Fritsch. 2019. Crip technoscience manifesto. Catalyst: Feminism, theory, technoscience 5, 1 (2019), 1-33.
  15. Chandrika Jayant, Hanjie Ji, Samuel White, and Jefrey P Bigham. 2011. Sup- porting blind photography. In The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility. 203-210.
  16. Mark T Keane, Eoin M Kenny, Eoin Delaney, and Barry Smyth. 2021. If only we had better counterfactual explanations: Five key defcits to rectify in the evaluation of counterfactual xai techniques. arXiv preprint arXiv:2103.01035 (2021). · arXiv
  17. Anja Kintsch and Rogerio DePaula. 2002. A framework for the adoption of assis- tive technology. SWAAAC 2002: Supporting learning through assistive technology 3 (2002), 1-10.
  18. Eva Feder Kittay. 1999. Love's Labor: Essays on Women, Equality and Dependency (1st ed.). Routledge.
  19. Zachary C Lipton. 2018. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue 16, 3 (2018), 31-57.
  20. Daniela Massiceti, Camilla Longden, Agnieszka Slowik, Samuel Wills, Martin Grayson, and Cecily Morrison. 2023. Explaining CLIP's performance disparities on data from blind/low vision users. arXiv preprint arXiv:2311.17315 (2023). · arXiv
  21. Tim Miller. 2019. Explanation in artifcial intelligence: Insights from the social sciences. Artifcial intelligence 267 (2019), 1-38.
  22. Julie Avril Minich. 2016. Enabling whom? Critical disability studies now. Lateral 5, 1 (2016).
  23. Joan Nwatu, Oana Ignat, and Rada Mihalcea. 2023. Bridging the digital divide: Performance variation across socio-economic factors in vision-language models. arXiv preprint arXiv:2311.05746 (2023). · arXiv
  24. Betsy Phillips and Hongxin Zhao. 1993. Predictors of assistive technology abandonment. Assistive technology 5, 1 (1993), 36-45.
  25. Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, and D Sculley. 2017. No classifcation without representation: Assessing geodiversity issues in open data sets for the developing world. arXiv preprint arXiv:1711.08536 (2017). · arXiv
  26. Renee Shelby, Shalaleh Rismani, Kathryn Henne, Ajung Moon, Negar Ros- tamzadeh, Paul Nicholas, YILLA-AKBARI N'MAH, Jess Gallegos, Andrew Smart, and Gurleen Virk. 2022. Identifying sociotechnical harms of algorithmic sys- tems: Scoping a taxonomy for harm reduction. arXiv preprint arXiv:2210.05791 (2022). · arXiv
  27. Margrit Shildrick. 1998. The Rejected Body: Feminist Philosophical Refections on Disability, by Susan Wendell; Feminist Approaches to Bioethics: Theoretical Refections and Practical Applications, by Rosemarie Tong. Women's Philosophy Review 18 (1998), 68-72.
  28. Tobin Siebers. 2008. Disability theory. University of Michigan Press.
  29. Ellen Simpson and Bryan Semaan. 2021. For you, or for" you"? Everyday LGBTQ+ encounters with TikTok. Proceedings of the ACM on human-computer interaction 4, CSCW3 (2021), 1-34.
  30. Erik Štrumbelj and Igor Kononenko. 2014. Explaining prediction models and individual predictions with feature contributions. Knowledge and information systems 41 (2014), 647-665.
  31. Rua M Williams. 2021. I, Misft: Empty Fortresses, Social Robots, and Peculiar Relations in Autism Research. Techne: Research in Philosophy & Technology 25, 3 (2021).
  32. Bess Williamson. 2019. Accessible America: A history of disability and design. In Accessible America. New York University Press.
  33. Austin P Wright, Zijie J Wang, Haekyu Park, Grace Guo, Fabian Sperrle, Men- natallah El-Assady, Alex Endert, Daniel Keim, and Duen Horng Chau. 2020. A comparative analysis of industry human-AI interaction guidelines. arXiv preprint arXiv:2010.11761 (2020). · arXiv
  34. Rui Zhang, Nathan J McNeese, Guo Freeman, and Geof Musick. 2021. " An ideal human" expectations of AI teammates in human-AI teaming. Proceedings of the ACM on Human-Computer Interaction 4, CSCW3 (2021), 1-25.

Cited by

Sorted by citations

Related

Authors
Alharbi, R., Lor, P., Herskovitz, J., Schoenebeck, S., & Brewer, R. N.
Affiliations
University of Michigan (US)
ORCID
Venue
The 26th International ACM SIGACCESS Conference on Computers and Accessibility
Publisher
ACM
Published
2024-10-27
Pages
1-17
DOI
10.1145/3663548.3675659
OpenAlex
W4402426945
Cited by
40
References
88
Citations by year
2026: 24 · 2025: 16
FWCI
21.85
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • Deep Blue (University of Michigan) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
National Science Foundation (1763297)
Topics
  • Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
  • Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
blind people, AI errors, disability studies, responsible AI, assistive technology

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review