How Multimodal Large Language Models Support Access to Visual Information: A Diary Study With Blind and Low Vision People
The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117
GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks
Patwardhan, T., Dias, R., Proehl, E., Kim, G., Wang, M., Watkins, O., Fishman, S. P., Aljubeh, M., Thacker, P., Fauconnet, L., et al. (2025) · arXiv · doi:10.48550/arxiv.2510.04374 · Cited by 189 · cited 1×
“Adaptive communication protocols : Establish early "information contracts" with users regarding preferred detail levels, preferred vocabulary, and bidirectional communication cues (e.g., user gestures like touching objects and system signals for when additional input is needed) (our work, [54]).”
Indoor navigation by blind people: behaviors and challenges in unfamiliar spaces and buildings
Jeamwatthanachai, W., Wald, M., & Wills, G. (2019) · British Journal of Visual Impairment · doi:10.1177/0264619619833723 · Cited by 105
VizLens: A Robust and Interactive Screen Reader for Interfaces in the Real World
Guo, A., Chen, X. '., Qi, H., White, S., Ghosh, S., Asakawa, C., & Bigham, J. P. (2016) · Proceedings of the 29th Annual Symposium on User Interface Software and Technology · doi:10.1145/2984511.2984518 · Cited by 99
"I look at it as the king of knowledge": How Blind People Use and Understand Generative AI Tools
Adnin, R., & Das, M. (2024) · The 26th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663548.3675631 · Cited by 73 · cited 1×
“We collected and analyzed 551 diary entries and 626 questions from 375 conversations with an MLLM.”
Cohn, M., Pushkarna, M., Olanubi, G. O., Moran, J. M., Padgett, D., Mengesha, Z., & Heldreth, C. (2024) · Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613905.3650818 · Cited by 71
(2019) · doi:10.18653/v1/w19-4807 · Cited by 43 · cited 1×
“Model developers must prioritize accuracy over engagement metrics and "teach" models what it means to be a "good visual assistant", system developers must configure AI for accessibility contexts in mind, and the broader community must incorporate BLV people’s perspectives throughout design.”
Nair, V., Zhu, H. '., & Smith, B. A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581302 · Cited by 29 · cited 1×
“…while we recruited a diverse set of participants across age and gender, all of our participants were expert users of visual interpretation systems, missing a critical perspective from beginners who have never engaged with visual interpretation systems (e.g., lack of awareness or access barriers).”
Sharma, T., Stangl, A., Zhang, L., Tseng, Y.-Y., Xu, I., Findlater, L., Gurari, D., & Wang, Y. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580922 · Cited by 23 · cited 1×
“Example: "I see an address depicted in this image.”
Be Friendly, Not Friends: How LLM Sycophancy Shapes User Trust
Sun, Y., & Wang, T. (2025) · arXiv · doi:10.48550/arxiv.2502.10844 · Cited by 16
Research works printed but not linked: 64
- 10.1145/3663548.3675659 · DOI · Paper page
- Peter Anderson Basura Fernando Mark Johnson and Stephen Gould. 2016. SPICE: Semantic Propositional Image Caption Evaluation. arxiv:https://arXiv.org/abs/1607.08822 [cs.CV] · DOI · Paper page
- Hammad A. Ayyubi Md. Mehrab Tanjim Julian J. McAuley and Garrison W. Cottrell. 2020. Generating Rationales in Visual Question Answering. arxiv:https://arXiv.org/abs/2004.02032 [cs.AI] https://arxiv.org/abs/2004.02032 · arXiv
- 10.1145/1866029.1866080 · DOI · Paper page
- 10.1109/CVPRW.2010.5543821 · DOI · Paper page
- Erin Brady. 2015. Using social microvolunteering to answer visual questions from blind users. 4 pages. 10.1145/2809904.2809910 · DOI
- Erin Brady Meredith Morris and Jeffrey Bigham. 2014. Friendsourcing for the Greater Good: Perceptions of Social Microvolunteering. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing 2 1 (Sep. 2014) 6–7. 10.1609/hcomp.v2i1.13174 · doi:10.1609/hcomp.v2i1.13174
- 10.1145/2470654.2481291 · DOI · Paper page
- 10.1145/2441776.2441915 · DOI · Paper page
- 10.1007/978-3-031-92089-9_22 · DOI
- 10.1109/HSI.2008.4581561 · DOI
- 10.1145/2384916.2384941 · DOI · Paper page
- Kilian Carolan Laura Fennelly and Alan F. Smeaton. 2024. A Review of Multi-Modal Large Language and Vision Models. arxiv:https://arXiv.org/abs/2404.01322 [cs.CL] https://arxiv.org/abs/2404.01322 · arXiv
- 10.1145/3654777.3676375 · DOI
- 10.1145/3663547.3746319 · DOI
- 10.1007/978-3-319-49655-9_2 · DOI
- 10.1007/978-3-031-73195-2_15 · DOI
- Lin Chen Jinsong Li Xiaoyi Dong Pan Zhang Conghui He Jiaqi Wang Feng Zhao and Dahua Lin. 2023. ShareGPT4V: Improving Large Multi-Modal Models with Better Captions. arxiv:https://arXiv.org/abs/2311.12793 [cs.CV] https://arxiv.org/abs/2311.12793 · arXiv
- 10.1145/3663547.3746393 · DOI · Paper page
- Kanzhi Cheng Wenpo Song Jiaxin Fan Zheng Ma Qiushi Sun Fangzhi Xu Chenyang Yan Nuo Chen Jianbing Zhang and Jiajun Chen. 2025. CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era. arxiv:https://arXiv.org/abs/2503.12329 [cs.CV] https://arxiv.org/abs/2503.12329 · arXiv
- Ben Cottier Robi Rahman Loredana Fattorini Nestor Maslej Tamay Besiroglu and David Owen. 2025. The rising costs of training frontier AI models. arxiv:https://arXiv.org/abs/2405.21015 [cs.CY] https://arxiv.org/abs/2405.21015 · arXiv
- 10.1145/3613904.3642325 · DOI · Paper page
- Vanja Garaj Rommanee Jirawimut Piotr Ptasinski Franjo Cecelja and Wamadeva Balachandran. 2003. A system for remote sighted guidance of visually impaired pedestrians. British Journal of Visual Impairment 21 2 (2003) 55–63. · DOI
- 10.1145/2982142.2982200 · DOI
- 10.1145/3613904.3642211 · DOI · Paper page
- Christina Granquist Susan Y Sun Sandra R Montezuma Tu M Tran Rachel Gage and Gordon E Legge. 2021. Evaluation and comparison of artificial intelligence vision aids: Orcam myeye 1 and seeing ai. Journal of Visual Impairment & Blindness 115 4 (2021) 277–285. · DOI · Paper page
- 10.1145/3663548.3675658 · DOI · Paper page
- Rex Hartson and Pardha S Pyla. 2012. The UX Book: Process and guidelines for ensuring a quality user experience. Morgan Kaufmann, San Diego, CA.
- 10.1145/3663548.3675635 · DOI · Paper page
- Mina Huh Fangyuan Xu Yi-Hao Peng Chongyan Chen Hansika Murugu Danna Gurari Eunsol Choi and Amy Pavel. 2024. Long-Form Answers to Visual Questions from Blind and Low Vision People. arxiv:https://arXiv.org/abs/2408.06303 [cs.CL] https://arxiv.org/abs/2408.06303 · arXiv
- Ziad Hunaiti Vanja Garaj and Wamadeva Balachandran. 2006. A remote vision guidance system for visually impaired pedestrians. The journal of navigation 59 3 (2006) 497–504. · DOI
- Adam Tauman Kalai Ofir Nachum Santosh S. Vempala and Edwin Zhang. 2025. Why Language Models Hallucinate. arxiv:https://arXiv.org/abs/2509.04664 [cs.CL] https://arxiv.org/abs/2509.04664 · arXiv
- 10.1145/3677846.3677860 · DOI
- 10.1145/3313831.3376823 · DOI · Paper page
- 10.1145/3544548.3581337 · DOI
- 10.1145/3517428.3544826 · DOI
- 10.1109/ICCE63647.2025.10929822 · doi:10.1109/icce63647.2025.10929822
- 10.1145/2513383.2517033 · DOI · Paper page
- 10.3115/1626355.1626389 · DOI · Paper page
- 10.1145/3441852.3476548 · DOI · Paper page
- Sooyeon Lee Madison Reddie Krish Gurdasani Xiying Wang Jordan Beck Mary Beth Rosson and John M. Carroll. 2018. Conversations for Vision: Remote Sighted Assistants Helping People with Visual Impairments. arxiv:https://arXiv.org/abs/1812.00148 [cs.HC] https://arxiv.org/abs/1812.00148 · arXiv
- 10.1145/3313831.3376591 · DOI · Paper page
- Saehyung Lee Seunghyun Yoon Trung Bui Jing Shi and Sungroh Yoon. 2025. Toward Robust Hyper-Detailed Image Captioning: A Multiagent Approach and Dual Evaluation Metrics for Factuality and Coverage. arxiv:https://arXiv.org/abs/2412.15484 [cs.CV] https://arxiv.org/abs/2412.15484 · arXiv
- 10.1145/3490099.3511113 · DOI · Paper page
- 10.1145/3613904.3642233 · DOI
- 10.1145/3663548.3675662 · DOI
- Kun Li George Vosselman and Michael Ying Yang. 2025. Multimodal Rationales for Explainable Visual Question Answering. arxiv:https://arXiv.org/abs/2402.03896 [cs.CV] https://arxiv.org/abs/2402.03896 · arXiv
- Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. Association for Computational Linguistics, Barcelona, Spain, 74–81. https://aclanthology.org/W04-1013/
- 10.1145/3706598.3713486 · DOI
- Brian J Nguyen William S Chen Allison J Chen Andrew Utt Emily Hill Ryan Apgar and Daniel L Chao. 2019. Large-scale assessment of needs in low vision individuals using the Aira assistive technology. Clinical Ophthalmology 13 (2019) 1853–1868. 10.2147/OPTH.S215658 · DOI
- 10.3115/1073083.1073135 · DOI · Paper page
- Johnny Saldaña. 2021. The Coding Manual for Qualitative Researchers (4th ed.). SAGE Publications Limited, Thousand Oaks, CA.
- 10.1145/3663547.3746335 · DOI
- Ramakrishna Vedantam C. Lawrence Zitnick and Devi Parikh. 2015. CIDEr: Consensus-based Image Description Evaluation. arxiv:https://arXiv.org/abs/1411.5726 [cs.CV] https://arxiv.org/abs/1411.5726 · arXiv
- Jingyi Xie Madison Reddie Sooyeon Lee Syed Masum Billah Zihan Zhou Chun-Hua Tsai and John M Carroll. 2022. Iterative design and prototyping of computer vision mediated remote sighted assistance. ACM Transactions on Computer-Human Interaction (TOCHI) 29 4 (2022) 1–40. · DOI
- 10.1145/3563657.3596019 · DOI
- 10.1145/3532106.3533560 · DOI
- 10.1145/3706598.3714210 · DOI · Paper page
- 10.1145/3613904.3642030 · DOI
- Jingyi Xie Rui Yu He Zhang Sooyeon Lee Syed Masum Billah and John M. Carroll. 2024. Emerging Practices for Large Multimodal Model (LMM) Assistance for People with Visual Impairments: Implications for Design. arxiv:https://arXiv.org/abs/2407.08882 [cs.HC] https://arxiv.org/abs/2407.08882 · arXiv
- Zhengyuan Yang Linjie Li Kevin Lin Jianfeng Wang Chung-Ching Lin Zicheng Liu and Lijuan Wang. 2023. The dawn of lmms: Preliminary explorations with gpt-4v (ision). arXiv preprint arXiv:https://arXiv.org/abs/2309.17421 9 1 (2023) 1. · arXiv
- Rui Yu Sooyeon Lee Jingyi Xie Syed Masum Billah and John M Carroll. 2024. Human–AI Collaboration for Remote Sighted Assistance: Perspectives from the LLM Era. Future Internet 16 7 (2024) 254. · DOI
- Omar Zaidan, Jason Eisner, and Christine Piatko. 2007. Using “Annotator Rationales” to Improve Machine Learning for Text Categorization. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, Candace Sidner, Tanja Schultz, Matthew Stone, and ChengXiang Zhai (Eds.). Association for Computational Linguistics, Rochester, New York, 260–267. https://aclanthology.org/N07-1033/
- 10.18653/v1/2025.acl-long.82 · DOI
Metadata
- Authors
- Gonzalez Penuela, R. E., Jung, C., Lin, S., Hu, R., & Azenkot, S.
- Venue
- Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2026-04-13
- Pages
- 1-24
- Cited by
- 6
- References
- 10
- Funders
- NSF (National Science Foundation) (1942693)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.