MAIDR Meets AI: Exploring Multimodal LLM-Based Data Visualization Interpretation by and with Blind and Low-Vision Users
Open accessAbstract
This paper investigates how blind and low-vision (BLV) users interact with multimodal large language models (LLMs) to interpret data visualizations. Building upon our previous work on the multimodal access and interactive data representation (MAIDR) framework, our mixed-visual-ability team co-designed maidrAI, an LLM extension providing multiple AI responses to users’ visual queries. To explore generative AI-based data representation, we conducted user studies with 8 BLV participants, tasking them with interpreting box plots using our system. We examined how participants personalize LLMs through prompt engineering, their preferences for data visualization descriptions, and strategies for verifying LLM responses. Our findings highlight three dimensions affecting BLV users’ decision-making process: modal preference, LLM customization, and multimodal data representation. This research contributes to designing more accessible data visualization tools for BLV users and advances the understanding of inclusive generative AI applications.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In research involving people with disabilities, participants mostly collaborate on the design or evaluation of access-supporting generative AI systems.
“Similarly, in research involving PwD, participants mostly collaborate on the design or evaluation of accesssupporting generative AI systems [2, 3, 22, 60, 61, 73, 98, 109].”
From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [98]
Cites
Cognitive Load During Problem Solving: Effects on Learning
Sweller, J. (1988) · Cognitive Science · doi:10.1016/0364-0213(88)90023-7 · Cited by 11,048 · see citing works
Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load
Sweller, J. (2010) · Educational Psychology Review · doi:10.1007/s10648-010-9128-5 · Cited by 2,020 · see citing works
VizWiz: nearly real-time answers to visual questions
Bigham, J. P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R. C., Miller, R., Tatarowicz, A., White, B., White, S., et al. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866080 · Cited by 601 · see citing works
Interdependence as a Frame for Assistive Technology Research and Design
Bennett, C. L., Brady, E., & Branham, S. M. (2018) · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3234695.3236348 · Cited by 407 · cited 2× · see citing works
“More specifcally, we adopted the inter-dependence framework [12] to facilitate the collaboration between our BLV and non-BLV team members.”
Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.04023 · Cited by 337 · cited 3× · see citing works
“For example, the hallucination efect [8, 17] in data visualization descriptions may become even more critical for BLV users, potentially misleading their informed decision process.”
Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content
Lundgard, A., & Satyanarayan, A. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3114770 · Cited by 155 · see citing works
VoxLens: Making Online Data Visualizations Accessible with an Interactive JavaScript Plug-In
Sharif, A., Wang, O. H., Muongchan, A. T., Reinecke, K., & Wobbrock, J. O. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517431 · Cited by 86 · cited 1× · see citing works
“Sharif et al. developed VoxLens , an open-source JavaScript library with a purpose of making web-based data visualizations accessible for BLV individuals by means of voice recognition-based question-and-answer, as well as sonifcation [35].”
Guiding Novice Web Workers in Making Image Descriptions Using Templates
Morash, V. S., Siu, Y.-T., Miele, J. A., Hasty, L., & Landau, S. (2015) · ACM Transactions on Accessible Computing · doi:10.1145/2764916 · Cited by 69 · see citing works
Sharif, A., Zhang, A. M., Reinecke, K., & Wobbrock, J. O. (2023) · 20th International Web for All Conference · doi:10.1145/3587281.3587284 · Cited by 21 · cited 1× · see citing works
“Future iterations were made to VoxLens to improve the drill-down information extraction from data visualizations by screen reader users [36], however this implementation does not extend to visualizations like box plots, scatter plots, and heatmaps.”
Flagged first, then by citations
Cited by
Sorted by citations
Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation
Thieme, A., Faia Marques, R., Grayson, M., Balachandar, S., Tyler Cassidy, C., Choksi, M. Z., Longden, C., Huda, R. S., Ileve Kalovwe, N., Mallon, C., et al. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790768 · Cited by 1 · see citing works
“Similarly, in research involving PwD, participants mostly collaborate on the design or evaluation of access-supporting generative AI systems [2, 3, 22, 60, 61, 73, 98, 109].”
- Authors
- Seo, J., Kamath, S. S., Zeidieh, A., Venkatesh, S., & McCurry, S.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- TransPerfect, United States
- Venue
- The 26th International ACM SIGACCESS Conference on Computers and Accessibility
- Publisher
- ACM
- Published
- 2024-10-27
- Pages
- 1-31
- OpenAlex
- W4403582792
- Cited by
- 25
- References
- 9
- Citations by year
- 2026: 19 · 2025: 6
- FWCI
- 11.68
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) (publishedVersion, cc-by)
- Funders
- The Institute of Museum and Library Services (IMLS), the Laura Bush 21st Century Librarian Program (#RE-254891-OLS-23)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Keywords
- blind and visually impaired users, multimodal large language models, accessible data visualization, prompt engineering, box plots, visual query, multimodal representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
