ImageExplorer: Multi-Layered Touch Exploration to Encourage Skepticism Towards Imperfect AI-Generated Image Captions
Abstract
Blind users rely on alternative text (alt-text) to understand an image; however, alt-text is often missing. AI-generated captions are a more scalable alternative, but they often miss crucial details or are completely incorrect, which users may still falsely trust. In this work, we sought to determine how additional information could help users better judge the correctness of AI-generated captions. We developed ImageExplorer, a touch-based multi-layered image exploration system that allows users to explore the spatial layout and information hierarchies of images, and compared it with popular text-based (Facebook) and touch-based (Seeing AI) image exploration systems in a study with 12 blind participants. We found that exploration was generally successful in encouraging skepticism towards imperfect captions. Moreover, many participants preferred ImageExplorer for its multi-layered and spatial information presentation, and Facebook for its summary and ease of use. Finally, we identify design improvements for effective and explainable image exploration systems for blind users.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Authors tend to over-trust automated or AI-generated content, rarely editing for accuracy.
“Mostly, authors tend to over-trust automated or AI-generated content, rarely editing for accuracy [44, 48].”
From How University Disability Services Professionals Write Image Descriptions for HCI Figures Using Generative AI (2026) · cites this paper as [44]
Studies applying generative AI as an automated method to produce alt text show varying performance on different types of images and call for integrating the author/writer's expertise.
“Recently, studies [44, 52, 53] have applied generative AI as an automated method to produce alt text. However, these studies show varying performance on different types of images and call for integrating the author/writer's expertise.”
From How University Disability Services Professionals Write Image Descriptions for HCI Figures Using Generative AI (2026) · cites this paper as [44]
Touch-based techniques and hierarchical organization benefit image understanding, supporting spatial awareness, agency, and user control, but have trade-offs in effort and efficiency compared to simpler text-based approaches.
“Their fi ndings reinforce the benefits of touch-based techniques and hierarchical organization for image understanding, particularly in supporting spatial awareness, agency, and user control, but also highlighted trade-offs in effort and efficiency when compared to simpler, text-based approaches.”
From From Screen Reading to "Scene Reading" in SceneVR: Touch-Based Interaction Techniques for Use in Virtual Reality by Blind and Low-Vision Users (2025) · cites this paper as [45]
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 46 · Cited by 61
- Authors
- Lee, J., Herskovitz, J., Peng, Y.-H., & Guo, A.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- University of Michigan (US)
- Carnegie Mellon University (US)
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-15
- OpenAlex
- W4224993535
- Cited by
- 61
- References
- 46
- Citations by year
- 2026: 11 · 2025: 18 · 2024: 18 · 2023: 14
- FWCI
- 18.22
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Keywords
- blind users, image exploration, touch interaction, skepticism, alternative text, accessibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
