CoChat
Sign Up
Cited by 29Digital Accessibility for DisabilitiesTactile and Sensory InteractionsAssistive Technology in Communication and Mobility

ImageAssist: Tools for Enhancing Touchscreen-Based Image Exploration Systems for Blind and Low Vision Users

Open access
Nair, Vishnu, Zhu, Hanxiu 'Hazel', Smith, Brian A.·2023·Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
DOI 10.1145/3544548.3581302
Open this paper in CoChat — free accountOpen PDF

Abstract

Blind and low vision (BLV) users often rely on alt text to understand what a digital image is showing. However, recent research has investigated how touch-based image exploration on touchscreens can supplement alt text. Touchscreen-based image exploration systems allow BLV users to deeply understand images while granting a strong sense of agency. Yet, prior work has found that these systems require a lot of effort to use, and little work has been done to explore these systems’ bottlenecks on a deeper level and propose solutions to these issues. To address this, we present ImageAssist, a set of three tools that assist BLV users through the process of exploring images by touch — scaffolding the exploration process. We perform a series of studies with BLV users to design and evaluate ImageAssist, and our findings reveal several implications for image exploration tools for BLV users.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3544548.3581302

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prior research in 2-D image exploration, BLV video game, and AR accessibility found that structured, menu-based navigation was favored for ease of use and reliably accessing all objects, while free-form exploration provided greater autonomy and spatial awareness, but was more difficult to control.

    “This fi nding aligns with prior research in 2-D image exploration [57] and BLV video game [55] and AR [33] accessibility, which found that structured, menu-based navigation was favored for ease of use and reliably accessing all objects, while free-form exploration provided greater autonomy and spatial awareness, but was more difficult to control.”

    From From Screen Reading to "Scene Reading" in SceneVR: Touch-Based Interaction Techniques for Use in Virtual Reality by Blind and Low-Vision Users (2025) · cites this paper as [57]

    For the 2-D image-exploration portion attributed to this paper, the Introduction and Related Work describe touch exploration as offering autonomy and spatial understanding but requiring more effort and risking missed objects. User Study Findings §§6.1 and 6.4 report that the menu and beacon helped users find more areas and achieve more complete coverage. The paper does not establish that users favored menu-based navigation over free-form exploration: §6.1 says most chose touch as their primary method and used the menu as a supplement. The video-game and AR portions have separate citations and are not judged here. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 23

  1. [1][1] Dragan Ahmetovic, Nahyun Kwon, Uran Oh, Cristian Bernareggi, and Sergio Mascetti. 2021. Touch Screen Exploration of Visual Artwork for Blind People. In Proceedings of the Web Conference 2021 (WWW '21). Association for Computing Machinery, New York, NY, USA, 2781–2791. https://doi.org/10.1145/3442381. 3449871 · DOI
  2. [2][2] Cynthia L. Bennett, Cole Gleason, Morgan Klaus Scheuerman, Jeffrey P. Bigham, Anhong Guo, and Alexandra To. 2021. "It's Complicated": Negotiating Accessibil- ity and (Mis)Representation in Image Descriptions of Race, Gender, and Disability. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Sys- tems (CHI '21). Association for Computing Machinery, New York, NY, USA, 1–19. https://doi.org/10.1145/3411764.3445498 · DOI · Paper page
  3. [3][3] Shuhan Chen, Xiuli Tan, Ben Wang, Huchuan Lu, Xuelong Hu, and Yun Fu. 2020. Reverse Attention-Based Residual Network for Salient Object Detection. IEEE Transactions on Image Processing 29 (2020), 3763–3776. https://doi.org/10.1109/ TIP.2020.2965989
  4. [6][6] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Ima- geNet: A Large-Scale Hierarchical Image Database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. 248–255. https://doi.org/10.1109/CVPR. 2009.5206848 · DOI
  5. [7][7] Julie Ducasse, Anke M. Brock, and Christophe Jouffrais. 2018. Accessible In- teractive Maps for Visually Impaired Users. In Mobility of Visually Impaired People: Fundamentals and ICT Assistive Technologies, Edwige Pissaloux and Ramiro Velazquez (Eds.). Springer International Publishing, Cham, 537–584. https://doi.org/10.1007/978-3-319-54446-5_17 · DOI · Paper page
  6. [9][9] Michael Figurnov, Maxwell D. Collins, Yukun Zhu, Li Zhang, Jonathan Huang, Dmitry Vetrov, and Ruslan Salakhutdinov. 2017. Spatially Adaptive Computa- tion Time for Residual Networks. https://doi.org/10.48550/arXiv.1612.02297 arXiv:1612.02297 [cs] · DOI
  7. [11][11] Cole Gleason, Patrick Carrington, Cameron Cassidy, Meredith Ringel Morris, Kris M. Kitani, and Jeffrey P. Bigham. 2019. "It's Almost like They're Trying to Hide It": How User-Provided Image Descriptions Have Failed to Make Twitter Accessible. In The World Wide Web Conference (WWW '19). Association for Computing Machinery, New York, NY, USA, 549–559. https://doi.org/10.1145/ 3308558.3313605
  8. [12][12] Cole Gleason, Amy Pavel, Xingyu Liu, Patrick Carrington, Lydia B. Chilton, and Jeffrey P. Bigham. 2019. Making Memes Accessible. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '19). Association for Computing Machinery, New York, NY, USA, 367–376. https://doi.org/10. 1145/3308561.3353792
  9. [13][13] Cole Gleason, Amy Pavel, Emma McCamey, Christina Low, Patrick Carrington, Kris M. Kitani, and Jeffrey P. Bigham. 2020. Twitter A11y: A Browser Extension to Make Twitter Images Accessible. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376728 · DOI
  10. [16][16] João Guerreiro, Dragan Ahmetovic, Kris M. Kitani, and Chieko Asakawa. 2017. Virtual Navigation for Blind People: Building Sequential Representations of the Real-World. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17). Association for Computing Machinery, New York, NY, USA, 280–289. https://doi.org/10.1145/3132525.3132545 · DOI
  11. [17][17] Richa Gupta, M. Balakrishnan, and P.V.M. Rao. 2017. Tactile Diagrams for the Visually Impaired. IEEE Potentials 36, 1 (Jan. 2017), 14–18. https://doi.org/10. 1109/MPOT.2016.2614754
  12. [19][19] Eric Hasper, Rogier A. Windhorst, Terri Hedgpeth, Leanne Van Tuyl, Ashleigh Gonzales, Britta Martinez, Hongyu Yu, Zoltan Farkas, and Debra P. Baluch. 2015. Methods for Creating and Evaluating 3D Tactile Images to Teach STEM Courses to the Visually Impaired. Journal of College Science Teaching 44, 6 (2015), 92–99.
  13. [20][20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep Residual Learning for Image Recognition. https://doi.org/10.48550/arXiv.1512.03385 arXiv:1512.03385 [cs] · DOI · Paper page
  14. [21][21] Xiaodong He and Li Deng. 2017. Deep Learning for Image-to-Text Generation: A Technical Overview. IEEE Signal Processing Magazine 34, 6 (Nov. 2017), 109–116. https://doi.org/10.1109/MSP.2017.2741510 · DOI
  15. [22][22] Leona Holloway, Kim Marriott, and Matthew Butler. 2018. Accessible Maps for the Blind: Comparing 3D Printed Models with Tactile Graphics. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi. org/10.1145/3173574.3173772 · DOI · Paper page
  16. [23][23] Mihail Ivanchev, Francis Zinke, and Ulrike Lucke. 2014. Pre-Journey Visualization of Travel Routes for the Blind on Refreshable Interactive Tactile Displays. In Computers Helping People with Special Needs (Lecture Notes in Computer Science), Klaus Miesenberger, Deborah Fels, Dominique Archambault, Petr Peňáz, and Wolfgang Zagler (Eds.). Springer International Publishing, Cham, 81–88. https: //doi.org/10.1007/978-3-319-08599-9_13 · DOI
  17. [25][25] Matthias Kümmerer, Lucas Theis, and Matthias Bethge. 2015. Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet. https: //doi.org/10.48550/arXiv.1411.1045 arXiv:1411.1045 [cs, q-bio, stat] · DOI
  18. [26][26] Jaewook Lee, Jaylin Herskovitz, Yi-Hao Peng, and Anhong Guo. 2022. Im- ageExplorer: Multi-Layered Touch Exploration to Encourage Skepticism To- wards Imperfect AI-Generated Image Captions. In CHI Conference on Human Factors in Computing Systems. ACM, New Orleans, LA, USA, 1–15. https: //doi.org/10.1145/3491102.3501966 · DOI · Paper page
  19. [29][29] Ying Lv, Wujie Zhou, Jingsheng Lei, Lv Ye, and Ting Luo. 2019. Attention-Based Fusion Network for Human Eye-Fixation Prediction in 3D Images. Optics Express 27, 23 (Nov. 2019), 34056–34066. https://doi.org/10.1364/OE.27.034056 · DOI
  20. [31][31] Haley MacLeod, Cynthia L. Bennett, Meredith Ringel Morris, and Edward Cutrell. 2017. Understanding Blind People's Experiences with Computer-Generated Captions of Social Media Images. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). Association for Computing Machinery, New York, NY, USA, 5988–5999. https://doi.org/10.1145/3025453. 3025814 · DOI
  21. [32][32] Karen McCall and Beverly Chagnon. 2022. Rethinking Alt Text to Improve Its Effectiveness. In Computers Helping People with Special Needs (Lecture Notes in Computer Science), Klaus Miesenberger, Georgios Kouroupetroglou, Katerina Mavrou, Roberto Manduchi, Mario Covarrubias Rodriguez, and Petr Penáz (Eds.). Springer International Publishing, Cham, 26–33. https://doi.org/10.1007/978-3- 031-08645-8_4 · DOI
  22. [33][33] Tom McEwan and Ben Weerts. 2007. ALT Text and Basic Accessibility. In Pro- ceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...but Not as We Know It - Volume 2 (BCS-HCI '07). BCS Learning & Develop- ment Ltd., Swindon, GBR, 71–74.
  23. [34][34] Chiara Meneghetti, Erika Borella, Irene Grasso, and Rossana De Beni. 2012. Learning a Route Using a Map and/or Description in Young and Older Adults. Journal of Cognitive Psychology 24, 2 (March 2012), 165–178. https://doi.org/10. 1080/20445911.2011.603694

Cited by

Sorted by citations

Authors
Nair, V., Zhu, H. '., & Smith, B. A.
Affiliations
Columbia University (US)
ORCID
Venue
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2023-04-19
Pages
1-17
DOI
10.1145/3544548.3581302
OpenAlex
W4321471976
Cited by
29
References
9
Citations by year
2026: 6 · 2025: 8 · 2024: 15
FWCI
12.16
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
  • Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
Keywords
blind and visually impaired users, alt text, accessibility, scaffolding, assistive technology, image understanding, user agency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review