Taming User-Interface Heterogeneity with Uniform Overlays for Blind Users
The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
For many blind users, interaction with computer applications using screen reader assistive technology is a frustrating and time-consuming affair, mostly due to the complexity and heterogeneity of applications’ user interfaces. An interview study revealed that many applications do not adequately convey their interface structure and controls to blind screen reader users, thereby placing additional burden on these users to acquire this knowledge on their own. This is often an arduous and tedious learning process given the one-dimensional navigation paradigm of screen readers. Moreover, blind users have to repeat this learning process multiple times, i.e., once for each application, since applications differ in their interface designs and implementations. In this paper, we propose a novel push-based approach to make non-visual computer interaction easy, efficient, and uniform across different applications. The key idea is to make screen reader interaction ‘structure-agnostic’, by automatically identifying and extracting all application controls and then instantly ‘pushing’ these controls on demand to the blind user via a custom overlay dashboard interface. Such a custom overlay facilitates uniform and efficient screen reader navigation across all applications. A user study showed significant improvement in user satisfaction and interaction efficiency with our approach compared to a state-of-the-art screen reader.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“with research indicating that they are frequently ten times slower than sighted individuals when performing similar computer tasks [13, 26, 46].”
From Enabling Uniform Computer Interaction Experience for Blind Users through Large Language Models (2024) · cites this paper as [46]
In the citing paper
“Moreover, sequential navigation of controls in the popup still requires considerable time and shortcut presses, especially in complex applications such as productivity tools that have hundreds of controls.”
From Enabling Uniform Computer Interaction Experience for Blind Users through Large Language Models (2024) · cites this paper as [46]
In the citing paper
“As modern computer applications all have two-dimensional graphical user interfaces primarily designed for sighted interaction, there is an inherent mismatch with the one-dimensional navigation supported by a screen reader, thereby resulting in numerous usability issues as uncovered by the findings of prior studies [9, 30, 46, 49, 50].”
From Enabling Uniform Computer Interaction Experience for Blind Users through Large Language Models (2024) · cites this paper as [46]
Citations
Cites
Kane, S. K., Jayant, C., Wobbrock, J. O., & Ladner, R. E. (2009) · Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1639642.1639663 · Cited by 383 · cited 1×
“ence for blind screen reader users have predominantly focused on web andmobile applications [6, 25, 27, 30], whereas desktop applications have been largely overlooked.”
BackgroundWhat Frustrates Screen Reader Users on the Web: A Study of 100 Blind Users
Lazar, J., Allen, A., Kleinman, J., & Malarkey, C. (2007) · International Journal of Human-Computer Interaction · doi:10.1080/10447310709336964 · Cited by 354 · cited 1×
“ence for blind screen reader users have predominantly focused on web andmobile applications [6, 25, 27, 30], whereas desktop applications have been largely overlooked.”
BackgroundComplete Counterbalancing of Immediate Sequential Effects in a Latin Square Design
Bradley, J. V. (1958) · Journal of the American Statistical Association · doi:10.1080/01621459.1958.10501456 · Cited by 343
More than meets the eye: a survey of screen-reader browsing strategies
Borodin, Y., Bigham, J. P., Dausch, G., & Ramakrishnan, I. V. (2010) · Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1806005 · Cited by 222
Self-citationUnblind your apps: predicting natural-language labels for mobile GUI components by deep learning
Chen, J., Chen, C., Xing, Z., Xu, X., Zhu, L., Li, G., & Wang, J. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380327 · Cited by 182
Interacting with mobile devices via VoiceOver: usability and accessibility issues
Leporini, B., Buzzi, M. C., & Buzzi, M. (2012) · Proceedings of the 24th Australian Computer-Human Interaction Conference · doi:10.1145/2414536.2414591 · Cited by 116
Dynamic Braille display using SMA coil actuator and magnetic latch
Haga, Y., Makishi, W., Iwami, K., Totsu, K., Nakamura, K., & Esashi, M. (2004) · Sensors and Actuators A Physical · doi:10.1016/j.sna.2004.10.001 · Cited by 95
A novel integrable microvalve for refreshable braille display system
Yobaş, L., Durand, D. M., Skebe, G. G., Lisy, F. J., & Huff, M. A. (2003) · Journal of Microelectromechanical Systems · doi:10.1109/jmems.2003.811754 · Cited by 94
The Graphical User Interface: Crisis, Danger, and Opportunity
Boyd, L. H., Boyd, W. L., & Vanderheiden, G. C. (1990) · Journal of Visual Impairment & Blindness · doi:10.1177/0145482x9008401002 · Cited by 85
Observing Sara: a case study of a blind person's interactions with technology
Shinohara, K., & Tenenberg, J. (2007) · Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1296843.1296873 · Cited by 59
Research works printed but not linked: 13
- Maria Claudia Buzzi, Marina Buzzi, Barbara Leporini, and Giulio Mori. 2012. Designing e-learning collaborative tools for blind people. E-Learning-Long-Distance and Lifelong Perspectives (2012), 125–144.
- Michael F Chiang, Roy G Cole, Suhit Gupta, Gail E Kaiser, and Justin B Starren. 2005. Computer and world wide web accessibility by visually disabled patients: Problems and solutions. Survey of ophthalmology 50, 4 (2005), 394–405.
- Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. Vol. 52. Elsevier, 139–183.
- 10.5555/1839214.1839217 · doi:10.5555/1839214.1839217
- Sri Hastuti Kurniawan Alistair G Sutcliffe and Paul Blenkhorn. 2003. How Blind Users’ Mental Models Affect Their Perceived Usability of an Unfamiliar Screen Reader.. In INTERACT Vol. 3. 631–638.
- Vladimir I Levenshtein. 1966. Binary codes capable of correcting deletions insertions and reversals. In Soviet physics doklady Vol. 10. 707–710.
- Giulio Mori, Maria Claudia Buzzi, Marina Buzzi, Barbara Leporini, and Victor MR Penichet. 2010. Making “Google Docs” user interface more accessible for blind people. In International Conference on Advances in New Technologies, Interactive Interfaces, and Communicability. Springer, 20–29.
- John Gerard Schoeberlein and Yuanqiong Wang. 2014. Usability Evaluation of an Accessible Collaborative Writing Prototype for Blind Users.Journal of Usability Studies 10, 1 (2014).
- Reeta Singh. 2012. Blind Handicapped Vs. Technology: How do Blind People use Computers?International Journal of Scientific & Engineering Research 3 4(2012) 1–7.
- Katie Wang, Laura G Barron, and Michelle R Hebl. 2010. Making those who cannot see look best: Effects of visual resume formatting on ratings of job applicants with blindness.Rehabilitation psychology 55, 1 (2010), 68.
- David Wicks. 2017. The coding manual for qualitative researchers. Qualitative research in organizations and management: an international journal (2017).
- Cheng Xu Ali Israr Ivan Poupyrev Olivier Bau and Chris Harrison. 2011. Tactile display for the visually impaired using TeslaTouch. In CHI’11 Extended Abstracts on Human Factors in Computing Systems. 317–322.
- Zoom. 2021. Zoom Meetings. https://explore.zoom.us/meetings.
Cited by
Enabling Uniform Computer Interaction Experience for Blind Users through Large Language Models
Kodandaram, S. R., Uckun, U., Bi, X., Ramakrishnan, IV, & Ashok, V. (2024) · The 26th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663548.3675605 · Cited by 24
Self-citationPerera, M., Ananthanarayan, S., Goncu, C., & Marriott, K. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713634 · Cited by 14
“…have aimed to improve the accessibility and usability of productivity applications for blind users by introducing feature enhancements [64, 91], interface customization [37, 64, 84, 90, 100, 114], auditory feedback [64, 109, 110], and multi-modal and interactive interfaces [35, 38, 117, 121, 122].”
Smale, M. C., Fox, J. D., & Fox, A. K. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108405 · Cited by 13
Screen Reading Enabled by Large Language Models
Ghosh, A., Reddy, M. P., Kodandaram, S. R., Uckun, U., Ashok, V. G., Bi, X., & Ramakrishnan, I. V. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3688491 · Cited by 8
Self-citationBeyond Bridging Divides: Examining the Goals of Digital Inclusion Practice in Post-Digital Societies
Parnaby, A. W., Kharrufa, A., & Crivellaro, C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713902 · Cited by 6
Metadata
- Authors
- Uckun, U., Tumkur Suresh, R., Ferdous, M. J., Bi, X., Ramakrishnan, I. V., & Ashok, V.
- Affiliations
- Stony Brook University (US)
- Stony Brook University, USA
- Old Dominion University (US)
- Venue
- Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization
- Publisher
- ACM
- Published
- 2022-07-04
- Pages
- 212-222
- OpenAlex
- W4283777838
- Cited by
- 7
- References
- 28
- Citations by year
- 2025: 4 · 2024: 3
- FWCI
- 2.55
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on User Modeling, Adaptation and Personalization (UMAP) (publishedVersion)
- Funders
- NIH (National Institutes of Health) (R01EY030085, R01HD097188)
- National Science Foundation (1805076, 1815514, 1936027, 2113485, 2125147)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Keywords
- blind users, screen readers, assistive technology, non-visual interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Taming User-Interface Heterogeneity with Uniform Overlays for Blind Users. https://app.cochat.ai/papers/10.1145/3503252.3531317
- The paper
- Uckun, U., Tumkur Suresh, R., Ferdous, M. J., Bi, X., Ramakrishnan, I. V., & Ashok, V. (2022). Taming User-Interface Heterogeneity with Uniform Overlays for Blind Users. Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization. https://doi.org/10.1145/3503252.3531317
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.