Nonvisual Interaction Techniques at the Keyboard Surface
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Web user interfaces today leverage many common GUI design patterns, including navigation bars and menus (hierarchical structure), tabular content presentation, and scrolling. These visual-spatial cues enhance the interaction experience of sighted users. However, the linear nature of screen translation tools currently available to blind users make it difficult to understand or navigate these structures. We introduce Spatial Region Interaction Techniques (SPRITEs) for nonvisual access: a novel method for navigating two-dimensional structures using the keyboard surface. SPRITEs 1) preserve spatial layout, 2) enable bimanual interaction, and 3) improve the end user experience. We used a series of design probes to explore different methods for keyboard surface interaction. Our evaluation of SPRITEs shows that three times as many participants were able to complete spatial tasks with SPRITEs than with their preferred current technology.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Common issues include inaccessible table formats [36], inadequate image descriptions that lack important visual context [19], insufficient navigation aids [29], and confusing or unclear explanations [13,16].”
From From Cluttered to Clear: Improving the Web Accessibility Design for Screen Reader Users in E-commerce With Generative AI (2025) · cites this paper as [29]
Citations
Cites
Laws of organization in perceptual forms.
Wertheimer, M. (1938) · A source book of Gestalt psychology. · doi:10.1037/11496-005 · Cited by 1,493
Direct Manipulation Interfaces
Hutchins, E. L., Hollan, J. D., & Norman, D. A. (1985) · Human–Computer Interaction · doi:10.1207/s15327051hci0104_2 · Cited by 1,307
(2013) · Practical Psychology · doi:10.4324/9781315009810-7 · Cited by 1,066
Gestalt Psychology: An Introduction to New Concepts in Modern Psychology
Köhler, W. (1970) · Medical Entomology and Zoology · Cited by 934
Tactons: structured tactile messages for non-visual information display
Brewster, S. A., & Brown, L. M. (2004) · Enlighten: Publications (University of Glasgow) · Cited by 697
Information foraging in information access environments
Pirolli, P., & Card, S. (1995) · Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '95 · doi:10.1145/223904.223911 · Cited by 464 · cited 1×
“The success of GUIs is much studied, and the best direct manipulation interfaces [14] support the brain’s ability to forage for information [9,24,25], recognize rather than recall information [6], form mental models [12], and make connections between related items [4,7].”
BackgroundUsing information scent to model user information needs and actions and the Web
Chi, E. H., Pirolli, P., Chen, K., & Pitkow, J. (2001) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/365024.365325 · Cited by 418
What Frustrates Screen Reader Users on the Web: A Study of 100 Blind Users
Lazar, J., Allen, A., Kleinman, J., & Malarkey, C. (2007) · International Journal of Human-Computer Interaction · doi:10.1080/10447310709336964 · Cited by 354 · cited 3×
“The two-dimensional visual structures that websites rely on to convey information are easily lost in translation, because presenting spatial and visual cues are challenging to represent in a linear format [ e.g., 3,20,22,23,27].”
BackgroundInternet browsing and searching: User evaluations of category map and concept space techniques
Chen, H., Houston, A. L., Sewell, R. R., & Schatz, B. R. (1998) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(1998)49:7<582::aid-asi2>3.0.co;2-v · Cited by 300
ScentTrails: Integrating browsing and searching on the Web
Olston, C., & Chi, E. H. (2003) · ACM Transactions on Computer-Human Interaction · doi:10.1145/937549.937550 · Cited by 249
More than meets the eye: a survey of screen-reader browsing strategies
Borodin, Y., Bigham, J. P., Dausch, G., & Ramakrishnan, I. V. (2010) · Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1806005 · Cited by 222
Manual and cognitive benefits of two-handed input: an experimental study
Leganchuk, A., Zhai, S., & Buxton, W. (1998) · ACM Transactions on Computer-Human Interaction · doi:10.1145/300520.300522 · Cited by 184
Designing Large Sets of Haptic Icons with Rhythm
Ternes, D., & MacLean, K. E. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-69057-3_24 · Cited by 163
Nonvisual presentation of graphical user interfaces: contrasting two approaches
Mynatt, E. D., & Weber, G. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191732 · Cited by 156 · cited 2×
“The two-dimensional visual structures that websites rely on to convey information are easily lost in translation, because presenting spatial and visual cues are challenging to represent in a linear format [ e.g., 3,20,22,23,27].”
BackgroundAccess overlays: improving non-visual access to large touch screens for blind users
Kane, S. K., Morris, M. R., Perkins, A. Z., Wigdor, D., Ladner, R. E., & Wobbrock, J. O. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047232 · Cited by 127 · cited 2×
“Past work has shown that for nonvisual interaction, edge projection carries similar benefits [16].”
BackgroundExtra-ordinary human-computer interaction: interfaces for users with disabilities
Edwards, A. D. N. (1995) · Cambridge University Press eBooks · Cited by 109
Touchplates: low-cost tactile overlays for visually impaired touch screen users
Kane, S. K., Morris, M. R., & Wobbrock, J. O. (2013) · Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/2513383.2513442 · Cited by 107
FingerReader: A Wearable Device to Explore Printed Text on the Go
Shilkrot, R., Huber, J., Meng Ee, W., Maes, P., & Nanayakkara, S. C. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702421 · Cited by 96
Mapping GUIs to auditory interfaces
Mynatt, E. D., & Edwards, W. K. (1992) · Proceedings of the 5th annual ACM symposium on User interface software and technology · doi:10.1145/142621.142629 · Cited by 96
V-braille: haptic braille perception using a touch-screen and vibration on mobile phones
Jayant, C., Acuario, C., Johnson, W., Hollier, J., & Ladner, R. (2010) · Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1878803.1878878 · Cited by 89
A tactile windowing system for blind users
Prescher, D., Weber, G., & Spindler, M. (2010) · Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1878803.1878821 · Cited by 73
Towards developing assistive haptic feedback for visually impaired internet users
Kuber, R., Yu, W., & McAllister, G. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240854 · Cited by 58
Chang, D., Nesbitt, K. V., & Wilkins, K. (2007) · Cited by 58
Emacspeak---a speech interface
Raman, T. V. (1996) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/238386.238405 · Cited by 54
Type-hover-swipe in 96 bytes: a motion sensing mechanical keyboard
Taylor, S., Keskin, C., Hilliges, O., Izadi, S., & Helmes, J. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557030 · Cited by 51
Interactive Audio-haptic Map Explorer on a Tactile Display
Zeng, L., Miao, M., & Weber, G. (2015) · Interacting with Computers · doi:10.1093/iwc/iwu006 · Cited by 43
The Tangible Desktop: A Multimodal Approach to Nonvisual Computing
Baldwin, M. S., Hayes, G. R., Haimson, O. L., Mankoff, J., & Hudson, S. E. (2017) · ACM Transactions on Accessible Computing · doi:10.1145/3075222 · Cited by 37
Self-citationAn architecture for transforming graphical interfaces
Edwards, W. K., & Mynatt, E. D. (1994) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/192426.192443 · Cited by 34
Haptic glyphs (hlyphs) - structured haptic objects for haptic visualization
Roberts, J. C., & Franklin, K. M. (2005) · doi:10.1109/whc.2005.68 · Cited by 30
GestKeyboard: enabling gesture-based interaction on ordinary physical keyboard
Zhang, H., & Li, Y. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557362 · Cited by 29
Non-visual Web Browsing: Beyond Web Accessibility
Ramakrishnan, I. V., Ashok, V., & Billah, S. M. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-58703-5_24 · Cited by 21
Assistive web browsing with touch interfaces
Ahmed, F., Islam, M. A., Borodin, Y., & Ramakrishnan, I. V. (2010) · Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1878803.1878848 · Cited by 16
Perception and Musical Preferences in Wishart's Work
Brattico, E., & Sassanelli, F. (2000) · Journal of New Music Research · doi:10.1076/jnmr.29.2.107.3099 · Cited by 13
Keyboard Surface Interaction: Making the keyboard into a pointing devicePreprint
Ramos, J., Li, Z., Rosas, J. M., Banović, N., Mankoff, J. C., & Dey, A. K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.04029 · Cited by 10
Self-citationThe importance of models in marking complex
Fischer, G. (1989) · Cited by 5
Research works printed but not linked: 2
- 10.5555/578027 · doi:10.5555/578027
- 10.5555/215596 · doi:10.5555/215596
Cited by
Das, M., Gergle, D., & Piper, A. M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359293 · Cited by 108
“, auditory [38, 73], speech-controlled [9, 86], haptic [16, 48] and spatial [44] interactions), personalized and opportunistic adaptations [8, 25, 37, 100], semantic web modeling [8, 9, 16], shared browsing [68], and collaborative [40, 78] and automated [35] evaluation techniques.”
BackgroundExamining Visual Semantic Understanding in Blind and Low-Vision Technology Users
Potluri, V., Grindeland, T. E., Froehlich, J. E., & Mankoff, J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445040 · Cited by 42
Self-citationVERSE: Bridging Screen Readers and Voice Assistants for Enhanced Eyes-Free Web Search
Vtyurina, A., Fourney, A., Morris, M. R., Findlater, L., & White, R. W. (2019) · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3308561.3353773 · Cited by 42
Ehtesham-Ul-Haque, M., Monsur, S. M., & Billah, S. M. (2022) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3526113.3545620 · Cited by 36
A Multi-Modal Approach for Blind and Visually Impaired Developers to Edit Webpage Designs
Potluri, V., He, L., Chen, C., Froehlich, J. E., & Mankoff, J. C. (2019) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3308561.3354626 · Cited by 33
Self-citation
Metadata
- Authors
- Khurana, R., McIsaac, D., Lockerman, E., & Mankoff, J.
- Affiliations
- Carnegie Mellon University (US)
- University of Washington (US)
- Venue
- Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2018-04-19
- Pages
- 1-12
- OpenAlex
- W2796081525
- Cited by
- 34
- References
- 35
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 6 · 2023: 4 · 2022: 5 · 2021: 4 · 2020: 4 · 2019: 7
- FWCI
- 6.46
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- NIDILRR (90DP5004-01-00)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- blind users, screen readers, spatial navigation, bimanual interaction, web accessibility, design probes, sprites
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.