The DOI matches its Crossref record, with a small difference in title or year; 56 of the 58 references it cites were found in a registry or an index.
The Tangible Desktop: A Multimodal Approach to Nonvisual Computing
Abstract
Audio-only interfaces, facilitated through text-to-speech screen reading software, have been the primary mode of computer interaction for blind and low-vision computer users for more than four decades. During this time, the advances that have made visual interfaces faster and easier to use, from direct manipulation to skeuomorphic design, have not been paralleled in nonvisual computing environments. The screen reader–dependent community is left with no alternatives to engage with our rapidly advancing technological infrastructure. In this article, we describe our efforts to understand the problems that exist with audio-only interfaces. Based on observing screen reader use for 4 months at a computer training school for blind and low-vision adults, we identify three problem areas within audio-only interfaces: ephemerality, linear interaction, and unidirectional communication. We then evaluated a multimodal approach to computer interaction called the Tangible Desktop that addresses these problems by moving semantic information from the auditory to the tactile channel. Our evaluation demonstrated that among novice screen reader users, Tangible Desktop improved task completion times by an average of 6 minutes when compared to traditional audio-only computer systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As modern computer applications all have two-dimensional graphical user interfaces primarily designed for sighted interaction, there is an inherent mismatch with the one-dimensional navigation supported by a screen reader, thereby resulting in numerous usability issues as uncovered by the findings of prior studies [9, 30, 46, 49, 50].”
From Enabling Uniform Computer Interaction Experience for Blind Users through Large Language Models (2024) · cites this paper as [9]
In the citing paper
“Increasing tactile feedback and tangible interaction on touchscreens can improve their accessibility.”
From Interactiles: 3D Printed Tactile Interfaces to Enhance Mobile Touchscreen Accessibility (2018) · cites this paper as [4]
In the citing paper
“consisted of many hardware components that were difficult to organize or carry [4]”
From Interactiles: 3D Printed Tactile Interfaces to Enhance Mobile Touchscreen Accessibility (2018) · cites this paper as [4]
In the citing paper
“or not focused on mobile systems [4,20]”
From Interactiles: 3D Printed Tactile Interfaces to Enhance Mobile Touchscreen Accessibility (2018) · cites this paper as [4]
In the citing paper
“The absence of tactile cues such as keys and buttons makes touchscreens difficult to navigate for people with visual impairments.”
From Interactiles: 3D Printed Tactile Interfaces to Enhance Mobile Touchscreen Accessibility (2018) · cites this paper as [4]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Tangible bits: towards seamless interfaces between people, bits and atoms
Ishii, H., & Ullmer, B. (1997) · Proceedings of the ACM SIGCHI Conference on Human factors in computing systems · doi:10.1145/258549.258715 · Cited by 3,853 · cited 7×
“Individually, concepts such as positional constraints [32], haptics [9, 55], and physical icons [20] have been shown to be meaningful for nonvisual and sighted users alike.”
BackgroundMethodInfluentialHollan, J., Hutchins, E., & Kirsh, D. (2000) · ACM Transactions on Computer-Human Interaction · doi:10.1145/353485.353487 · Cited by 2,300
Nielsen, J. (1994) · Cited by 1,897
Earcons and Icons: Their Structure and Common Design Principles
Blattner, M., Sumikawa, D., & Greenberg, R. (1989) · Human-Computer Interaction · doi:10.1207/s15327051hci0401_1 · Cited by 980
"Constant, constant, multi-tasking craziness": managing multiple working spheres
González, V. M., & Mark, G. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985707 · Cited by 715
Tactons: structured tactile messages for non-visual information display
Brewster, S. A., & Brown, L. M. (2004) · Enlighten: Publications (University of Glasgow) · Cited by 697
As-rigid-as-possible shape manipulation
Igarashi, T., Moscovich, T., & Hughes, J. F. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073323 · Cited by 675
TeslaTouch: electrovibration for touch surfaces
Bau, O., Poupyrev, I., Israr, A., & Harrison, C. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866074 · Cited by 664
Tangible User Interfaces: Past, Present, and Future Directions
Shaer, O. (2009) · Foundations and Trends® in Human–Computer Interaction · doi:10.1561/1100000026 · Cited by 581
Tangible User Interfaces: Past, Present, and Future Directions
Shaer, O. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601983299 · Cited by 414
What Frustrates Screen Reader Users on the Web: A Study of 100 Blind Users
Lazar, J., Allen, A., Kleinman, J., & Malarkey, C. (2007) · International Journal of Human-Computer Interaction · doi:10.1080/10447310709336964 · Cited by 354
Empowering individuals with do-it-yourself assistive technology
Hurst, A., & Tobias, J. (2011) · The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/2049536.2049541 · Cited by 302
AIREAL: interactive tactile experiences in free air
Sodhi, R., Poupyrev, I., Glisson, M., & Israr, A. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2462007 · Cited by 276
Distributed cognition, representation, and affordance
Zhang, J., & Patel, V. L. (2006) · Pragmatics & Cognition · doi:10.1075/pc.14.2.12zha · Cited by 265
A First Investigation into the Effectiveness of Tactons
Brown, L. M., Brewster, S. A., & Purchase, H. C. (2005) · First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems · doi:10.1109/whc.2005.6 · Cited by 224
Review of research on the intelligibility and comprehension of accelerated speech.
Foulke, E., & Sticht, T. G. (1969) · Psychological Bulletin · doi:10.1037/h0027575 · Cited by 224
More than meets the eye: a survey of screen-reader browsing strategies
Borodin, Y., Bigham, J. P., Dausch, G., & Ramakrishnan, I. V. (2010) · Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1806005 · Cited by 222
Mankoff, J., Fait, H., & Tran, T. (2005) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1054972.1054979 · Cited by 207
Self-citationSLAP widgets: bridging the gap between virtual and physical controls on tabletops
Weiss, M., Wagner, J., Jansen, Y., Jennings, R., Khoshabeh, R., Hollan, J. D., & Borchers, J. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518779 · Cited by 196
Auditory memory in congenitally blind adults: a behavioral-electrophysiological investigation
Röder, B., Rösler, F., & Neville, H. J. (2001) · Cognitive Brain Research · doi:10.1016/s0926-6410(01)00002-7 · Cited by 181
Midas: fabricating custom capacitive touch sensors to prototype interactive objects
Savage, V., Zhang, X., & Hartmann, B. (2012) · Proceedings of the 25th annual ACM symposium on User interface software and technology · doi:10.1145/2380116.2380189 · Cited by 176
Distributed cognition, representation, and affordance
Zhang, J., & Patel, V. L. (2008) · Benjamins Current Topics · doi:10.1075/bct.16.11zha · Cited by 174
Bored mondays and focused afternoons: the rhythm of attention and online activity in the workplace
Mark, G. J., Iqbal, S. T., Czerwinski, M. P., & Johns, P. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557204 · Cited by 169
Zuckerman, O., & Gal-Oz, A. (2013) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2013.04.003 · Cited by 159
Nonvisual presentation of graphical user interfaces: contrasting two approaches
Mynatt, E. D., & Weber, G. (1994) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/191666.191732 · Cited by 156
Klatzky, R. L., & Lederman, S. J. (2003) · Wiley eBooks · doi:10.1002/0471264385.wei0406 · Cited by 150
Implementation of Assistive Technology with Students who are Visually Impaired: Teachers’ Readiness
Abner, G. H., & Lahm, E. A. (2002) · Journal of Visual Impairment & Blindness · doi:10.1177/0145482x0209600204 · Cited by 150
Tactile display for the visually impaired using TeslaTouch
Cheng, X., Israr, A., Poupyrev, I., Bau, O., & Harrison, C. (2011) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1979742.1979705 · Cited by 138
Why do i keep interrupting myself?: environment, habit and self-interruption
Dabbish, L., Mark, G., & González, V. M. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979405 · Cited by 132 · cited 1×
“As noted in prior work [10, 15, 27], distractions in the workplace can be problematic for any computing user.”
BackgroundWeiss, M., Wacharamanotham, C., Voelker, S., & Borchers, J. (2011) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2047196.2047277 · Cited by 122
AirWave: non-contact haptic feedback using air vortex rings
Gupta, S., Morris, D., Patel, S., & Tan, D. S. (2013) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2493432.2493463 · Cited by 109
Zhou, L., Parker, A. T., Smith, D. W., & Griffin-Shirley, N. (2011) · Journal of Visual Impairment & Blindness · doi:10.1177/0145482x1110500402 · Cited by 104
Accessibility: a Web engineering approach
Plessers, P., Casteleyn, S., Yeşilada, Y., Troyer, O. D., Stevens, R., Harper, S., & Goble, C. (2005) · International World Wide Web Conference (WWW) · doi:10.1145/1060745.1060799 · Cited by 92
As-rigid-as-possible shape manipulation
(2005) · doi:10.1145/1186822.1073323 · Cited by 87
Transforming Graphical Interfaces Into Auditory Interfaces for Blind Users
Mynatt, E. D. (1997) · Human-Computer Interaction · doi:10.1207/s15327051hci1201&2_2 · Cited by 82
Preparing for Future Learning with a Tangible User Interface: The Case of Neuroscience
Schneider, B., Wallace, J. L., Blikstein, P., & Pea, R. D. (2013) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2013.15 · Cited by 80
A tactile windowing system for blind users
Prescher, D., Weber, G., & Spindler, M. (2010) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1878803.1878821 · Cited by 75
Savidis, A., Stephanidis, C., Korte, A., Crispien, K., & Fellbaum, K. (1996) · doi:10.1145/228347.228366 · Cited by 74
The design of auditory user interfaces for blind users
Donker, H., Klante, P., & Gorny, P. (2002) · Nordic Conference on Human-Computer Interaction · doi:10.1145/572020.572038 · Cited by 73
Yeşilada, Y., Harper, S., Goble, C., & Stevens, R. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-27834-4_55 · Cited by 54
Emacspeak---a speech interface
Raman, T. V. (1996) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/238386.238405 · Cited by 54
(1998) · doi:10.1145/288392.288597 · Cited by 48
Auditory and tactile interfaces for representing the visual effects on the web
Asakawa, C., Takagi, H., Ino, S., & Ifukube, T. (2002) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/638249.638263 · Cited by 36
Web accessibility as a side effect
Richards, J. T., Montague, K., & Hanson, V. L. (2012) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2384916.2384931 · Cited by 35
(2002) · doi:10.1145/638249.638260 · Cited by 33
Challenging information foraging theory
Vigo, M., & Harper, S. (2013) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/2481492.2481499 · Cited by 28
The Effectiveness of Haptic Cues as an Assistive Technology for Human Memory
Kuznetsov, S., Dey, A. K., & Hudson, S. E. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-01516-8_12 · Cited by 22
Self-citationA snapshot of the first encounters of visually disabled users with the Web
Vigo, M., & Harper, S. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.01.045 · Cited by 20 · cited 1×
“The large corpus of input commands required to operate audio interfaces impose significant hurdles as users first familiarize themselves, only to become a constraint on efficiency with increasing expertise [47].”
BackgroundRepresenting Graphical User Interfaces with Sound: A Review of Approaches
Ratanasit, D., & Moore, M. M. (2005) · Journal of Visual Impairment & Blindness · doi:10.1177/0145482x0509900202 · Cited by 19
Spatial gestures using a tactile-proprioceptive display
Folmer, E., & Morelli, T. (2012) · International Conference on Tangible, Embedded, and Embodied Interaction (TEI) · doi:10.1145/2148131.2148161 · Cited by 17
Graphical user interfaces and visually disabled users
(1995) · doi:10.1109/86.372894 · Cited by 16
A survey of blind users on the usability of email applications
Wentz, B., Hochheiser, H., & Lazar, J. (2013) · Universal Access in the Information Society · doi:10.1007/s10209-012-0285-9 · Cited by 11
Tactile Web Browsing for Blind Users
Kuber, R., Yu, W. M., & O'Modhrain, M. S. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15841-4_9 · Cited by 3
Research works printed but not linked: 3
- Ängeslevä Jussi, 2003, Body mnemonics portable device interaction design concept, Proceedings of UIST, 3, 2
- Brewster Stephen, Proceedings of the Australasian User Interface Conference (AUIC’04)
- Vitense Holly S., Proceedings of the 5th International ACM Conference on Assistive Technologies. ACM, 49–56
Cited by
A Place to Play: The (Dis)Abled Embodied Experience for Autistic Children in Online Spaces
Ringland, K. E. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300518 · Cited by 93
Editing spatial layouts through tactile templates for people with visual impairments.
Li, J., Kim, S., Miele, J. A., Agrawala, M., & Follmer, S. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300436 · Cited by 62
“They have employed 3D printing to let blind users access works of art [39], STEM education [11, 20], and traditional WIMP desktop interface elements [4].”
MethodFunctional programming for compiling and decompiling computer-aided design
Nandi, C., Wilcox, J. R., Panchekha, P., Blau, T., Grossman, D., & Tatlock, Z. (2018) · Proceedings of the ACM on Programming Languages · doi:10.1145/3236794 · Cited by 64
Li, F. M., Chen, D. L., Fan, M., & Truong, K. N. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445321 · Cited by 32
Examining Visual Semantic Understanding in Blind and Low-Vision Technology Users
Potluri, V., Grindeland, T. E., Froehlich, J. E., & Mankoff, J. C. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445040 · Cited by 52
“For example, studies of beginning users highlight challenges with understanding visual context, such as when the cursor has switched to a new application [8].”
BackgroundSelf-citation
Metadata
- Authors
- Baldwin, M. S., Hayes, G. R., Haimson, O. L., Mankoff, J., & Hudson, S. E.
- Affiliations
- University of California, Irvine (US)
- Carnegie Mellon University (US)
- Venue
- ACM Transactions on Accessible Computing
- Published
- 2017-08-11
- Volume
- 10
- Issue
- 3
- Pages
- 1-28
- ISSN
- 1936-7228 (ISSN-L)
- 1936-7236
- OpenAlex
- W2744896498
- Cited by
- 37
- References
- 58
- Citations by year
- 2026: 1 · 2025: 7 · 2024: 6 · 2023: 1 · 2022: 3 · 2021: 4 · 2020: 5 · 2019: 6 · 2018: 3 · 2017: 1
- FWCI
- 2.35
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Accessible Computing (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Keywords
- screen readers, blind and visually impaired users, tactile feedback, multimodal interaction, transience, linear interaction, one-way communication, assistive technology, task completion time
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.