Developing Dynamic Audio Navigation UIs to Pinpoint Elements in Tactile Graphics
Open accessThe DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
Access to complex graphical information is essential when connecting blind and visually impaired (BVI) people with the world. Tactile graphics readers enable access to graphical data through audio-tactile user interfaces (UIs), but these have yet to mature. A challenging task for blind people is locating specific elements–areas in detailed tactile graphics. To this end, we developed three audio navigation UIs that dynamically guide the user’s hand to a specific position using audio feedback. One is based on submarine sonar sounds, another relies on the target’s coordinate plan x and y-axis, and the last uses direct voice instructions. The UIs were implemented in the Tactonom Reader device, a new tactile graphic reader that enhances swell paper graphics with pinpointed audio explanations. To evaluate the effectiveness of the three different dynamic navigation UIs, we conducted a within-subject usability test that involved 13 BVI participants. Beyond comparing the effectiveness of the different UIs, we observed and recorded the interaction of the visually impaired participants with the different navigation UI to further investigate their behavioral patterns during the interaction. We observed that user interfaces that required the user to move their hand in a straight direction were more likely to provoke frustration and were often perceived as challenging for blind and visually impaired people. The analysis revealed that the voice-based navigation UI guides the participant the fastest to the target and does not require prior training. This suggests that a voice-based navigation strategy is a promising approach for designing an accessible user interface for the blind.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“People with BVI have highlighted the need for an assistive interface to pinpoint elements on tactile surfaces (9, 20–24).”
From SONOICE! a Sonar–Voice dynamic user interface for assisting individuals with blindness and visual impairment in pinpointing elements in 2D tactile readers (2024) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“Past research (22) has revealed that a Voice-based is more efficient and effective in assisting users in pinpointing elements in tactile graphics readers.”
From SONOICE! a Sonar–Voice dynamic user interface for assisting individuals with blindness and visual impairment in pinpointing elements in 2D tactile readers (2024) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“While this strategy has the potential to give more details about the target location, it requires that users move their hands in a straight line through the axis, which is a difficult task for people with BVI (22).”
From SONOICE! a Sonar–Voice dynamic user interface for assisting individuals with blindness and visual impairment in pinpointing elements in 2D tactile readers (2024) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“However, users revealed dissatisfaction with the repetitive and potentially irritating nature of using voice-based feedback (22, 24, 35, 38, 39).”
From SONOICE! a Sonar–Voice dynamic user interface for assisting individuals with blindness and visual impairment in pinpointing elements in 2D tactile readers (2024) · cites this paper as [22]
The cited source says what the claim attributes to it.
In the citing paper
“In addition to addressing the issue of moving the fingertip in a straight direction, which was present in the previous study (22), the Sonoice method offers further usefulness.”
From SONOICE! a Sonar–Voice dynamic user interface for assisting individuals with blindness and visual impairment in pinpointing elements in 2D tactile readers (2024) · cites this paper as [22]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
(2003) · doi:10.1145/642611 · Cited by 905
The Midas Touch: The Effects of Interpersonal Touch on Restaurant Tipping
Crusco, A. H., & Wetzel, C. G. (1984) · Personality and Social Psychology Bulletin · doi:10.1177/0146167284104003 · Cited by 473
Interactivity improves usability of geographic maps for visually impaired people
Brock, A. M., Truillet, P., Oriola, B., Picard, D., & Jouffrais, C. (2015) · Human–Computer Interaction · doi:10.1080/07370024.2014.924412 · Cited by 195
Access overlays: improving non-visual access to large touch screens for blind users
Kane, S. K., Morris, M. R., Perkins, A. Z., Wigdor, D., Ladner, R. E., & Wobbrock, J. O. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047232 · Cited by 127
Analysis of navigability of web applications for improving blind usability
Takagi, H., Saito, S., Fukuda, K., & Asakawa, C. (2007) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1279700.1279703 · Cited by 116
Gravvitas: generic multi-touch presentation of accessible graphics
Goncu, C., & Marriott, K. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-23774-4_5 · Cited by 112
Tactile Graphics Revised: The Novel BrailleDis 9000 Pin-Matrix Device with Multitouch Input
Völkel, T., Weber, G., & Baumann, U. A. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-70540-6_124 · Cited by 98
The Tactile Graphics Helper: Providing Audio Clarification for Tactile Graphics Using Machine Vision
Fusco, G., & Morash, V. S. (2015) · Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility - ASSETS '15 · doi:10.1145/2700648.2809868 · Cited by 91
LucentMaps: 3D Printed Audiovisual Tactile Maps for Blind and Visually Impaired People
Götzelmann, T. (2016) · Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/2982142.2982163 · Cited by 86
Helping visually impaired users properly aim a camera.
Vázquez, M., & Steinfeld, A. (2012) · Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/2384916.2384934 · Cited by 81
Refreshable tactile graphics applied to schoolbook illustrations for students with visual impairment
Petit, G., Dufresne, A., Lévesque, V., Vincent Hayward, B., & Trudeau, N. (2008) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1414471.1414489 · Cited by 77
A tactile windowing system for blind users
Prescher, D., Weber, G., & Spindler, M. (2010) · Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1878803.1878821 · Cited by 73
Speech technologies for blind and low vision persons
Freitas, D., & Κουρουπέτρογλου, Γ. (2008) · Technology and Disability · doi:10.3233/tad-2008-20208 · Cited by 70
Melfi, G., Müller, K., Schwarz, T., Jaworek, G., & Stiefelhagen, R. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376508 · Cited by 63
Editing spatial layouts through tactile templates for people with visual impairments.
Li, J., Kim, S., Miele, J. A., Agrawala, M., & Follmer, S. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300436 · Cited by 62
Tactile graphics with a voice: using QR codes to access text in tactile graphics
Baker, C. M., Milne, L. R., Scofield, J., Bennett, C. L., & Ladner, R. E. (2014) · Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility - ASSETS '14 · doi:10.1145/2661334.2661366 · Cited by 51
Collaborative Creation of Digital Tactile Graphics
Bornschein, J., Prescher, D., & Weber, G. (2015) · Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility - ASSETS '15 · doi:10.1145/2700648.2809869 · Cited by 49
Interactive audio-tactile maps for visually impaired people
Brock, A., & Jouffrais, C. (2015) · ACM SIGACCESS Accessibility and Computing · doi:10.1145/2850440.2850441 · Cited by 49
(2001) · Information technology and disabilities · Cited by 47
A mobile phone application enabling visually impaired users to find and read product barcodes.
Tekin, E., & Coughlan, J. M. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-14100-3_43 · Cited by 43
SmartTactMaps: a smartphone-based approach to support blind persons in exploring tactile maps
Götzelmann, T., & Winkler, K. (2015) · Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments · doi:10.1145/2769493.2769497 · Cited by 33
Accessible on-line floor plans.
Goncu, C., Madugalla, A., Marinai, S., & Marriott, K. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741660 · Cited by 31
Touching graphical applications: bimanual tactile interaction on the HyperBraille pin-matrix display
Prescher, D., Bornschein, J., Köhlmann, W., & Weber, G. (2017) · Universal Access in the Information Society · doi:10.1007/s10209-017-0538-8 · Cited by 29
Fingersight: Fingertip haptic sensing of the visual environment
Horvath, S., Galeotti, J., Wu, B., Klatzky, R., Siegel, M., & Stetten, G. (2014) · IEEE Journal of Translational Engineering in Health and Medicine · doi:10.1109/jtehm.2014.2309343 · Cited by 28
Holdable haptic device for 4-dof motion guidance.
Walker, J. M., Zemiti, N., Poignet, P., & Okamura, A. M. (2019) · 2019 IEEE World Haptics Conference (WHC) · doi:10.1109/whc.2019.8816171 · Cited by 28
Pantoguide: a haptic and audio guidance system to support tactile graphics exploration.
Chase, E. D. Z., Siu, A. F., Boadi-Agyemang, A., Kim, G. S.-H., Gonzalez, E. J., & Follmer, S. (2020) · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3373625.3418023 · Cited by 25
Guideband: Intuitive 3d multilevel force guidance on a wristband in virtual reality.
Tsai, H.-R., Chang, Y.-C., Wei, T.-Y., Tsao, C.-A., Koo, X. C.-Y., Wang, H.-C., & Chen, B.-Y. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445262 · Cited by 25
(2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-41264-1 · Cited by 22
Radecki, A., Bujacz, M., Skulimowski, P., & Strumiłło, P. (2020) · British Journal of Educational Technology · doi:10.1111/bjet.12852 · Cited by 20
Jagdish, D., Sawhney, R., Gupta, M., & Nangia, S. (2008) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1378773.1378824 · Cited by 17
Creating accessible online floor plans for visually impaired readers
Madugalla, A., Marriott, K., Marinai, S., Capobianco, S., & Goncu, C. (2020) · ACM Transactions on Accessible Computing · doi:10.1145/3410446 · Cited by 17
Redesigning Input Controls of a Touch-Sensitive Pin-Matrix Device
Prescher, D. (2014) · Cited by 16
FluxMarker: Enhancing Tactile Graphics with Dynamic Tactile MarkersPreprint
Suzuki, R., Stangl, A. J., Gross, M. D., & YEH, T. P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.03783 · Cited by 15
(2022) · Lecture notes in computer science · doi:10.1007/978-3-031-08648-9 · Cited by 15
Touchpen: rich interaction technique for audio-tactile charts by means of digital pens.
Engel, C., Konrad, N., & Weber, G. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58796-3_52 · Cited by 11
BrailleIO - A Tactile Display Abstraction Framework.
Bornschein, J. (2014) · Cited by 7
Miesenberger, K., Klaus, J., & Zagler, W. L. (2002) · Lecture notes in computer science · doi:10.1007/11788713 · Cited by 6
Locating Widgets in Different Tactile Information Visualizations
Prescher, D., & Weber, G. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-41267-2_14 · Cited by 4
Correction to: Computers Helping People with Special Needs
Miesenberger, K., Κουρουπέτρογλου, Γ., Mavrou, K., Manduchi, R., Covarrubias, M., & Peňáz, P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-08645-8_64 · Cited by 1
Research works printed but not linked: 4
- 2022, Classification of 2D refreshable tactile user interfaces, Assist. Technol. Access. Incl. ICCHP AAATE, 1, 186
- Poppinga, B., Magnusson, C., Pielot, M., and Rassmus-Gröhn, K. (September, January 30). TouchOver Map: Audio-Tactile Exploration of Interactive Maps. Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services, Stockholm, Sweden.
- He, L., Wan, Z., Findlater, L., and Froehlich, J.E. (November, January 20). TacTILE: A Preliminary Toolchain for Creating Accessible Graphics with 3D-Printed Overlays and Auditory Annotations. Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility, Baltimore, MD, USA.
- Yang, 2016, Graille: Design research of graphical tactile display for the visually impaired, Decorate, 1, 94
Cited by
Artificial Intelligence in Special Education
Harkins-Brown, A. R., Carling, L. Z., & Peloff, D. C. (2025) · Encyclopedia · doi:10.3390/encyclopedia5010011 · Cited by 61
“Additionally, in recent years, the use of dynamic user interfaces with similar technologies have shown early results that provide a promising strategy to increase accessibility for individuals who are visually impaired [14].”
TADA: Making Node-link Diagrams Accessible to Blind and Low-Vision People
Zhao, Y., Nacenta, M. A., Sukhai, M. A., & Somanath, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642222 · Cited by 46
“Five-finger Dome: Users activate the Five-finger Dome by placing all the fingers of the hand on the tablet’s surface, which defines a pentagonal area.”
BackgroundMethodInfluentialTouchpilot: designing a guidance system that assists blind people in learning complex 3d structures.
Wang, X., Kayukawa, S., Takagi, H., & Asakawa, C. (2023) · The 25th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3597638.3608426 · Cited by 17
“To aid navigation, our system provides direct voice instructions such as “go up” or “go down,” which were more efective than sonar or axis-based sounds in previous studies [41].”
MethodRamôa, G., Moured, O., Schwarz, T., Müller, K., & Stiefelhagen, R. (2023) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3594806.3594818 · Cited by 15
“3% of the population in Europe [4], have mobility problems, and high unemployment rates [5, 12, 17, 19].”
BackgroundSelf-citationRamôa, G., Schmidt, V., Schwarz, T., Stiefelhagen, R., & König, P. (2024) · Frontiers in Rehabilitation Sciences · doi:10.3389/fresc.2024.1368983 · Cited by 5
Self-citation
Metadata
- Authors
- Ramôa, G., Schmidt, V., & König, P.
- Affiliations
- Karlsruhe Institute of Technology (DE)
- Osnabrück University (DE)
- Institute of Cognitive Science, Osnabrück University, 49090 Osnabrück, Germany
- Corresponding author
- Ramôa, Gaspar
- Venue
- Multimodal Technologies and Interaction
- Publisher
- MDPI AG
- Published
- 2022-12-18
- Volume
- 6
- Issue
- 12
- Pages
- 113
- ISSN
- 2414-4088 (ISSN-L)
- OpenAlex
- W4313405271
- Semantic Scholar
- 2cfd645f1797f56557d308f4c40c25bdf77d9250
- Cited by
- 14
- References
- 39
- Citations by year
- 2026: 4 · 2025: 3 · 2024: 3 · 2023: 4
- FWCI
- 1.29
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Multimodal Technologies and Interaction (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- European Union’s Horizon 2020 (861166)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- tactile graphics, audio navigation, blind and visually impaired users, assistive technology, voice-based navigation, accessibility, usability evaluation, user frustration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.