A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research
Open accessThe DOI matches its Crossref record; 60 of the 62 references it cites were found in a registry or an index.
Abstract
This paper summarizes recent developments in audio and tactile feedback based assistive technologies targeting the blind community. Current technology allows applications to be efficiently distributed and run on mobile and handheld devices, even in cases where computational requirements are significant. As a result, electronic travel aids, navigational assistance modules, text-to-speech applications, as well as virtual audio displays which combine audio with haptic channels are becoming integrated into standard mobile devices. This trend, combined with the appearance of increasingly user-friendly interfaces and modes of interaction has opened a variety of new perspectives for the rehabilitation and training of users with visual impairments. The goal of this paper is to provide an overview of these developments based on recent advances in basic research and application development. Using this overview as a foundation, an agenda is outlined for future research in mobile interaction design with respect to users with special needs, as well as ultimately in relation to sensor-bridging applications in general.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Thanks to the rise of accessibility features and applications for mainstream devices, the number of people with vision impairment using smartphones is increasing [14,20].”
From Understanding Authentication Method Use on Mobile Devices by People with Vision Impairment (2018) · cites this paper as [14]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Earcons and Icons: Their Structure and Common Design Principles
Blattner, M., Sumikawa, D., & Greenberg, R. (1989) · Human-Computer Interaction · doi:10.1207/s15327051hci0401_1 · Cited by 980
An experimental system for auditory image representations
Meijer, P. B. L. (1992) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.121642 · Cited by 900
Definition and synergies of cognitive infocommunications
Bárányi, P., & Csapó, Á. (2012) · SZTAKI Publication Repository (Hungarian Academy of Sciences) · Cited by 748
Self-citationTactons: structured tactile messages for non-visual information display
Brewster, S. A., & Brown, L. M. (2004) · Enlighten: Publications (University of Glasgow) · Cited by 697
Auditory Icons: Using Sound in Computer Interfaces
Gaver, W. (1986) · Human-Computer Interaction · doi:10.1207/s15327051hci0202_3 · Cited by 637
Visual Capture of Touch: Out-of-the-Body Experiences With Rubber Gloves.
Pavani, F., Spence, C., & Driver, J. (2000) · Psychological Science · doi:10.1111/1467-9280.00270 · Cited by 608
The SonicFinder: An Interface That Uses Auditory Icons
Gaver, W. (1989) · Human-Computer Interaction · doi:10.1207/s15327051hci0401_3 · Cited by 587
Serafin, S., Franinović, K., Hermann, T. M., Lemaître, G., Rinott, M., & Rocchesso, D. (2011) · Cited by 572
Auditory Display: Sonification, Audification, and Auditory Interfaces
Perkis, T., & Kramer, G. (1995) · Computer Music Journal · doi:10.2307/3680606 · Cited by 536
Waypoint navigation with a vibrotactile waist belt
Erp, J. B. F. V., Veen, H. A. H. C. V., Jansen, C., & Dobbins, T. (2005) · ACM Transactions on Applied Perception · doi:10.1145/1060581.1060585 · Cited by 477
From software product lines to software ecosystems
Bosch, J. (2009) · Software Product Lines · doi:10.5555/1753235.1753251 · Cited by 474
Medusa: a programming framework for crowd-sensing applications
Ra, M.-R., Liu, B., La Porta, T. F., & Govindan, R. (2012) · Proceedings of the 10th international conference on Mobile systems, applications, and services · doi:10.1145/2307636.2307668 · Cited by 330
Optical-to-Tactile Image Conversion for the Blind
Bliss, J., Katcher, M., Rogers, C., & Shepard, R. (1970) · IEEE Transactions on Man Machine Systems · doi:10.1109/tmms.1970.299963 · Cited by 306
A sonar aid to enhance spatial perception of the blind: engineering design and evaluation
Kay, L. (1974) · Radio and Electronic Engineer · doi:10.1049/ree.1974.0148 · Cited by 259
Brain Plasticity: ‘Visual’ Acuity of Blind Persons via the Tongue
Sampaio, E., Maris, S., & Bach-y-Rita, P. (2001) · Brain Research · doi:10.1016/s0006-8993(01)02667-1 · Cited by 258 · cited 1×
“Experiments have confirmed the viability of this approach in supporting the recognition of basic shapes [15,16], as well as reading [17,18] and localization tasks [19].”
Biocca, F., Kim, J., & Choi, Y. (2001) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474601300343595 · Cited by 256
Gyrophone: recognizing speech from gyroscope signals
Michalevsky, Y., Boneh, D., & Nakibly, G. (2014) · Cited by 251
Bach-y-Rita, P., Tyler, M. E., & Kaczmarek, K. A. (2003) · International Journal of Human-Computer Interaction · doi:10.1207/s15327590ijhc1502_6 · Cited by 215
Capelle, C., Trullemans, C., Arno, P., & Veraart, C. (1998) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.720206 · Cited by 210
Perceptual Design of Haptic Icons
MacLean, K. E., & Enriquez, M. J. (2003) · Cited by 200
SWAN: System for Wearable Audio Navigation
Wilson, J., Walker, B. N., Lindsay, J., Cambias, C., & Dellaert, F. (2007) · 2007 11th IEEE International Symposium on Wearable Computers · doi:10.1109/iswc.2007.4373786 · Cited by 200 · cited 1×
“which was developed to serve as a safe pedestrian navigation and orientation aid for persons with temporary or permanent visual impairments [36,37].”
BackgroundCombining crowdsourcing and google street view to identify street-level accessibility problems
Hara, K., Le, V., & Froehlich, J. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470744 · Cited by 196
Sensory dominance in combinations of audio, visual and haptic stimuli
Hecht, D., & Reiner, M. (2009) · Experimental Brain Research · doi:10.1007/s00221-008-1626-z · Cited by 193 · cited 1×
“Many studies have been conducted on intersensory integration, sensory cross-effects and sensory dominance effects in visually oriented desktop and virtual applications [104–107].”
BackgroundWalker, B. N., & Lindsay, J. (2006) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872006777724507 · Cited by 170
Tactile display and vibrotactile pattern recognition on the torso
Jones, L. A., Lockyer, B., & Piateski, E. (2006) · Advanced Robotics · doi:10.1163/156855306778960563 · Cited by 162
Pygmalion: A Computer program to Model and Stimulate Creative Thought
Smith, D. C. (1975) · Medical Entomology and Zoology · Cited by 157
Barrass, S. (1997) · ANU Open Research (Australian National University) · doi:10.25911/5d7a2acd2b71d · Cited by 141
Everyday listening and auditory icons
Gaver, W. W., & Norman, D. A. (1988) · Cited by 137
Proceedings of the 13th International Software Product Line Conference
Muthig, D., & McGregor, J. D. (2009) · Cited by 135
Chapman *, R. L., & Corso, M. (2005) · Production Planning & Control · doi:10.1080/09537280500063269 · Cited by 122
The hapticon editor: a tool in support of haptic communication research
Enriquez, M. J., & MacLean, K. E. (2003) · 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003. HAPTICS 2003. Proceedings. · doi:10.1109/haptic.2003.1191310 · Cited by 102
A crowdsourcing platform for the construction of accessibility maps
Cardonha, C., Gallo, D. S., Avegliano, P., Herrmann, R., Koch, F. L., & Borger, S. (2013) · International Cross-Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/2461121.2461129 · Cited by 98
Kaczmarek, K. A., & Haase, S. J. (2003) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2003.810421 · Cited by 86
Human Spatial Navigation via a Visuo-Tactile Sensory Substitution System
Segond, H., Weiss, D., & Sampaio, E. (2005) · Perception · doi:10.1068/p3409 · Cited by 79
A unified terminology for the structure and semantics of CogInfoCom channels
Csapó, Á., & Bárányi, P. (2012) · SZTAKI Publication Repository (Hungarian Academy of Sciences) · Cited by 74
Self-citationDissemination in opportunistic mobile ad-hoc networks: The power of the crowd
Zyba, G., Voelker, G. M., Ioannidis, S., & Diot, C. (2011) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.2011.5934896 · Cited by 72
Overview of auditory representations in human-machine interfaces
Csapó, Á., & Wersényi, G. (2013) · ACM Computing Surveys · doi:10.1145/2543581.2543586 · Cited by 70
Self-citationVibrotactile Pattern Perception: Extraordinary Observers
Craig, J. C. (1977) · Science · doi:10.1126/science.850791 · Cited by 65
Real-time assistance prototype — A new navigation aid for blind people
Dunai, L., Peris‐Fajarnés, G., Santiago-Praderas, V., Defez, B., & Lengua, I. L. (2010) · Annual Conference of the IEEE Industrial Electronics Society (IECON) · doi:10.1109/iecon.2010.5675535 · Cited by 64
Generalized Learning of Visual-to-Auditory Substitution in Sighted Individuals
Kim, J.-K., & Zatorre, R. J. (2008) · Brain Research · doi:10.1016/j.brainres.2008.06.038 · Cited by 62
Jansson, G. (1983) · International Journal of Neuroscience · doi:10.3109/00207458309148644 · Cited by 57
Braille Display by Lateral Skin Deformation with the STReSS2 Tactile Transducer
Lévesque, V., Pasquero, J., & Vincent Hayward, B. (2007) · IEEE World Haptics Conference (WHC) · doi:10.1109/whc.2007.25 · Cited by 52
Real-time emergency response: improved management of real-time information during crisis situations
Blum, J. R., Eichhorn, A., Smith, S., Sterle-Contala, M., & Cooperstock, J. R. (2013) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-013-0139-7 · Cited by 50
Tactile letter recognition: Pattern duration and modes of pattern generation
Craig, J. C. (1981) · Perception & Psychophysics · doi:10.3758/bf03202007 · Cited by 37
Rice, M. T., Aburizaiza, A. O., Jacobson, D., Shore, B. M., & Paez, F. I. (2012) · Transactions in GIS · doi:10.1111/j.1467-9671.2012.01318.x · Cited by 35
An application of bio-feedback in the rehabilitation of the blind
Bissitt, D., & Heyes, A. D. (1980) · Applied Ergonomics · doi:10.1016/0003-6870(80)90118-0 · Cited by 32
Collaborative navigation of visually impaired
Balata, J., Franc, J., Míkovec, Z., & Slavík, P. (2013) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-013-0137-9 · Cited by 29
Towards Real-time Emergency Response using Crowd Supported Analysis of Social Media
Rogstadius, J., Kostakos, V., Laredo, J., & Vuković, M. (2011) · doi:10.13140/rg.2.2.25881.83047 · Cited by 26
The Spiral Discovery Method: An Interpretable Tuning Model for CogInfoCom Channels
Csapó, Á., & Bárányi, P. (2012) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2012.p0358 · Cited by 24
Self-citationBalan, O. C., Moldoveanu, A. D. B., Moldoveanu, F., & Dascălu, M.-I. (2014) · International Conference on System Theory, Control and Computing (ICSTCC) · doi:10.1109/icstcc.2014.6982496 · Cited by 22
THE WALKING STRAIGHT MOBILE APPLICATION: HELPING THE VISUALLY IMPAIRED AVOID VEERING
Panëels, S., Varenne, D., Blum, J. R., & Cooperstock, J. R. (2013) · SMARTech Repository (Georgia Institute of Technology) · Cited by 21
A computer-vision based sensory substitution device for the visually impaired (See ColOr)
Valencia, G., & Diego, J. (2014) · Archive ouverte UNIGE (University of Geneva) · doi:10.13097/archive-ouverte/unige:34568 · Cited by 16
Characteristics of Reading Rate and Manual Scanning Patterns of Blind Optacon Readers
Hislop, D. W., Zuber, B. L., & TRIMBLE, J. L. (1983) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872088302500403 · Cited by 15
The design of an audio film for the visually impaired
López, M., & Pauletto, S. (2009) · Cited by 14
AUDIO GAMES- A NOVEL APPROACH TOWARDS EFFECTIVE LEARNING IN THE CASE OF VISUALLY-IMPAIRED PEOPLE
Balan, O. C., Moldoveanu, A. D. B., Moldoveanu, F., & Dascălu, M.-I. (2014) · Cited by 13
Oversketching and associated audio-based feedback channels for a virtual sketching application
Csapó, Á., Israel, J. H., & Belaifa, O. (2013) · IEEE International Conference on Cognitive Infocommunications · doi:10.1109/coginfocom.2013.6719300 · Cited by 10
Self-citationAccessing Audiotactile Images with HFVE Silooet
Dewhurst, D. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-04076-4_7 · Cited by 7
Spectral Discrimination Thresholds Comparing Audio and Haptics for Complex Stimuli
Picinali, L., Feakes, C., Mauro, D. A., & Katz, B. F. G. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-32796-4_14 · Cited by 5
Tone-2 tones discrimination task comparing audio and haptics
Picinali, L., Feakes, C., Mauro, D. A., & Katz, B. F. G. (2012) · IEEE International Workshop on Haptic Audio visual Environments and Games · doi:10.1109/have.2012.6374432 · Cited by 5
Perceptual Interpolation and Open-Ended Exploration of Auditory Icons and Earcons
Csapó, Á., & Bárányi, P. (2011) · SMARTech Repository (Georgia Institute of Technology) · Cited by 4
Self-citation
Research works printed but not linked: 6
- 7. Hermann T, Hunt A, Neuhoff JG (2011) The sonification hand- book. Logos, Berlin
- 14. Picinali L, Katz BF (2010) Spectral discrimination thresholds comparing audio and haptics. In: Proceedings of haptic and audi- tory interaction design workshop, Copenhagen, pp 1-2
- 36. Dewhurst D (2007) An audiotactile vision-substitution system. In: Proceedings of second international workshop on interactive sonification, York, pp 1-4
- 38. Dewhurst D (2010) Creating and accessing audio-tactile images with "HFVE" vision substitution software. In: Proceedings of the third interactive sonification workshop. KTH, Stockholm, pp 101- 104
- 56. Balan O, Moldoveanu A, Moldoveanu F, Dascalu M-I (2014) Nav- igational 3D audio-based game-training towards rich auditory spatial representation of the environment. In: Proceedings of the 18th international conference on system theory, control and com- puting, Sinaia 91. http://www.gmagames.com/sod.html. Accessed Mar 2015
- 61. Biocca F, Inoue Y, Polinsky H, Lee A, Tang A (2002) Visual cues and virtual touch: role of visual stimuli and intersensory integra- tion in cross-modal haptic illusions and the sense of presence. In: Gouveia F (ed) Proceedings of presence, Porto 106.
Cited by
Bhowmick, A., & Hazarika, S. M. (2017) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-016-0235-6 · Cited by 271
Tools and Technologies for Blind and Visually Impaired Navigation Support: A Review
Kuriakose, B., Shrestha, R., & Sandnes, F. E. (2022) · IETE Technical Review · doi:10.1080/02564602.2020.1819893 · Cited by 267
“[16], the largest and most common mobile platforms are rapidly evolving as the de-facto standards for the implementation of assistive technologies.”
BackgroundHuman augmentation: Past, present and future
Raisamo, R., Rakkolainen, I., Majaranta, P., Salminen, K., Rantala, J., & Farooq, A. (2019) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2019.05.008 · Cited by 236
Wearable assistive devices for visually impaired: A state of the art survey
Tapu, R., Mocanu, B., & Zaharia, T. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.10.031 · Cited by 211
Khan, A., & Khusro, S. (2021) · Universal Access in the Information Society · doi:10.1007/s10209-020-00733-8 · Cited by 177
“Subsequently, mobile computing, advances in sensors technology, crowed/friend sourcing and data integration capabilities are leading perspectives of smartphone-based assistive technologies [39].”
Background
Metadata
- Authors
- Csapó, Á., Wersényi, G., Nagy, H., & Stockman, T.
- Affiliations
- HUN-REN Institute for Computer Science and Control (HU)
- Széchenyi István University (HU)
- Queen Mary University of London (GB)
- Corresponding author
- Csapó, Ádám
- Venue
- Journal on Multimodal User Interfaces
- Published
- 2015-06-18
- Volume
- 9
- Issue
- 4
- Pages
- 275-286
- ISSN
- 1783-7677 (ISSN-L)
- 1783-8738
- OpenAlex
- W1958876462
- Cited by
- 181
- References
- 62
- Citations by year
- 2026: 1 · 2025: 12 · 2024: 11 · 2023: 15 · 2022: 23 · 2021: 27 · 2020: 24 · 2019: 18 · 2018: 18 · 2017: 24 · 2016: 6 · 2015: 2
- FWCI
- 10.02
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Journal on Multimodal User Interfaces (publishedVersion, cc-by)
- SZTAKI Publication Repository (Hungarian Academy of Sciences) (submittedVersion)
- Funders
- European Commission (-2012-0001, A/2-11-1-2012-0001, TÁMOP‐4.2.4.A/2‐11/1‐2012‐0001, HORIZON2020, 4.2.4.A/2-11-1-2012-0001, 643636)
- European Social Fund (A/2–11/1-2012-0001, TÁMOP 4.2.4, -2012-0001)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Keywords
- assistive technology, blind users, visual impairment, mobile devices, audio feedback, tactile feedback, haptic feedback, text-to-speech, electronic travel aids, navigation aids, mobile interaction design, rehabilitation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.