Sensory substitution and the human-machine interface
Abstract
Recent advances in the instrumentation technology of sensory substitution have presented new opportunities to develop systems for compensation of sensory loss. In sensory substitution (e.g. of sight or vestibular function), information from an artificial receptor is coupled to the brain via a human-machine interface. The brain is able to use this information in place of that usually transmitted from an intact sense organ. Both auditory and tactile systems show promise for practical sensory substitution interface sites. This research provides experimental tools for examining brain plasticity and has implications for perceptual and cognition studies more generally.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Available evidence suggests that there is great potential in sensory substitution methods.
“Notably, although the available evidence suggests that there is great potential in sensory substitution methods (see Bach-y-Rita & Kercel, 2003; Merabet & Pascual-Leone, 2010; Proulx et al., 2016; Shull & Damian, 2015; Nagel et al., 2005, for reviews)”
From Designing sensory-substitution devices: Principles, pitfalls and potential1 (2016) · cites this paper as Bach-y-Rita & Kercel, 2003
Even if two-point thresholds are exceeded, perceptual interpretation can compensate, functionally resulting in better resolution than 2-point thresholds dictate.
“Note however that even if two-point thresholds are exceeded it is possible that perceptual interpretation can compensate for this (see e.g. Novich & Eagleman 2015; and discussion in Bach-Y Rita, 2003). The pattern of stimulation may be detected and perceptual interpretation can functionally result in better resolution than 2-point thresholds dictate (Bach-Y-Rita & Kercel, 2003).”
From Designing sensory-substitution devices: Principles, pitfalls and potential1 (2016) · cites this paper as Bach-Y Rita, 2003
Cites
A sensorimotor account of vision and visual consciousness
O'Regan, J. K., & Noë, A. (2001) · Behavioral and Brain Sciences · doi:10.1017/s0140525x01000115 · Cited by 3,126 · see citing works
Vision substitution by tactile image projection
BACH-Y-RITA, P., COLLINS, C. C., SAUNDERS, F. A., WHITE, B., & SCADDEN, L. (1969) · Nature · doi:10.1038/221963a0 · Cited by 956 · see citing works
Self-citationJenkins, W. M., Merzenich, M. M., Ochs, M. T., Allard, T., & Guic-Robles, E. (1990) · Journal of Neurophysiology · doi:10.1152/jn.1990.63.1.82 · Cited by 937 · see citing works
An experimental system for auditory image representations
Meijer, P. B. L. (1992) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.121642 · Cited by 900 · see citing works
How brains make up their minds
(2001) · Choice Reviews Online · doi:10.5860/choice.39-0920 · Cited by 786 · see citing works
Chronic, multisite, multielectrode recordings in macaque monkeys
Nicolelis, M. A. L., Dimitrov, D., Carmena, J. M., Crist, R., Lehew, G., Kralik, J. D., & Wise, S. P. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1934665100 · Cited by 740 · see citing works
Different activation patterns in the visual cortex of late and congenitally blind subjects
Buchel, C. (1998) · Brain · doi:10.1093/brain/121.3.409 · Cited by 426 · see citing works
Brain mechanisms in sensory substitution
Bach‐y‐Rita, P. (1972) · Cited by 408 · see citing works
Self-citationNeural networks for Braille reading by the blind
Sadato, N. (1998) · Brain · doi:10.1093/brain/121.7.1213 · Cited by 352 · see citing works
Brain mechanisms in sensory substitution
Krieg, W. J. S. (1973) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(73)90016-3 · Cited by 336 · see citing works
Neural Plasticity and Consciousness
Hurley, S. L., & Noë, A. (2003) · Biology & Philosophy · doi:10.1023/a:1023308401356 · Cited by 305 · see citing works
Form perception with a 49-point electrotactile stimulus array on the tongue: a technical note.
Bach‐y‐Rita, P., Kaczmarek, K. A., Tyler, M. E., & Garcia-Lara, J. (1998) · PubMed · Cited by 304 · see citing works
Self-citationDarlington, C. L., & Darlington, C. (2003) · CRC Press eBooks · doi:10.1201/b12615-8 · Cited by 259 · see citing works
Brain Plasticity: ‘Visual’ Acuity of Blind Persons via the Tongue
Sampaio, E., Maris, S., & Bach-y-Rita, P. (2001) · Brain Research · doi:10.1016/s0006-8993(01)02667-1 · Cited by 258 · see citing works
Self-citationWhite, B. W., Saunders, F. A., Scadden, L., Bach-Y-Rita, P., & Collins, C. C. (1970) · Perception & Psychophysics · doi:10.3758/bf03210126 · Cited by 232 · see citing works
Self-citationBach‐y‐Rita, P., Tyler, M. E., & Kaczmarek, K. A. (2003) · International Journal of Human-Computer Interaction · doi:10.1207/s15327590ijhc1502_6 · Cited by 215 · see citing works
Self-citationCapelle, C., Trullemans, C., Arno, P., & Veraart, C. (1998) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.720206 · Cited by 210 · see citing works
CLOSING AN OPEN-LOOP CONTROL SYSTEM: VESTIBULAR SUBSTITUTION THROUGH THE TONGUE
Tyler, M. E., Danilov, Y. P., & Bach‐y‐Rita, P. (2003) · Journal of Integrative Neuroscience · doi:10.1142/s0219635203000263 · Cited by 209 · see citing works
Self-citationBrain–machine interfaces: computational demands and clinical needs meet basic neuroscience
Mussa-Ivaldi, F. A. (2003) · Trends in Neurosciences · doi:10.1016/s0166-2236(03)00121-8 · Cited by 174 · see citing works
Suaning, G. J., & Lovell, N. H. (2001) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.909646 · Cited by 164 · see citing works
Arno, P., De Volder, A. G., Vanlierde, A., Wanet-Defalque, M.-C., Streel, E., Robert, A., Sanabria-Bohórquez, S., & Veraart, C. (2001) · NeuroImage · doi:10.1006/nimg.2000.0731 · Cited by 148 · see citing works
Auditory Triggered Mental Imagery of Shape Involves Visual Association Areas in Early Blind Humans
Volder, A. D., Toyama, H., Kimura, Y., Kiyosawa, M., Nakano, H., Vanlierde, A., Wanet-Defalque, M.-C., Mishina, M., Oda, K., Ishiwata, K., et al. (2001) · NeuroImage · doi:10.1006/nimg.2001.0782 · Cited by 144 · see citing works
Moser, P. K. (1984) · International Philosophical Quarterly · doi:10.5840/ipq198424336 · Cited by 141 · see citing works
Changes in occipital cortex activity in early blind humans using a sensory substitution device
De Volder, A. G., Catalan-Ahumada, M., Robert, A., Bol, A., Labar, D., Coppens, A., Michel, C., & Veraart, C. (1999) · Brain Research · doi:10.1016/s0006-8993(99)01275-5 · Cited by 114 · see citing works
Kaczmarek, K. A., & Bach‐y‐Rita, P. (1995) · Cited by 101 · see citing works
Self-citationStack, G. J., & Morgan, M. J. (1979) · Philosophy and Phenomenological Research · doi:10.2307/2106332 · Cited by 100 · see citing works
Auditory Coding of Visual Patterns for the Blind
Arno, P., Capelle, C., Wanet-Defalque, M.-C., Catalán-Ahumada, M., & Veraart, C. (1999) · Perception · doi:10.1068/p281013 · Cited by 84 · see citing works
Form perception with a 49-point electrotactile stimulus array on the tongue
Bach‐y‐Rita, P. (1998) · Medical Entomology and Zoology · Cited by 84 · see citing works
Self-citationKaczmarek, K. A., & Haase, S. J. (2003) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2003.810421 · Cited by 78 · see citing works
Kuc, R. B. (2002) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2002.803561 · Cited by 76 · see citing works
Cerebral Reorganization of Function After Brain Damage
Johnston, K. M. (2000) · doi:10.1093/oso/9780195120264.001.0001 · Cited by 68 · see citing works
Electrotactile adaptation on the abdomen: preliminary results
Kaczmarek, K. A. (2000) · IEEE Transactions on Rehabilitation Engineering · doi:10.1109/86.895953 · Cited by 68 · see citing works
Transmission of Pictorial Information Through the Skin
Collins, C. C., & Bach‐y‐Rita, P. (1973) · Advances in biological and medical physics · doi:10.1016/b978-0-12-005214-1.50010-8 · Cited by 60 · see citing works
Self-citationNonsynaptic diffusion, neurotransmission and late brain reorganization
Niedermeyer, E. (1995) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(95)90290-2 · Cited by 53 · see citing works
SmartTouch - augmentation of skin sensation with electrocutaneous display
Kajimoto, H., İnami, M., Kawakami, N., & Tachi, S. (2003) · Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems · doi:10.1109/haptic.2003.1191225 · Cited by 51 · see citing works
Nonsynaptic Diffusion Neurotransmission and Late Brain Reorganization
Bach‐y‐Rita, P. (1995) · Cited by 49 · see citing works
Self-citationPattern identification as a function of stimulation on a fingertip-scanned electrotactile display
Kaczmarek, K. A., & Haase, S. J. (2003) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2003.816874 · Cited by 42 · see citing works
Representation of locomotor space by the blind
Veraart, C., & Wanet-Defalque, M.-C. (1987) · Perception & Psychophysics · doi:10.3758/bf03210501 · Cited by 41 · see citing works
Kajimoto, H., Kawakami, N., & Tachi, S. (2003) · Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems · doi:10.1109/haptic.2002.998973 · Cited by 40 · see citing works
(1997) · Presses Universitaires de France eBooks · doi:10.3917/puf.prous.1997.01 · Cited by 25 · see citing works
The self-composing brain: Towards a glial–neuronal brain theory
Mitterauer, B., & Kopp, K. L. (2003) · Brain and Cognition · doi:10.1016/s0278-2626(03)00043-5 · Cited by 22 · see citing works
Kercel, S. W. (2004) · Journal of Integrative Neuroscience · doi:10.1142/s0219635204000385 · Cited by 15 · see citing works
Sensori-"motor" coupling by observed and imagined movement
Bach‐y‐Rita, P., & Kercel, S. W. (2002) · Intellectica Revue de l Association pour la Recherche Cognitive · doi:10.3406/intel.2002.1671 · Cited by 10 · see citing works
Self-citationRossetti, Y. (1997) · Presses Universitaires de France eBooks · doi:10.3917/puf.prous.1997.01.0179 · Cited by 10 · see citing works
Du problème de Molyneux au problème de Bach-y-Rita
Pacherie, É. (1997) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 9 · see citing works
Bizarre hierarchy of brain function
Kercel, S. W., Caulfield, H. J., & Bach‐y‐Rita, P. (2003) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.496988 · Cited by 5 · see citing works
Self-citationKercel, S. W. (2003) · doi:10.1109/smcia.2003.1231339 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Supersizing the Mind: Embodiment, Action, and Cognitive Extension
Clark, A. (2008) · doi:10.1093/acprof:oso/9780195333213.001.0001 · Cited by 2,201 · see citing works
Control strategies for active lower extremity prosthetics and orthotics: a review
Tucker, M. R., Olivier, J., Pagel, A., Bleuler, H., Bouri, M., Lambercy, O., Millán, J. D. R., Riener, R., Vallery, H., & Gassert, R. (2015) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/1743-0003-12-1 · Cited by 1,047 · see citing works
Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex
Amedi, A., Stern, W. M., Camprodon, J. A., Bermpohl, F., Merabet, L., Rotman, S., Hemond, C., Meijer, P., & Pascual-Leone, A. (2007) · Nature Neuroscience · doi:10.1038/nn1912 · Cited by 451 · see citing works
Ptito, M. (2005) · Brain · doi:10.1093/brain/awh380 · Cited by 310 · see citing works
Mediated social touch: a review of current research and future directions
Haans, A., & IJsselsteijn, W. (2006) · Virtual Reality · doi:10.1007/s10055-005-0014-2 · Cited by 303 · see citing works
Compensatory plasticity and cross-modal reorganization following early visual deprivation
Kupers, R., & Ptito, M. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.08.001 · Cited by 293 · see citing works
Haptic wearables as sensory replacement, sensory augmentation and trainer – a review
Shull, P. B., & Damian, D. D. (2015) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-015-0055-z · Cited by 245 · see citing works
Lewis, P. M., Ackland, H. M., Lowery, A. J., & Rosenfeld, J. V. (2015) · Brain Research · doi:10.1016/j.brainres.2014.11.020 · Cited by 228 · see citing works
Beyond sensory substitution—learning the sixth sense
Nagel, S. K., Carl, C., Kringe, T., Märtin, R., & König, P. (2005) · Journal of Neural Engineering · doi:10.1088/1741-2560/2/4/r02 · Cited by 225 · see citing works
Neural correlates of virtual route reCognition in congenital blindness
Kupers, R., Chebat, D. R., Madsen, K. H., Paulson, O. B., & Ptito, M. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1006199107 · Cited by 187 · see citing works
Han, J. H., Kwak, J.-H., Joe, D. J., Hong, S. K., Wang, H. S., Park, J. H., Hur, S., & Lee, K. J. (2018) · Nano Energy · doi:10.1016/j.nanoen.2018.08.053 · Cited by 128 · see citing works
Kupers, R., Fumal, A., de Noordhout, A. M., Gjedde, A., Schoenen, J., & Ptito, M. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0602925103 · Cited by 122 · see citing works
‘Visual’ Acuity of the Congenitally Blind Using Visual-to-Auditory Sensory Substitution
Striem-Amit, E., Guendelman, M., & Amedi, A. (2012) · PLoS ONE · doi:10.1371/journal.pone.0033136 · Cited by 117 · see citing works
Designing sensory-substitution devices: Principles, pitfalls and potential1
Kristjánsson, Á., Moldoveanu, A., Jóhannesson, Ó. I., Balan, O., Spagnol, S., Valgeirsdóttir, V. V., & Unnthorsson, R. (2016) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-160647 · Cited by 116 · see citing works
“The pattern of stimulation may be detected and perceptual interpretation can functionally result in better resolution than 2-point thresholds dictate (Bach-Y-Rita & Kercel, 2003).”
BackgroundResultTMS of the occipital cortex induces tactile sensations in the fingers of blind Braille readers
Ptito, M., Fumal, A., de Noordhout, A. M., Schoenen, J., Gjedde, A., & Kupers, R. (2008) · Experimental Brain Research · doi:10.1007/s00221-007-1091-0 · Cited by 113 · see citing works
There is something out there: Distal attribution in sensory substitution, twenty years later
AUVRAY, M., HANNETON, S., LENAY, C., & O'REGAN, K. (2005) · Journal of Integrative Neuroscience · doi:10.1142/s0219635205001002 · Cited by 112 · see citing works
Tactile-‘visual’ acuity of the tongue in early blind individuals
Chebat, D.-R., Rainville, C., Kupers, R., & Ptito, M. (2007) · Neuroreport · doi:10.1097/wnr.0b013e3282f2a63 · Cited by 88 · see citing works
Proulx, M. J., Stoerig, P., Ludowig, E., & Knoll, I. (2008) · PLoS ONE · doi:10.1371/journal.pone.0001840 · Cited by 81 · see citing works
Fast, accurate reaching movements with a visual-to-auditory sensory substitution device
Levy-Tzedek, S., Hanassy, S., Abboud, S., Maidenbaum, S., & Amedi, A. (2012) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-2012-110219 · Cited by 79 · see citing works
Klatzky, R. L., Giudice, N. A., Bennett, C. R., & Loomis, J. M. (2014) · Multisensory Research · doi:10.1163/22134808-00002447 · Cited by 74 · see citing works
How well do you see what you hear? The acuity of visual-to-auditory sensory substitution
Haigh, A., Brown, D. J., Meijer, P., & Proulx, M. J. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00330 · Cited by 72 · see citing works
Ortiz, T., Poch, J., Santos, J. M., Requena, C., Martínez, A. M., Ortiz-Terán, L., Turrero, A., Barcia, J., Nogales, R., Calvo, A., et al. (2011) · PLoS ONE · doi:10.1371/journal.pone.0023264 · Cited by 63 · see citing works
Multichannel electrotactile feedback for simultaneous and proportional myoelectric control
Patel, G. K., Dosen, S., Castellini, C., & Farina, D. (2016) · Journal of Neural Engineering · doi:10.1088/1741-2560/13/5/056015 · Cited by 58 · see citing works
Proulx, M. J., Gwinnutt, J., Dell’Erba, S., Levy-Tzedek, S., de Sousa, A. A., & Brown, D. J. (2015) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-150541 · Cited by 36 · see citing works
The role of visual deprivation and experience on the performance of sensory substitution devices
Stronks, H. C., Nau, A. C., Ibbotson, M. R., & Barnes, N. (2015) · Brain Research · doi:10.1016/j.brainres.2015.06.033 · Cited by 31 · see citing works
Hu, D., Wang, D., Li, X., Nie, F., & Wang, Q. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00816 · Cited by 21 · see citing works
An investigation into the effectiveness of using acoustic touch to assist people who are blind
Zhu, H. Y., Hossain, S. N., Jin, C., Singh, A. K., Nguyen, M. T. D., Deverell, L., Nguyen, V., Gates, F. S., Fernandez, I. G., Melencio, M. V., et al. (2023) · PLoS ONE · doi:10.1371/journal.pone.0290431 · Cited by 16 · see citing works
A Sensory Substitution Device Inspired by the Human Visual System
Osinski, D., & Hjelme, D. R. (2018) · 2018 11th International Conference on Human System Interaction (HSI) · doi:10.1109/hsi.2018.8431078 · Cited by 15 · see citing works
Guidelines for quantitative and qualitative studies of sensory substitution experience
Kałwak, W., Reuter, M., Łukowska, M., Majchrowicz, B., & Wierzchoń, M. (2018) · Adaptive Behavior · doi:10.1177/1059712318771690 · Cited by 14 · see citing works
Buchs, G., Maidenbaum, S., Levy-Tzedek, S., & Amedi, A. (2015) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-150592 · Cited by 8 · see citing works
- Authors
- Bach-y-Rita, P., & W. Kercel, S.
- Venue
- Trends in Cognitive Sciences
- Publisher
- Elsevier BV
- Published
- 2003-12
- Volume
- 7
- Issue
- 12
- Pages
- 541-546
- ISSN
- 1364-6613 (ISSN-L)
- OpenAlex
- W2131294823
- Cited by
- 858
- References
- 47
- Citations by year
- 2026: 36 · 2025: 31 · 2024: 40 · 2023: 77 · 2022: 32 · 2021: 54 · 2020: 37 · 2019: 38 · 2018: 48 · 2017: 52 · 2016: 46 · 2015: 47 · 2014: 61 · 2013: 51 · 2012: 36
- FWCI
- 1.26
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Trends in Cognitive Sciences (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Keywords
- sensory substitution, human-machine interaction, neuroplasticity, synthetic receptors, perceptual learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
