Visual experiences in the blind induced by an auditory sensory substitution device
Open accessAbstract
In this report, the phenomenology of two blind users of a sensory substitution device - "The vOICe" - that converts visual images to auditory signals is described. The users both report detailed visual phenomenology that developed within months of immersive use and has continued to evolve over a period of years. This visual phenomenology, although triggered through use of The vOICe, is likely to depend not only on online visualization of the auditory signal but also on the users' previous (albeit distant) experience of veridical vision (e.g. knowledge of shapes and visual perspective). Once established, the sensory substitution mapping between the auditory and visual domains is not confined to when the device is worn and, thus, may constitute an example of acquired synaesthesia.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Externalization with the haptic stimulator array only occurred for observers who actively trained with the device.
“Note importantly, that this only occurred for observers who actively trained with the device, consistent with the findings of Maravita et al. (2001, 2003; see also discussion in Proulx et al., 2008; Ward & Meijer, 2010; Bach-Y-Rita & Kercel, 2003; Stiles, Zheng & Shimojo, 2015).”
From Designing sensory-substitution devices: Principles, pitfalls and potential1 (2016) · cites this paper as Ward & Meijer, 2010
A user of the vOICe SSD exclaimed "It is sight!" about their experiences with auditory feedback.
“Ward and Meijer (2010) reported that a user of the vOICe SSD exclaimed “It is sight!” about their experiences with auditory feedback.”
From Designing sensory-substitution devices: Principles, pitfalls and potential1 (2016) · cites this paper as Ward and Meijer (2010)
Later, following training for several months or even years, they experienced smooth movement, and depth, although the SSD did not convey this type of information.
“Later, following training for several months or even years, they experienced smooth movement, and depth, although the SSD did not convey this type of information.”
From Designing sensory-substitution devices: Principles, pitfalls and potential1 (2016) · cites this paper as Ward and Meijer (2010)
At first, both users reported that visual details provided by the device were weak, flat and monochrome.
“At first, both users reported that visual details provided by the device were weak, flat and monochrome.”
From Designing sensory-substitution devices: Principles, pitfalls and potential1 (2016) · cites this paper as Ward and Meijer (2010)
One participant reported seeing color, but this probably reflected the participants' memory of color.
“One participant even reported seeing color, but this probably reflected the participants’ memory of color (the participant became blind at 21).”
From Designing sensory-substitution devices: Principles, pitfalls and potential1 (2016) · cites this paper as Ward and Meijer (2010)
Ward and Meijer (2010) found two blind users of the vOICe reported visual-like perception (externalization) acquired over several months of use.
“Ward and Meijer (2010) found that two blind users of the vOICe reported visual-like perception (externalization) acquired over several months of use.”
From Designing sensory-substitution devices: Principles, pitfalls and potential1 (2016) · cites this paper as Ward and Meijer (2010)
Ward and Meijer argued that even though the vOICe system enabled visualization through training, users' experiences depend on previous visual experience.
“Ward and Meijer argued that even though the vOICe system enabled visualization through training, users’ experiences depend on previous visual experience.”
From Designing sensory-substitution devices: Principles, pitfalls and potential1 (2016) · cites this paper as Ward and Meijer (2010)
Cites
A sensorimotor account of vision and visual consciousness
O'Regan, J. K., & Noë, A. (2001) · Behavioral and Brain Sciences · doi:10.1017/s0140525x01000115 · Cited by 3,126 · see citing works
An experimental system for auditory image representations
Meijer, P. B. L. (1992) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.121642 · Cited by 900 · see citing works
Self-citationShape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex
Amedi, A., Stern, W. M., Camprodon, J. A., Bermpohl, F., Merabet, L., Rotman, S., Hemond, C., Meijer, P., & Pascual-Leone, A. (2007) · Nature Neuroscience · doi:10.1038/nn1912 · Cited by 451 · see citing works
Self-citationBrain Plasticity: ‘Visual’ Acuity of Blind Persons via the Tongue
Sampaio, E., Maris, S., & Bach-y-Rita, P. (2001) · Brain Research · doi:10.1016/s0006-8993(01)02667-1 · Cited by 258 · see citing works
Kupers, R., Fumal, A., de Noordhout, A. M., Gjedde, A., Schoenen, J., & Ptito, M. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0602925103 · Cited by 122 · see citing works
Proulx, M. J., Stoerig, P., Ludowig, E., & Knoll, I. (2008) · PLoS ONE · doi:10.1371/journal.pone.0001840 · Cited by 81 · see citing works
Generalized Learning of Visual-to-Auditory Substitution in Sighted Individuals
Kim, J.-K., & Zatorre, R. J. (2008) · Brain Research · doi:10.1016/j.brainres.2008.06.038 · Cited by 62 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Cognition does not affect perception: Evaluating the evidence for “top-down” effects
Firestone, C., & Scholl, B. J. (2016) · Behavioral and Brain Sciences · doi:10.1017/s0140525x15000965 · Cited by 1,200 · see citing works
Maidenbaum, S., Abboud, S., & Amedi, A. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.11.007 · Cited by 230 · see citing works
Abboud, S., Hanassy, S., Levy-Tzedek, S., Maidenbaum, S., & Amedi, A. (2014) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-130338 · Cited by 199 · see citing works
Designing sensory-substitution devices: Principles, pitfalls and potential1
Kristjánsson, Á., Moldoveanu, A., Jóhannesson, Ó. I., Balan, O., Spagnol, S., Valgeirsdóttir, V. V., & Unnthorsson, R. (2016) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-160647 · Cited by 116 · see citing works
“Ward and Meijer (2010) found that two blind users of the vOICe reported visual-like perception (externalization) acquired over several months of use.”
BackgroundResultFast, accurate reaching movements with a visual-to-auditory sensory substitution device
Levy-Tzedek, S., Hanassy, S., Abboud, S., Maidenbaum, S., & Amedi, A. (2012) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-2012-110219 · Cited by 79 · see citing works
How well do you see what you hear? The acuity of visual-to-auditory sensory substitution
Haigh, A., Brown, D. J., Meijer, P., & Proulx, M. J. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00330 · Cited by 72 · see citing works
Ortiz, T., Poch, J., Santos, J. M., Requena, C., Martínez, A. M., Ortiz-Terán, L., Turrero, A., Barcia, J., Nogales, R., Calvo, A., et al. (2011) · PLoS ONE · doi:10.1371/journal.pone.0023264 · Cited by 63 · see citing works
Proulx, M. J., Gwinnutt, J., Dell’Erba, S., Levy-Tzedek, S., de Sousa, A. A., & Brown, D. J. (2015) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-150541 · Cited by 36 · see citing works
SoundSight: a mobile sensory substitution device that sonifies colour, distance, and temperature
Hamilton-Fletcher, G., Alvarez, J., Obrist, M., & Ward, J. (2022) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-021-00376-w · Cited by 28 · see citing works
Self-citationSpatial navigation with horizontally spatialized sounds in early and late blind individuals
Paré, S., Bleau, M., Djerourou, I., Malotaux, V., Kupers, R., & Ptito, M. (2021) · PLOS ONE · doi:10.1371/journal.pone.0247448 · Cited by 20 · see citing works
Guidelines for quantitative and qualitative studies of sensory substitution experience
Kałwak, W., Reuter, M., Łukowska, M., Majchrowicz, B., & Wierzchoń, M. (2018) · Adaptive Behavior · doi:10.1177/1059712318771690 · Cited by 14 · see citing works
Buchs, G., Maidenbaum, S., Levy-Tzedek, S., & Amedi, A. (2015) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-150592 · Cited by 8 · see citing works
Analysis and Validation of Cross-Modal Generative Adversarial Network for Sensory Substitution
Kim, M., Park, Y., Moon, K., & Jeong, C. Y. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18126216 · Cited by 8 · see citing works
Autoencoding sensory substitution
Tóth, V., & Parkkonen, L. (2019) · Unpublished · doi:10.13140/rg.2.2.10576.87048 · Cited by 7 · see citing works
- Authors
- Ward, J., & Meijer, P.
- Affiliations
- University of Sussex (GB)
- Corresponding author
- Ward, Jamie
- ORCID
- Ward, Jamie
- Venue
- Consciousness and Cognition
- Publisher
- Elsevier BV
- Published
- 2010-03
- Volume
- 19
- Issue
- 1
- Pages
- 492-500
- ISSN
- 1053-8100 (ISSN-L)
- OpenAlex
- W2127751554
- Cited by
- 247
- References
- 7
- Citations by year
- 2026: 8 · 2025: 10 · 2024: 10 · 2023: 42 · 2022: 9 · 2021: 20 · 2020: 16 · 2019: 18 · 2018: 13 · 2017: 18 · 2016: 10 · 2015: 22 · 2014: 16 · 2013: 10 · 2012: 12
- FWCI
- 4.04
- Type
- journal article
- Language
- en
- Open access
- green
- figshare.com
- Licence
- Locations
- Consciousness and Cognition (publishedVersion)
- PubMed (publishedVersion)
- Figshare (submittedVersion, cc-by)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › Hallucinations in medical conditions
- Keywords
- sensory substitution, The Voice, visual phenomenology, blind users, cross-modal perception
- MeSH terms
- Acoustic Stimulation, Auditory Perception, Blindness, Brain, Cognition, Female, Humans, Imagination, Functional Laterality, Middle Aged, Neuronal Plasticity, Sensory Aids, Sensory Deprivation, Software Design, Visual Acuity, Visual Cortex, Visual Perception, Touch Perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
