Integration and binding in rehabilitative sensory substitution: Increasing resolution using a new Zooming-in approach
Open accessAbstract
PURPOSE: To visually perceive our surroundings we constantly move our eyes and focus on particular details, and then integrate them into a combined whole. Current visual rehabilitation methods, both invasive, like bionic-eyes and non-invasive, like Sensory Substitution Devices (SSDs), down-sample visual stimuli into low-resolution images. Zooming-in to sub-parts of the scene could potentially improve detail perception. Can congenitally blind individuals integrate a 'visual' scene when offered this information via different sensory modalities, such as audition? Can they integrate visual information -perceived in parts - into larger percepts despite never having had any visual experience? METHODS: We explored these questions using a zooming-in functionality embedded in the EyeMusic visual-to-auditory SSD. Eight blind participants were tasked with identifying cartoon faces by integrating their individual components recognized via the EyeMusic's zooming mechanism. RESULTS: After specialized training of just 6-10 hours, blind participants successfully and actively integrated facial features into cartooned identities in 79±18% of the trials in a highly significant manner, (chance level 10% ; rank-sum P < 1.55E-04). CONCLUSIONS: These findings show that even users who lacked any previous visual experience whatsoever can indeed integrate this visual information with increased resolution. This potentially has important practical visual rehabilitation implications for both invasive and non-invasive methods.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Buchs et al. (2015) found even users with no visual experience can integrate information across successive zooms.
“Even users with no visual experience can integrate information across successive zooms (Buchs et al., 2015).”
From Designing sensory-substitution devices: Principles, pitfalls and potential1 (2016) · cites this paper as Buchs et al., 2015
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
IARBUS, A. L. (1967) · Medical Entomology and Zoology · Cited by 3,319 · see citing works
Yarbus, A. L. (1967) · doi:10.1007/978-1-4899-5379-7 · Cited by 3,304 · see citing works
An experimental system for auditory image representations
Meijer, P. B. L. (1992) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.121642 · Cited by 900 · see citing works
Sensory substitution and the human-machine interface
Bach-y-Rita, P., & W. Kercel, S. (2003) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2003.10.013 · Cited by 858 · see citing works
A trial studying approach to predict college achievement
Meijer, R. R., & Niessen, A. S. M. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00887 · Cited by 711 · see citing works
Interim results from the international trial of Second Sight's visual prosthesis
Humayun, M. S., Dorn, J. D., da Cruz, L., Dagnelie, G., Sahel, J.-A., Stanga, P. E., Cideciyan, A. V., Duncan, J. L., Eliott, D., Filley, E., et al. (2012) · Ophthalmology · doi:10.1016/j.ophtha.2011.09.028 · Cited by 700 · see citing works
Prioritizing spatial accuracy in high-resolution fMRI data using multivariate feature weight mapping
Stelzer, J., Buschmann, T., Lohmann, G., Margulies, D. S., Trampel, R., & Turner, R. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00066 · Cited by 638 · see citing works
An investigation into component and configural processes underlying face perception
Sergent, J. (1984) · British Journal of Psychology · doi:10.1111/j.2044-8295.1984.tb01895.x · Cited by 493 · see citing works
da Cruz, L., Coley, B. F., Dorn, J., Merlini, F., Filley, E., Christopher, P., Chen, F. K., Wuyyuru, V., Sahel, J., Stanga, P., et al. (2013) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2012-301525 · Cited by 359 · see citing works
Form perception with a 49-point electrotactile stimulus array on the tongue: a technical note.
Bach‐y‐Rita, P., Kaczmarek, K. A., Tyler, M. E., & Garcia-Lara, J. (1998) · PubMed · Cited by 304 · see citing works
Multisensory Integration: Flexible Use of General Operations
van Atteveldt, N., Murray, M. M., Thut, G., & Schroeder, C. E. (2014) · Neuron · doi:10.1016/j.neuron.2014.02.044 · Cited by 269 · see citing works
Visual experiences in the blind induced by an auditory sensory substitution device
Ward, J., & Meijer, P. (2010) · Consciousness and Cognition · doi:10.1016/j.concog.2009.10.006 · Cited by 247 · see citing works
Maidenbaum, S., Abboud, S., & Amedi, A. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.11.007 · Cited by 230 · see citing works
Self-citationAbboud, S., Hanassy, S., Levy-Tzedek, S., Maidenbaum, S., & Amedi, A. (2014) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-130338 · Cited by 199 · see citing works
Self-citationRetinal Prostheses: Current Clinical Results and Future Needs
Weiland, J. D., Cho, A. K., & Humayun, M. S. (2011) · Ophthalmology · doi:10.1016/j.ophtha.2011.08.042 · Cited by 196 · see citing works
Retinal implants: a systematic review: Table 1
Chuang, A., Margo, C. E., & Greenberg, P. B. (2014) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2013-303708 · Cited by 192 · see citing works
The functional performance of the Argus II retinal prosthesis
Stronks, H. C., & Dagnelie, G. (2014) · Expert Review of Medical Devices · doi:10.1586/17434440.2014.862494 · Cited by 189 · see citing works
Arno, P., De Volder, A. G., Vanlierde, A., Wanet-Defalque, M.-C., Streel, E., Robert, A., Sanabria-Bohórquez, S., & Veraart, C. (2001) · NeuroImage · doi:10.1006/nimg.2000.0731 · Cited by 148 · see citing works
Multisensory perceptual learning and sensory substitution
Proulx, M. J., Brown, D. J., Pasqualotto, A., & Meijer, P. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.11.017 · Cited by 148 · see citing works
A number-form area in the blind
Abboud, S., Maidenbaum, S., Dehaene, S., & Amedi, A. (2015) · Nature Communications · doi:10.1038/ncomms7026 · Cited by 128 · see citing works
Self-citationReading the World through the Skin and Ears: A New Perspective on Sensory Substitution
Deroy, O. D., & Auvray, M. (2012) · Frontiers in Psychology · doi:10.3389/fpsyg.2012.00457 · Cited by 94 · see citing works
Is there a future for sensory substitution outside academic laboratories?
Elli, G. V., Benetti, S., & Collignon, O. (2014) · Multisensory Research · doi:10.1163/22134808-00002460 · Cited by 90 · see citing works
Proulx, M. J., Stoerig, P., Ludowig, E., & Knoll, I. (2008) · PLoS ONE · doi:10.1371/journal.pone.0001840 · Cited by 81 · see citing works
Fast, accurate reaching movements with a visual-to-auditory sensory substitution device
Levy-Tzedek, S., Hanassy, S., Abboud, S., Maidenbaum, S., & Amedi, A. (2012) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-2012-110219 · Cited by 79 · see citing works
Self-citationHow well do you see what you hear? The acuity of visual-to-auditory sensory substitution
Haigh, A., Brown, D. J., Meijer, P., & Proulx, M. J. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00330 · Cited by 72 · see citing works
Sensory rehabilitation in the plastic brain
Collignon, O., Champoux, F., Voss, P., & Leporé, F. (2011) · Progress in brain research · doi:10.1016/b978-0-444-53752-2.00003-5 · Cited by 70 · see citing works
Dagnelie, G. (2011) · Current Opinion in Neurology · doi:10.1097/wco.0b013e32834f02c3 · Cited by 70 · see citing works
Auditory substitution of vision: pattern recognition by the blind
Arno, P., Vanlierde, A., Streel, E., Wanet-Defalque, M.-C., Bohorquez, S. M. S., & Veraart, C. (2001) · Applied Cognitive Psychology · doi:10.1002/acp.720 · Cited by 70 · see citing works
Retina implant—a bioMEMS challenge
Meyer, J.-U. (2002) · Sensors and Actuators A Physical · doi:10.1016/s0924-4247(01)00807-x · Cited by 69 · see citing works
Motor-Sensory Confluence in Tactile Perception
Saig, A., Gordon, G., Assa, E., Arieli, A., & Ahissar, E. (2012) · Journal of Neuroscience · doi:10.1523/jneurosci.2432-12.2012 · Cited by 66 · see citing works
Nau, A. C., Bach, M., & Fisher, C. (2013) · Translational Vision Science & Technology · doi:10.1167/tvst.2.3.1 · Cited by 54 · see citing works
Renier, L. A., & Volder, A. D. (2010) · Disability and Rehabilitation Assistive Technology · doi:10.3109/17483100903253936 · Cited by 51 · see citing works
Levy‐Tzedek, S., Novick, I., Arbel, R., Abboud, S., Maidenbaum, S., Vaadia, E., & Amedi, A. (2012) · Scientific Reports · doi:10.1038/srep00949 · Cited by 51 · see citing works
Self-citationAn interactive tactile graphics display
Maucher, T., Meier, K., & Schemmel, J. (2002) · International Symposium on Signal Processing and Its Applications (ISSPA) · doi:10.1109/isspa.2001.949809 · Cited by 33 · see citing works
Representing Colour Through Hearing and Touch in Sensory Substitution Devices
Hamilton-Fletcher, G., & Ward, J. (2013) · Multisensory Research · doi:10.1163/22134808-00002434 · Cited by 31 · see citing works
Color improves “visual†acuity via sound
Levy-Tzedek, S., Riemer, D., & Amedi, A. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00358 · Cited by 30 · see citing works
Self-citationSella, I., Reiner, M., & Pratt, H. (2013) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2013.11.003 · Cited by 25 · see citing works
WATANABE, T., YAMAGUCHI, T., & MINATANI, K. (2015) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2014edp7317 · Cited by 24 · see citing works
Maidenbaum, S., Arbel, R., Buchs, G., Shapira, S., & Amedi, A. (2014) · Mediterranean Conference on Control and Automation (MED) · doi:10.1109/med.2014.6961368 · Cited by 22 · see citing works
Self-citationAuvray, M., Philipona, D., O'Regan, J. K., & Spence, C. J. (2007) · Perception · doi:10.1068/p5800 · Cited by 22 · see citing works
Stiles, N. R. B., Zheng, Y., & Shinsuke Shimojo, A. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00842 · Cited by 14 · see citing works
See ColOr: an extended sensory substitution device for the visually impaired
Gómez, J. D., Bologna, G., & Pun, T. (2014) · Journal of Assistive Technologies · doi:10.1108/jat-08-2013-0025 · Cited by 14 · see citing works
Maidenbaum, S., Abboud, S., Buchs, G., & Amedi, A. (2015) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr.2015.7223381 · Cited by 8 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Kim D. (1984). Functional roles of the inner and outer hair cell subsystems in the cochlea and brainstem In Berlin C.I. (Ed.), Hearing science: Recent advantages (pp. 241– 261). San Diego: College-Hill Press.
Cited by
Sorted by citations
Designing sensory-substitution devices: Principles, pitfalls and potential1
Kristjánsson, Á., Moldoveanu, A., Jóhannesson, Ó. I., Balan, O., Spagnol, S., Valgeirsdóttir, V. V., & Unnthorsson, R. (2016) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-160647 · Cited by 116 · see citing works
“Even users with no visual experience can integrate information across successive zooms (Buchs et al., 2015).”
BackgroundWaist-up protection for blind individuals using the EyeCane as a primary and secondary mobility aid
Buchs, G., Simon, N., Maidenbaum, S., & Amedi, A. (2017) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-160686 · Cited by 25 · see citing works
Self-citationA self-training program for sensory substitution devices
Buchs, G., Haimler, B., Kerem, M., Maidenbaum, S., Braun, L., & Amedi, A. (2021) · PLoS ONE · doi:10.1371/journal.pone.0250281 · Cited by 13 · see citing works
Self-citationVisuo-spatial imagery in dreams of congenitally and early blind: a systematic review
Ilić, K., Bertani, R., Lapteva, N., Drakatos, P., Delogu, A., Raheel, K., Soteriou, M., Mutti, C., Steier, J. S., Carmichael, D. W., et al. (2023) · Frontiers in Integrative Neuroscience · doi:10.3389/fnint.2023.1204129 · Cited by 12 · see citing works
Maimon, A., Wald, I. Y., Oz, M. B., Codron, S., Netzer, O., Heimler, B., & Amedi, A. (2023) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2022.1058093 · Cited by 9 · see citing works
Self-citationAutoencoding sensory substitution
Tóth, V., & Parkkonen, L. (2019) · Unpublished · doi:10.13140/rg.2.2.10576.87048 · Cited by 7 · see citing works
Advances in Technology-Assisted Visual Rehabilitation Therapies: Scoping Review
Briceño, M. S., López, M. P. L., Suárez, L. V. B., Hernández, J. C. T., & Rincón, E. H. H. (2026) · Journal of Medical Systems · doi:10.1007/s10916-026-02410-4
Park, S., Bak, S., & Lee, H. (2026) · British Journal of Visual Impairment · doi:10.1177/02646196261450976
- Authors
- Buchs, G., Maidenbaum, S., Levy-Tzedek, S., & Amedi, A.
- Affiliations
- Hebrew University of Jerusalem (IL)
- Recanati School for Community Health Professions, Department of Physical Therapy, Ben Gurion University of the Negev, Beer-Sheva, Israel
- Institut de la Vision (FR)
- Corresponding author
- Amedi, Amir
- Venue
- Restorative Neurology and Neuroscience
- Publisher
- SAGE Publications
- Published
- 2015-10-28
- Volume
- 34
- Issue
- 1
- Pages
- 97-105
- ISSN
- 0922-6028 (ISSN-L)
- 1878-3627
- PMC
- PMC4927841
- OpenAlex
- W1863754572
- Cited by
- 8
- References
- 43
- Citations by year
- 2026: 2 · 2023: 2 · 2021: 1 · 2019: 1 · 2017: 1 · 2016: 1
- FWCI
- 0.40
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Restorative Neurology and Neuroscience (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- Leona M. and Harry B. Helmsley Charitable Trust
- Israel Science Foundation (1684/08)
- Ben-Gurion University of the Negev
- FP7 Ideas: European Research Council (310809)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Keywords
- sensory substitution devices, visual rehabilitation, congenital blindness, face recognition (cognition), perceptual integration
- MeSH terms
- Acoustic Stimulation, Adult, Algorithms, Blindness, Face, Female, Humans, Photic Stimulation, Practice, Psychological, User-Computer Interface
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
