The DOI matches its Crossref record; 224 of the 229 references it cites were found in a registry or an index; 30 of the 150 sources its claims cite were checked for support.
Designing sensory-substitution devices: Principles, pitfalls and potential1
Open accessAbstract
An exciting possibility for compensating for loss of sensory function is to augment deficient senses by conveying missing information through an intact sense. Here we present an overview of techniques that have been developed for sensory substitution (SS) for the blind, through both touch and audition, with special emphasis on the importance of training for the use of such devices, while highlighting potential pitfalls in their design. One example of a pitfall is how conveying extra information about the environment risks sensory overload. Related to this, the limits of attentional capacity make it important to focus on key information and avoid redundancies. Also, differences in processing characteristics and bandwidth between sensory systems severely constrain the information that can be conveyed. Furthermore, perception is a continuous process and does not involve a snapshot of the environment. Design of sensory substitution devices therefore requires assessment of the nature of spatiotemporal continuity for the different senses. Basic psychophysical and neuroscientific research into representations of the environment and the most effective ways of conveying information should lead to better design of sensory substitution systems. Sensory substitution devices should emphasize usability, and should not interfere with other inter- or intramodal perceptual function. Devices should be task-focused since in many cases it may be impractical to convey too many aspects of the environment. Evidence for multisensory integration in the representation of the environment suggests that researchers should not limit themselves to a single modality in their design. Finally, we recommend active training on devices, especially since it allows for externalization, where proximal sensory stimulation is attributed to a distinct exterior object.
Overview
CoChat Summary
TL;DR
This review argues that sensory substitution devices for blindness should be designed around psychophysical limits, attentional capacity, task focus, and multisensory integration, with active training to promote externalization. It surveys haptic and auditory approaches and highlights pitfalls such as sensory overload, bandwidth mismatches, and insufficient basic research.
Key findings
- Sensory bandwidth differs greatly: the optic nerve contains over 1 million fibers versus around 30,000 in the auditory nerve, and vision has four orders of magnitude greater bandwidth than haptic perception; two-point tactile thresholds range from ∼3–4 mm on the fingers to over 40 mm on the thighs and calves.
- Training and externalization are central: Bach-y-Rita et al. (1969) found object recognition after 5–15 h of training; Sampaio et al. (2001) reported tongue-SSD acuity of 20/860 on average, doubling after 9 h of training; Striem-Amit et al. (2012a) used 73 h of vOICe training, with 55% showing acuity above the blindness threshold; Auvray et al. (2007) found object identification after 15 h of vOICe training.
- Design should avoid sensory overload and task-irrelevant information; attentional limits and within-modality interference mean devices should convey critical information and not interfere with natural audition, while multisensory integration suggests combining haptics and audition may help.
- Neural evidence indicates that “visual” areas can serve supramodal functions in blind users, with activation of regions such as LOC, VWFA, and FFA during tactile or auditory tasks, and active interaction may produce more visual-cortex activity than passive processing.
Methods
- Design: narrative review/overview of sensory substitution for blindness through touch and audition, emphasizing training, design pitfalls, and potential.
- Data or sample: synthesizes prior psychophysical, neuroimaging, TMS, animal-deprivation, and SSD studies; no new empirical sample is reported.
- Main analysis: thematic discussion of principles, pitfalls, and potential, including bandwidth, attention, externalization, multisensory integration, and training.
Limitations
- The paper states that the literature lacks clear and precise training guidelines for SSDs and that no studies have yet addressed training systematically; evidence for enhanced brain capacity through training is scarce, and video-game/brain-training claims are confounded or task-specific.
- The paper notes controversy over whether blind individuals have better tactile acuity and caveats that multisensory integration can be worse with impaired senses, with visual experience possibly needed for crossmodal calibration.
- Evident from the text: as a narrative review, it does not report a systematic search or selection protocol or primary data, and many conclusions rely on cited studies with varying designs and populations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
26 Valid · 4 Not valid · 74 Not checked · 40 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Brown, Simpson and Proulx (2014) found that there is an upper limit to representation of auditory objects due to intrinsic limits of transmission in auditory cortical areas.”
§ Designing sensory-substitution devices: Principles, pitfalls and potential1 > Key general considerations for sensory substitution
What the sources say
- Partly supportedcites Brown, Simpson and Proulx (2014) sourceExperimental — accuracy not yet measured
The Discussion reports an apparent 8 × 8-pixel ceiling in object recognition by naïve sensory-substitution users and proposes that limits in the ascending auditory hierarchy may explain it. It does not establish intrinsic cortical transmission limits as the cause; the authors describe that explanation as a postulate. Page numbers are not provided in our copy. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Such evidence suggests that the loss of one perceptual channel can be compensated for by higher resolution activity in another (Gougoux et al., 2004; Stevens & Weaver, 2009).”
§ Designing sensory-substitution devices: Principles, pitfalls and potential1 > Neural evidence regarding sensory substitution
What the sources say
- Not checked — no full text of the sourcecites Gougoux et al., 2004 source
- Partly supportedcites Stevens & Weaver, 2009 sourceExperimental — accuracy not yet measured
The paper’s discussion of early-onset blindness (page unavailable in our copy) reports superior auditory abilities, including greater temporal and spectral processing, and proposes that auditory cortical plasticity may contribute. Its own results concern altered activity during passive listening, not a measured increase in auditory resolution. The claim also generalizes from vision and hearing to any pair of perceptual channels. (LLM, conf=0.84)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“The literature on multimodal interactions (see e.g. Deroy, Chen & Spence, 2014; Driver & Spence, 2000; Koelewijn, Bronkhorst & Theeuwes, 2010; Kristjánsson, Thorvaldsson, & Kristjánsson, 2014; Stein & Meredith, 1993) makes clear that a holistic picture of perceptual representation cannot be constructed without understanding multisensory interactions.”
§ Designing sensory-substitution devices: Principles, pitfalls and potential1 > Research on similarities and differences between perceptual processing in different modalities
What the sources say
- Supportedcites Deroy, Chen & Spence, 2014 source
The Introduction argues that the paradigmatic case of perceptual awareness is likely to be multisensory and criticizes theories that study awareness one sense at a time. This supports the citing sentence’s broad point that an account of perception needs to consider multisensory interactions; the paper does not assume that those interactions always produce a unified percept. (LLM, conf=0.88)
- Not checked — no full text of the sourcecites Driver & Spence, 2000 source
- Not matchedcites Stein & Meredith, 1993
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Although this presents many challenges, a feedback system that codes the environment through haptics and audition in conjunction may compensate for the drawbacks of using only each individual sense if strengths are strategically utilized and weaknesses avoided (Shull & Damian, 2015).”
§ Designing sensory-substitution devices: Principles, pitfalls and potential1 > The possibility of combining haptics and audition for sensory substitution
What the sources say
- Partly supportedcites Shull & Damian, 2015 sourceExperimental — accuracy not yet measured
The “Vision aid for the blind” section (page not provided) describes haptic feedback that conveys information about objects and obstacles in the environment. The Conclusions (page not provided) advocate combining multiple haptic modalities, but the copy does not say that jointly coding the environment through haptics and audition compensates for the drawbacks of either sense alone. (LLM, conf=0.83)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Neural activity related to multisensory interactions can be seen surprisingly close to cortical areas typically considered unimodal (Driver & Noesselt, 2008).”
§ Designing sensory-substitution devices: Principles, pitfalls and potential1 > Research on similarities and differences between perceptual processing in different modalities
What the sources say
- Supportedcites Driver & Noesselt, 2008 source
The section ‘Different Accounts and Possible Architectures for Multisensory Influences on “Sensory-Specific” Processing’ describes multisensory zones adjacent to sensory-specific cortex and rapid multisensory effects in and around auditory cortex. This supports the claim that such activity occurs close to areas traditionally considered unimodal; no page number is supplied in our copy. (LLM, conf=0.92)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
86 more claims · 18 with their cited sources checked
Create a free account to see all 144 claims and every source verdict.
- Probably not a substantial claim. 30
- It may state two separate claims as one. 32
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Designing sensory-substitution devices: Principles, pitfalls and potential1 > Neural evidence regarding sensory substitution
“Poirier et al. (2007; see also Renier et al., 2005) show how tactile stimulation can cause activity change in visual areas.”
§ Designing sensory-substitution devices: Principles, pitfalls and potential1 > References
“Poirier C., De Volder A., Tranduy D., & Scheiber C.,. 2007 Pattern recognition using a device substituting audition for vision in blindfolded sighted subjects.”
Citations
Cites
The fusiform face area: a module in human extrastriate cortex specialized for face perception
Kanwisher, N., McDermott, J., & Chun, M. M. (1997) · The Journal of Neuroscience · doi:10.1523/jneurosci.17-11-04302.1997 · Cited by 7,738 · cited 11×
“Furthermore, recruitment of unused visual cortex can help with compensation for the loss of vision (Pasqualotto & Proulx, 2012; Merabet & PascualLeone, 2010).”
BackgroundInfluentialHearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183 · cited 2×
“The McGurk effect is another example of a multimodal illusion where lip movements and heard speech interact (McGurk & MacDonald, 1976).”
BackgroundControlled and automatic human information processing: I. Detection, search, and attention.
Schneider, W., & Shiffrin, R. M. (1977) · Psychological Review · doi:10.1037/0033-295x.84.1.1 · Cited by 5,902 · cited 1×
“Humans can only process a limited amount of the information hitting their senses at a given time, since attentional capacity is severely limited (Broadbent, 1958; Treisman, 1960; Neisser, 1967; Most et al., 2005; Kristjánsson, 2006; Schneider & Shiffrin, 1977).”
BackgroundHumans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844 · cited 1×
“In another study, Ernst and Banks (2002) observed that while both haptics and vision conveyed accurate information about the height of a bar this was much less reliable for touch than vision.”
Background(2016) · Interpersonal Communication · doi:10.4324/9781315506173-8 · Cited by 4,693
Broadbent, D. E. (1958) · doi:10.1037/10037-000 · Cited by 4,337
PENFIELD, W., & BOLDREY, E. (1937) · Brain · doi:10.1093/brain/60.4.389 · Cited by 4,188 · cited 2×
“They found that cross-modal plasticity developed rapidly and that the occipital cortex was involved in tactile discrimination.”
BackgroundHenry, M. A. (1995) · The Journal of the American Dental Association · doi:10.14219/jada.archive.1995.0108 · Cited by 3,416
The period of susceptibility to the physiological effects of unilateral eye closure in kittens
Hubel, D. H., & Wiesel, T. N. (1970) · The Journal of Physiology · doi:10.1113/jphysiol.1970.sp009022 · Cited by 3,030 · cited 1×
“The brain is capable of considerable reorganization of neural function (Held et al., 2011; Rauschecker, 1995), while younger brains are typically more flexible (Hubel & Wiesel, 1970; Merabet & Pascual-Leone, 2010).”
BackgroundTo See or not to See: The Need for Attention to Perceive Changes in Scenes
Rensink, R. A., O'Regan, J. K., & Clark, J. J. (1997) · Psychological Science · doi:10.1111/j.1467-9280.1997.tb00427.x · Cited by 2,338
Research works printed but not linked: 5
- Grimes, 1996, Vancouver studies in cognitive science: 5. Perception, 89
- Neisser, 1967, Cognitive Psychology.
- Stein, 1993, The Merging of the Senses.
- Weinstein, 1968, The Skin Senses, 195
- Wurtz, 2000, of neural science, 523
Cited by
Human augmentation: Past, present and future
Raisamo, R., Rakkolainen, I., Majaranta, P., Salminen, K., Rantala, J., & Farooq, A. (2019) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2019.05.008 · Cited by 236
“Augmented senses can also enable sensory substitution or sensory prosthesis, in which information from one sense can be mediated through a different sense (see examples of sensory substitution: Kristjánsson et al., 2016; Wright et al., 2015).”
BackgroundEXTENDED REALITY (XR) ENVIRONMENTS
Stanney, K. M., Nye, H., Haddad, S. D., Hale, K. S., Padron, C. K., & Cohn, J. V. (2021) · Wiley eBooks · doi:10.1002/9781119636113.ch30 · Cited by 93
Developments in the human machine interface technologies and their applications: a review
Singh, H. P., & Kumar, P. (2021) · Journal of Medical Engineering & Technology · doi:10.1080/03091902.2021.1936237 · Cited by 73
E-Skins: Biomimetic Sensing and Encoding for Upper Limb Prostheses
Iskarous, M. M., & Thakor, N. V. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2939369 · Cited by 60
“[17] Á.”
BackgroundInfluentialHoffmann, R., Valgeirsdóttir, V. V., Jóhannesson, Ó. I., Unnþórsson, R., & Kristjansson, A. K. (2018) · Experimental Brain Research · doi:10.1007/s00221-018-5387-z · Cited by 54
Self-citation
Metadata
- Authors
- Kristjánsson, Á., Moldoveanu, A., Jóhannesson, Ó. I., Balan, O., Spagnol, S., Valgeirsdóttir, V. V., & Unnthorsson, R.
- Affiliations
- Laboratory of Visual Perception and Visuomotor control, University of Iceland, Faculty of Psychology, School of Health Sciences, Reykjavik, Iceland
- Universitatea Națională de Știință și Tehnologie Politehnica București (RO)
- University of Iceland (IS)
- Faculty of Industrial Engineering, Mechanical Engineering and Computer Science, School of Engineering and Natural Sciences, Reykjavik, Iceland
- Venue
- Restorative Neurology and Neuroscience
- Publisher
- SAGE Publications
- Published
- 2016-09-21
- Volume
- 34
- Issue
- 5
- Pages
- 769-787
- ISSN
- 0922-6028 (ISSN-L)
- 1878-3627
- PMC
- PMC5044782
- OpenAlex
- W2510919644
- Cited by
- 116
- References
- 229
- Citations by year
- 2026: 13 · 2025: 14 · 2024: 6 · 2023: 16 · 2022: 14 · 2021: 14 · 2020: 9 · 2019: 12 · 2018: 14 · 2017: 4
- FWCI
- 1.07
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Restorative Neurology and Neuroscience (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- AirIuav (Università Iuav di Venezia) (submittedVersion, other-oa)
- Opin vísindi (Opin vísindi) (submittedVersion, cc-by-nc)
- Funders
- European Commission (HORIZON2020, 643636)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Keywords
- sensory substitution, blind people, sensory deprivation, sensory overload, attentional capacity, spatiotemporal continuity, multisensory integration, device training, externalization, usability
- MeSH terms
- Biofeedback, Psychology, Humans, Perception, Psychophysics, Sensation Disorders, Sensory Aids, Neuroimaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Designing sensory-substitution devices: Principles, pitfalls and potential1. https://app.cochat.ai/papers/10.3233/rnn-160647
- The paper
- Kristjánsson, Á., Moldoveanu, A., Jóhannesson, Ó. I., Balan, O., Spagnol, S., Valgeirsdóttir, V. V., & Unnthorsson, R. (2016). Designing sensory-substitution devices: Principles, pitfalls and potential1. Restorative Neurology and Neuroscience. https://doi.org/10.3233/rnn-160647
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.