CoChat
Sign Up
Cited by 25Tactile and Sensory InteractionsMultisensory perception and integrationVirtual Reality Applications and Impacts

Sensory augmentation: integration of an auditory compass signal into human perception of space

Open access
Schumann, Frank, O’Regan, J. Kevin·2017·Scientific Reports
DOI 10.1038/srep42197PMC PMC5307328
Open this paper in CoChat — free accountOpen PDF

Abstract

Bio-mimetic approaches to restoring sensory function show great promise in that they rapidly produce perceptual experience, but have the disadvantage of being invasive. In contrast, sensory substitution approaches are non-invasive, but may lead to cognitive rather than perceptual experience. Here we introduce a new non-invasive approach that leads to fast and truly perceptual experience like bio-mimetic techniques. Instead of building on existing circuits at the neural level as done in bio-mimetics, we piggy-back on sensorimotor contingencies at the stimulus level. We convey head orientation to geomagnetic North, a reliable spatial relation not normally sensed by humans, by mimicking sensorimotor contingencies of distal sounds via head-related transfer functions. We demonstrate rapid and long-lasting integration into the perception of self-rotation. Short training with amplified or reduced rotation gain in the magnetic signal can expand or compress the perceived extent of vestibular self-rotation, even with the magnetic signal absent in the test. We argue that it is the reliability of the magnetic signal that allows vestibular spatial recalibration, and the coding scheme mimicking sensorimotor contingencies of distal sounds that permits fast integration. Hence we propose that contingency-mimetic feedback has great potential for creating sensory augmentation devices that achieve fast and genuinely perceptual experiences.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/srep42197

Cites

20 of 20 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Human direction hearing is a naturally integrated sense requiring less training to perceive spatial information accurately.

    “In contrast, human direction hearing is a naturally integrated sense requiring less training to perceive spatial information accurately [17–19].”

    From An investigation into the effectiveness of using acoustic touch to assist people who are blind (2023) · cites this paper as [18]

    The Discussion (page not available) says the auditory compass can interface with existing spatial processes through natural auditory localization, and reports that most participants immediately perceived its sound as coming from a stable direction. Experiment 2 reports accurate use after 25 minutes of training. The paper does not establish the broader claim that human direction hearing generally requires less training to perceive spatial information accurately. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 75

  1. Bach-y-Rita P., Collins C. C., Saunders F. A., White B. & Scadden L. Vision substitution by tactile image projection. Nature 221, 963– 4 (1969). 5818337
  2. Deroy O. & Auvray M. Reading the World through the Skin and Ears: A New Perspective on Sensory Substitution. Front. Psychol. 3, 1– 13 (2012). 23162506 PMC3491585
  3. Renier L.. Cross-modal activation of visual cortex during depth perception using auditory substitution of vision. Neuroimage 26, 573– 80 (2005). 15907314
  4. Auvray M., Hanneton S. & O’Regan J. K. Learning to perceive with a visuo-auditory substitution system: localisation and object recognition with ‘the vOICe’. Perception 36, 416– 30 (2007). 17455756
  5. Bermejo F., Di Paolo E., Hug M. X. & Arias C. Sensorimotor strategies for recognizing geometrical shapes: a comparative study with different sensory substitution devices. Front. Psychol. 6, 1– 20 (2015). 26106340 PMC4460306
  6. Froese T., McGann M., Bigge W., Spiers A. & Seth A. K. The Enactive Torch: A New Tool for the Science of Perception. IEEE Trans. Haptics doi: 10.1109/TOH.2011.57 (2011). 26964133 · DOI
  7. Norimoto H. & Ikegaya Y. Visual cortical prosthesis with a geomagnetic compass restores spatial navigation in blind rats. Curr. Biol. 25, 1091– 1095 (2015). 25843028
  8. König S. U.. Learning New Sensorimotor Contingencies: Effects of Long-Term Use of Sensory Augmentation on the Brain and Conscious Perception. PLoS One 11, e0166647 (2016). 27959914 PMC5154504
  9. Kaspar K., König S., Schwandt J. & König P. The experience of new sensorimotor contingencies by sensory augmentation. Conscious. Cogn. 28, 47– 63 (2014). 25038534 PMC4154453
  10. Proulx M. J., Brown D. J., Pasqualotto A. & Meijer P. Multisensory perceptual learning and sensory substitution. Neurosci. Biobehav. Rev. 41, 16– 25 (2014). 23220697
  11. Renier L. & De Volder A. G. Vision substitution and depth perception: early blind subjects experience visual perspective through their ears. Disabil. Rehabil. Assist. Technol. 5, 175– 183 (2010). 20214472
  12. Capelle C., Trullemans C., Arno P. & Veraart C. A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution. IEEE Trans. Biomed. Eng. 45, 1279– 1293 (1998). 9775542
  13. Ahissar M. & Hochstein S. Task difficulty and the specificity of perceptual learning. Nature 387, 401– 6 (1997). 9163425
  14. Pascual-Leone A. & Hamilton R. The metamodal organization of the brain. Vis. From Neurons to Cogn. 134, 427– 445 (2001). 11702559
  15. Collignon O., Lassonde M., Lepore F., Bastien D. & Veraart C. Functional cerebral reorganization for auditory spatial processing and auditory substitution of vision in early blind subjects. Cereb. Cortex 17, 457– 65 (2007). 16581983
  16. Collignon O., Voss P., Lassonde M. & Lepore F. Cross-modal plasticity for the spatial processing of sounds in visually deprived subjects. Exp. Brain Res. 192, 343– 58 (2009). 18762928
  17. Poirier C., De Volder A. G. & Scheiber C. What neuroimaging tells us about sensory substitution. Neurosci. Biobehav. Rev. 31, 1064– 1070 (2007). 17688948
  18. Ward J. & Meijer P. Visual experiences in the blind induced by an auditory sensory substitution device. Conscious. Cogn. 19, 492– 500 (2010). 19955003
  19. Tabot G.. Restoring the sense of touch with a prosthetic hand through a brain interface. Proc. Natl. Acad. Sci. 110, 18279– 18284 (2013). 24127595 PMC3831459
  20. Oddo C. M.. Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans. Elife 5, 1– 27 (2016). PMC4798967 26952132
  21. Dadarlat M. C.. A learning-based approach to artificial sensory feedback leads to optimal integration. Nat. Neurosci. 18, 138– 144 (2014). 25420067 PMC4282864
  22. Wightman F. L. & Kistler D. J. Headphone simulation of free-field listening. I: Stimulus synthesis. J. Acoust. Soc. Am. 85, 858– 67 (1989). 2926000
  23. Seemungal B. M., Glasauer S., Gresty M. & Bronstein A. M. Vestibular perception and navigation in the congenitally blind. J. Neurophysiol. 97, 4341– 56 (2007). 17392406
  24. Seemungal B. M., Rizzo V., Gresty M. A., Rothwell J. C. & Bronstein A. M. Cortical processing in vestibular navigation. Prog. Brain Res. 171, 339– 46 (2008). 18718324
  25. Wolbers T., Wiener J. M., Mallot H. A. & Büchel C. Differential recruitment of the hippocampus, medial prefrontal cortex, and the human motion complex during path integration in humans. J. Neurosci. 27, 9408– 16 (2007). 17728454 PMC6673121
  26. Viaud-Delmon I.. Sex, lies and virtual reality. Nat. Neurosci. 1, 15– 6 (1998). 10195102
  27. Srikulwong M. & O’Neill E. Wearable tactile display of directions for pedestrian navigation: Comparative lab and field evaluations. In 2013 World Haptics Conference (WHC) 503– 508 doi: 10.1109/WHC.2013.6548459 (IEEE, 2013). · DOI
  28. Leone L. M. & McCourt M. E. Dissociation of perception and action in audiovisual multisensory integration. Eur. J. Neurosci. 42, 2915– 2922 (2015). 26417674 PMC4715611
  29. Parise C. V. & Ernst M. O. Correlation detection as a general mechanism for multisensory integration. Nat. Commun. 7, 11543 (2016). 27265526 PMC4897755
  30. Kercel S. W., Reber A. S. & Manges W. W. Some radical implications of Bach-y-Rita’s discoveries. J. Integr. Neurosci. 4, 551– 65 (2005). 16385647
  31. Bach-y-Rita P. & Kercel S. W. Sensory substitution and the human–machine interface. Trends Cogn. Sci. 7, 541– 546 (2003). 14643370
  32. Reich L., Maidenbaum S. & Amedi A. The brain as a flexible task machine. Curr. Opin. Neurol. 25, 86– 95 (2012). 22157107
  33. Striem-Amit E., Cohen L., Dehaene S. & Amedi A. Reading with sounds: sensory substitution selectively activates the visual word form area in the blind. Neuron 76, 640– 52 (2012). 23141074
  34. Leopold D. A. & Logothetis N. K. Multistable phenomena: changing views in perception. Trends Cogn. Sci. 6613, 254– 264 (1999). 10377540
  35. Tong F. Primary visual cortex and visual awareness. Nat. Rev. Neurosci. 4, 219– 29 (2003). 12612634
  36. Tong F., Meng M. & Blake R. Neural bases of binocular rivalry. Trends Cogn. Sci. 10, 502– 11 (2006). 16997612
  37. Wunderlich K., Schneider K. A. & Kastner S. Neural correlates of binocular rivalry in the human lateral geniculate nucleus. Nat Neurosci 8, 1595– 1602 (2005). 16234812 PMC1470662
  38. Berthoz A. & Jones M. Adaptive Mechanisms in Gaze Control. (Elsevier Publishing Company, 1985).
  39. Jürgens R. & Becker W. Perception of angular displacement without landmarks: evidence for Bayesian fusion of vestibular, optokinetic, podokinesthetic, and cognitive information. Exp. Brain Res. 174, 528– 543 (2006). 16832684
  40. Angelaki D. E. & Cullen K. E. Vestibular system: the many facets of a multimodal sense. Annu. Rev. Neurosci. 31, 125– 50 (2008). 18338968
  41. Ivanenko Y. P., Viaud-Delmon I., Siegler I., Israël I. & Berthoz A. The vestibulo-ocular reflex and angular displacement perception in darkness in humans: adaptation to a virtual environment. Neurosci. Lett. 241, 167– 170 (1998). 9507947
  42. Väljamäe A. Auditorily-induced illusory self-motion: A review. Brain Res. Rev. doi: 10.1016/j.brainresrev.2009.07.001 (2009). 19619584 · DOI
  43. Alais D. & Burr D. Ventriloquist Effect Results from Near-Optimal Bimodal Integration. Curr. Biol. 14, 257– 262 (2004). 14761661
  44. Zwiers M. P., Van Opstal A. J. & Paige G. D. Plasticity in human sound localization induced by compressed spatial vision. Nat. Neurosci. 6, 175– 81 (2003). 12524547
  45. Ehrenstein W. H. & Reinhardt-Rutland A. H. A cross-modal aftereffect: auditory displacement following adaptation to visual motion. Percept. Mot. Skills 82, 23– 6 (1996). 8668481
  46. Keating P. & King A. J. Sound localization in a changing world. Curr. Opin. Neurobiol. 35, 35– 43 (2015). 26126152
  47. Takahashi T. T.. The synthesis and use of the owl’s auditory space map. Biol. Cybern. 89, 378– 387 (2003). 14669018
  48. Kondo H. M., Pressnitzer D., Toshima I. & Kashino M. Effects of self-motion on auditory scene analysis. Proc. Natl. Acad. Sci. 109, 6775– 6780 (2012). 22493250 PMC3340062
  49. Goossens H. H. L. M. & van Opstal A. J. Influence of head position on the spatial representation of acoustic targets. J. Neurophysiol. 81, 2720– 2736 (1999). 10368392
  50. Ernst M. O. & Banks M. S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429– 33 (2002). 11807554
  51. Witten I. B. & Knudsen E. I. Why seeing is believing: merging auditory and visual worlds. Neuron 48, 489– 96 (2005). 16269365
  52. Wilke C., Synofzik M. & Lindner A. Sensorimotor Recalibration Depends on Attribution of Sensory Prediction Errors to Internal Causes. PLoS One 8, e54925 (2013). 23359818 PMC3554678
  53. Senna I., Parise C. V. & Ernst M. O. Hearing in slow-motion: Humans underestimate the speed of moving sounds. Sci. Rep. 5, 14054 (2015). 26370720 PMC4570192
  54. Pretto P., Bresciani J. P., Rainer G. & Bülthoff H. H. Foggy perception slows us down. Elife 2012, 1– 12 (2012). PMC3479833 23110253
  55. Jetzschke S., Ernst M. O., Moscatelli A. & Boeddeker N. Going round the bend: Persistent personal biases in walked angles. Neurosci. Lett. 617, 72– 75 (2016). 26854843
  56. Souman J. L., Frissen I., Sreenivasa M. N. & Ernst M. O. Walking Straight into Circles. Curr. Biol. 19, 1538– 1542 (2009). 19699093
  57. Moser I., Grabherr L., Hartmann M. & Mast F. W. Self-motion direction discrimination in the visually impaired. Exp. Brain Res. 233, 3221– 3230 (2015). 26223579
  58. Dieterich M., Bauermann T., Best C., Stoeter P. & Schlindwein P. Evidence for cortical visual substitution of chronic bilateral vestibular failure (an fMRI study). Brain 130, 2108– 2116 (2007). 17575279
  59. Cuturi L. F., Aggius-Vella E., Campus C., Parmiggiani A. & Gori M. From science to technology: Orientation and mobility in blind children and adults. Neurosci. Biobehav. Rev. 71, 240– 251 (2016). 27608959
  60. Gori M., Cappagli G., Tonelli A., Baud-Bovy G. & Finocchietti S. Devices for visually impaired people: High technological devices with low user acceptance and no adaptability for children. Neurosci. Biobehav. Rev. 69, 79– 88 (2016). 27484870
  61. Cassinelli A., Reynolds C. & Ishikawa M. Augmenting spatial awareness with Haptic Radar. In 2006 10th IEEE International Symposium on Wearable Computers 61– 64 doi: 10.1109/ISWC.2006.286344 (IEEE, 2006). · DOI
  62. Henze N., Heuten W. & Boll S. Non-intrusive somatosensory navigation support for blind pedestrians. In Proc. EuroHaptics 459– 464 (2006).
  63. Hayward V. A brief taxonomy of tactile illusions and demonstrations that can be done in a hardware store. Brain Res. Bull. 75, 742– 752 (2008). 18394520
  64. Hayward V. Tactile illusions. Scholarpedia 10, 8245 (2015).
  65. Moscatelli A., Naceri A. & Ernst M. O. Path integration in tactile perception of shapes. Behav. Brain Res. 274, 355– 364 (2014). 25151621
  66. Bicchi A., Scilingo E. P., Ricciardi E. & Pietrini P. Tactile flow explains haptic counterparts of common visual illusions. Brain Res. Bull. 75, 737– 741 (2008). 18394519
  67. O’Regan J. K. Why Red Doesn’t Sound Like a Bell: Understanding the feel of consciousness. doi: 10.1093/acprof:oso/9780199775224.001.0001 (Oxford University Press, 2011). · DOI · Paper page
  68. Held R. & Hein A. Movement-produced stimulation in the development of visually guided behavior. J. Comp. Physiol. Psychol. 56, 872– 876 (1963). 14050177
  69. Mriganka S., Garraghty P. E., Roe A. W. & Sur M. Experimentally induced visual projections into auditory thalamus and cortex. Science (80-.). 242, 1437– 1441 (1988). 2462279
  70. Pascual-Leone A., Amedi A., Fregni F. & Merabet L. B. The plastic human brain cortex. Annu. Rev. Neurosci. 28, 377– 401 (2005). 16022601
  71. Aoki C., Romeo R. D. & Smith S. S. Adolescence as a Critical Period for Developmental Plasticity. Brain Res. 1654, 85– 86 (2017). 27955766
  72. Hubel T. W. & D. Effects of visual deprivation on morphology and physiology of cells in the cat’s lateral geniculate body. J Neurophysiol 26, 978– 993 (1963). 14084170
  73. Sur M. & Leamey C. A. Development and plasticity of cortical areas and networks. Neuroscience 2 (2001). 11283748
  74. Maidenbaum S., Abboud S. & Amedi A. Sensory substitution: closing the gap between basic research and widespread practical visual rehabilitation. Neurosci. Biobehav. Rev. 41, 3– 15 (2014). 24275274
  75. Ivanenko Y., Grasso R., Israël I. & Berthoz A. Spatial orientation in humans: Perception of angular whole-body displacements in two-dimensional trajectories. Exp. Brain Res. 117, 419– 427 (1997). 9438709

Cited by

Sorted by citations

Authors
Schumann, F., & O’Regan, J. K.
Affiliations
Centre National de la Recherche Scientifique (FR)
Corresponding author
Schumann, Frank
ORCID
Schumann, Frank
Venue
Scientific Reports
Publisher
Springer Science and Business Media LLC
Published
2017-02-14
Volume
7
Issue
1
Pages
42197-42197
ISSN
2045-2322 (ISSN-L)
DOI
10.1038/srep42197
PMC
PMC5307328
OpenAlex
W2589035303
Cited by
25
References
20
Citations by year
2026: 1 · 2025: 1 · 2024: 4 · 2023: 4 · 2022: 4 · 2021: 5 · 2020: 1 · 2019: 1 · 2018: 4
FWCI
0.94
Type
journal article
Language
en
Update notices
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Edinburgh Research Explorer (University of Edinburgh) (publishedVersion, cc-by)
Funders
  • Université Paris Descartes
  • FP7 Ideas: European Research Council (323674)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
  • Physical Sciences › Computer Science › Human-Computer Interaction › Virtual Reality Applications and Impacts
Keywords
sensory augmentation, sensorimotor contingencies, head-related transfer functions, perceptual integration
MeSH terms
Adult, Auditory Perception, Female, Head, Humans, Male, Orientation, Rotation, Space Perception, Vestibule, Labyrinth, Feedback, Sensory, Magnetic Fields

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review