The perception of visual images encoded in musical form: a study in cross-modality information transfer
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
This study demonstrates the ability of blind (previously sighted) and blindfolded (sighted) subjects in reconstructing and identifying a number of visual targets transformed into equivalent musical representations. Visual images are deconstructed through a process which selectively segregates different features of the image into separate packages. These are then encoded in sound and presented as a polyphonic musical melody which resembles a Baroque fugue with many voices, allowing subjects to analyse the component voices selectively in combination, or separately in sequence, in a manner which allows a subject to patch together and bind the different features of the object mentally into a mental percept of a single recognizable entity. The visual targets used in this study included a variety of geometrical figures, simple high-contrast line drawings of man-made objects, natural and urban scenes, etc., translated into sound and presented to the subject in polyphonic musical form.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An experimental system for auditory image representations
Meijer, P. B. L. (1992) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.121642 · Cited by 900
Gregory, R. L. (1980) · Philosophical Transactions of the Royal Society of London. B, Biological Sciences · doi:10.1098/rstb.1980.0090 · Cited by 789
Self-citation
Cited by
Auvray, M., Hanneton, S., & O'Regan, J. K. (2007) · Perception · doi:10.1068/p5631 · Cited by 268
Cortical plasticity and preserved function in early blindness
Renier, L., De Volder, A. G., & Rauschecker, J. P. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.01.025 · Cited by 144
Sensory substitution as an artificially acquired synaesthesia
Ward, J., & Wright, T. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.07.007 · Cited by 94
Proulx, M. J., Stoerig, P., Ludowig, E., & Knoll, I. (2008) · PLoS ONE · doi:10.1371/journal.pone.0001840 · Cited by 81
How well do you see what you hear? The acuity of visual-to-auditory sensory substitution
Haigh, A., Brown, D. J., Meijer, P., & Proulx, M. J. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00330 · Cited by 72
- Sensory Substitution of Vision: Importance of Perceptual and Cognitive Processing
(2018) · Assistive Technology for Blindness and Low Vision · doi:10.1201/9781315216935-12 · Cited by 51
Navigating from a Depth Image Converted into Sound
Stoll, C., Palluel-Germain, R., Fristot, V., Pellerin, D., Alleysson, D., & Graff, C. (2015) · Applied Bionics and Biomechanics · doi:10.1155/2015/543492 · Cited by 45
Blind subjects construct conscious mental images of visual scenes encoded in musical form
Cronly–Dillon, J., Persaud, K. C., & Blore, R. (2000) · Proceedings of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rspb.2000.1273 · Cited by 35
Self-citationColor improves “visual†acuity via sound
Levy-Tzedek, S., Riemer, D., & Amedi, A. (2014) · Frontiers in Neuroscience · doi:10.3389/fnins.2014.00358 · Cited by 30
SoundSight: a mobile sensory substitution device that sonifies colour, distance, and temperature
Hamilton-Fletcher, G., Alvarez, J., Obrist, M., & Ward, J. (2022) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-021-00376-w · Cited by 28
“Beyond the vOICe, this spectrogram-like approach of using pitch-height, panning-laterality, and loudness-brightness has underpinned many other SSD design choices with minor variations, such as presenting the information all-atonce (PSVA [19]; The Vibe [20]), musically (SmartSight [21]), or adding colour information th…”
BackgroundTesting vision is not testing for vision
Peli, E. (2020) · Translational Vision Science & Technology · doi:10.1167/tvst.9.13.32 · Cited by 14
Analysis and Validation of Cross-Modal Generative Adversarial Network for Sensory Substitution
Kim, M., Park, Y., Moon, K., & Jeong, C. Y. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18126216 · Cited by 8
Metadata
- Authors
- Cronly-Dillon, J., Persaud, K., & Gregory, R. P. F.
- Venue
- Proceedings of the Royal Society of London. Series B: Biological Sciences
- Publisher
- The Royal Society
- Published
- 1999-12-07
- Volume
- 266
- Issue
- 1436
- Pages
- 2427-2433
- ISSN
- 0080-4649 (ISSN-L)
- 0962-8452
- 1471-2954
- OpenAlex
- W2054647727
- Cited by
- 76
- References
- 2
- FWCI
- 0.71
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.