The evolution of a visual-to-auditory sensory substitution device using interactive genetic algorithms
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Sensory substitution is a promising technique for mitigating the loss of a sensory modality. Sensory substitution devices (SSDs) work by converting information from the impaired sense (e.g., vision) into another, intact sense (e.g., audition). However, there are a potentially infinite number of ways of converting images into sounds, and it is important that the conversion takes into account the limits of human perception and other user-related factors (e.g., whether the sounds are pleasant to listen to). The device explored here is termed “polyglot” because it generates a very large set of solutions. Specifically, we adapt a procedure that has been in widespread use in the design of technology but has rarely been used as a tool to explore perception—namely, interactive genetic algorithms. In this procedure, a very large range of potential sensory substitution devices can be explored by creating a set of “genes” with different allelic variants (e.g., different ways of translating luminance into loudness). The most successful devices are then “bred” together, and we statistically explore the characteristics of the selected-for traits after multiple generations. The aim of the present study is to produce design guidelines for a better SSD. In three experiments, we vary the way that the fitness of the device is computed: by asking the user to rate the auditory aesthetics of different devices (Experiment 1), and by measuring the ability of participants to match sounds to images (Experiment 2) and the ability to perceptually discriminate between two sounds derived from similar images (Experiment 3). In each case, the traits selected for by the genetic algorithm represent the ideal SSD for that task. Taken together, these traits can guide the design of a better SSD.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Solving the “real” mysteries of visual perception: The world as an outside memory
O'Regan, J. K. (1992) · Canadian Journal of Psychology / Revue canadienne de psychologie · doi:10.1037/h0084327 · Cited by 997 · cited 1×
“In order for the participant to link an auditory component (e.g., a high-pitch sound in the second time point) to an external location/object that can be acted upon, they must also “ embody ” the device itself (Brown et al., 2011; O ’ Regan, 1992).”
Vision substitution by tactile image projection
BACH-Y-RITA, P., COLLINS, C. C., SAUNDERS, F. A., WHITE, B., & SCADDEN, L. (1969) · Nature · doi:10.1038/221963a0 · Cited by 956
An experimental system for auditory image representations
Meijer, P. B. L. (1992) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.121642 · Cited by 900
Artificial evolution for computer graphics
Sims, K. (1991) · Proceedings of the 18th annual conference on Computer graphics and interactive techniques · doi:10.1145/122718.122752 · Cited by 741 · cited 1×
“Other examples include computer graphics and animations (Sims, 1991) as well as architecture and product design (Soddu, 2002).”
Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex
Amedi, A., Stern, W. M., Camprodon, J. A., Bermpohl, F., Merabet, L., Rotman, S., Hemond, C., Meijer, P., & Pascual-Leone, A. (2007) · Nature Neuroscience · doi:10.1038/nn1912 · Cited by 451 · cited 1×
“Expert blind users recruit “ visual ” cortices to process the substituted sense (Amedi et al., 2007; Merabet et al., 2009; Poirier, De Volder, Tranduy, & Scheiber, 2007; cf. Pollok, Schnitzler, Stoerig, Mierdorf, & Schnitzler, 2005).”
Parise, C. V., & Spence, C. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005664 · Cited by 272 · cited 5×
“…perspective, there are reliable “ rules ” that people adopt when linking together auditory and visual features — for instance, between pitch and size (Parise & Spence, 2009), pitch and space (Melara & O ’ Brien, 1987; Pratt, 1930), loudness and luminance (Marks et al., 1987), and pitch and…”
InfluentialNeural correlates of virtual route reCognition in congenital blindness
Kupers, R., Chebat, D. R., Madsen, K. H., Paulson, O. B., & Ptito, M. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1006199107 · Cited by 187 · cited 1×
“Although visually impaired people tend to perform better (Arno et al., 2001) and undergo functional changes to their brains (Kupers, Chebat, Madsen, Paulson, & Ptito, 2010; Ortiz et al., 2011), blind-folded sighted participants can complete sensory substitution tasks and are not necessarily…”
There is something out there: Distal attribution in sensory substitution, twenty years later
AUVRAY, M., HANNETON, S., LENAY, C., & O'REGAN, K. (2005) · Journal of Integrative Neuroscience · doi:10.1142/s0219635205001002 · Cited by 112 · cited 1×
“…the Vibe, is similar to the tactile systems in that it presents the whole fi eld of view at once and relies on the natural localization abilities of the ear by expressing horizontal (left/right) position by controlling the relative amplitude in each ear (Auvray, Hanneton, Lenay, & O ’ Regan, 2005).”
Mesz, B., Trevisan, M. A., & Sigman, M. (2011) · Perception · doi:10.1068/p6801 · Cited by 108 · cited 1×
“Consider a recent study by Mesz, Trevisan, and Sigman (2011) in which a group of composers were asked to create musical pieces to denote tastes (e.g., salty music, sour music).”
Ortiz, T., Poch, J., Santos, J. M., Requena, C., Martínez, A. M., Ortiz-Terán, L., Turrero, A., Barcia, J., Nogales, R., Calvo, A., et al. (2011) · PLoS ONE · doi:10.1371/journal.pone.0023264 · Cited by 63
Cited by
Maidenbaum, S., Abboud, S., & Amedi, A. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2013.11.007 · Cited by 230
Abboud, S., Hanassy, S., Levy-Tzedek, S., Maidenbaum, S., & Amedi, A. (2014) · Restorative Neurology and Neuroscience · doi:10.3233/rnn-130338 · Cited by 199
Sensory substitution as an artificially acquired synaesthesia
Ward, J., & Wright, T. (2014) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.07.007 · Cited by 94
Self-citationAutoencoding sensory substitution
Tóth, V., & Parkkonen, L. (2019) · Unpublished · doi:10.13140/rg.2.2.10576.87048 · Cited by 7
“In order to optimize the V2A conversion method of TV, Wright and Ward [154] ran a genetic algorithm tied to the performance of sighted participants, who were solving SS tasks.”
Method
Metadata
- Authors
- Wright, T., & Ward, J.
- Venue
- Quarterly Journal of Experimental Psychology
- Publisher
- SAGE Publications
- Published
- 2013-08
- Volume
- 66
- Issue
- 8
- Pages
- 1620-1638
- ISSN
- 0033-555X (ISSN-L)
- 1747-0218
- 1747-0226
- OpenAlex
- W2126636462
- Cited by
- 20
- References
- 12
- FWCI
- 1.23
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The evolution of a visual-to-auditory sensory substitution device using interactive genetic algorithms. https://app.cochat.ai/papers/10.1080/17470218.2012.754911
- The paper
- Wright, T., & Ward, J. (2013). The evolution of a visual-to-auditory sensory substitution device using interactive genetic algorithms. Quarterly Journal of Experimental Psychology. https://doi.org/10.1080/17470218.2012.754911
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.