Depth and motion cues with phosphene patterns for prosthetic vision
Abstract
Recent research demonstrates that visual prostheses are able to provide visual perception to people with some kind of blindness. In visual prostheses, image information from the scene is transformed to a phosphene pattern to be sent to the implant. This is a complex problem where the main challenge is the very limited spatial and intensity resolution. Moreover, depth perception, which is relevant to perform agile navigation, is lost and codifying the semantic information to phosphene patterns remains an open problem. In this work, we consider the framework of perception for navigation where aspects such as obstacle avoidance are critical. We propose using a head-mounted RGB-D camera to detect free-space, obstacles and scene direction in front of the user. The main contribution is a new approach to represent depth information and provide motion cues by using particular phosphene patterns. The effectiveness of this approach is tested in simulation with real data from indoor environments.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Many computational models of prosthetic vision assume a linear relationship between stimulus parameters (e.g. amplitude) and phosphene appearance (e.g. brightness).
“Many computational models of prosthetic vision (Chen et al 2009, Perez-Yus et al 2017, Sanchez-Garcia et al 2019, Granley and Beyeler 2021) assume a linear relationship between stimulus parameters (e.g. amplitude) and phosphene appearance (e.g. brightness).”
From Axonal stimulation affects the linear summation of single-point perception in three Argus II users (2024) · cites this paper as Perez-Yus et al 2017
Cites
Fischler, M. A., & Bolles, R. C. (1981) · Communications of the ACM · doi:10.1145/358669.358692 · Cited by 25,893 · see citing works
The sensations produced by electrical stimulation of the visual cortex
Brindley, G. S., & Lewin, W. S. (1968) · The Journal of Physiology · doi:10.1113/jphysiol.1968.sp008519 · Cited by 1,170 · cited 1× · see citing works
“Since 1968, different research works have found that electrical stimulation of the visual cortex or other parts of the visual pathway (such as retina) caused patients to perceive bright dots of light called phosphenes [6].”
BackgroundWearable Obstacle Avoidance Electronic Travel Aids for Blind: A Survey
Dakopoulos, D., & Bourbakis, N. (2009) · IEEE Transactions on Systems Man and Cybernetics Part C (Applications and Reviews) · doi:10.1109/tsmcc.2009.2021255 · Cited by 657 · see citing works
Recovering the spatial layout of cluttered rooms
Hedau, V., Hoiem, D., & Forsyth, D. A. (2009) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459411 · Cited by 514 · see citing works
Manhattan World: compass direction from a single image by Bayesian inference
Coughlan, J. M., & Yuille, A. (1999) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1999.790349 · Cited by 497 · see citing works
Geometric reasoning for single image structure recovery
Lee, D. C., Hebert, M., & Takeo Kanade, A. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206872 · Cited by 414 · see citing works
Ahuja, A. K., Dorn, J. D., Caspi, A., McMahon, M. J., Dagnelie, G., daCruz, L., Stanga, P., Humayun, M. S., Greenberg, R. J., & Argus II Study Group (2011) · British Journal of Ophthalmology · doi:10.1136/bjo.2010.179622 · Cited by 289 · see citing works
Navigation Assistance for the Visually Impaired Using RGB-D Sensor With Range Expansion
Aladren, A., Nicolás, G. L., Puig, L., & Guerrero, J. (2014) · IEEE Systems Journal · doi:10.1109/jsyst.2014.2320639 · Cited by 232 · see citing works
Self-citationFacial Recognition Using Simulated Prosthetic Pixelized Vision
Thompson, R. W., Barnett, G. D., Humayun, M. S., & Dagnelie, G. (2003) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.03-0341 · Cited by 176 · see citing works
Simulating prosthetic vision: I. Visual models of phosphenes
Chen, S. C., Suaning, G. J., Morley, J. W., & Lovell, N. H. (2009) · Vision Research · doi:10.1016/j.visres.2009.02.003 · Cited by 172 · see citing works
Reading speed with a pixelized vision system
Cha, K., Boman, D. K., Horch, K., & Normann, R. A. (1992) · Journal of the Optical Society of America A · doi:10.1364/josaa.9.000673 · Cited by 168 · see citing works
Wang, H.-C., Katzschmann, R. K., Teng, S., Araki, B., Giarre, L., & Rus, D. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989772 · Cited by 167 · see citing works
Manhattan scene understanding using monocular, stereo, and 3D features
Flint, A., Murray, D., & Reid, I. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126501 · Cited by 132 · see citing works
Simulating prosthetic vision: II. Measuring functional capacity
Chen, S. C., Suaning, G. J., Morley, J. W., & Lovell, N. H. (2009) · Vision Research · doi:10.1016/j.visres.2009.07.003 · Cited by 59 · see citing works
Stairs detection with odometry-aided traversal from a wearable RGB-D camera
Pérez-Yus, A., Gutiérrez-Gómez, D., Nicolás, G. L., & Guerrero, J. J. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.04.007 · Cited by 53 · see citing works
Self-citationActive confocal imaging for visual prostheses
Jung, J., Aloni, D., Yitzhak Yitzhaky, B., & Eli Peli, A. (2014) · Vision Research · doi:10.1016/j.visres.2014.10.023 · Cited by 51 · see citing works
Mobility and low contrast trip hazard avoidance using augmented depth
McCarthy, C., Walker, J. G., Lieby, P., Scott, A., & Barnes, N. (2015) · Journal of Neural Engineering · doi:10.1088/1741-2560/12/1/016003 · Cited by 49 · see citing works
Dagnelie, G. (2011) · Springer eBooks · doi:10.1007/978-1-4419-0754-7 · Cited by 48 · see citing works
Simulating prosthetic vision: Optimizing the information content of a limited visual display
van Rheede, J. J., Christopher Kennard, C., & Hicks, S. L. (2010) · Journal of Vision · doi:10.1167/10.14.32 · Cited by 46 · see citing works
3D obstacle detection and avoidance in vegetated off-road terrain
Schafer, H., Hach, A., Proetzsch, M., et al. (2008) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2008.4543323 · Cited by 36 · see citing works
Application of stereovision in a navigation aid for blind people
Wong, F., Nagarajan, R., & Yaacob, S. (2003) · doi:10.1109/icics.2003.1292553 · Cited by 34 · see citing works
Gutiérrez-Gómez, D., Mayol-Cuevas, W. W., & Guerrero, J. J. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7138984 · Cited by 33 · see citing works
Self-citationGround surface segmentation for navigation with a low resolution visual prosthesis
McCarthy, C., Barnes, N., & Lieby, P. (2011) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2011.6091105 · Cited by 33 · see citing works
Peasley, B., & Birchfield, S. T. (2013) · doi:10.1109/worv.2013.6521938 · Cited by 32 · see citing works
Semantic labeling for prosthetic vision
Horne, L., Alvarez, J., McCarthy, C., Salzmann, M., & Barnes, N. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.02.015 · Cited by 30 · see citing works
Transformative Reality: Improving bionic vision with robotic sensing
Lui, W. L. D., Browne, D., Kleeman, L., Drummond, T., & Li, W. H. (2012) · 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society · doi:10.1109/embc.2012.6345929 · Cited by 23 · see citing works
Face recognition in simulated prosthetic vision: face detection-based image processing strategies
Wang, J., Wu, X., Lü, Y., Wu, H., Kan, H., & Chai, X. (2014) · Journal of Neural Engineering · doi:10.1088/1741-2560/11/4/046009 · Cited by 23 · see citing works
Reading with a Simulated 60-Channel Implant
Fornos, A. P., Sommerhalder, J., & Pelizzone, M. (2011) · Frontiers in Neuroscience · doi:10.3389/fnins.2011.00057 · Cited by 22 · see citing works
An indoor navigation aid designed for visually impaired people
Öktem, R., Aydin, E. U., & Çağıltay, N. E. (2008) · doi:10.1109/iecon.2008.4758435 · Cited by 20 · see citing works
Wearable Computer Vision Systems for a Cortical Visual Prosthesis
Li, W. H. (2013) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2013.63 · Cited by 18 · see citing works
A salient information processing system for bionic eye with application to obstacle avoidance
Stacey, A., Li, Y., & Barnes, N. (2011) · doi:10.1109/iembs.2011.6091267 · Cited by 17 · see citing works
Smart image processing system for retinal prosthesis
Weiland, J. D., Parikh, N., Pradeep, V., & Medioni, G. G. (2012) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2012.6345928 · Cited by 16 · see citing works
Vergnieux, V., Macé, M. J. ‐., & Jouffrais, C. (2014) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2014.6944151 · Cited by 15 · see citing works
Psychophysics testing of bionic vision image processing algorithms using an FPGA Hatpack
Josh, H., Mann, C., Kleeman, L., & Lui, W. L. D. (2013) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2013.6738319 · Cited by 13 · see citing works
Simulated Prosthetic Vision: Improving text accessibility with retinal prostheses
Denis, G., Jouffrais, C., Mailhes, C., & Macé, M. J. ‐. (2014) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2014.6943939 · Cited by 13 · see citing works
Enhancing scene structure in prosthetic vision using iso-disparity contour perturbance maps
Feng, D. D., & McCarthy, C. D. (2013) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2013.6610741 · Cited by 11 · see citing works
Image segmentation for enhancing symbol recognition in prosthetic vision
Horne, L., Barnes, N. M., McCarthy, C. D., & He, X. (2012) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2012.6346544 · Cited by 11 · see citing works
An overview of vision processing in implantable prosthetic vision
Barnes, N. M. (2013) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2013.6738315 · Cited by 11 · see citing works
Augmenting intensity to enhance scene structure in prosthetic vision
McCarthy, C. D., Feng, D. D., & Barnes, N. M. (2013) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew.2013.6618430 · Cited by 10 · see citing works
Zapf, M. P., Boon, M. Y., Lovell, N. H., & Suaning, G. J. (2016) · Journal of Neural Engineering · doi:10.1088/1741-2560/13/2/026022 · Cited by 10 · see citing works
Embracing the irregular: A patient-specific image processing strategy for visual prostheses
Kiral-Kornek, F. I., Savage, C. O., O'Sullivan-Greene, E., Burkitt, A. N., & Grayden, D. B. (2013) · 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2013.6610312 · Cited by 9 · see citing works
What limits spatial perception with retinal implants?
Hamish Meffin, A. (2013) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2013.6738318 · Cited by 7 · see citing works
Analysis and design of an embedded system to aid the navigation of the visually impaired
Guimarães, C. S. S., Henriques, R. V. B., & Pereira, C. E. (2013) · ISSNIP Biosignals and Biorobotics Conference: Biosignals and Robotics for Better and Safer Living · doi:10.1109/brc.2013.6487454 · Cited by 6 · see citing works
On Just Noticeable Difference for Bionic Eye
Li, Y., McCarthy, C. D., & Barnes, N. (2012) · doi:10.1109/embc.2012.6346585 · Cited by 6 · see citing works
A Fast and Flexible Computer Vision System for Implanted Visual Prostheses
Li, W. H. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16199-0_48 · Cited by 6 · see citing works
RGB-D Computer Vision Techniques for Simulated Prosthetic Vision
Bermúdez-Cameo, J., Badías, A., Guerrero-Viu, M., Nicolás, G. L., & Guerrero, J. J. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-58838-4_47 · Cited by 5 · see citing works
Self-citationImage processing for visual prostheses: A clinical perspective
Ayton, L. N., Luu, C. D., Bentley, S. A., Allen, P. J., & Guymer, R. H. (2013) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2013.6738317 · Cited by 4 · see citing works
Simulations of Cortical Prosthetic Vision
Srivastava, N. (2011) · Springer eBooks · doi:10.1007/978-1-4419-0754-7_18 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Semantic and structural image segmentation for prosthetic vision
Sanchez-Garcia, M., Martinez-Cantin, R., & Guerrero, J. J. (2020) · PLOS ONE · doi:10.1371/journal.pone.0227677 · Cited by 62 · see citing works
Self-citationBeyeler, M., & Sanchez-Garcia, M. (2022) · Journal of Neural Engineering · doi:10.1088/1741-2552/aca69d · Cited by 34 · see citing works
Axonal stimulation affects the linear summation of single-point perception in three Argus II users
Hou, Y., Nanduri, D., Granley, J., Weiland, J. D., & Beyeler, M. (2024) · Journal of Neural Engineering · doi:10.1088/1741-2552/ad31c4 · Cited by 14 · see citing works
Deep Learning--Based Scene Simplification for Bionic Vision
Han, N., Srivastava, S., Xu, A., Klein, D., & Beyeler, M. (2021) · arXiv · doi:10.48550/arxiv.2102.00297 · Cited by 4 · see citing works
- Authors
- Perez-Yus, A., Bermudez-Cameo, J., Guerrero, J. J., & Lopez-Nicolas, G.
- Affiliations
- Universidad de Zaragoza (ES)
- ORCID
- Guerrero, Jose J.
- Venue
- 2017 IEEE International Conference on Computer Vision Workshops (ICCVW)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 1516-1525
- ISSN
- 2473-9936 (ISSN-L)
- OpenAlex
- W2771683651
- Cited by
- 24
- References
- 48
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 4 · 2023: 1 · 2022: 4 · 2021: 4 · 2020: 3 · 2019: 1 · 2018: 1
- FWCI
- 5.33
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Computer Vision Workshops (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neuroscience and Neural Engineering
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Keywords
- visual prosthesis, depth perception, motion cues, obstacle avoidance, RGB-D camera, blind navigation, free space detection, spatial resolution
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
