Deep reinforcement learning for task-based feature learning in prosthetic vision
Abstract
Developing hand-crafted visual features to enhance perception with prosthetic vision devices can often miss important aspects of a given task. Retinal implants suffer from the need to create low-dimensional features for elaborate tasks such as navigation in complex environments. Using Deep Reinforcement Learning (DRL), visual features are learnt through task-based simulations that remove the ambiguity of inferring the visual information most crucial to a specific activity. Learning task-based features ensures that the visual information is salient to the tasks an implant recipient may be undertaking and eliminates potentially redundant features. In this paper, we focus specifically on basic orientation and mobility, and the methods for feature learning and visualisation in structured 3D environments. We propose a new model for learning visual features through task-based simulations and show that learnt features can be transferred directly to real RGB-D images. We demonstrate this new scalable approach for feature learning in simulation and open the possibility for more complex simulations of more complex tasks in the future.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Of the five end-to-end studies identified, two relied on auto-encoding strategies and three entailed reinforcement learning approaches.
“Of the five end-to-end studies that we identified, two relied on auto-encoding strategies,88,91 where the task to be learnt is reconstructing an original image, and three entailed reinforcement learning approaches,87,89,90 for which the scope of learned tasks is wider.”
From Visual Prostheses in the Era of Artificial Intelligence Technology (2025) · cites this paper as [87]
Computational optimization approaches can aid in the development of safe stimulation protocols because they allow faster exploration of the large parameter space and enable task-driven optimization of image processing strategies.
“Computational optimization approaches can also aid in the development of safe stimulation protocols, because they allow a faster exploration of the large parameter space and enable task-driven optimization of image processing strategies (Granley et al., 2022a; Fauvel and Chalk, 2022; White et al., 2019; Küçükoğlu et al., 2022; de Ruyter van Steveninck et al., 2022a; Ghaffari et al., 2021).”
From Towards biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses (2024) · cites this paper as White et al., 2019
This kind of AI learning is often performed in virtual environments due to the lack of real data, but current methods have employed virtual platforms with low realism, limiting their real-world usability.
“This kind of AI learning is often performed in virtual environments due to the lack of real data, but current methods have employed virtual platforms with low realism, limiting their real-world usability.87,89,90”
From Visual Prostheses in the Era of Artificial Intelligence Technology (2025) · cites this paper as [87]
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · see citing works
Benchmarking Deep Reinforcement Learning for Continuous Control
Duan, Y., Chen, X., Houthooft, R., Schulman, J., & Abbeel, P. (2016) · arXiv · doi:10.48550/arxiv.1604.06778 · Cited by 1,114 · see citing works
Subretinal electronic chips allow blind patients to read letters and combine them to words
Zrenner, E., Bartz-Schmidt, K. U., Benav, H., Besch, D., Bruckmann, A., Gabel, V.-P., Gekeler, F., Greppmaier, U., Harscher, A., Kibbel, S., et al. (2011) · Proceedings of the Royal Society B: Biological Sciences · doi:10.1098/rspb.2010.1747 · Cited by 819 · see citing works
Learning to Navigate in Complex EnvironmentsPreprint
Mirowski, P., Pascanu, R., Viola, F., Soyer, H., Ballard, A. J., Banino, A., Denil, M., Goroshin, R., Sifre, L., Kavukcuoglu, K., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.03673 · Cited by 679 · see citing works
Fujikado, T., Kamei, M., Sakaguchi, H., Kanda, H., Morimoto, T., Ikuno, Y., Nishida, K., Kishima, H., Maruo, T., Konoma, K., et al. (2011) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.10-6836 · Cited by 216 · see citing works
Visualizing and Understanding Atari AgentsPreprint
Greydanus, S., Koul, A., Dodge, J. E., & Fern, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.00138 · Cited by 216 · see citing works
Weiland, J. D., & Humayun, M. S. (2014) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2014.2314733 · Cited by 186 · see citing works
Learning to Navigate Through Complex Dynamic Environment With Modular Deep Reinforcement Learning
Wang, Y., He, H., & Sun, C. (2018) · IEEE Transactions on Games · doi:10.1109/tg.2018.2849942 · Cited by 99 · see citing works
Time-to-contact maps for navigation with a low resolution visual prosthesis
McCarthy, C., & Barnes, N. M. (2012) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2012.6346541 · Cited by 15 · see citing works
Self-citation
Flagged first, then by citations
1 of the 11 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
End-to-end optimization of prosthetic vision
de Ruyter van Steveninck, J., Güçlü, U., van Wezel, R., & van Gerven, M. (2022) · Journal of Vision · doi:10.1167/jov.22.2.20 · Cited by 54 · see citing works
van der Grinten, M., de Ruyter van Steveninck, J., Lozano, A., Pijnacker, L., Rueckauer, B., Roelfsema, P., van Gerven, M., van Wezel, R., Güçlü, U., & Güçlütürk, Y. (2024) · eLife · doi:10.7554/elife.85812 · Cited by 39 · see citing works
“…they allow a faster exploration of the large parameter space and enable task-driven optimization of image processing strategies ( Granley et al., 2022a ; Fauvel and Chalk, 2022 ; White et al., 2019 ; Küçüko ğ lu et al., 2022 ; de Ruyter van Steveninck et al., 2022a ; Ghaffari et al., 2021 ).”
BackgroundVision Processing for Assistive Vision: A Deep Reinforcement Learning Approach
White, J., Kameneva, T., & McCarthy, C. D. (2021) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3121661 · Cited by 29 · see citing works
Self-citationThe application of computer vision to visual prosthesis
Wang, J., Zhu, H., Liu, J., Li, H., Han, Y., Zhou, R., & Zhang, Y. (2021) · Artificial Organs · doi:10.1111/aor.14022 · Cited by 26 · see citing works
Optimization of Neuroprosthetic Vision via End-to-End Deep Reinforcement Learning
Küçükoğlu, B., Rueckauer, B., Ahmad, N., van Steveninck, J. D. R., Güçlü, U., & van Gerven, M. A. J. (2022) · International Journal of Neural Systems · doi:10.1142/s0129065722500526 · Cited by 25 · see citing works
Optimization of Neuroprosthetic Vision via End-to-end Deep Reinforcement Learning
Küçükoğlu, B., Rueckauer, B., Ahmad, N., de Ruyter van Steveninck, J., Güçlü, U., & van Gerven, M. (2022) · doi:10.1101/2022.02.25.482017 · Cited by 25 · see citing works
Wang, C., Fang, C., Zou, Y., Yang, J., & Sawan, M. (2023) · Journal of Neural Engineering · doi:10.1088/1741-2552/acb295 · Cited by 19 · see citing works
Semantic translation of face image with limited pixels for simulated prosthetic vision
Xia, X., He, X., Lu, F., Pan, X., Li, N., Zhang, J., Pang, X., Yu, F., & Ding, N. (2022) · Information Sciences · doi:10.1016/j.ins.2022.07.094 · Cited by 11 · see citing works
Clinical Progress and Optimization of Information Processing in Artificial Visual Prostheses
Wang, J., Zhao, R., Li, P., Fang, Z., Li, Q., Han, Y., Zhou, R., & Zhang, Y. (2022) · Sensors · doi:10.3390/s22176544 · Cited by 10 · see citing works
Pre‐processing visual scenes for retinal prosthesis systems: A comprehensive review
Holiel, H. A., Fawzi, S. A., & Al‐Atabany, W. (2024) · Artificial Organs · doi:10.1111/aor.14824 · Cited by 9 · see citing works
End-to-end learning of safe stimulation parameters for cortical neuroprosthetic vision
Küçükoğlu, B., Rueckauer, B., van Steveninck, J. D. R., van der Grinten, M., Güçlütürk, Y., Roelfsema, P. R., Güçlü, U., & van Gerven, M. A. J. (2025) · Journal of Neural Engineering · doi:10.1088/1741-2552/ade918 · Cited by 5 · see citing works
Self-Attention Based Vision Processing for Prosthetic Vision
White, J., Ruiz-Serra, J., Petrie, S., Kameneva, T., & McCarthy, C. (2023) · 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) · doi:10.1109/embc40787.2023.10341053 · Cited by 3 · see citing works
Self-citationVisual Prostheses in the Era of Artificial Intelligence Technology
Sarbout, I., Gungor, A., Ounissi, M., Zaher, S., Ptito, M., Kupers, R., Racoceanu, D., & Milea, D. (2025) · Eye and Brain · doi:10.2147/eb.s524322 · Cited by 3 · see citing works
“Of the five end-to-end studies that we identified, two relied on auto-encoding strategies, 88,91 where the task to be learnt is reconstructing an original image, and three entailed reinforcement learning approaches, 87,89,90 for which the scope of learned tasks is wider.”
Dynamic Manipulation Skill Learning for Tactile Myoelectric Prosthetic Hands in Tool Handling
Li, B., Wu, S., You, T., Wang, S., Chen, Z., Liu, Y., Guo, D., Sun, F., Xu, G., Jiang, D., et al. (2026) · Cyborg and Bionic Systems · doi:10.34133/cbsystems.0572 · Cited by 2 · see citing works
Vincent, J., Germany, T. U. B., & Germany, T. U. B. (2022) · Journal of Vision · doi:10.1167/jov.22.3.20 · Cited by 2 · see citing works
End-to-end optimization of prosthetic vision
de Ruyter van Steveninck, J., Güçlü, U., van Wezel, R., & van Gerven, M. (2020) · doi:10.1101/2020.12.19.423601 · Cited by 54 · see citing works
“For instance, with reinforcement learning strategies (see White et al., 2019), the model could be extended to perform tasks that more closely related to the everyday actions that need to be performed by the end-user, such as object manipulation (Levine, Finn, Darrell, & Abbeel, 2015) or object…”
BackgroundMethod
- Authors
- White, J., Kameneva, T., & McCarthy, C.
- Affiliations
- Swinburne University of Technology (AU)
- The University of Melbourne (AU)
- Venue
- 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
- Publisher
- IEEE
- Published
- 2019-07
- Volume
- 2019
- Pages
- 2809-2812
- ISSN
- 2375-7477 (ISSN-L)
- OpenAlex
- W2979370604
- Cited by
- 14
- References
- 11
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 2 · 2023: 2 · 2022: 5 · 2021: 2
- FWCI
- 0.67
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (publishedVersion)
- PubMed (publishedVersion)
- Swinburne figshare (Swinburne University of Technology) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neuroscience and Neural Engineering
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- prosthetic vision, retinal prosthesis, visual feature learning, deep reinforcement learning, orientation and mobility, RGB-D
- MeSH terms
- Learning, Orientation, Reinforcement, Psychology, Vision, Ocular, Prostheses and Implants
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
