Evaluating Deep Human-in-the-Loop Optimization for Retinal Implants Using Sighted Participants
Open accessAbstract
Human-in-the-loop optimization (HILO) is a promising approach for personalizing visual prostheses by iteratively refining stimulus parameters based on user feedback. Previous work demonstrated HILO's efficacy in simulation, but its performance with human participants remains untested. Here we evaluate HILO using sighted participants viewing simulated prosthetic vision to assess its ability to optimize stimulation strategies under realistic conditions. Participants selected between phosphenes generated by competing encoders to iteratively refine a deep stimulus encoder (DSE). We tested HILO in three conditions: standard optimization, threshold misspecifications, and out-of-distribution parameter sampling. Participants consistently preferred HILO-generated stimuli over both a naive encoder and the DSE alone, with log odds favoring HILO across all conditions. We also observed key differences between human and simulated decision-making, highlighting the importance of validating optimization strategies with human participants. These findings support HILO as a viable approach for adapting visual prostheses to individuals. Clinical relevance: Validating HILO with sighted participants viewing simulated prosthetic vision is an important step toward personalized calibration of future visual prostheses.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
This approach allowed inclusion of patient's preferences for stimulation parameters optimization and was proposed in two studies, one using virtual patients and one using real sighted patients under SPV.
“This approach allowed inclusion of patient’s preferences for stimulation parameters optimization and was proposed in two studies, one using virtual patients96 and one using real sighted patients under SPV.99”
From Visual Prostheses in the Era of Artificial Intelligence Technology (2025) · cites this paper as [99]
The Introduction and Task sections describe sighted participants viewing simulated prosthetic vision and using their preferences to optimize simulated stimulation. However, the Participants section identifies them as undergraduate students with normal or corrected-to-normal vision and no reported visual impairment, not patients. The separate virtual-patient study is attached to a different citation. (LLM, conf=0.88)
Experimental — accuracy not yet measuredAnother approach consisted in using human-in-the-loop optimization to improve the perceived image in a simulated prosthetic vision setting.
“Another approach consisted in using human-in-the-loop optimization to improve the perceived image in a simulated prosthetic vision setting.96,99”
From Visual Prostheses in the Era of Artificial Intelligence Technology (2025) · cites this paper as [99]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The sensations produced by electrical stimulation of the visual cortex
Brindley, G. S., & Lewin, W. S. (1968) · The Journal of Physiology · doi:10.1113/jphysiol.1968.sp008519 · Cited by 1,170 · cited 1× · see citing works
“To account for the limited dynamic range of computer monitors, numerical brightness values were displayed alongside each stimulus.”
Dobelle, W. H., & Mladejovsky, M. G. (1974) · The Journal of Physiology · doi:10.1113/jphysiol.1974.sp010766 · Cited by 468 · see citing works
The Argus® II Retinal Prosthesis System
Luo, Y. H.-L., & da Cruz, L. (2016) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2015.09.003 · Cited by 351 · see citing works
Dynamic Stimulation of Visual Cortex Produces Form Vision in Sighted and Blind Humans
Beauchamp, M. S., Oswalt, D., Sun, P., Foster, B. L., Magnotti, J. F., Niketeghad, S., Pouratian, N., Bosking, W. H., & Yoshor, D. (2020) · Cell · doi:10.1016/j.cell.2020.04.033 · Cited by 257 · see citing works
Fernández, E., Alfaro, A., Soto-Sánchez, C., Gonzalez-Lopez, P., Lozano, A. M., Peña, S., Grima, M. D., Rodil, A., Gómez, B., Chen, X., et al. (2021) · Journal of Clinical Investigation · doi:10.1172/jci151331 · Cited by 250 · see citing works
Nanduri, D., Fine, I., Horsager, A., Boynton, G. M., Humayun, M. S., Greenberg, R. J., & Weiland, J. D. (2012) · Investigative Opthalmology & Visual Science · doi:10.1167/iovs.11-8401 · Cited by 163 · cited 1× · see citing works
“This model represents phosphenes as multivariate Gaussian blobs, whose size, shape, and brightness are determined by both electrode location and stimulus properties such as amplitude, frequency, and pulse duration [9], [25], [26].”
Simultaneous perception of prosthetic and natural vision in AMD patients
Palanker, D., Le Mer, Y., Mohand-Said, S., & Sahel, J. A. (2022) · Nature Communications · doi:10.1038/s41467-022-28125-x · Cited by 140 · see citing works
Development of visual Neuroprostheses: trends and challenges.
Fernandez, E. (2018) · Bioelectronic Medicine · doi:10.1186/s42234-018-0013-8 · Cited by 114 · see citing works
Predicting visual sensitivity in retinal prosthesis patients
Horsager, A., Greenwald, S. H., Weiland, J. D., Humayun, M. S., Greenberg, R. J., McMahon, M. J., Boynton, G. M., & Fine, I. (2009) · Investigative Opthalmology & Visual Science · doi:10.1167/iovs.08-2595 · Cited by 107 · see citing works
Biomimetic encoding model for restoring touch in bionic hands through a nerve interface
Okorokova, E. V., He, Q., & Bensmaı̈a, S. J. (2018) · Journal of Neural Engineering · doi:10.1088/1741-2552/aae398 · Cited by 90 · see citing works
Electrical Stimulation of the Retina to Produce Artificial Vision
Weiland, J. D., Walston, S. T., & Humayun, M. S. (2016) · Annual Review of Vision Science · doi:10.1146/annurev-vision-111815-114425 · Cited by 85 · see citing works
Erickson-Davis, C., & Korzybska, H. (2021) · PLoS ONE · doi:10.1371/journal.pone.0229189 · Cited by 83 · see citing works
Acoustic Simulations of Combined Electric and Acoustic Hearing (EAS)
Dorman, M. F., Spahr, A. J., Loizou, P. C., Dana, C. J., & Schmidt, J. S. (2005) · Ear and Hearing · doi:10.1097/00003446-200508000-00001 · Cited by 82 · see citing works
End-to-end optimization of prosthetic vision
de Ruyter van Steveninck, J., Güçlü, U., van Wezel, R., & van Gerven, M. (2022) · Journal of Vision · doi:10.1167/jov.22.2.20 · Cited by 54 · see citing works
van der Grinten, M., de Ruyter van Steveninck, J., Lozano, A., Pijnacker, L., Rueckauer, B., Roelfsema, P., van Gerven, M., van Wezel, R., Güçlü, U., & Güçlütürk, Y. (2024) · eLife · doi:10.7554/elife.85812 · Cited by 39 · see citing works
Deep Learning–Based Perceptual Stimulus Encoder for Bionic Vision
Relic, L., Zhang, B., Tuan, Y.-L., & Beyeler, M. (2022) · Augmented Humans International Conference (AHs) · doi:10.1145/3519391.3524034 · Cited by 18 · see citing works
Self-citationAligning Visual Prosthetic Development With Implantee Needs
Nadolskis, L., Turkstra, L. M., Larnyo, E., & Beyeler, M. (2024) · Translational Vision Science & Technology · doi:10.1167/tvst.13.11.28 · Cited by 16 · see citing works
Self-citationHuman-in-the-loop optimization of visual prosthetic stimulation
Fauvel, T., & Chalk, M. (2022) · Journal of Neural Engineering · doi:10.1088/1741-2552/ac7615 · Cited by 15 · see citing works
Axonal stimulation affects the linear summation of single-point perception in three Argus II users
Hou, Y., Nanduri, D., Granley, J., Weiland, J. D., & Beyeler, M. (2024) · Journal of Neural Engineering · doi:10.1088/1741-2552/ad31c4 · Cited by 14 · see citing works
Self-citationSequential epiretinal stimulation improves discrimination in simple shape discrimination tasks only
Christie, B., Sadeghi, R., Kartha, A., Caspi, A., Tenore, F. V., Klatzky, R. L., Dagnelie, G., & Billings, S. (2022) · Journal of Neural Engineering · doi:10.1088/1741-2552/ac7326 · Cited by 10 · see citing works
Adapting Brain-Like Neural Networks for Modeling Cortical Visual ProsthesesPreprint
Granley, J., Riedel, A., & Beyeler, M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.13561 · Cited by 5 · see citing works
Self-citationBrain Co-processors: Using AI to Restore and Augment Brain Function
Rao, R. P. N. (2023) · Handbook of Neuroengineering · doi:10.1007/978-981-16-5540-1_32 · Cited by 5 · see citing works
End-to-end learning of safe stimulation parameters for cortical neuroprosthetic vision
Küçükoğlu, B., Rueckauer, B., van Steveninck, J. D. R., van der Grinten, M., Güçlütürk, Y., Roelfsema, P. R., Güçlü, U., & van Gerven, M. A. J. (2025) · Journal of Neural Engineering · doi:10.1088/1741-2552/ade918 · Cited by 5 · see citing works
Beyond Sight: Probing Alignment Between Image Models and Blind V1
Granley, J., Pogoncheff, G., Rodil, A., et al. (2024) · doi:10.48550/arxiv.2403.12990 · Cited by 1
Self-citation
Flagged first, then by citations
Research works printed but not linked: 13
- K. Stingl, R. Schippert, K. U. Bartz-Schmidt, D. Besch, C. L. Cottriall, T. L. Edwards, F. Gekeler, U. Greppmaier, K. Kiel, A. Koitschev, L. Kühlewein, R. E. MacLaren, J. D. Ramsden, J. Roider, A. Rothermel, H. Sachs, G. S. Schröder, J. Tode, N. Troelenberg, and E. Zrenner, "Interim Results of a Multicenter Trial with the New Electronic Subretinal Implant Alpha AMS in 15 Patients Blind from Inherited Retinal Degenerations," Frontiers in Neuroscience, vol. 11, 2017, publisher: Frontiers. [Online]. Available: https://www.frontiersin.org/articles/10.3389/fnins.2017.00445/full · DOI
- M. P. Barry, R. Sadeghi, V. L. Towle, K. Stipp, H. Puhov, W. Diaz, P. Grant, F. T. Collison, F. J. Lane, J. P. Szlyk et al., "Preliminary visual function for the first human with the Intracortical Visual Prosthesis (ICVP)," Investigative Ophthalmology & Visual Science, vol. 64, no. 8, pp. 2842-2842, 2023.
- M. Beyeler, D. Nanduri, J. D. Weiland, A. Rokem, G. M. Boynton, and I. Fine, "A model of ganglion axon pathways accounts for percepts elicited by retinal implants," Scientific Reports, vol. 9, no. 1, pp. 1-16, Jun. 2019. [Online]. Available: https://www.nature.com/articles/s41598-019-45416-4
- N. C. Sinclair, M. N. Shivdasani, T. Perera, L. N. Gillespie, H. J. McDermott, L. N. Ayton, and P. J. Blamey, "The Appearance of Phosphenes Elicited Using a Suprachoroidal Retinal Prosthesis," Investigative Ophthalmology & Visual Science, vol. 57, no. 11, pp. 4948-4961, Sep. 2016. [Online]. Available: https: //iovs.arvojournals.org/article.aspx?articleid=2556019
- M. P. Barry, M. Armenta Salas, U. Patel, V. Wuyyuru, S. Niketeghad, W. H. Bosking, D. Yoshor, J. D. Dorn, and N. Pouratian, "Video-mode percepts are smaller than sums of single-electrode phosphenes with the Orion® visual cortical prosthesis," Investigative Ophthalmology & Visual Science, vol. 61, no. 7, p. 927, Jun. 2020.
- J. Granley and M. Beyeler, "A Computational Model of Phosphene Appearance for Epiretinal Prostheses," Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference, vol. 2021, pp. 4477-4481, Nov. 2021. [Online]. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/ PMC9255280/
- I. Fine and G. M. Boynton, "A virtual patient simulation modeling the neural and perceptual effects of human visual cortical stimulation, from pulse trains to percepts," Scientific Reports, vol. 14, no. 1, p. 17400, Jul. 2024, publisher: Nature Publishing Group. [Online]. Available: https://www.nature.com/articles/s41598-024-65337-1
- J. Granley, L. Relic, and M. Beyeler, "Hybrid Neural Autoencoders for Stimulus Encoding in Visual and Other Sensory Neuroprostheses," in Advances in Neural Information Processing Systems, vol. 35, Dec. 2022, pp. 22 671-22 685. [Online]. Available: https://papers.nips.cc/paper files/paper/2022/hash/ 8e9a6582caa59fda0302349702965171-Abstract-Conference.html
- J. Granley, T. Fauvel, M. Chalk, and M. Beyeler, "Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prostheses," in Advances in Neural Information Processing Systems, Nov. 2023. [Online]. Available: https://openreview.net/forum?id=ZED5wdGous
- E. Prashnani, H. Cai, Y. Mostofi, and P. Sen, "PieAPP: Perceptual Image-Error Assessment Through Pairwise Preference," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp. 1808-1817, conference Name: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) ISBN: 9781538664209 Place: Salt Lake City, UT Publisher: IEEE. [Online]. Available: https://ieeexplore.ieee.org/document/8578292/
- L. M. Turkstra, T. Bhatia, A. Van Os, and M. Beyeler, "Assistive technology use in domestic activities by people who are blind," Scientific Reports, vol. 15, no. 1, p. 7486, Mar. 2025, publisher: Nature Publishing Group. [Online]. Available: https://www.nature.com/articles/s41598-025-91755-w
- F. Drakopoulos, A. Van Den Broucke, and S. Verhulst, "A DNN-Based Hearing-Aid Strategy For Real-Time Processing: One Size Fits All," in ICASSP 2023 -2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Jun. 2023, pp. 1-5, iSSN: 2379-190X. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10094887
- N. A. Lesica, N. Mehta, J. G. Manjaly, L. Deng, B. S. Wilson, and F.-G. Zeng, "Harnessing the power of artificial intelligence to transform hearing healthcare and research," Nature Machine Intelligence, vol. 3, no. 10, pp. 840-849, Oct. 2021, publisher: Nature Publishing Group. [Online]. Available: https: //www.nature.com/articles/s42256-021-00394-z
Cited by
Sorted by citations
Visual Prostheses in the Era of Artificial Intelligence Technology
Sarbout, I., Gungor, A., Ounissi, M., Zaher, S., Ptito, M., Kupers, R., Racoceanu, D., & Milea, D. (2025) · Eye and Brain · doi:10.2147/eb.s524322 · Cited by 3 · see citing works
“96,99 This approach allowed inclusion of patient’s preferences for stimulation parameters optimization and was proposed in two studies, one using virtual patients 96 and one using real sighted patients under SPV. 99 This approach might allow simultaneous optimization of the saliency extraction and the phosphenes consi…”
Fuzzing the brain: automated stress testing for the safety of ML-driven neurostimulation
Downing, M., Peng, M., Granley, J., Beyeler, M., & Bultan, T. (2026) · Journal of Neural Engineering · doi:10.1088/1741-2552/ae4927 · Cited by 1 · see citing works
Self-citationAniorte, A., Gea, M., Garrigós, J., Martínez, J. J., & Ferrández, J. M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-27314-7_38
- Authors
- Schoinas, E., Rastogi, A., Carter, A., Granley, J., & Beyeler, M.
- Affiliations
- University of California, Santa Barbara (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- Volume
- 2025
- Pages
- 1-6
- ISSN
- 2375-7477 (ISSN-L)
- arXiv
- 2502.00177
- OpenAlex
- W4416962888
- Cited by
- 3
- References
- 21
- Citations by year
- 2026: 2 · 2025: 1
- FWCI
- 18.82
- Type
- conference paper
- Language
- en
- Open access
- Licence
- Locations
- Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Institutes of Health
- U.S. National Library of Medicine (DP2LM014268)
- HORIZON EUROPE Health
- Topics
- Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neuroscience and Neural Engineering
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Keywords
- human-in-the-loop optimization, retinal prosthesis, phosphenes, personalized stimulation, simulated prosthetic vision, user feedback
- MeSH terms
- Adult, Female, Humans, Male, Visual Prosthesis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
