CoChat
Sign Up
Cited by 60Neuroscience and Neural EngineeringTactile and Sensory InteractionsEEG and Brain-Computer Interfaces

End-to-end optimization of prosthetic vision

Open access
de Ruyter van Steveninck, Jaap, Güçlü, Umut, van Wezel, Richard, van Gerven, Marcel·2022·Journal of Vision
DOI 10.1167/jov.22.2.20PMC PMC8899855

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Neural prosthetics may provide a promising solution to restore visual perception in some forms of blindness. The restored prosthetic percept is rudimentary compared to normal vision and can be optimized with a variety of image preprocessing techniques to maximize relevant information transfer. Extracting the most useful features from a visual scene is a nontrivial task and optimal preprocessing choices strongly depend on the context. Despite rapid advancements in deep learning, research currently faces a difficult challenge in finding a general and automated preprocessing strategy that can be tailored to specific tasks or user requirements. In this paper, we present a novel deep learning approach that explicitly addresses this issue by optimizing the entire process of phosphene generation in an end-to-end fashion. The proposed model is based on a deep auto-encoder architecture and includes a highly adjustable simulation module of prosthetic vision. In computational validation experiments, we show that such an approach is able to automatically find a task-specific stimulation protocol. The results of these proof-of-principle experiments illustrate the potential of end-to-end optimization for prosthetic vision. The presented approach is highly modular and our approach could be extended to automated dynamic optimization of prosthetic vision for everyday tasks, given any specific constraints, accommodating individual requirements of the end-user.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper presents a deep autoencoder that optimizes the entire phosphene-generation pipeline end-to-end, including a differentiable simulator of cortical prosthetic vision. In four computational experiments, it automatically finds task-specific stimulation protocols and can incorporate constraints such as sparsity or custom phosphene mappings.

Key findings

  • In Experiment 1, the model reconstructed synthetic character images from a 32×32 binary phosphene representation with validation MSE = 0.018, SSIM = 0.139, and PSNR = 17.59; it adopted an “inverted” encoding strategy with average electrode activity of 95.65% in two of five random restarts and across all 234 validation images.
  • In Experiment 2, adding an L1 sparsity loss across 13 values of κ reduced the percentage of active electrodes but impaired reconstruction quality for larger κ; even low κ led to more ecologically useful phosphene patterns than the unconstrained model.
  • In Experiment 3 on ADE20K naturalistic images, the end-to-end model achieved the highest intensity-reconstruction MSE/SSIM/FSIM and perceptual-reconstruction SSIM/FSIM among compared methods; for semantic boundary prediction it reached accuracy 0.697, sensitivity 0.722, specificity 0.695, precision 0.165, and AUC 0.785, versus Canny AUC 0.746 and HED AUC 0.724.
  • In Experiment 4 with custom phosphene mappings, reconstruction performance was lower than in Experiment 3, and reducing the number of phosphenes from 650 to 488 or 325 was associated with reduced reconstruction performance; AUC differed significantly among phosphene resolutions.

Methods

  • The model is an end-to-end autoencoder with three components: a fully convolutional encoder producing a binary stimulation protocol via a Heaviside step function and straight-through estimator, a parameter-free phosphene simulator mapping a 32×32 protocol to a 256×256 simulated visual field with distorted grid positions, random gain 0.5–1.5, and Gaussian blur (spacing 8 pixels, sigma 1.5), and a residual-network decoder.
  • Training used PyTorch 1.3.1, an NVIDIA GeForce GTX 1080 TI GPU, Adam, and five random restarts per model, reporting the best validation-loss model unless stated otherwise.
  • Four computational experiments were run: synthetic 128×128 black-and-white lowercase letters in 47 fonts (38 train, 9 validation) with MSE; sparsity-constrained optimization over 13 κ values; ADE20K naturalistic images with MSE, VGG16 feature loss at d = 3, and weighted BCE semantic boundary tasks compared against Canny and HED using paired t-tests with Bonferroni correction at α = 0.05; and custom phosphene mappings with 650, 488, or 325 phosphenes in the lower left quadrant.

Limitations

  • The paper states that subjective quality of phosphene representations was not addressed, that the simulated prosthetic vision remains simplified and omits stimulation dynamics such as pulse frequency, inter-stimulation interval, and electrode interactions, that the model was trained only on static images, and that the optimization tasks remain basic.
  • The paper also states that the lower performance in Experiment 4 may partly be explained by the reduced number of phosphenes, but this was not formally tested.
  • The text makes evident that validation is computational only: no behavioral experiments, patient data, or direct perceptual testing are reported.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1167/jov.22.2.20

Cites

74 of 74 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 40 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    In Experiment 1, the reconstruction loss was successfully minimized until convergence after 39 epochs.

    “As can be observed, the reconstruction loss is successfully minimized until convergence after 39 epochs.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Overall reconstruction performance in Experiment 4 was lower compared to Experiment 3.

    “The overall reconstruction performance was lower compared to Experiment 3 (see Figure 7).”
    • Also stated in: § Tailored optimization to realistic phosphene mappings · plain
      “The overall reconstruction performance with the customized phosphene mapping was lower compared to the regular phosphene mapping that was used in Experiment 3.”
    Quote in the paperNot a substantial claim
  • Findingown data

    The model successfully converged to an optimal solution in Experiment 4.

    “The model successfully converged to an optimal solution and the results of Experiment 4 are displayed in Figure 8.”
    Quote in the paperNot a substantial claim
  • Findingown data

    In Experiment 2, addition of sparsity loss led to phosphene patterns that more naturally encode the presence of pixels by the presence of phosphenes, in contrast to Experiment 1.

    “In contrast to the results of Experiment 1, addition of a sparsity loss led to phosphene patterns that more naturally encode the presence of pixels by the presence of phosphenes (instead of vice versa).”
    Quote in the paperSubstantial claim
  • Findingown data

    The model achieved a low MSE of 0.018 on the validation dataset.

    “The model achieved adequate reconstruction performance, as indicated by the low MSE of 0.018 on the validation dataset.”
    Quote in the paperSubstantial claim
  • May combine two claims. 13
  • Classified as probably not a substantial claim. 29

  • May combine two claims.

    Findingown data

    In Experiment 1, the model adopted an inverted encoding strategy where the presence of the alphabetic character is encoded with the absence of phosphenes and vice versa, resulting in an average electrode activity of 95.65%.

    “Notably, the model adopted an encoding strategy where the presence of the alphabetic character (white pixels) is encoded with the absence of phosphenes and vice versa, resulting in an average electrode activity (percentage of active electrodes) of 95.65%.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In Experiment 3, the end-to-end model scored the highest performance for intensity-based reconstruction (MSE, SSIM, and FSIM) and perceptual reconstruction (SSIM and FSIM) compared to existing approaches.

    “Compared to the reconstructions of the SPV-encodings using existing approaches, our end-to-end model achieved adequate reconstruction performance: our model scored the highest performance for the intensity-based reconstruction (MSE, SSIM, and FSIM) and the perceptual reconstruction (SSIM and FSIM).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    A larger sparsity weight κ results in fewer active electrodes but also in impaired reconstruction performance.

    “Notably, a larger sparsity weight κ results in fewer active electrodes, but also in impaired reconstruction performance.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In Experiment 3, the perceptual reconstruction task had higher FSIM and lower MSE compared to the intensity-based reconstruction task.

    “The higher FSIM and the lower MSE in the perceptual reconstruction task, compared to the intensity-based reconstruction task, indicate that information about the higher-level perceptual features is favored over pixel-intensity information.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The phosphene encodings found by the model in the semantic boundary condition are comparable to those found with traditional preprocessing approaches and yield similar reconstruction quality compared to edge detection or holistic contour detection.

    “The phosphene encodings that were found by the model in this condition are comparable to those found with traditional preprocessing approaches (see Figure 6e), and yield similar reconstruction quality (see Table 3) compared to edge detection (Canny, 1986) or holistic contour detection (Xie & Tu, 2017).”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Canny, 1986 source

      Not checked in the free run.

    • cites Xie & Tu, 2017 source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    In Experiment 3, in the semantic boundary reconstruction task, the end-to-end model scored average for accuracy, sensitivity, specificity, and precision, and had the largest AUC compared to the other models.

    “In the semantic boundary reconstruction task, our model scored average for the accuracy, sensitivity, specificity, and precision, and had the largest area under the receiver-operator curve (AUC) compared to the other models.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The inverted encoding strategy in Experiment 1 was found for two out of five random restart initializations and observed for all 234 images in the validation dataset.

    “Such an “inverted” encoding strategy was found for two out of the five random restart initializations and observed for all 234 images in the validation dataset.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Visual prosthetics may provide a promising solution that can restore a rudimentary form of vision in some forms of blindness.

    “For some forms of blindness, visual prosthetics may provide a promising solution that can restore a rudimentary form of vision (Chen, Wang Fernandez, & Roelfsema, 2020; Fernandez, 2018; Lewis et al., 2016; Lozano et al., 2020; Riazi-Esfahani, Maghami, Sodagar, Lashay, & riazi-Esfahani, 2014; Roelfsema, Denys, & Klink, 2018; Shepherd, Shivdasani, Nayagam, Williams, & Blamey, 2013).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Chen, Wang Fernandez, & Roelfsema, 2020 source

      Not checked in the free run.

    • cites Fernandez, 2018 source

      Not checked in the free run.

    • cites Lewis et al., 2016 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    V1 visual implants are reported to have enormous potential in future treatment of visual impairment.

    “In this paper, we focus on visual implants that reside in the primary visual cortex (V1), which are reported to have an enormous potential in future treatment of visual impairment (Beauchamp et al., 2020; Beauchamp & Yoshor, 2020; Chen et al., 2020; Lewis, Ackland, Lowery, & Rosenfeld, 2015).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Beauchamp et al., 2020

      Not checked in the free run.

    • cites Beauchamp & Yoshor, 2020

      Not checked in the free run.

    • cites Chen et al., 2020 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Premisenone given

    Practical, medical, or biophysical limitations of the neural interface may motivate tailoring stimulation parameters to additional constraints.

    “Second, as a result of practical, medical, or biophysical limitations of the neural interface, one might want to tailor the stimulation parameters to additional constraints.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Premisereasoning

    The choice of filtering is nontrivial and the definition of useful information strongly depends on the context.

    “The choice of filtering is nontrivial and the definition of useful information will strongly depend on the context.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The percept restored with visual prostheses is very rudimentary compared to normal vision, and resolution remains relatively limited even with relatively high numbers of electrodes.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    In naturalistic stimuli, not all features may be considered relevant.

    “Here, not all features may be considered relevant”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisereasoning

    Research currently faces a difficult challenge in finding a general preprocessing strategy that can be automatically tailored to specific tasks and requirements of the user.

    “Despite the rapid advancements in these fields, research currently faces a difficult challenge in finding a general preprocessing strategy that can be automatically tailored to specific tasks and requirements of the user.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The percept restored with visual prostheses is very rudimentary compared to normal vision, and resolution remains relatively limited even with relatively high numbers of electrodes.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The efficient encoding of a complex visual environment into a rudimentary phosphene representation can be considered a dimensionality reduction problem aiming to maximally preserve information in the latent SPV representation.

    “In this view, the efficient encoding of the rather complex visual environment into a relatively rudimentary phosphene representation can be considered a dimensionality reduction problem in which we aim to maximally preserve the information that is present in the latent SPV representation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    SSIM and FSIM better reflect perceptual quality than MSE and PSNR.

    “Where MSE and PSNR are image quality assessment metrics that operate on pixel intensity, SSIM and FSIM are popular alternatives that better reflect perceptual quality (Preedanan, Kondo, Bunnun, & Kumazawa, 2018).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Preedanan, Kondo, Bunnun, & Kumazawa, 2018

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The decoder is an image-to-image conversion model based on a residual network architecture, which is known for its useful training properties.

    “The decoder is an image-to-image conversion model that is based on a residual network architecture (He, Zhang, Ren, & Sun, 2016), which is known for its useful training properties (Ebrahimi & Abadi, 2018; Huang, Wang, Tao, & Zhao, 2020).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites He, Zhang, Ren, & Sun, 2016 source

      Not checked in the free run.

    • cites Ebrahimi & Abadi, 2018

      Not checked in the free run.

    • cites Huang, Wang, Tao, & Zhao, 2020

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The proposed method enables the implementation of virtually any type of additional constraint that can be incorporated in the optimization procedure.

    “Importantly, the proposed method enables the implementation of virtually any type of additional constraint that can be incorporated in the optimization procedure.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: An important requirement for automated optimization with deep learning is that all components of the artificial neural network make use of differentiable operations.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    The proposed end-to-end framework allows for optimization to virtually any type of task that can be formalized as a loss function.

    “Note that the proposed end-to-end framework allows for optimization to virtually any type of task that can be formalized as a loss function.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    The proposed end-to-end architecture merely requires supervision to the output reconstructions, and the labels do not directly control the phosphene representations themselves.

    “Different from the aforementioned traditional strategies, the proposed end-to-end architecture, merely requires supervision to the output reconstructions and the labels do not directly control the phosphene representations themselves.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisereasoning

    Image preprocessing may help maximize the usefulness and interpretability of phosphene representations by selectively filtering the visual environment.

    “By selective filtering of the visual environment, image preprocessing may help to maximize the usefulness and interpretability of phosphene representations.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The percept restored with visual prostheses is very rudimentary compared to normal vision, and resolution remains relatively limited even with relatively high numbers of electrodes.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    A random restarts approach was used to reduce the likelihood that a suitable parameter configuration is missed due to a specific weights initialization and learning rate.

    “To account for potential convergence of the model parameters toward local optima (i.e. to reduce the likelihood that a suitable parameter configuration of the network is missed due to a combination of a specific weights initialization and learning rate), a “random restarts” approach was used.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Each model was trained five times with different random weight initializations, and only the best performing model on the validation dataset is reported unless stated otherwise.

    “That is, each model was trained five times, each time randomly starting with a different weight initialization. The results only show the best performing one out of these five models (i.e. the one with the lowest loss on the validation dataset), unless stated otherwise.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The limited amount of information that can be conveyed allows for only selective visualization of the surroundings.

    “The limited amount of information that can be conveyed allows for only selective visualization of the surroundings.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The percept restored with visual prostheses is very rudimentary compared to normal vision, and resolution remains relatively limited even with relatively high numbers of electrodes.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    The VGG feature loss and BCE loss represent basic functions that are normally performed in the brain.

    “The VGG feature loss and BCE loss that were implemented in this paper were not chosen only because of their well-established application in optimization problems (Asgari Taghanaki, Abhishek, Cohen, Cohen-Adad, & Hamarneh, 2021; Zhang, Isola, Efros, Shechtman, & Wang, 2018) but also because they represent basic functions that are normally performed in the brain.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Asgari Taghanaki, Abhishek, Cohen, Cohen-Adad, & Hamarneh, 2021

      Not checked in the free run.

    • cites Zhang, Isola, Efros, Shechtman, & Wang, 2018 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    In the unconstrained setting, the model merely maximizes information transfer and has no knowledge about practical requirements such as sparsity.

    “Note, that in the unconstrained setting, the model merely maximizes information transfer and has no knowledge about practical requirements, such as sparsity.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Prosthetic engineers need to speculate or make assumptions about what visual features are crucial for the task and the ways in which these features can be transformed into a suitable stimulation protocol.

    “We illustrate this with two issues: first, prosthetic engineers need to speculate or make assumptions about what visual features are crucial for the task and the ways in which these features can be transformed into a suitable stimulation protocol.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Residual networks demonstrate computational similarities with recurrent networks found in the biological visual system.

    “Furthermore, residual networks demonstrate computational similarities with recurrent networks that are found in the biological visual system (Kubilius et al., 2019; Liao & Poggio, 2016; Schrimpf et al., 2018).”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Kubilius et al., 2019

      Not checked in the free run.

    • cites Liao & Poggio, 2016

      Not checked in the free run.

    • cites Schrimpf et al., 2018

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The proposed method provides a generalized approach that opens the possibility for task-specific and tailored optimization.

    “In addition to that, our proposed method provides a generalized approach that opens the possibility for task-specific and tailored optimization.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Preprocessing with semantic segmentation may successfully improve object recognition performance in simulated prosthetic vision compared to conventional edge detection techniques.

    “Note that a variation on these approaches is investigated in Sanchez-Garcia et al. (2020), who demonstrated that preprocessing with semantic segmentation may successfully improve object recognition performance in simulated prosthetic vision (compared to preprocessing with conventional edge detection techniques).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sanchez-Garcia et al. (2020) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Accurate simulation of these stimulation dynamics requires further adaptations to the phosphene simulator.

    “Accurate simulation of these characteristics requires further adaptations to our phosphene simulator.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The simulated prosthetic vision used in the current study is a simplified model of reality and does not address several stimulation dynamics, such as pulse frequency, inter-stimulation interval, and interactions between electrodes.

    “Second, the simulated prosthetic vision that was used in the current study is still a simplified model of the reality, and it does not address several stimulation dynamics, such as pulse frequency, inter-stimulation interval, and interactions between electrodes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Fully optimizing the interaction between prosthetic stimulation and the downstream visual processing requires a deep understanding of the biological networks involved.

    “Still, fully optimizing the interaction between prosthetic stimulation and the downstream visual processing, requires a deep understanding of the biological networks involved.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    The presented approach is highly modular.

    “The presented approach is highly modular and could be extended to dynamically optimize prosthetic vision for everyday tasks and requirements of the end-user.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Boundary detection is one of the processing steps needed for segregation of objects from background.

    “and boundary detection is one of these processing steps needed for segregation of objects from background (Roelfsema, Lamme, Spekreijse, & Bosch, 2002).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Roelfsema, Lamme, Spekreijse, & Bosch, 2002

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Due to inadvertent activation of underlying axon pathways, phosphenes generated with retinal prostheses may demonstrate distorted shapes that vary across subjects and even individual electrodes.

    “For instance, due to inadvertent activation of underlying axon pathways, phosphenes generated with retinal prostheses may demonstrate distorted shapes that vary across subjects and even individual electrodes (Beyeler et al., 2019).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Beyeler et al., 2019 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    In this paper, the model is trained on static images.

    “Third, in this paper, the model is trained on static images.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Study design and conductnone given

    The optimization tasks used in the current paper remain basic.

    “Finally, the optimization tasks that were used in the current paper remain basic.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The proposed end-to-end approach is designed in a modular way.

    “The proposed end-to-end approach is designed in a modular way and future research can extend the concept with virtually any reconstruction model and task.”
    Quote in the paperNot a substantial claim
38 of 38

  • The authors state this as an assumption.

    The model assumes binary rather than graded electrode activation.

    “Therefore, in this paper, we assume binary instead of graded electrode activation.”

  • The paper relies on this without support.

    The percept restored with visual prostheses is very rudimentary compared to normal vision, and resolution remains relatively limited even with relatively high numbers of electrodes.

    “Compared to normal vision, the percept that can be restored with visual prostheses is very rudimentary and the resolution remains relatively limited, even with relatively high numbers of electrodes.”

  • The paper relies on this without support.

    The V1 implantation site has a relatively large surface area that allows for stimulation with many electrodes.

    “Due to the relatively large surface area, this implantation site allows for stimulation with many electrodes.”

  • The paper relies on this without support.

    The exact phosphene coverage achieved in prosthetic vision depends on many factors, including electrode placement and the cortical anatomy of the patient, which is variable across people.

    “In reality, however, the exact phosphene coverage that is achieved in prosthetic vision will depend on many factors, including the electrode placement and the cortical anatomy of the patient, which is variable across people.”

  • The paper relies on this without support.

    In contrast with convolutional layers, spatial information is lost in a fully connected artificial neural network layer.

    “In contrast with convolutional layers, spatial information is lost in a fully connected artificial neural network layer.”

  • The paper relies on this without support.

    An important requirement for automated optimization with deep learning is that all components of the artificial neural network make use of differentiable operations.

    “An important requirement for automated optimization with deep learning, is that all components of the artificial neural network make use of differentiable operations.”

  • The paper relies on this without support.

    Due to relative complexity and the presence of non-relevant information, encoding naturalistic scenes into phosphenes remains a challenge and requires task-dependent processing.

    “Due to the relative complexity, and the presence of non-relevant information, the encoding of naturalistic scenes into phosphenes remains a challenge and it requires task-dependent processing.”

  • The paper relies on this without support.

    The precise characteristics of the artificial percept that can be generated with visual prosthetics depend on many factors, including electrode placement and the patient's visual cortex.

    “The precise characteristics of the artificial percept that can be generated with visual prosthetics will depend on many factors, including the electrode placement and the visual cortex of the patient.”
8 of 8

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Alternative stimulation strategies have not been validated in vivo in humans using perceptually relevant pulse-train stimulation, and most do not address the substantial day-to-day variability in cortical responses that limits clinical reliability.

    “While alternative stimulation strategies have been proposed [25–39], none have been validated in vivo in humans using perceptually relevant pulse-train stimulation, and most do not address the substantial day-to-day variability in cortical responses that limits clinical reliability.”

    From Deep Learning–Based Control of Electrically Evoked Activity in Human Visual Cortex (2025) · cites this paper as [34]

    Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Recent computational models of prosthetic vision assume linear summation of phosphenes.

    “Moreover, recent computational models of prosthetic vision assume linear summation of phosphenes (Spencer et al 2019, de Ruyter van Steveninck et al 2022), but this has yet to be demonstrated empirically.”

    From Axonal stimulation affects the linear summation of single-point perception in three Argus II users (2024) · cites this paper as de Ruyter van Steveninck et al 2022

  • Machine learning approaches such as convolutional neural networks and deep learning could recognise multiple objects in complex scenes, helping navigation, obstacle avoidance and target object finding.

    “Machine learning approaches, such as convolutional neural networks [189] and deep learning [190,191], could recognise multiple objects in complex scenes, helping navigation, obstacle avoidance and target object finding.”

    From Advances in visual prostheses: engineering and biological challenges (2022) · cites this paper as [191]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Perceptual reconstruction MSE is higher, not lower

    Minor

    The discussion says the perceptual task has lower MSE than the intensity-based task. Table 3 reports 0.064 for perceptual reconstruction and 0.034 for intensity reconstruction; only the stated FSIM comparison is correct.

  • Boundary-pixel classification rate confused with overall accuracy

    Minor

    The reported 69.7% is overall accuracy, not the proportion of boundary pixels classified correctly. Table 3 defines the latter as sensitivity and reports 72.2% for the end-to-end model.

2 of 2

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
de Ruyter van Steveninck, J., Güçlü, U., van Wezel, R., & van Gerven, M.
Affiliations
  • Department of Artificial Intelligence, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
  • Radboud University Nijmegen (NL)
  • Department of Biophysics, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands
ORCID
Güçlü, Umut
Venue
Journal of Vision
Publisher
Association for Research in Vision and Ophthalmology (ARVO)
Published
2022-02-28
Volume
22
Issue
2
Pages
20
ISSN
1534-7362 (ISSN-L)
DOI
10.1167/jov.22.2.20
PMC
PMC8899855
OpenAlex
W4214727024
Cited by
54
References
72
Citations by year
2026: 12 · 2025: 15 · 2024: 12 · 2023: 7 · 2022: 7 · 2021: 1
FWCI
4.53
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Vision (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • University of Twente Research Information (submittedVersion, cc-by)
  • Radboud Repository (Radboud University) (submittedVersion)
  • Radboud Repository (Radboud University) (submittedVersion, other-oa)
Funders
  • Radboud Universiteit
  • Nederlandse Organisatie voor Wetenschappelijk Onderzoek
  • Department of Artificial Intelligence, Korea University
Topics
  • Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neuroscience and Neural Engineering
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
  • Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
Keywords
neuroprosthetics, prosthetic vision, vision restoration, blindness, end-to-end optimization, autoencoder, image preprocessing, stimulation protocol optimization, visual perception
MeSH terms
Blindness, Computer Simulation, Humans, Phosphenes, Vision Disorders, Visual Perception

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review