A virtual patient simulation modeling the neural and perceptual effects of human visual cortical stimulation, from pulse trains to percepts

Model predictions correlate with patient apparent brightness ratings across pulse trains varying in pulse width, frequency, pulse-train duration and amplitude in three surface electrodes (r(42) = 0.771, p < 0.0001).

“There is a strong correlation between simulation predictions and patient data (r(42) = 0.771, p < 0.0001).”

• AssumptionThe authors state this as an assumption: Threshold and brightness are determined by the maximum phosphene brightness over time and space.
• AssumptionThe authors state this as an assumption: Responses reach perceptual detection threshold when the maximum response over time is greater than threshold.

In simulations, nearby electrodes are not spatially resolvable.

“Consistent with preliminary data86, our simulations predict that nearby electrodes are not spatially resolvable.”

• cites [86] Barry, M. P. et al. Contributed session III: Characteristics of electrically-induced visual percepts in the first human with the intracortical visual prosthesis. J. Vis. 23, 35–35. https://doi.org/10.1167/jov.23.11.35 (2023). sourceNot checked yet

  Not checked in the free run.

The model produces perceptual predictions for shape recognition that are very close to patient reports.

“As shown in Fig. 6, our model, based on a prediction of phosphene locations based on aligning the electrode array to a cortical anatomical model (see “STAR methods”), produces perceptual predictions that are very close to patient reports”

• AssumptionRests on an assumption: The percept resulting from stimulation of neuronal populations is based on a linear sum of each cell’s receptive field, weighted by the neural signal strength at that location at each moment in time.

The computational model based on the neurophysiological architecture of V1 successfully predicts the perceptual experience of participants across a wide range of previously published human cortical stimulation studies describing the location, size, brightness and spatiotemporal shape of electrically induced percepts in humans.

“Here we describe a computational model or ‘virtual patient’, based on the neurophysiological architecture of V1, which successfully predicts the perceptual experience of participants across a wide range of previously published human cortical stimulation studies describing the location, size, brightness and spatiotemporal shape of electrically induced percepts in humans.”

• Also stated in: § Introduction · plain

  “Here, for the first time, we show that a great deal of the literature on human electrical stimulation of early visual cortex can be modeled using a simple model based on the neurophysiological architecture of V1.”

• AssumptionThe authors state this as an assumption: The contribution of on cells is weighted more heavily than the contribution of off cells (ωon–off = 0.8), enabling capture of the phenomenon that phosphenes are occasionally dark at threshold but consistently bright as current increases above threshold.
• AssumptionThe authors state this as an assumption: Receptive field size linearly increases with eccentricity.
• AssumptionThe authors state this as an assumption: Threshold and brightness are determined by the maximum phosphene brightness over time and space.
• AssumptionThe authors state this as an assumption: Responses reach perceptual detection threshold when the maximum response over time is greater than threshold.
• AssumptionRests on an assumption: The percept resulting from stimulation of neuronal populations is based on a linear sum of each cell’s receptive field, weighted by the neural signal strength at that location at each moment in time.
• AssumptionThe authors state this as an assumption: A spiking refractory period is followed by a slower integration stage and a compressive nonlinearity.
• AssumptionThe authors state this as an assumption: The resting state membrane potential change is proportional to the second-order spatial derivative of the extracellular potential over the cell.

Simulations are consistent with informal experimental observations that stimulation of individual or multiple electrodes separated by 0.4-1.85 mm in cortex results in irregularly shaped ("amoeba" or "crosses") phosphenes of roughly a half degree in size that contain dark regions.

“Our simulations are consistent with informal experimental observations in patients that stimulation of individual or multiple electrodes separated by 0.4–1.85 mm in cortex result in irregularly shaped (“amoeba” or “crosses”) phosphenes of roughly a half degree in size that contain dark regions.”

Model predictions correlate with experimental thresholds as a function of pulse width (r(43) = 0.804, p < 0.0001).

“There was a strong correlation between model predictions and experimental thresholds: (r(43) = 0.804, p < 0.0001).”

• AssumptionThe authors state this as an assumption: Threshold and brightness are determined by the maximum phosphene brightness over time and space.
• AssumptionThe authors state this as an assumption: Responses reach perceptual detection threshold when the maximum response over time is greater than threshold.

Between 0.25 and 1 mm, phosphenes begin to approximate a "Gaussian blob".

“Between 0.25 and 1 mm, the phosphenes begin to approximate a “Gaussian blob”, but the size of the phosphene is still primarily determined by receptive field sizes rather than the extent of stimulated tissue.”

Model predictions correlate with patient data for phosphene size as a function of amplitude (r(77) = 0.833, p < p < 0.0001).

“Once again, there is a strong correlation between simulation predictions and patient data (r(77) = 0.833, p < p < 0.0001).”

• AssumptionThe authors state this as an assumption: Threshold and brightness are determined by the maximum phosphene brightness over time and space.
• AssumptionThe authors state this as an assumption: Responses reach perceptual detection threshold when the maximum response over time is greater than threshold.

For small electrodes with limited current spread, resulting in less than ~0.25 mm radius of cortical tissue being stimulated, phosphenes tend to have a complicated structure.

“For small electrodes with limited current spread, resulting in less than ~ 0.25 mm radius of cortical tissue being stimulated, phosphenes tend to have a complicated structure (upper panels of Panel B) and the size of the electrode has little effect on the appearance or size of the percept.”

Model predictions based on Keliris et al. receptive field size estimates are well correlated with patient data for phosphene size as a function of eccentricity (r(41) = 0.880, p < 0.0001).

“Simulations based on the Keliris et al. data were well correlated with the patient data, (r(41) = 0.880, p < 0.0001).”

• cites [44] Keliris, G. A., Li, Q., Papanikolaou, A., Logothetis, N. K. & Smirnakis, S. M. Estimating average single-neuron visual receptive field sizes by fMRI. Proc. Natl. Acad. Sci. U. S. A. 116, 6425–6434. https://doi.org/10.1073/pnas.1809612116 (2019). sourceNot checked yet

  Not checked in the free run.

• AssumptionThe authors state this as an assumption: Receptive field size linearly increases with eccentricity.

For small electrodes with limited current spread (<~0.25 mm radius stimulated), the size of the electrode has little effect on the appearance or size of the percept.

“For small electrodes with limited current spread, resulting in less than ~ 0.25 mm radius of cortical tissue being stimulated, phosphenes tend to have a complicated structure (upper panels of Panel B) and the size of the electrode has little effect on the appearance or size of the percept.”

Model predictions correlate with experimental thresholds as a function of pulse frequency (r(34) = 0.774, p < 0.0001).

“nonetheless there was a strong correlation between model predictions and experimental thresholds: (r(34) = 0.774, p < 0.0001).”

• AssumptionThe authors state this as an assumption: Threshold and brightness are determined by the maximum phosphene brightness over time and space.
• AssumptionThe authors state this as an assumption: Responses reach perceptual detection threshold when the maximum response over time is greater than threshold.

The model replicates data from Bosking et al. examining phosphene size as a function of eccentricity for 93 surface electrodes implanted in 13 patients.

“Our model also replicates data from Bosking et al.71 who examined phosphene size as a function of eccentricity for 93 surface electrodes (0.25 mm radius) implanted in 13 patients.”

• cites [71] Bosking, W. H., Beauchamp, M. S. & Yoshor, D. Electrical stimulation of visual cortex: Relevance for the development of visual cortical prosthetics. Annu. Rev. Vis. Sci. 3, 141–166. https://doi.org/10.1146/annurev-vision-111815-114525 (2017). sourceNot checked yet

  Not checked in the free run.

• AssumptionThe authors state this as an assumption: Receptive field size linearly increases with eccentricity.

It is not until electrodes have radii above 1 mm that the size of the electrode has an appreciable impact on the size of the phosphene.

“It is not until electrodes have radii above 1 mm that the size of the electrode has an appreciable impact on the size of the phosphene.”

Between 0.25 and 1 mm, phosphene size is still primarily determined by receptive field sizes rather than the extent of stimulated tissue.

“Between 0.25 and 1 mm, the phosphenes begin to approximate a “Gaussian blob”, but the size of the phosphene is still primarily determined by receptive field sizes rather than the extent of stimulated tissue.”

The model successfully predicts phosphene size as a function of current amplitude.

“Our model also successfully predicts phosphene size as a function of amplitude.”

• AssumptionThe authors state this as an assumption: Threshold and brightness are determined by the maximum phosphene brightness over time and space.
• AssumptionThe authors state this as an assumption: Responses reach perceptual detection threshold when the maximum response over time is greater than threshold.

The model predictions are similar to those made by others using simpler models that assume phosphene size is inversely proportional to cortical magnification.

“our model predictions are similar to those made by others24,83 using simpler models that assume that phosphene size is inversely proportional to cortical magnification.”

• cites [24] Bosking, W. H. et al. Saturation in phosphene size with increasing current levels delivered to human visual cortex. J. Neurosci. 37, 7188–7197. https://doi.org/10.1523/JNEUROSCI.2896-16.2017 (2017). sourceNot checked yet

  Not checked in the free run.

• cites [83] Grinten, M. V. D. et al. Biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses. bioRxiv https://doi.org/10.1101/2022.12.23.521749 (2022). sourceNot checked yet

  Not checked in the free run.

Given a single scaling parameter, the shape of experimental strength-duration curves showed little variation across a wide range of experimental protocols.

“As can be seen in Figure 3A, consistent with our model, given a single scaling parameter, the shape of experimental strength-duration curves showed little variation across a wide range of experimental protocols.”

Across the entire visual field, receptive fields impose a neurophysiological 'lower limit' on phosphene size.

“Critically, our simulations suggest that, across the entire visual field, receptive fields impose a neurophysiological ‘lower limit’ on phosphene size.”

Once phosphene size is normalized to the maximum size, the effect of eccentricity on the curves relating phosphene size to stimulation amplitude is small.

“Once phosphene size is normalized to the maximum size, the effect of eccentricity (difference between square and circle symbols) on the curves relating phosphene size to stimulation amplitude is small.”

The model successfully predicts that phosphene size increases as a function of eccentricity in the visual field.

“Our model successfully predicts the finding that phosphene size increases as a function of eccentricity in the visual field.”

• AssumptionThe authors state this as an assumption: Receptive field size linearly increases with eccentricity.

An electrode array arranged to produce a regular tiling in visual space underrepresents the fovea, producing a sparse collection of tiny phosphenes in the fovea.

“Figure 8A shows electrodes arranged to produce a regular tiling in visual space. This array clearly underrepresents the fovea – producing a sparse collection of tiny phosphenes in the fovea.”

The model replicates previous findings that simultaneous stimulation results in less interpretable percepts than sequential stimulation.

“Despite this, our model does replicate previous findings that simultaneous stimulation results in less interpretable percepts then sequential stimulation.”

For a fixed electrode size, phosphene size increases linearly as a function of eccentricity.

“Both data and simulations suggest that, for a fixed electrode size, phosphene size increases linearly as a function of eccentricity.”

The model successfully predicts a wide range of psychophysical and electrophysiological cortical stimulation data.

“Despite its simplicity and lack of fitted parameters, our model successfully predicts a wide range of psychophysical and electrophysiological cortical stimulation data.”

For larger electrode sizes, cortical magnification and the extent of stimulated cortex also play a role in the relationship between phosphene size and eccentricity.

“For electrode radii less than 0.25 mm this linear relationship is primarily due to the increase in receptive field sizes as a function of eccentricity. It is only for larger electrode sizes that cortical magnification and the extent of stimulated cortex also plays a role.”

An electrode array arranged regularly on the cortical surface over-tiles electrodes within the fovea, producing a cluster of heavily overlapping receptive fields within the fovea.

“Figure 8B shows electrodes arranged regularly on the cortical surface. This configuration suffers from the opposite problem—an over-tiling of electrodes within the fovea, producing a cluster of heavily overlapping receptive fields within the fovea, as has been informally observed in one recent study86.”

• cites [86] Barry, M. P. et al. Contributed session III: Characteristics of electrically-induced visual percepts in the first human with the intracortical visual prosthesis. J. Vis. 23, 35–35. https://doi.org/10.1167/jov.23.11.35 (2023). sourceNot checked yet

  Not checked in the free run.

For electrode radii less than 0.25 mm, the linear relationship between phosphene size and eccentricity is primarily due to the increase in receptive field sizes as a function of eccentricity.

“For electrode radii less than 0.25 mm this linear relationship is primarily due to the increase in receptive field sizes as a function of eccentricity. It is only for larger electrode sizes that cortical magnification and the extent of stimulated cortex also plays a role.”

The simulated ocular dominance map's columns overlap with the orientation map in a manner consistent with results from optical imaging data from Obermayer and Blasdel.

“The result is an ocular dominance map whose columns overlap with the orientation map in a manner consistent with results from optical imaging data from Obermayer and Blasdel41.”

• cites [41] Obermayer, K. & Blasdel, G. G. Geometry of orientation and ocular dominance columns in monkey striate cortex. J. Neurosci. 13, 4114–4129 (1993). sourceNot checked yet

  Not checked in the free run.

The fitted cortical shape and array position parameters that best predicted the locations of all 24 perceived phosphenes fell within typical variation observed in human cortical maps.

“We used function minimization to find the cortical shape (a = 0.15, k = 16.6, squish = 0.63) and array position (x = − 68.4, y = − 6.85, and angle = -2.2) that best predicted the location of all 24 perceived phosphenes. These parameters fell within typical variation observed in human cortical maps106.”

• cites [106] Duncan, R. O. & Boynton, G. M. Cortical magnification within human primary visual cortex correlates with acuity thresholds. Neuron 38, 659–671 (2003). sourceNot checked yet

  Not checked in the free run.

Stimulation with extremely small electrodes will potentially stimulate neurons tuned for similar orientations, creating percepts that are elongated, or have complicated structure.

“As a result, stimulation with extremely small electrodes will potentially stimulate neurons tuned for similar orientations, creating percepts that are elongated, or have complicated structure.”

Results from the wide collection of studies on human electrical stimulation of early visual cortex have been almost entirely descriptive to date.

“However, to date, results from this wide collection of studies have been almost entirely descriptive.”

• cites Table 1Not checked yet

  Not checked in the free run.

In the model, the shape of the function relating threshold as a function of pulse width-the strength-duration curve-is entirely determined by the first integration stage of the temporal model, and is independent of electrode size, frequency or pulse train duration.

“In our model the shape of the function relating threshold as a function of pulse width—the ‘strength-duration’ curve69 is entirely determined by the first integration stage of our temporal model, and is independent of electrode size, frequency or pulse train duration.”

• cites [69] Lapicque, L. L’excitabilité en Fonction du Temps: La Chronaxie, sa Signification et sa Mesure Vol. 4 (Les presses universitaires de France, 1926).Not checked yet

  Not checked in the free run.

• AssumptionThe authors state this as an assumption: A spiking refractory period is followed by a slower integration stage and a compressive nonlinearity.

Orientation and on-off dominance columns are relatively large (> 2 mm for a full ocular dominance/pinwheel map).

“Our predictions reflect the fact that orientation and on–off dominance columns are relatively large (> 2 mm for a full ocular dominance/pinwheel map41,42).”

• cites [41] Obermayer, K. & Blasdel, G. G. Geometry of orientation and ocular dominance columns in monkey striate cortex. J. Neurosci. 13, 4114–4129 (1993). sourceNot checked yet

  Not checked in the free run.

• cites [42] Adams, D. L., Sincich, L. C. & Horton, J. C. Complete pattern of ocular dominance columns in human primary visual cortex. J. Neurosci. 27, 10391–10403. https://doi.org/10.1523/JNEUROSCI.2923-07.2007 (2007). sourceNot checked yet

  Not checked in the free run.

The primary difficulty with simultaneous stimulation may be due to a 'Gestalt' failure to correctly group phosphenes.

“Because our model does not include electrical or complex neuronal spatiotemporal interactions these results suggest that the primary difficulty with simultaneous stimulation may be due to a ‘Gestalt’ failure to correctly group phosphenes.”

Previous experimental studies have found it extremely difficult to generate recognizable shapes through stimulation of multiple electrodes.

“Previous experimental studies have found it extremely difficult to generate recognizable shapes through stimulation of multiple electrodes34,84.”

• cites [34] Schmidt, E. M. et al. Feasibility of a visual prosthesis for the blind based on intracortical microstimulation of the visual cortex. Brain 119, 507–522 (1996). sourceNot checked yet

  Not checked in the free run.

• cites [84] Dobelle, W. H. Artificial vision for the blind by connecting a television camera to the visual cortex. ASAIO J. 46, 3–9 (2000). sourceNot checked yet

  Not checked in the free run.

In the model, threshold as a function of frequency is determined by the second stage of the temporal model and the refractory period.

“In our model, threshold as a function of frequency is determined by the second stage of our temporal model and the refractory period.”

• AssumptionThe authors state this as an assumption: A spiking refractory period is followed by a slower integration stage and a compressive nonlinearity.

In the foreseeable future, the perceptual quality of cortical prosthetic devices is likely to be limited by the neurophysiological organization of visual cortex rather than engineering constraints.

“Our simulations suggest that, in the foreseeable future the perceptual quality of cortical prosthetic devices is likely to be limited by the neurophysiological organization of visual cortex, rather than engineering constraints.”

Subjects can identify simple forms when multiple electrodes are stimulated in sequence even though those same shapes are uninterpretable when electrodes are simultaneously stimulated.

“Recently, Beauchamp et al.25 showed that subjects can identify simple forms when multiple electrodes are stimulated in sequence even though those same shapes are uninterpretable when electrodes are simultaneously stimulated.”

• cites [25] Beauchamp, M. S. et al. Dynamic stimulation of visual cortex produces form vision in sighted and blind humans. Cell 181, 774-783e775. https://doi.org/10.1016/j.cell.2020.04.033 (2020). sourceNot checked yet

  Not checked in the free run.

Electrode sensitivity (s) may play a more major role in influencing the current amplitude-size curve than eccentricity.

“Our simulations suggest that electrode sensitivity (s) may play a more major role in influencing the current amplitude-size curve.”

The model is capable of providing insight into the likely perceptual experience of novel technologies.

“Our ability to replicate such a wide range of data suggests that this model is capable of providing insight into the likely perceptual experience of novel technologies—one of the more important uses of ‘virtual patients’.”

A basic model describing the transformation from pulse trains to perceptual intensity over time can successfully predict both thresholds and brightness ratings across a wide range of pulse train parameters, electrode locations and sizes.

“Thus, the data from Fig. 3, collated across a wide variety of studies, supports the notion that a basic model describing the transformation from pulse trains to perceptual intensity over time can successfully predict both thresholds and brightness ratings across a wide range of pulse train parameters, electrode locations and sizes.”

The sensitivity parameter s likely mediates the effects of a multitude of factors affecting sensitivity including distance of the electrode from the cortical surface, electrode size, and distribution of current over the electrode surface.

“This parameter likely mediates the effects of a multitude of factors affecting sensitivity including the distance of the electrode from the cortical surface, electrode size (which alters current density), and the distribution of current over the electrode surface.”

Linear averaging may, in practice, capture the essential structure of how neuronal messages are encoded by the brain.

“Thus our linear averaging may, in practice, capture the essential structure of how neuronal messages are encoded by the brain91.”

• cites [91] Brackbill, N. et al. Reconstruction of natural images from responses of primate retinal ganglion cells. eLife 9, e58516. https://doi.org/10.7554/eLife.58516 (2020). sourceNot checked yet

  Not checked in the free run.

The assumption often made in the cortical stimulation literature, that the massive expansion of the foveal representation in V1 might allow for relatively high sampling of spatial position, is probably incorrect.

“Thus, the assumption often made in the cortical stimulation literature, that the massive expansion of the foveal representation in V1 might allow for relatively high sampling of spatial position, is probably incorrect.”

Overlapping receptive fields in the fovea seem to offer little to no benefit in terms of resolution.

“These overlapping receptive fields seem to offer little to no benefit in terms of resolution: In the region of cortex representing the foveal confluence nearby electrodes project to almost the same location of visual space – so small shifts in electrode locations on the cortical surface produces imperceptible shifts in phosphene location relative to the size of foveal phosphenes.”

High-resolution vision is based on interpreting the complex pattern of responses across a population of neurons with receptive fields tuned to multiple locations, orientations and sizes.

“However, as discussed more fully below, our ability to see in high resolution is based on interpreting the complex pattern of responses across a population of neurons with receptive fields tuned to multiple locations, orientations and sizes.”

For smaller electrodes, it is receptive field sizes that limit acuity.

“For smaller electrodes, it is receptive field sizes that limit acuity.”

At a single cell level, once a spike is elicited, there is little effect of increasing the current amplitude further.

“Thus, at a single cell level, once a spike is elicited, there is little effect of increasing the current amplitude further.”

Optimal spacing calculations are based on estimated average receptive field sizes.

“Our optimal spacing calculations are based on estimated average receptive field sizes.”

It remains probable that electrical and neuronal interactions play an additional complicating role.

“Nonetheless, it remains probable that electrical and neuronal interactions play an additional complicating role54.”

• cites [54] Nowak, L. G. & Bullier, J. Axons, but not cell bodies, are activated by electrical stimulation in cortical gray matter. I. Evidence from chronaxie measurements. Exp. Brain Res. 118, 477–488. https://doi.org/10.1007/s002210050304 (1998). sourceNot checked yet

  Not checked in the free run.

Our ability to perceive a single point of light, a fine grating, or the offset in a thin line is not limited by minimum receptive field sizes.

“Our ability to perceive a single point of light, a fine grating, or the offset in a thin line is not limited by minimum receptive field sizes.”

Cortical magnification varies logarithmically.

“Because receptive fields vary linearly with eccentricity67 while cortical magnification varies logarithmically64,82, the optimal configuration packs electrodes less tightly in regions of the cortex representing the fovea than regions representing the periphery (see “STAR methods”).”

• cites [64] Schwartz, E. L. Cerebral Cortex (Plenum Press, 1994).Not checked yet

  Not checked in the free run.

• cites [82] Stevens, C. F. Predicting functional properties of visual cortex from an evolutionary scaling law. Neuron 36, 139–142. https://doi.org/10.1016/s0896-6273(02)00902-9 (2002). sourceNot checked yet

  Not checked in the free run.

Increases in current amplitude also cause cells further from the electrode to reach their depolarization threshold, due to current spread, increasing the size of the phosphene.

“However, increases in current amplitude also cause cells further from the electrode to reach their depolarization threshold, due to current spread, increasing the size of the phosphene.”

The only factor allowed to vary across experiments was a sensitivity parameter (s) that linearly scaled current amplitude.

“The only factor that was allowed to vary across experiments was a sensitivity parameter (s) that linearly scaled current amplitude.”

Neurons vary in the sensitivity of their activating function because the capacitance and resistance of individual neurons vary based on their size, shape and myelination.

“However, because the capacitance and resistance of individual neurons vary, based on their size, shape and myelination, neurons vary in the sensitivity of their activating function.”

If one interprets the responses of early visual areas as approximate to a wavelet analysis, the resolution of small spots of light would be mediated by a population response across hundreds or thousands of V1 neurons with a wide range of receptive field shapes and sizes.

“If one interprets the responses of early visual areas as being approximate to a wavelet analysis, one would expect the resolution of small spots of light to be mediated by a population response across hundreds or thousands of V1 neurons with a wide range of receptive field shapes and sizes.”

A one-stage leaky integrator provides a reasonable approximation to cellular depolarization due to extracellular stimulation.

“It is well established that this one-stage leaky integrator provides a reasonable approximation to cellular depolarization due to extracellular stimulation60–62.”

• cites [60] Rattay, F. Analysis of models for external stimulation of axons. IEEE Trans. Biomed. Eng. https://doi.org/10.1109/TBME.1986.325670 (1986). sourceNot checked yet

  Not checked in the free run.

• cites [61] Jensen, R. J., Rizzo, J. F. III., Ziv, O. R., Grumet, A. & Wyatt, J. Thresholds for activation of rabbit retinal ganglion cells with an ultrafine, extracellular microelectrode. Investig. Ophthalmol. Vis. Sci. 44, 3533–3543 (2003). sourceNot checked yet

  Not checked in the free run.

• cites [62] Jensen, R. J., Ziv, O. R. & Rizzo, J. F. 3rd. Thresholds for activation of rabbit retinal ganglion cells with relatively large, extracellular microelectrodes. Investig. Ophthalmol. Vis. Sci. 46, 1486–1496 (2005). sourceNot checked yet

  Not checked in the free run.

As the current amplitude of a pulse increases, gradually more and more cells under the electrode will reach their depolarization threshold.

“Consequently, as the current amplitude of a pulse increases, gradually more and more cells under the electrode will reach their depolarization threshold.”

The current model only includes cortical area V1.

“Finally, our current model only includes cortical area V1.”

Neurophysiological rather than engineering constraints are likely to limit the spatial resolution of cortical prostheses for the foreseeable future.

“Overall, our simulations suggest that neurophysiological rather than engineering constraints are likely to limit the spatial resolution of cortical prostheses for the foreseeable future.”

Extremely fine spatial discriminations are based on interpreting a complex pattern of responses across a population of neurons with different receptive fields.

“Extremely fine spatial discriminations are based on interpreting a complex pattern of responses across a population of neurons with different receptive fields.”

Receptive fields vary linearly with eccentricity.

“Because receptive fields vary linearly with eccentricity67 while cortical magnification varies logarithmically64,82, the optimal configuration packs electrodes less tightly in regions of the cortex representing the fovea than regions representing the periphery (see “STAR methods”).”

• cites [67] Freeman, J. & Simoncelli, E. P. Metamers of the ventral stream. Nat. Neurosci. 14, 1195–1201. https://doi.org/10.1038/nn.2889 (2011). sourceNot checked yet

  Not checked in the free run.

Generating a punctate percept through electrical stimulation would require appropriately stimulating many hundreds or thousands of neurons.

“Conversely, generating a punctate percept through electrical stimulation would require appropriately stimulating many hundreds or thousands of neurons95.”

• cites [95] Gogliettino, A. R. et al. High-fidelity reproduction of visual signals by electrical stimulation in the central primate retina. J. Neurosci. 43, 4625–4641. https://doi.org/10.1523/jneurosci.1091-22.2023 (2023). sourceNot checked yet

  Not checked in the free run.

For large electrodes used in some simulated studies, current intensity can be five-fold higher at the edge of the electrode than in the middle.

“However in practice current “pools” around the edge of electrodes87,88 – for the large electrodes used in some of the studies we simulated, current intensity can be five-fold higher at the edge of the electrode than in the middle.”

• cites [87] Rubinstein, J. T., Spelman, F. A., Soma, M. & Suesserman, M. F. Current density profiles of surface mounted and recessed electrodes for neural prostheses. IEEE Trans. Biomed. Eng. 34, 864–875 (1987). sourceNot checked yet

  Not checked in the free run.

• cites [88] Suesserman, M. F., Spelman, F. A. & Rubinstein, J. T. In vitro measurement and characterization of current density profiles produced by non-recessed, simple recessed, and radially varying recessed stimulating electrodes. IEEE Trans Biomed. Eng. 38, 401–408 (1991). sourceNot checked yet

  Not checked in the free run.

At the fovea, cortical magnification reaches a maximum, and therefore receptive field sizes reach a minimum, somewhere between 0.02 and 0.5 degrees in radius.

“At the fovea, cortical magnification reaches a maximum, and therefore receptive field sizes reach a minimum, somewhere between 0.02 and 0.5 degrees in radius.”

In the model, the output of the first stage, 'spike response strength', should be thought of as reflecting the recruitment of spikes from a population of cells with activation functions that vary in sensitivity, not as representing spikes per se.

“Thus, in our model, the output of the first stage, ‘spike response strength’ should not be thought of as representing spikes per se, but as reflecting the recruitment of spikes from a population of cells with activation functions that vary in sensitivity.”

The optimal configuration packs electrodes less tightly in regions of the cortex representing the fovea than regions representing the periphery.

“Because receptive fields vary linearly with eccentricity67 while cortical magnification varies logarithmically64,82, the optimal configuration packs electrodes less tightly in regions of the cortex representing the fovea than regions representing the periphery (see “STAR methods”).”

Across much of cortex, receptive field areas approximate the areal cortical magnification to the −2/3 power.

“Across much of cortex receptive field areas approximate the areal cortical magnification to the − 2/3 power82; explaining the previous observation of Bosking et al.24 that the size of the phosphenes drawn by their patients could be predicted by cortical magnification.”

• cites [82] Stevens, C. F. Predicting functional properties of visual cortex from an evolutionary scaling law. Neuron 36, 139–142. https://doi.org/10.1016/s0896-6273(02)00902-9 (2002). sourceNot checked yet

  Not checked in the free run.

Simulations suggest that electrodes should be placed more sparsely in regions of cortex representing the foveal confluence, the opposite of the common intuition that electrodes should be placed more densely in foveal regions.

“Thus, our simulations suggest that electrodes should actually be placed more sparsely in regions of cortex representing the foveal confluence – the opposite of the common intuition that electrodes should be placed more densely in foveal regions.”

The previous observation of Bosking et al. that the size of the phosphenes drawn by their patients could be predicted by cortical magnification is explained by the relationship between receptive field areas and cortical magnification.

“Across much of cortex receptive field areas approximate the areal cortical magnification to the − 2/3 power82; explaining the previous observation of Bosking et al.24 that the size of the phosphenes drawn by their patients could be predicted by cortical magnification.”

• cites [24] Bosking, W. H. et al. Saturation in phosphene size with increasing current levels delivered to human visual cortex. J. Neurosci. 37, 7188–7197. https://doi.org/10.1523/JNEUROSCI.2896-16.2017 (2017). sourceNot checked yet

  Not checked in the free run.

The model should be considered an approximation that will not generalize to longer stimulation protocols.

“Our model should therefore be considered an approximation that will not generalize to longer stimulation protocols.”

Simulations such as these allow one to predict the best possible perceptual performance for any given array, based on neurophysiological constraints.

“More generally, simulations such as these allow one to predict the best possible perceptual performance for any given array, based on neurophysiological constraints.”

At a single cell level, once depolarized to 'threshold' a neuron will almost always produce a single spike rather than multiple spikes.

“At a single cell level, once depolarized to ‘threshold’ a neuron will (almost always) produce a single spike rather than multiple spikes (since the pulse durations used in electrical stimulation are short compared to the refractory period).”

At the retinal level, as experimental sampling of the population of neurons converges to a complete tiling, the reconstruction filters generated by the optimal reconstruction filter approach come to resemble receptive fields as measured using white noise stimuli.

“However, at the retinal level it has been found that as experimental sampling of the population of neurons converges to a complete tiling, the reconstruction filters generated by this approach come to resemble receptive fields as measured using white noise stimuli.”

• cites [91] Brackbill, N. et al. Reconstruction of natural images from responses of primate retinal ganglion cells. eLife 9, e58516. https://doi.org/10.7554/eLife.58516 (2020). sourceNot checked yet

  Not checked in the free run.

The optimal spacing on the cortex is approximately equal to the size of the phosphene in visual space multiplied by the slope of the cortical mapping function y(x).

“Thus, the optimal spacing on the cortex is approximately equal to the size of the phosphene in visual space multiplied by the slope of the cortical mapping function y(x).”

For realistic phosphene sizes and a feasible map between visual space and cortex, optimal cortical sampling should be less dense toward the foveal representation of the visual field, despite the large expansion of cortex devoted to foveal vision.

“Paradoxically, for realistic phosphene sizes and a feasible map between visual space and cortex, optimal cortical sampling should be less dense toward the foveal representation of the visual field, despite the large expansion of cortex devoted to foveal vision.”

The current model counterintuitively suggests limited advantages to small electrode sizes and dense implantation in regions of cortex devoted to the fovea.

“For example, as described above, our current model counterintuitively suggests limited advantages to small electrode sizes and dense implantation in regions of cortex devoted to the fovea.”

Without extending virtual prototyping to include the basic physiology of early visual areas, it is impossible to predict perceptual outcomes.

“However, without extending virtual prototyping to include the basic physiology of early visual areas it is impossible to predict perceptual outcomes.”

Optimal cortical sampling depends on the mapping function from visual space to visual cortex and the size of phosphenes as a function of visual eccentricity.

“Optimal cortical sampling depends on the mapping function from visual space to visual cortex and the size of phosphenes as a function of visual eccentricity.”

The model is likely to provide a reasonable approximation of predicted perceptual outcomes for future implants.

“Our simple model is successful at predicting a wide range of cortical electrical stimulation results, suggesting that it is likely to provide a reasonable approximation of predicted perceptual outcomes for future implants.”