Precise control of neural activity using dynamically optimized electrical stimulation

The dynamically optimized stimulation approach substantially enhanced performance compared to existing approaches based on static mapping between visual stimulus intensity and current amplitude.

“The dynamically optimized stimulation approach substantially enhanced performance compared to existing approaches based on static mapping between visual stimulus intensity and current amplitude.”

• Also stated in: § Introduction · plain

  “The method produced substantial improvements in stimulus reconstruction compared to existing methods, by appropriately activating ON and OFF RGCs over space.”

• Also stated in: § Greedy temporal dithering outperforms existing static methods · hedged

  “Even using this generous upper bound, static pixel-wise mapping resulted in significantly less accurate reconstruction of the target image (Figure 3D, H), likely due to coactivation of overlapping ON and OFF cell types.”

Many electrical stimuli simultaneously activated two or more cells, often due to axonal stimulation.

“In general, even though some electrical stimuli selectively activated one cell, many stimuli simultaneously activated two or more cells, often due to axonal stimulation (Figure 1C; Grosberg et al., 2017).”

• cites Grosberg et al., 2017 sourceNot checked yet

  Not checked in the free run.

Examination of many electrode pairs over a range of distances and multiple cells revealed a systematic decrease of the interaction between stimulating electrodes as a function of distance.

“Examination of many electrode pairs over a range of distances and multiple cells (Figure 5B) revealed a systematic decrease of the interaction between stimulating electrodes as a function of distance.”

The reconstruction performance benefited from differential activation of ON and OFF cells over space in a way that reflected the spatial distribution of intensities in each target image.

“Importantly, the reconstruction performance benefited from differential activation of ON and OFF cells over space in a way that reflected the spatial distribution of intensities in each target image (Figure 4E, F).”

A minimal (<5%) increase in reconstruction error was observed if the number of available electrodes was reduced by up to 50%.

“However, a minimal (<5%) increase in reconstruction error was observed if the number of available electrodes was reduced by up to 50% (Figure 6B–F).”

Reconstructions using evoked RGC responses were less accurate than reconstructions based on calibrated responses to single-electrode stimulation.

“However, these reconstructions were less accurate than the reconstructions based on the calibrated responses to single-electrode stimulation (which were used to compute the temporal dithering stimulation sequence during the experiment).”

The reconstruction error obtained using the RGC responses evoked by the stimulation sequence was higher than the error obtained using calibrated responses to single-electrode stimulation.

“Second, the reconstruction error obtained using the RGC responses evoked by the stimulation sequence was higher than the error obtained using calibrated responses to single-electrode stimulation (Figure 4G, bottom), presumably reflecting non-stationarity in the ex vivo recordings and/or failures of independence due to temporal dithering (Figure 4G, see Discussion).”

The reconstructed visual scene was smoother and more accurate (lower reconstruction error) when fixational eye movements were simulated along with saccades.

“Interestingly, the reconstructed visual scene was smoother and more accurate (lower reconstruction error) when fixational eye movements were simulated along with saccades (Figure 7C).”

These reconstructions reached asymptotic quality with 225 ± 29 stimulations.

“These reconstructions reached asymptotic quality with 225 ± 29 stimulations (not shown), a significantly smaller number than in tests obtained with macaque retina (Figure 3G), likely in part because of the smaller number of cells in the rat recordings (see Discussion).”

Reconstructions using evoked RGC responses captured much of the spatial structure in each target image.

“The reconstructions using these evoked RGC responses captured much of the spatial structure in each target image.”

The performance gap between the greedy algorithm and the optimal algorithm was low (<10%) across multiple target images.

“Across multiple target images, the gap was low (<10%, ‘optimal algorithm’ in Figure 3E, H), suggesting that the approximate nature of the greedy algorithm is not a substantial source of error in the present conditions.”

The reconstruction error obtained using calibrated responses to single-electrode stimulation was higher than the error obtained under an assumption of 'perfect control'.

“First, the reconstruction error obtained using calibrated responses to single-electrode stimulation was higher than the error obtained under an assumption of ‘perfect control’ (i.e. that all recorded cells can be activated independently), reflecting the limitations of single-electrode stimulation (Figure 4G, top), as shown earlier (Figure 3H).”

High-amplitude stimuli tended to evoke spikes in cells with receptive fields off the electrode array via their axons.

“Also, high-amplitude stimuli tended to evoke spikes in cells with receptive fields off the electrode array via their axons.”

The reconstructions obtained with the evoked RGC responses captured much of the spatial structure in each target image.

“The reconstructions obtained with the evoked RGC responses captured much of the spatial structure in each target image (Figure 4A–D).”

The spatial structure of reconstructions using calibrated responses to single-electrode stimulation and the RGC responses evoked by the stimulation sequence were similar.

“Notably, the spatial structure of reconstructions using the calibrated responses to single-electrode stimulation (‘calibrated’ in Figure 4, similar to Figures 2 and 3) and the RGC responses evoked by the stimulation sequence (‘evoked’ in Figure 4) were similar.”

SSIM optimization produced higher-quality reconstructions when the number of electrical stimuli was limited.

“However, SSIM optimization produced higher-quality reconstructions when the number of electrical stimuli was limited (Figure 8C), a constraint that may be relevant with short visual integration times or low power limits in an implant.”

For the same final reconstruction error, a ~4× reduction in the number of required saccades and hence the number of required electrical stimuli was observed in the presence of fixational eye movements.

“Specifically, for the same final reconstruction error, a ~4× reduction in the number of required saccades and hence the number of required electrical stimuli was observed in the presence of fixational eye movements (Figure 7D).”

SSIM and MSE metrics produced similar reconstructions when the number of electrical stimulation patterns was unlimited.

“The SSIM and MSE metrics produced similar reconstructions when the number of electrical stimulation patterns was unlimited (Figure 8B).”

The approach frequently exploits non-selective activation, in order to evoke desired spiking activity in fewer time steps.

“In fact, the approach frequently exploits non-selective activation, in order to evoke desired spiking activity in fewer time steps (Figure 6).”

• AssumptionThe authors state this as an assumption: Distant electrodes generate independent responses.

The assembled visual scene closely matched the target, capturing many of its fine details.

“The assembled visual scene closely matched the target, capturing many of its fine details (Figure 7C, Figure 7—video 1).”

The greedy temporal dithering and spatial multiplexing approach outperforms existing approaches.

“The greedy temporal dithering and spatial multiplexing approach outperforms existing approaches (Figures 2—5) and enables efficient neural interface development (Figure 6), potentially incorporating naturalistic viewing with eye movements (Figure 7) and/or perceptual similarity metrics (Figure 8).”

• AssumptionThe authors state this as an assumption: Downstream processing of RGC responses is slow, integrating over tens of milliseconds, so perception depends primarily on the total number of evoked spikes within this time interval.
• AssumptionThe authors state this as an assumption: Brief (150 µs) and low (<4 µA) current pulses used in this study typically evoke a single spike with low and precise latencies.
• AssumptionThe authors state this as an assumption: Electrical stimuli at the same electrode separated by at least 10 ms generate approximately independent responses.
• AssumptionThe authors state this as an assumption: Distant electrodes generate independent responses.

On average, the activation probability of a cell in response to single-electrode stimulation was affected relatively little (<5% fractional change in threshold) for a secondary electrode 200μm away.

“On average, the activation probability of a cell in response to single-electrode stimulation was affected relatively little (<5% fractional change in threshold) for a secondary electrode 200µm away.”

The greedy temporal dithering algorithm chooses electrodes in a spatially non-uniform manner over the array.

“Because the greedy temporal dithering algorithm chooses electrodes in a spatially non-uniform manner over the array (Figure 6A), restricting stimulation to a more frequently chosen subset of electrodes could enhance efficiency.”

The increase in reconstruction error with electrode subsampling was not due to the greedy nature of stimulation, because a lower bound computed for an optimal algorithm showed similar behavior.

“Note that this increase was not due to the greedy nature of stimulation, because a lower bound computed for an optimal algorithm showed similar behavior (Figure 6B).”

The performance gap between a single-electrode dictionary and perfect control was substantial (>40%) across multiple targets.

“Across multiple targets, this gap was substantial (>40%, ‘perfect control’ in Figure 3F, H).”

During the stimulation sequence, the reconstructed image slowly built up to a spatially smooth version of the target image.

“During the stimulation sequence, the reconstructed image slowly built up to a spatially smooth version of the target image (Figure 2).”

The improvement of dynamic optimization over static methods is due to appropriately activating ON and OFF RGCs over space.

“The method produced substantial improvements in stimulus reconstruction compared to existing methods, by appropriately activating ON and OFF RGCs over space.”

The ex vivo lab prototype provides reliable experimental access to complete populations of ON and OFF parasol cell types in macaque retina.

“These data provide reliable experimental access to complete populations of ON and OFF parasol cell types in macaque retina, so these two cell types are the focus of the empirical analysis.”

• cites Field and Chichilnisky, 2007Not checked yet

  Not checked in the free run.

• cites Jepson et al., 2013 sourceNot checked yet

  Not checked in the free run.

• cites Grosberg et al., 2017 sourceNot checked yet

  Not checked in the free run.

Epiretinal implants electrically activate retinal ganglion cells (RGCs) that have survived degeneration, causing them to send artificial visual signals to the brain.

“Epiretinal implants electrically activate retinal ganglion cells (RGCs) that have survived degeneration, causing them to send artificial visual signals to the brain.”

• cites Humayun et al., 2012 sourceNot checked yet

  Not checked in the free run.

• cites Beyeler et al., 2019 sourceNot checked yet

  Not checked in the free run.

• cites Bloch, 2020 sourceNot checked yet

  Not checked in the free run.

A direct closed-loop test of the approach supported its potential use in vision restoration.

“A direct closed-loop test of the approach supported its potential use in vision restoration.”

Spatial multiplexing of stimuli outside a spatial exclusion radius is a practical strategy for high-throughput stimulation with temporal dithering.

“This suggests that spatial multiplexing of stimuli outside a spatial exclusion radius is a practical strategy for high-throughput stimulation with temporal dithering.”

The static pixel-wise mapping simulation provides a generous upper bound to the performance of existing retinal implants because it uses actual response probabilities to optimize each sigmoidal function rather than relying on limited patient feedback.

“This approach provides a generous upper bound to the performance of existing implants, because it uses actual response probabilities to optimize each sigmoidal function rather than relying on much more limited patient feedback, as is the case in existing retinal implants.”

The stimulus reconstruction filter approximation matches an optimal linear decoder only if the receptive fields of the cells are orthogonal.

“Note that because the stimulus reconstruction filter is proportional to the receptive field, this approximation matches an optimal linear decoder only if the receptive fields of the cells are orthogonal.”

Restricting stimulation to a more frequently chosen subset of electrodes could enhance efficiency.

“Because the greedy temporal dithering algorithm chooses electrodes in a spatially non-uniform manner over the array (Figure 6A), restricting stimulation to a more frequently chosen subset of electrodes could enhance efficiency.”

Two electrodes more than 200μm apart were unlikely to substantially influence the activation probability of the same cell(s).

“Thus, two electrodes more than this distance apart were unlikely to substantially influence the activation probability of the same cell(s).”

Such a reduction in the set of stimulated electrodes could lead to reduced memory access and power consumption, with little loss in performance.

“Such a reduction in the set of stimulated electrodes could lead to reduced memory access and power consumption, with little loss in performance.”

The physiological similarities between human and macaque retina, including their responses to electrical stimulation, suggest that the benefits of the present stimulation approach could translate from the ex vivo lab prototype with healthy macaque retina to an in vivo implant in the degenerated human retina.

“The physiological similarities between human and macaque retina (Cowan et al., 2019; Kling et al., 2020; Soto et al., 2020; Rodieck, 1998), including their responses to electrical stimulation (Madugula et al., 2022), suggest that the benefits of the present stimulation approach could translate from the ex vivo lab prototype with healthy macaque retina to an in vivo implant in the degenerated human”

• cites Cowan et al., 2019 sourceNot checked yet

  Not checked in the free run.

• cites Kling et al., 2020 sourceNot checked yet

  Not checked in the free run.

• cites Madugula et al., 2022 sourceNot checked yet

  Not checked in the free run.

More accurate image reconstruction is possible with fixational eye movements.

“Hence, the performance of greedy temporal dithering translates to natural viewing conditions and reveals that more accurate image reconstruction is possible with fixational eye movements (Wu et al., 2024).”

• cites Wu et al., 2024Not checked yet

  Not checked in the free run.

A single-electrode stimulus generally cannot create a pattern of activity across the RGC population that accurately encodes a visual image.

“Because a single-electrode stimulus generally cannot create a pattern of activity across the RGC population that accurately encodes a visual image, multiple stimuli must be combined, while also avoiding the nonlinear interactions mentioned above.”

The approach can effectively exploit non-selective activation.

“Second, the approach can effectively exploit non-selective activation.”

• AssumptionThe authors state this as an assumption: Distant electrodes generate independent responses.

The hardware requirements of the proposed closed-loop procedure are less stringent than those of a real-time closed-loop system.

“First, the hardware requirements of the proposed closed-loop procedure – calibration of single-electrode stimuli followed by dynamically optimized stimulation for encoding visual scenes – are less stringent than those of a real-time closed-loop system.”

The greedy temporal dithering and spatial multiplexing approach enables efficient neural interface development.

“The greedy temporal dithering and spatial multiplexing approach outperforms existing approaches (Figures 2—5) and enables efficient neural interface development (Figure 6), potentially incorporating naturalistic viewing with eye movements (Figure 7) and/or perceptual similarity metrics (Figure 8).”

• AssumptionThe authors state this as an assumption: Downstream processing of RGC responses is slow, integrating over tens of milliseconds, so perception depends primarily on the total number of evoked spikes within this time interval.

The choice of reconstruction error metric may not be important for an implant that can stimulate at high rates.

“This suggests that the choice of reconstruction error metric may not be important for an implant that can stimulate at high rates.”

The approximate nature of the greedy algorithm is not a substantial source of error in the present conditions.

“Across multiple target images, the gap was low (<10%, ‘optimal algorithm’ in Figure 3E, H), suggesting that the approximate nature of the greedy algorithm is not a substantial source of error in the present conditions.”

Electrical stimulation which approximates naturalistic RGC responses is important in comparison with static pixel-wise approaches used in existing implants.

“This observation highlights the importance of electrical stimulation which approximates naturalistic RGC responses, in comparison with the static pixel-wise approaches used in existing implants.”

The performance of greedy temporal dithering translates to natural viewing conditions.

“Hence, the performance of greedy temporal dithering translates to natural viewing conditions and reveals that more accurate image reconstruction is possible with fixational eye movements (Wu et al., 2024).”

• cites Wu et al., 2024Not checked yet

  Not checked in the free run.

The optimization problem is NP complete due to the integer constraints on w.

“The optimization problem is NP complete due to the integer constraints on w.”

Both studies only considered simple visual stimuli which can be described by lines (such as English letters and numbers) or a few dots.

“However, both studies only considered simple visual stimuli which can be described by lines (such as English letters and numbers) or a few dots.”

• cites Chen et al., 2020 sourceNot checked yet

  Not checked in the free run.

• cites Beauchamp et al., 2020 sourceNot checked yet

  Not checked in the free run.

The less accurate reconstruction by static pixel-wise mapping is likely due to coactivation of overlapping ON and OFF cell types.

“Even using this generous upper bound, static pixel-wise mapping resulted in significantly less accurate reconstruction of the target image (Figure 3D, H), likely due to coactivation of overlapping ON and OFF cell types.”

Greedy temporal dithering and spatial multiplexing address the challenges of precisely controlling the activity of diverse cell types in a neural population by rapidly delivering a sequence of simple electrical stimuli with independent effects within a visual integration window.

“Greedy temporal dithering and spatial multiplexing address the challenges of precisely controlling the activity of diverse cell types in a neural population by rapidly delivering a sequence of simple electrical stimuli with independent effects within a visual integration window.”

• AssumptionThe authors state this as an assumption: Downstream processing of RGC responses is slow, integrating over tens of milliseconds, so perception depends primarily on the total number of evoked spikes within this time interval.
• AssumptionThe authors state this as an assumption: Brief (150 µs) and low (<4 µA) current pulses used in this study typically evoke a single spike with low and precise latencies.
• AssumptionThe authors state this as an assumption: Electrical stimuli at the same electrode separated by at least 10 ms generate approximately independent responses.
• AssumptionThe authors state this as an assumption: Distant electrodes generate independent responses.

The greedy dithering approach with a perceptually accurate reconstruction metric could lead to higher performance in an implanted device.

“Thus, the greedy dithering approach with a perceptually accurate reconstruction metric could lead to higher performance in an implanted device, though additional developments will be needed before such an optimization can be performed in real time.”

A strategy for efficient implant operation is to identify the most frequently used ~50% of electrodes during calibration and permanently turn off the remaining electrodes during run-time usage.

“These observations suggest a strategy for efficient implant operation in a retina-specific manner: identify the most frequently used ~50% of electrodes during calibration and permanently turn off the remaining electrodes during run-time usage.”

RGC receptive fields form a uniform mosaic sampling the visual field with little overlap.

“This is approximately true: RGC receptive fields form a uniform mosaic sampling the visual field with little overlap (Devries and Baylor, 1997; Chichilnisky and Kalmar, 2002).”

• cites Devries and Baylor, 1997Not checked yet

  Not checked in the free run.

• cites Chichilnisky and Kalmar, 2002Not checked yet

  Not checked in the free run.

One study deployed a dynamic stimulation approach that demonstrated impressive performance in human participants.

“Electrical stimulation of the visual cortex has also been tested for vision restoration (Chen et al., 2020), and one study (Beauchamp et al., 2020) deployed a dynamic stimulation approach that demonstrated impressive performance in human participants.”

• cites Beauchamp et al., 2020 sourceNot checked yet

  Not checked in the free run.

Applying the algorithmic framework to the ex vivo lab prototype leads to insights relevant to the development of an in vivo implant.

“Thus, applying the algorithmic framework to the ex vivo lab prototype leads to insights relevant to the development of an in vivo implant.”

Each evoked spike influences the reconstruction at multiple subsequent time steps.

“Hence, each evoked spike influences the reconstruction at multiple subsequent time steps.”