Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index; 27 of the 44 sources its claims cite were checked for support.

Connectome-constrained networks predict neural activity across the fly visual system

Open access

Abstract

With experimental measurements of only the connectivity of a biological neural network, we can predict the neural activity underlying a specified neural computation. We constructed a model neural network with the experimentally determined connectivity for 64 cell types in the motion pathways of the fruit fly optic lobe but with unknown parameters for the single-neuron and single-synapse properties, and task-optimized it on visual motion detection.

Overview

CoChat Summary

## TL;DR

The paper shows that a deep mechanistic network constrained only by the fly optic-lobe connectome and optimized for optic-flow computation can predict neural responses across the visual system, matching measurements from 26 prior studies. It also argues that sparse connectivity and task optimization are key to making single-neuron predictions from connectome data.

## Key findings

  • The model includes 45,669 neurons, 1,513,231 connections, 64 cell types, and only 734 free parameters because cell-type parameters and synapse-count-based constraints are shared across the tiled hexagonal lattice.
  • In validation against prior physiology, the ensemble predicted the correct ON/OFF contrast selectivity for all 32 cell types with experimentally established contrast preferences; the best single model got 30 of 32 correct.
  • For motion tuning, the ensemble correctly predicted that T4 neurons are ON-motion selective and T5 neurons are OFF-motion selective, and also predicted the lack of motion tuning in the listed input neurons to T4/T5.
  • In synthetic-connectome experiments, sparse networks improved prediction from connectivity alone: median Pearson correlation was 0.85 for 10% connectivity versus 0.38 for 80% connectivity; with connection-strength estimates, correlations were >0.9 across connectivities.

## Methods

  • Built a connectome-constrained, task-optimized deep mechanistic network (DMN) for the fly visual system using local reconstructions from FlyEM/FIB datasets, tiled into a spatially invariant hexagonal lattice.
  • Trained the model end-to-end on optic-flow prediction from 23 Sintel video sequences (with augmentations), using a recurrent DMN plus a feedforward decoder and backpropagation through time.
  • Evaluated model predictions by simulating responses to flash stimuli, moving edges, and naturalistic video, then comparing predicted tuning to reported experimental data from 26 studies.

## Limitations

  • The paper explicitly notes that the model is reductionist and cannot account for electrical synapses, nonlinear chemical synapses, neuromodulation, or glia.
  • The connectome was approximated as perfectly periodic / translation invariant across the visual field; this is a modeling simplification stated in the paper.
  • The text makes evident that the model is limited to the motion pathways and the optic-flow task only, so its scope does not cover all computations of the fly visual system.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41586-024-07939-3

Cites

72 of 72 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Connectome-constrained and task-optimized models of the fly visual system. a, DMNs aim to satisfy three constraints: the architecture is based on connectome measurements (b–e); cellular and synaptic dynamics are given by simple mechanistic models (f); and free parameters are task-optimized by training the model to perform optic flow estimation (g). Graphics of fruit fly and microscope were created with BioRender.com. b, Schematic of optic lobe of D. melanogaster with several processing stages (neuropils) and cell types, including photoreceptor (R1–R8), lamina monopolar (L), lamina wide-field (Lawf), medulla intrinsic (Mi), transmedullary (Tm) and T-shaped (T) neurons (adapted from ref. 25, Springer Nature). Scale bar, 10 μm. c, Identified connectivity between 64 modelled cell types, represented by total number of synapses from all neurons of a given presynaptic cell type to a postsynaptic cell of a given type. Amacrine (Am), centrifugal (C2–C3) and complex tangential (CT) neurons are also included. Blue (red) colour indicates putative hyperpolarizing (depolarizing) inputs; size of squares indicates number of input synapses. d, Retinotopic hexagonal lattice columnar organization of visual system model. Each lattice represents a cell type; each hexagon represents an individual cell. Positions of photoreceptor ommatidia are aligned with downstream columns. The model comprises synapses from all neuropils (Supplementary Fig. 1). e, Example of convolutional filter, representing Mi9 inputs onto T4d cells. Values represent the average number of synapses projecting from presynaptic Mi9 cells in columns with indicated offset onto the postsynaptic dendrite of T4d cells. f, Single-neuron and synaptic dynamics are given by simple mechanistic models. Free parameters (magenta) are optimized by training the recurrent network model to perform optic flow estimation. g, Illustration of DMN performing optic flow estimation. Each hexagonal lattice shows a snapshot of simulated voltage levels of all cells of each type in response to input to the photoreceptors (R1–R8). Edges illustrate connectivity between cell types. A decoder receives the simulated neural activity of all output neurons to compute optic flow. Parameters of DMN and decoder are optimized using deep learning.
Fig. 1 Connectome-constrained and task-optimized models of the fly visual system. a, DMNs aim to satisfy three constraints: the architecture is based on connectome measurements (b–e); cellular and synaptic dynamics are given by simple mechanistic models (f); and free parameters are task-optimized by training the model to perform optic flow estimation (g). Graphics of fruit fly and microscope were created with BioRender.com. b, Schematic of optic lobe of D. melanogaster with several processing stages (neuropils) and cell types, including photoreceptor (R1–R8), lamina monopolar (L), lamina wide-field (Lawf), medulla intrinsic (Mi), transmedullary (Tm) and T-shaped (T) neurons (adapted from ref. 25, Springer Nature). Scale bar, 10 μm. c, Identified connectivity between 64 modelled cell types, represented by total number of synapses from all neurons of a given presynaptic cell type to a postsynaptic cell of a given type. Amacrine (Am), centrifugal (C2–C3) and complex tangential (CT) neurons are also included. Blue (red) colour indicates putative hyperpolarizing (depolarizing) inputs; size of squares indicates number of input synapses. d, Retinotopic hexagonal lattice columnar organization of visual system model. Each lattice represents a cell type; each hexagon represents an individual cell. Positions of photoreceptor ommatidia are aligned with downstream columns. The model comprises synapses from all neuropils (Supplementary Fig. 1). e, Example of convolutional filter, representing Mi9 inputs onto T4d cells. Values represent the average number of synapses projecting from presynaptic Mi9 cells in columns with indicated offset onto the postsynaptic dendrite of T4d cells. f, Single-neuron and synaptic dynamics are given by simple mechanistic models. Free parameters (magenta) are optimized by training the recurrent network model to perform optic flow estimation. g, Illustration of DMN performing optic flow estimation. Each hexagonal lattice shows a snapshot of simulated voltage levels of all cells of each type in response to input to the photoreceptors (R1–R8). Edges illustrate connectivity between cell types. A decoder receives the simulated neural activity of all output neurons to compute optic flow. Parameters of DMN and decoder are optimized using deep learning.CC BY · PubMed Central
Fig. 2 Ensembles of DMNs predict tuning properties. a, We task-optimized 50 connectome-constrained DMNs, yielding different solutions for the biophysical parameters (magenta), and compared the tuning properties of their cell types to experimental measurements. Inset: distribution of task errors. Blue: ten best models; also shown in b–d. b, ON- and OFF-contrast selectivity indices (FRI) for each cell type from ten models with best task performance (ten worst models in Extended Data Fig. 6). Yellow: cell types known to be ON-selective. Violet: known OFF-selective types. Black: selectivity not yet established experimentally. Bold: inputs to optic flow decoder. c, Direction selectivity index (DSI) from neural responses to moving edges; same ten models as above. d, Correlations between measurements and predictions of neural activity from seven types of DMN with different connectome and parameter constraints: FRI (pink); motion-tuning curves for T4 (red) and T5 (turquoise); dashes indicate median correlation across models. The first DMN type on the left corresponds to the main DMNs analysed in b,c and all subsequent figures; the remaining six DMNs incorporate fewer constraints. Ten best task-performing models from each DMN type.
Fig. 2 Ensembles of DMNs predict tuning properties. a, We task-optimized 50 connectome-constrained DMNs, yielding different solutions for the biophysical parameters (magenta), and compared the tuning properties of their cell types to experimental measurements. Inset: distribution of task errors. Blue: ten best models; also shown in b–d. b, ON- and OFF-contrast selectivity indices (FRI) for each cell type from ten models with best task performance (ten worst models in Extended Data Fig. 6). Yellow: cell types known to be ON-selective. Violet: known OFF-selective types. Black: selectivity not yet established experimentally. Bold: inputs to optic flow decoder. c, Direction selectivity index (DSI) from neural responses to moving edges; same ten models as above. d, Correlations between measurements and predictions of neural activity from seven types of DMN with different connectome and parameter constraints: FRI (pink); motion-tuning curves for T4 (red) and T5 (turquoise); dashes indicate median correlation across models. The first DMN type on the left corresponds to the main DMNs analysed in b,c and all subsequent figures; the remaining six DMNs incorporate fewer constraints. Ten best task-performing models from each DMN type.CC BY · PubMed Central
Fig. 3 Cluster analysis of DMN ensembles enables hypothesis generation and suggests experimental tests. We clustered 50 DMNs after carrying out nonlinear dimensionality reduction of their responses to naturalistic scenes for each cell type, and aimed to identify whether clusters correspond to qualitatively different tuning mechanisms. a, Responses of T4c cells exhibit three clusters, two with ON-motion direction selectivity (circular and triangular marker) and one (square marker) without. b, T4c tuning in the three clusters. Circular marker: upwards tuning (cluster with lowest average task error 5.297; black: known tuning of T4c). Triangular marker: downwards (5.316 error). Square marker: no motion tuning (5.357 error). c, Schematic of corresponding ON-motion detection pathway. d, Connectivity of major input elements to T4c. Blue and red colour: putative hyper- and depolarizing inputs. Saturation: average number of input synapses for each offset location. e, Tuning properties within each cluster reveal dependencies between T4 tuning and that of Mi4 and Mi9 cells in the ensemble: switching Mi4 (known ON-contrast selective) and Mi9 (known OFF-contrast selective) contrast preferences results in directionally opposite motion-tuning solutions in T4. DMNs in first cluster (T4c in DMN upwards tuned, circle) exhibit ON-selectivity for Mi1, Tm3, Mi4 and CT1(M10), and OFF-selectivity for Mi9. For ON-motion stimuli, in these DMNs T4c receives central depolarizing input from Mi1 and Tm3 and dorsal hyperpolarizing input from Mi4 and CT1(M10).
Fig. 3 Cluster analysis of DMN ensembles enables hypothesis generation and suggests experimental tests. We clustered 50 DMNs after carrying out nonlinear dimensionality reduction of their responses to naturalistic scenes for each cell type, and aimed to identify whether clusters correspond to qualitatively different tuning mechanisms. a, Responses of T4c cells exhibit three clusters, two with ON-motion direction selectivity (circular and triangular marker) and one (square marker) without. b, T4c tuning in the three clusters. Circular marker: upwards tuning (cluster with lowest average task error 5.297; black: known tuning of T4c). Triangular marker: downwards (5.316 error). Square marker: no motion tuning (5.357 error). c, Schematic of corresponding ON-motion detection pathway. d, Connectivity of major input elements to T4c. Blue and red colour: putative hyper- and depolarizing inputs. Saturation: average number of input synapses for each offset location. e, Tuning properties within each cluster reveal dependencies between T4 tuning and that of Mi4 and Mi9 cells in the ensemble: switching Mi4 (known ON-contrast selective) and Mi9 (known OFF-contrast selective) contrast preferences results in directionally opposite motion-tuning solutions in T4. DMNs in first cluster (T4c in DMN upwards tuned, circle) exhibit ON-selectivity for Mi1, Tm3, Mi4 and CT1(M10), and OFF-selectivity for Mi9. For ON-motion stimuli, in these DMNs T4c receives central depolarizing input from Mi1 and Tm3 and dorsal hyperpolarizing input from Mi4 and CT1(M10).CC BY · PubMed Central
Fig. 4 Task-optimal DMNs largely recapitulate known mechanisms of motion computation. a, Responses to moving edges for T4 and T5 subtypes from task-optimal model clusters, and comparison with experimental measurements37,39 (null-contrasts in Supplementary Fig. 4). a.u., arbitrary units. b, Voltage of T4c neuron (top) and contributions from major input cells (bottom) during movement of an ON-edge across the visual field in preferred (solid) and null (dashed) direction. c, Major cell types and connectivity in the ON-motion (T4) and OFF-motion (T5) detection pathways (simplified). d, Spatial receptive fields of major motion detector input neurons revealed by single-ommatidium flashes and comparison with experimental measurements28,43. e, Single-ommatidium flash responses agree with experimental measurements41,43, with the exception of Tm4 (red cross). f, Stimulus sequence predicted to elicit the strongest responses in T4c and T5c cells. A central OFF-disc followed by an ON-edge moving upwards elicits the strongest response in a T4c cell (ON-disc followed by OFF-edge for T5c).
Fig. 4 Task-optimal DMNs largely recapitulate known mechanisms of motion computation. a, Responses to moving edges for T4 and T5 subtypes from task-optimal model clusters, and comparison with experimental measurements37,39 (null-contrasts in Supplementary Fig. 4). a.u., arbitrary units. b, Voltage of T4c neuron (top) and contributions from major input cells (bottom) during movement of an ON-edge across the visual field in preferred (solid) and null (dashed) direction. c, Major cell types and connectivity in the ON-motion (T4) and OFF-motion (T5) detection pathways (simplified). d, Spatial receptive fields of major motion detector input neurons revealed by single-ommatidium flashes and comparison with experimental measurements28,43. e, Single-ommatidium flash responses agree with experimental measurements41,43, with the exception of Tm4 (red cross). f, Stimulus sequence predicted to elicit the strongest responses in T4c and T5c cells. A central OFF-disc followed by an ON-edge moving upwards elicits the strongest response in a T4c cell (ON-disc followed by OFF-edge for T5c).CC BY · PubMed Central
Fig. 5 Connectome measurements constrain neural networks in circuits with sparse connectivity. a, We constructed synthetic ‘ground-truth connectome’ networks with varying degrees of sparse connectivity for classifying handwritten digits. For each ground-truth connectome network, we simulated connectome measurements and constructed a connectome-constrained and task-optimized the unknown parameters (magenta) in the ‘simulated network’ (Methods). We measured the correlation of the neural response vector, across all stimuli, between a ground-truth (dark green) and a simulated (light green) network. b, Median neural response correlation coefficients from 100 randomly sampled neuron pairs from each layer and across 25 network pairs. Two conditions were considered in which connectome measurements revealed either only binary connectivity (blue) or also connection strength (orange). The fly visual system model presented here probably falls in the region between the two curves, as measured synapse counts inform relative connection strengths between pairs of neurons for the same pair of cell types, but not absolute connection strength.
Fig. 5 Connectome measurements constrain neural networks in circuits with sparse connectivity. a, We constructed synthetic ‘ground-truth connectome’ networks with varying degrees of sparse connectivity for classifying handwritten digits. For each ground-truth connectome network, we simulated connectome measurements and constructed a connectome-constrained and task-optimized the unknown parameters (magenta) in the ‘simulated network’ (Methods). We measured the correlation of the neural response vector, across all stimuli, between a ground-truth (dark green) and a simulated (light green) network. b, Median neural response correlation coefficients from 100 randomly sampled neuron pairs from each layer and across 25 network pairs. Two conditions were considered in which connectome measurements revealed either only binary connectivity (blue) or also connection strength (orange). The fly visual system model presented here probably falls in the region between the two curves, as measured synapse counts inform relative connection strengths between pairs of neurons for the same pair of cell types, but not absolute connection strength.CC BY · PubMed Central
Extended Data Fig. 1 Cell connectivity. The matrix shows how cells of the 64 cell types within the inner 91 columns (of 721) of the recurrent convolutional DMN connect (Supplementary Data file 1), either by excitatory connections (red) or inhibitory connections (blue).
Extended Data Fig. 1 Cell connectivity. The matrix shows how cells of the 64 cell types within the inner 91 columns (of 721) of the recurrent convolutional DMN connect (Supplementary Data file 1), either by excitatory connections (red) or inhibitory connections (blue).CC BY · PubMed Central
Extended Data Fig. 2 DMN benchmark of connectomic constraints. (a-d) How would incomplete knowledge of connectome affect the tuning predictions? We artificially varied DMNs with random parameters, connectome-constrained or task-optimized parameters. Five experiments: Four ‘Synapse-optimized models’, one ‘Fully optimized’. Details in Methods. How would incomplete knowledge of cell types affect the tuning predictions? We artificially assumed some cell types to be indistinguishable, with shared physiological parameters (resting potentials, time constants, and unitary synapse strengths). Two experiments: (1) ‘Full DMN Merge T4, T5’ assumes that T4 and T5 subtypes were indistinguishable, reducing the number of cell types to 58. (2) ‘Full DMN Merge E/I’ assumes that we had three cell types, excitatory (37 cell types), inhibitory (22 cell types) or both (4 cell types), based on our knowledge of synapse signs. Tuning predictions are shown in comparison to the Full DMN and the DMN with random parameters. (a) Task error. (b) Predicted correlations to flash response indices, T4, and T5 motion-tuning curves (10 best models). (c) Predicted correlations to known direction selectivity indices. (d) Distances between known preferred directions and predicted preferred directions for T4 and T5 neurons. (e) Better task-performing models predict motion-tuning neurons better. We correlate predicted tuning metrics from each model to the known tuning properties to understand when better performing models give us better tuning predictions. (orange) When correlating the direction selectivity index of each model to the binary known properties for T4 and T5 and their input cell types, we find that this correlation is higher for better performing models (Pearson correlation, r=−0.60, p =2.6×10⁻⁶, t=r√(df/(1−r²)), 95% CI = [−1, −0.42], df =48). (magenta) While the models predicted the known contrast preferences generally well, the correlation of flash response index to the binary known contrast preferences of 31 cell types did not significantly increase with better performing models.
Extended Data Fig. 2 DMN benchmark of connectomic constraints. (a-d) How would incomplete knowledge of connectome affect the tuning predictions? We artificially varied DMNs with random parameters, connectome-constrained or task-optimized parameters. Five experiments: Four ‘Synapse-optimized models’, one ‘Fully optimized’. Details in Methods. How would incomplete knowledge of cell types affect the tuning predictions? We artificially assumed some cell types to be indistinguishable, with shared physiological parameters (resting potentials, time constants, and unitary synapse strengths). Two experiments: (1) ‘Full DMN Merge T4, T5’ assumes that T4 and T5 subtypes were indistinguishable, reducing the number of cell types to 58. (2) ‘Full DMN Merge E/I’ assumes that we had three cell types, excitatory (37 cell types), inhibitory (22 cell types) or both (4 cell types), based on our knowledge of synapse signs. Tuning predictions are shown in comparison to the Full DMN and the DMN with random parameters. (a) Task error. (b) Predicted correlations to flash response indices, T4, and T5 motion-tuning curves (10 best models). (c) Predicted correlations to known direction selectivity indices. (d) Distances between known preferred directions and predicted preferred directions for T4 and T5 neurons. (e) Better task-performing models predict motion-tuning neurons better. We correlate predicted tuning metrics from each model to the known tuning properties to understand when better performing models give us better tuning predictions. (orange) When correlating the direction selectivity index of each model to the binary known properties for T4 and T5 and their input cell types, we find that this correlation is higher for better performing models (Pearson correlation, r=−0.60, p =2.6×10⁻⁶, t=r√(df/(1−r²)), 95% CI = [−1, −0.42], df =48). (magenta) While the models predicted the known contrast preferences generally well, the correlation of flash response index to the binary known contrast preferences of 31 cell types did not significantly increase with better performing models.CC BY · PubMed Central
Extended Data Fig. 3 DMNs suggest that TmY3 neurons compute motion independently of T4 and T5 neurons. (a) We clustered 50 DMNs after performing nonlinear dimensionality reduction of their responses to naturalistic scenes for each cell type, and aimed to identify whether clusters correspond to qualitatively different tuning mechanisms. (b) Dimensionality reduction on TmY3 responses to naturalistic stimuli reveals 4 clusters of DMNs with average task errors 5.298 (circle), 5.317 (triangle), 5.328 (square) and 5.331 (star). Across clusters, TmY3 shows different strengths of direction selectivity (evaluated with moving edge stimuli). ON-edge direction selectivity is strong in the first and the third cluster. (c) Normalized peak responses of TmY3 to moving edge stimuli in the DMNs of each cluster. (d) Major cell types and synaptic connections in the pathway that projects onto TmY3 (simplified). (e) The input elements of TmY3 with the highest amount of synapses are L4, L5, Tm2, Tm3, Mi1, Mi9, and Mi4. The asymmetries of their projective fields could allow TmY3 to become motion selective. (f) Dependencies between TmY3 tuning and the contrast preference of its input cells. For clusters in which TmY3 is motion selective, cluster 1 (TmY3 tuning to downwards/front-to-back motion, circular marker) indicates ON-selectivity for Tm3, Mi1, and Mi4 cells, and OFF-selectivity for L4, Tm2, and Mi9 cells, in agreement with known selectivities. In contrast, cluster 3 (TmY3 tuning to upwards/back-to-front motion, square marker) indicates ON-selectivity for Mi9 in contradiction to the known selectivities and hence ruling out the third TmY3 tuning solution.
Extended Data Fig. 3 DMNs suggest that TmY3 neurons compute motion independently of T4 and T5 neurons. (a) We clustered 50 DMNs after performing nonlinear dimensionality reduction of their responses to naturalistic scenes for each cell type, and aimed to identify whether clusters correspond to qualitatively different tuning mechanisms. (b) Dimensionality reduction on TmY3 responses to naturalistic stimuli reveals 4 clusters of DMNs with average task errors 5.298 (circle), 5.317 (triangle), 5.328 (square) and 5.331 (star). Across clusters, TmY3 shows different strengths of direction selectivity (evaluated with moving edge stimuli). ON-edge direction selectivity is strong in the first and the third cluster. (c) Normalized peak responses of TmY3 to moving edge stimuli in the DMNs of each cluster. (d) Major cell types and synaptic connections in the pathway that projects onto TmY3 (simplified). (e) The input elements of TmY3 with the highest amount of synapses are L4, L5, Tm2, Tm3, Mi1, Mi9, and Mi4. The asymmetries of their projective fields could allow TmY3 to become motion selective. (f) Dependencies between TmY3 tuning and the contrast preference of its input cells. For clusters in which TmY3 is motion selective, cluster 1 (TmY3 tuning to downwards/front-to-back motion, circular marker) indicates ON-selectivity for Tm3, Mi1, and Mi4 cells, and OFF-selectivity for L4, Tm2, and Mi9 cells, in agreement with known selectivities. In contrast, cluster 3 (TmY3 tuning to upwards/back-to-front motion, square marker) indicates ON-selectivity for Mi9 in contradiction to the known selectivities and hence ruling out the third TmY3 tuning solution.CC BY · PubMed Central
Extended Data Fig. 4 TmY3 motion detection mechanisms hypothesized by the model. (a) Responses to PD and ND ON-edge motion and contributions from input elements. (b) PD enhancement and ND suppression in TmY3 in the model.
Extended Data Fig. 4 TmY3 motion detection mechanisms hypothesized by the model. (a) Responses to PD and ND ON-edge motion and contributions from input elements. (b) PD enhancement and ND suppression in TmY3 in the model.CC BY · PubMed Central
Extended Data Fig. 5 Statistics of inhibitory and excitatory synapse inputs. (a) Number of input cell types per cell type. (b) Center of mass offsets of synaptic input. (c) Average excitatory and (d) inhibitory center of mass offset of synaptic inputs against median predicted direction selectivity index for all cell types. Datapoints for cell types that were predicted as significantly motion selective are labeled.
Extended Data Fig. 5 Statistics of inhibitory and excitatory synapse inputs. (a) Number of input cell types per cell type. (b) Center of mass offsets of synaptic input. (c) Average excitatory and (d) inhibitory center of mass offset of synaptic inputs against median predicted direction selectivity index for all cell types. Datapoints for cell types that were predicted as significantly motion selective are labeled.CC BY · PubMed Central
Extended Data Fig. 6 Predicted tuning with respect to task-performance. (a) Flash response index computed as the max-abs-scaled peak response to an off flash subtracted from the max-abs-scaled peak response to an on flash – both of approximately 35° radius and presented for 1 s after 2 s of grey input. Values above 0 indicate on-polarity, values below zero indicate off-polarity. Known on-polar and off-polar cell types are colored in yellow and magenta. (b) Single-cell type direction selectivity of best 20% task-performing models versus worst 20% task-performing models of an ensemble of 50 models as a result of peak voltage responses in central columns to on-edges and off-edges moving towards all possible directions on grey background.
Extended Data Fig. 6 Predicted tuning with respect to task-performance. (a) Flash response index computed as the max-abs-scaled peak response to an off flash subtracted from the max-abs-scaled peak response to an on flash – both of approximately 35° radius and presented for 1 s after 2 s of grey input. Values above 0 indicate on-polarity, values below zero indicate off-polarity. Known on-polar and off-polar cell types are colored in yellow and magenta. (b) Single-cell type direction selectivity of best 20% task-performing models versus worst 20% task-performing models of an ensemble of 50 models as a result of peak voltage responses in central columns to on-edges and off-edges moving towards all possible directions on grey background.CC BY · PubMed Central
Extended Data Fig. 7 T4 motion detection mechanisms hypothesized by the model. (a) Across all T4 cell types (here T4c, Supplementary Fig. 5 for other T4 types), our model predicts that T4 depolarization in response to PD ON-motion (black, solid) is driven by excitatory Mi1 current inputs (darkest red, solid) from roughly a two-column radius of Mi1 cells. Excitatory inputs from neighboring T4 cells of the same type increase the T4 PD-motion response (third darkest red, solid). Tm3 and Mi1 cells excite T4 agnostic to PD vs. ND motion. For ND-motion, Mi4 cells cancel excitatory currents from Mi1 with matching inhibition from the trailing side of the receptive field (darkest blue, dashed). The inhibition from Mi4 cells is delayed for PD-motion (darkest blue, solid), allowing strong depolarization of T4. CT1 shadows the Mi4 mechanism with similar but weaker inhibition from the same location of the receptive field (second darkest, blue). Our model suggests mechanisms involving Mi9 cells and TmY15 cells: both contribute to T4 motion detection by different inhibitory mechanisms for PD-motion with respect to ND-motion. (b) ‘Measured’: Predicted T4c responses to bars moving in PD (left column) and in ND (right column) at varied speeds (saturated red and blue). ‘Linear sum’: linear sum of responses to individually flashed frames that constitute the moving bar video (faint red and blue). Faint grey traces in background of first panel show individual flash responses before linear summation. Flash duration in each location matched to length of stay at the location in the moving bar video. Bars were approximately 9° wide and 20.25° high and moved across 45° with respect to receptive field in the center. This figure should be compared to Gruntman et al.27, Fig. 4f.
Extended Data Fig. 7 T4 motion detection mechanisms hypothesized by the model. (a) Across all T4 cell types (here T4c, Supplementary Fig. 5 for other T4 types), our model predicts that T4 depolarization in response to PD ON-motion (black, solid) is driven by excitatory Mi1 current inputs (darkest red, solid) from roughly a two-column radius of Mi1 cells. Excitatory inputs from neighboring T4 cells of the same type increase the T4 PD-motion response (third darkest red, solid). Tm3 and Mi1 cells excite T4 agnostic to PD vs. ND motion. For ND-motion, Mi4 cells cancel excitatory currents from Mi1 with matching inhibition from the trailing side of the receptive field (darkest blue, dashed). The inhibition from Mi4 cells is delayed for PD-motion (darkest blue, solid), allowing strong depolarization of T4. CT1 shadows the Mi4 mechanism with similar but weaker inhibition from the same location of the receptive field (second darkest, blue). Our model suggests mechanisms involving Mi9 cells and TmY15 cells: both contribute to T4 motion detection by different inhibitory mechanisms for PD-motion with respect to ND-motion. (b) ‘Measured’: Predicted T4c responses to bars moving in PD (left column) and in ND (right column) at varied speeds (saturated red and blue). ‘Linear sum’: linear sum of responses to individually flashed frames that constitute the moving bar video (faint red and blue). Faint grey traces in background of first panel show individual flash responses before linear summation. Flash duration in each location matched to length of stay at the location in the moving bar video. Bars were approximately 9° wide and 20.25° high and moved across 45° with respect to receptive field in the center. This figure should be compared to Gruntman et al.27, Fig. 4f.CC BY · PubMed Central
Extended Data Fig. 8 T5 motion detection mechanisms hypothesized by the model. (a) Across all T5 cell types (here T5c, Supplementary Fig. 6 for other T5 types), our model predicts that T5 depolarization in response to PD OFF-motion (black, solid) is driven by excitatory Tm1 and Tm9 input currents (darkest and second darkest red). Tm1 currents come from a centered, two-column radius of Tm1 cells. Tm9 inputs come from cells offset by one column towards the leading side of the receptive field. We observe delayed excitation from Tm9 cells for ND-motion. The PD-motion response is increased through excitatory inputs from the neighboring T5 cells of the same type (as for T4 cells), not providing excitation for ND-motion. CT1(Lo1) cells cancel excitatory currents with strong inhibitory currents from the trailing side of the receptive field leading to the weak ND response. For PD-motion, inhibition from CT1(Lo1) cells is delayed allowing strong T5 depolarization. (b) ‘Measured’: Predicted T5c responses to bars moving in PD (left column) and in ND (right column) at varied speeds (saturated red and blue). ‘Linear sum’: linear sum of responses to individually flashed frames that constitute the moving bar video (faint red and blue). Faint grey traces in background of first panel show individual flash responses before linear summation. Flash duration in each location matched to length of stay at the location in the moving bar video. Bars were approximately 9° wide and 20.25° high and moved across 45° with respect to receptive field in the center.
Extended Data Fig. 8 T5 motion detection mechanisms hypothesized by the model. (a) Across all T5 cell types (here T5c, Supplementary Fig. 6 for other T5 types), our model predicts that T5 depolarization in response to PD OFF-motion (black, solid) is driven by excitatory Tm1 and Tm9 input currents (darkest and second darkest red). Tm1 currents come from a centered, two-column radius of Tm1 cells. Tm9 inputs come from cells offset by one column towards the leading side of the receptive field. We observe delayed excitation from Tm9 cells for ND-motion. The PD-motion response is increased through excitatory inputs from the neighboring T5 cells of the same type (as for T4 cells), not providing excitation for ND-motion. CT1(Lo1) cells cancel excitatory currents with strong inhibitory currents from the trailing side of the receptive field leading to the weak ND response. For PD-motion, inhibition from CT1(Lo1) cells is delayed allowing strong T5 depolarization. (b) ‘Measured’: Predicted T5c responses to bars moving in PD (left column) and in ND (right column) at varied speeds (saturated red and blue). ‘Linear sum’: linear sum of responses to individually flashed frames that constitute the moving bar video (faint red and blue). Faint grey traces in background of first panel show individual flash responses before linear summation. Flash duration in each location matched to length of stay at the location in the moving bar video. Bars were approximately 9° wide and 20.25° high and moved across 45° with respect to receptive field in the center.CC BY · PubMed Central
Extended Data Fig. 9 Spatio-temporal receptive fields mapped with ON- and OFF-impulses and maximally excitatory stimuli. (a) Spatiotemporal receptive fields for motion detector neurons agree with experimental measurements (Gruntman et al.27). (b) Spatio-temporal receptive field mapping with single ommatidium OFF-impulses. (c) Maximally excitatory stimuli and baseline-subtracted responses. Including full-field naturalistic, regularized naturalistic, artificial, and moving edge stimuli and responses. Moving edge angle and speed maximize the central cell peak response. Artificial stimuli are optimized from initial noise to maximize the central cell activity using gradient ascent plus full-field regularization towards grey. The last row shows the baseline-subtracted central cell responses. Peak central cell responses at time point zero.
Extended Data Fig. 9 Spatio-temporal receptive fields mapped with ON- and OFF-impulses and maximally excitatory stimuli. (a) Spatiotemporal receptive fields for motion detector neurons agree with experimental measurements (Gruntman et al.27). (b) Spatio-temporal receptive field mapping with single ommatidium OFF-impulses. (c) Maximally excitatory stimuli and baseline-subtracted responses. Including full-field naturalistic, regularized naturalistic, artificial, and moving edge stimuli and responses. Moving edge angle and speed maximize the central cell peak response. Artificial stimuli are optimized from initial noise to maximize the central cell activity using gradient ascent plus full-field regularization towards grey. The last row shows the baseline-subtracted central cell responses. Peak central cell responses at time point zero.CC BY · PubMed Central

Verification

Claims the paper makes

16 Valid · 1 Not valid · 5 Not checked · 67 Own results and interpretations

46 more claims

Create a free account to see all 89 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 26
  • Probably not a substantial claim. 18

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 69 references

Research works printed but not linked: 7

  1. Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT Press, 2016).
  2. Raghu, M., Unterthiner, T., Kornblith, S., Zhang, C. & Dosovitskiy, A. Do vision transformers see like convolutional neural networks? Adv. Neural Inf. Process. Syst. 34, 12116–12128 (2021).
  3. Reichardt, W. in Sensory Communication (ed. Rosenblith, W. A.) Ch. 17 (MIT Press, 1961).
  4. Mi, L. et al. Connectome-constrained latent variable model of whole-brain neural activity. In Proc. International Conference on Learning Representations https://openreview.net/forum?id=CJzi3dRlJE- (2022).
  5. Chen, R. T. Q., Rubanova, Y., Bettencourt, J. & Duvenaud, D. K. Neural ordinary differential equations. In Proc. Advances in Neural Information Processing Systems Vol. 31 (eds Bengio, S. et al.) 6571–6583 (Curran Associates, 2018).
  6. Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. In Proc. Advances in Neural Information Processing Systems (eds Pereira, F. et al.) 1097–1105 (Curran Associates, 2012).
  7. Frankle, J. & Carbin, M. The lottery ticket hypothesis: finding sparse, trainable neural networks. In Proc. International Conference on Learning Representations https://openreview.net/forum?id=rJl-b3RcF7 (2018).

Cited by

See all 128

Metadata

Authors
Lappalainen, J. K., Tschopp, F. D., Prakhya, S., McGill, M., Nern, A., Shinomiya, K., Takemura, S.-Y., Gruntman, E., Macke, J. H., & Turaga, S. C.
Affiliations
  • Howard Hughes Medical Institute (US)
  • California Institute of Technology (US)
  • Max Planck Institute for Intelligent Systems (DE)
Corresponding author
Turaga, Srinivas C.
Venue
Nature
Published
2024-09-11
Volume
634
Issue
8036
Pages
1132-1140
ISSN
  • 0028-0836 (ISSN-L)
  • 1476-4687
OpenAlex
W4402437891
Cited by
128
References
69
Citations by year
2026: 64 · 2025: 44 · 2024: 17
FWCI
30.28
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
Funders
  • Howard Hughes Medical Institute
  • International Max Planck Research School for Environmental, Cellular and Molecular Microbiology
  • European Commission (101089288)
  • Deutsche Forschungsgemeinschaft (206/4-1, CC BY 4.0, 390727645, SFB1233)
  • Bundesministerium für Bildung und Forschung (01IS18039A, CC BY 4.0, FKZ: 01IS18039A)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
  • Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurobiology and Insect Physiology Research
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
Keywords
connectome, neural network model, neural activity prediction, visual motion detection, sparse connectivity, neural circuit function, parameter optimization, deep learning, Drosophila
MeSH terms
Deep Learning, Animals, Drosophila melanogaster, Models, Neurological, Motion Perception, Nerve Net, Neurons, Optic Lobe, Nonmammalian, Synapses, Visual Pathways, Neural Networks, Computer, Connectome

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Connectome-constrained networks predict neural activity across the fly visual system. https://app.cochat.ai/papers/10.1038/s41586-024-07939-3
The paper
Lappalainen, J. K., Tschopp, F. D., Prakhya, S., McGill, M., Nern, A., Shinomiya, K., Takemura, S.-Y., Gruntman, E., Macke, J. H., & Turaga, S. C. (2024). Connectome-constrained networks predict neural activity across the fly visual system. Nature. https://doi.org/10.1038/s41586-024-07939-3

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.