CoChat
Sign Up
PreprintCited by 55

The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.

Automated Reconstruction of a Serial-Section EM Drosophila Brain with Flood-Filling Networks and Local Realignment

Li, Peter H., Lindsey, Larry F., Januszewski, Michał, Zheng, Zhihao, Bates, Alexander Shakeel, Taisz, István, Tyka, Mike, Nichols, Matthew, Li, Feng, Perlman, Eric, Maitin-Shepard, Jeremy, Blakely, Tim, Leavitt, Laramie, Jefferis, Gregory S.X.E., Bock, Davi, Jain, Viren·2019
DOI 10.1101/605634
Open this paper in CoChat — free account

Abstract

Reconstruction of neural circuitry at single-synapse resolution is a key target for improving understanding of the nervous system in health and disease. Serial section transmission electron microscopy (ssTEM) is among the most prolific imaging methods employed in pursuit of such reconstructions. We demonstrate how Flood-Filling Networks (FFNs) can be used to computationally segment a forty-teravoxel whole-brain Drosophila ssTEM volume. To compensate for data irregularities and imperfect global alignment, FFNs were combined with procedures that locally re-align serial sections as well as dynamically adjust and synthesize image content. The proposed approach produced a largely merger-free segmentation of the entire ssTEM Drosophila brain, which we make freely available. As compared to manual tracing using an efficient skeletonization strategy, the segmentation enabled circuit reconstruction and analysis workflows that were an order of magnitude faster.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1101/605634

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Dense segmentation of the entire fly brain via FFNs. (A) 3D rendering of a smoothed tissue mask of the FAFB dataset. Arbitrary coronal sections (dataset XY plane) reveal the FAFB-FFN1 segmentation throughout the interior. In the lower right, two recent dense segmentation benchmarks, from the songbird tectum (Michał Januszewski et al. 2018) and Drosophila optic lobe (Takemura et al. 2015) , for comparison. Scale bar 200 μm. Exterior surface of FAFB is pseudo-textured. (B-E) Increasing zoom levels of coronal sections intersecting the mushroom body peduncular tract (right hemisphere), FAFB-FFN1 segmentation overlaid on CLAHE processed raw data. Scale bars 200, 50, 10, 1 μm. d, dorsal; v, ventral; a, anterior; p, posterior; r, right hemisphere; l, left hemisphere.
Figure 1 Dense segmentation of the entire fly brain via FFNs. (A) 3D rendering of a smoothed tissue mask of the FAFB dataset. Arbitrary coronal sections (dataset XY plane) reveal the FAFB-FFN1 segmentation throughout the interior. In the lower right, two recent dense segmentation benchmarks, from the songbird tectum (Michał Januszewski et al. 2018) and Drosophila optic lobe (Takemura et al. 2015) , for comparison. Scale bar 200 μm. Exterior surface of FAFB is pseudo-textured. (B-E) Increasing zoom levels of coronal sections intersecting the mushroom body peduncular tract (right hemisphere), FAFB-FFN1 segmentation overlaid on CLAHE processed raw data. Scale bars 200, 50, 10, 1 μm. d, dorsal; v, ventral; a, anterior; p, posterior; r, right hemisphere; l, left hemisphere.CC-BY-NC-ND · From the PDF
Image 2
CC-BY-NC-ND · From the PDF
Figure 3 Local realignment (LR). (A) LR procedure; a local subvolume is requested for FFN inference (1-2); this is aligned according to section image content (3) and then cropped and segmented (4); the segmentation output is then dealigned (5) and cropped back to the original requested subvolume dimensions for incorporation into a coherent output volume (6). The expansion of the subvolume bounds in (2) is computed to account for the two subsequent rounds of cropping. (B) YZ view of FFN segmentation at for example location in FAFB without LR. (C) The same segmentation result as (B), with LR applied post-hoc . This reveals more clearly the impact of the discontinuity in section alignment, including many clear split errors as well as a merge error (stars). (D) Segmentation of the same location as (B) and (C), now with LR applied as a preprocessing step prior to segmentation. This fixes the merge error as well as most splits. (E) Impact of LR on skeleton metrics for 16x16x40 nm resolution FFN segmentation of the Sample E mushroom body cutout. LR fixes many merge and split errors, but cannot correct heavily distorted areas or other types of data irregularity such as damaged or occluded sections. In combination with movement restriction to address these other cases, LR reduced merge errors by an order of magnitude while minimizing splits and omitted areas.
Figure 3 Local realignment (LR). (A) LR procedure; a local subvolume is requested for FFN inference (1-2); this is aligned according to section image content (3) and then cropped and segmented (4); the segmentation output is then dealigned (5) and cropped back to the original requested subvolume dimensions for incorporation into a coherent output volume (6). The expansion of the subvolume bounds in (2) is computed to account for the two subsequent rounds of cropping. (B) YZ view of FFN segmentation at for example location in FAFB without LR. (C) The same segmentation result as (B), with LR applied post-hoc . This reveals more clearly the impact of the discontinuity in section alignment, including many clear split errors as well as a merge error (stars). (D) Segmentation of the same location as (B) and (C), now with LR applied as a preprocessing step prior to segmentation. This fixes the merge error as well as most splits. (E) Impact of LR on skeleton metrics for 16x16x40 nm resolution FFN segmentation of the Sample E mushroom body cutout. LR fixes many merge and split errors, but cannot correct heavily distorted areas or other types of data irregularity such as damaged or occluded sections. In combination with movement restriction to address these other cases, LR reduced merge errors by an order of magnitude while minimizing splits and omitted areas.CC-BY-NC-ND · From the PDF
Image 4
CC-BY-NC-ND · From the PDF
Figure 5 Overall FAFB-FFN1 segmentation pipeline. (A) Detailed pipeline steps and major stages, bold numerals highlight selected intermediate volumes in the pipeline used for evaluations in (B). Rounded boxes indicate FFN inference stages. Each FFN step indicates the model checkpoint used. (B) Evaluation metrics relative to ground truth skeletons at progressive stages of the pipeline, columns corresponding to pipeline stages with bold numeral labels in (A). Top, percent remaining skeleton edge errors by subtype. Bottom, expected run length.
Figure 5 Overall FAFB-FFN1 segmentation pipeline. (A) Detailed pipeline steps and major stages, bold numerals highlight selected intermediate volumes in the pipeline used for evaluations in (B). Rounded boxes indicate FFN inference stages. Each FFN step indicates the model checkpoint used. (B) Evaluation metrics relative to ground truth skeletons at progressive stages of the pipeline, columns corresponding to pipeline stages with bold numeral labels in (A). Top, percent remaining skeleton edge errors by subtype. Bottom, expected run length.CC-BY-NC-ND · From the PDF
Figure 6 Segmentation-assisted neuron tracing. (A) Tracing speed (person-minutes per 𝜇 m path length) for Kenyon cells in the mushroom body calyx. Cells are grouped according to the reconstruction methodology: 'full manual' (gray points, n=545) or 'FFN concatenation' (blue points, n=601), in which FFN segmentation-derived skeleton fragments were linked together. Inset illustrates for one cell the targeted level of reconstruction completeness for the concatenation methodology (blue) versus the complete dendritic arbor (gray). (B) Tracing speeds for various neurons in the lateral horn and gnathal ganglion, traced to lower levels of completeness primarily for cell type identification (mean path length for different completeness indicated for reference). Cells are grouped by reconstruction methodology as in (A). Linear fits indicate tracing speed-ups from using FFN concatenation of 5.4x (A) and 14.6x (B).
Figure 6 Segmentation-assisted neuron tracing. (A) Tracing speed (person-minutes per 𝜇 m path length) for Kenyon cells in the mushroom body calyx. Cells are grouped according to the reconstruction methodology: 'full manual' (gray points, n=545) or 'FFN concatenation' (blue points, n=601), in which FFN segmentation-derived skeleton fragments were linked together. Inset illustrates for one cell the targeted level of reconstruction completeness for the concatenation methodology (blue) versus the complete dendritic arbor (gray). (B) Tracing speeds for various neurons in the lateral horn and gnathal ganglion, traced to lower levels of completeness primarily for cell type identification (mean path length for different completeness indicated for reference). Cells are grouped by reconstruction methodology as in (A). Linear fits indicate tracing speed-ups from using FFN concatenation of 5.4x (A) and 14.6x (B).CC-BY-NC-ND · From the PDF
Figure 7 Trans-synaptic circuit tracing. (A) For a set of manually traced neurons in the lateral horn (LH, n=82), the number of incoming synapses recovered per upstream partner by concatenating FFN segments, versus ground truth synapse total from manual search. Upstream partners have either axonal
Figure 7 Trans-synaptic circuit tracing. (A) For a set of manually traced neurons in the lateral horn (LH, n=82), the number of incoming synapses recovered per upstream partner by concatenating FFN segments, versus ground truth synapse total from manual search. Upstream partners have either axonalCC-BY-NC-ND · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 10 references

Research works printed but not linked: 41

  1. 10.3390/molecules17010355 · DOI
  2. 10.1016/j.neuron.2015.09.003 · DOI
  3. 10.1038/nature09802 · DOI
  4. Buhmann, Julia , Renate Krause , Rodrigo Ceballos Lentini , Nils Eckstein , Matthew Cook , Srinivas Turaga , and Jan Funke . 2018. “Synaptic Partner Prediction from Point Annotations in Insect Brains.” arXiv [cs.CV]. arXiv. http://arxiv.org/abs/1806.08205. · DOI · Paper page
  5. Buhmann, Julia , Arlo Sheridan , Stephan Gerhard , Renate Krause , Tri Nguyen , Larissa Heinrich , Philipp Schlegel , et al. 2019. “Automatic Detection of Synaptic Partners in a Whole-Brain Drosophila EM Dataset.” bioRxiv. https://doi.org/10.1101/2019.12.12.874172. · DOI · Paper page
  6. 2019, Neural Arbors Are Pareto Optimal, Proceedings of the Royal Society B: Biological Sciences, 286, 20182727, 10.1098/rspb.2018.2727 · DOI
  7. 10.1016/j.neuron.2016.06.012 · DOI · Paper page
  8. 2018, Communication from Learned to Innate Olfactory Processing Centers Is Required for Memory Retrieval in Drosophila, Neuron, 100, 651, 10.1016/j.neuron.2018.08.037 · DOI · Paper page
  9. Heinrich, Larissa , Jan Funke , Constantin Pape , Juan Nunez-Iglesias , and Stephan Saalfeld . 2018. “Synaptic Cleft Segmentation in Non-Isotropic Volume Electron Microscopy of the Complete Drosophila Brain.” arXiv [cs.CV]. arXiv. http://arxiv.org/abs/1805.02718. · DOI · Paper page
  10. He, Kaiming , Xiangyu Zhang , Shaoqing Ren , and Jian Sun . 2016. “Deep Residual Learning for Image Recognition.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–78.
  11. 2016, Organization of Descending Neurons in Drosophila Melanogaster, Scientific Reports, 6, 20259, 10.1038/srep20259 · DOI
  12. 2018, Fully-Automatic Synapse Prediction and Validation on a Large Data Set, Frontiers in Neural Circuits, 12, 87, 10.3389/fncir.2018.00087 · DOI · Paper page
  13. Huoviala, Paavo , Michael-John Dolan , Fiona Love , Shahar Frechter , Ruairi J. V. Roberts , Zane Mitrevica , Philipp Schlegel , et al. 2018. “Neural Circuit Basis of Aversive Odour Processing inDrosophilafrom Sensory Input to Descending Output.” https://doi.org/10.1101/394403. · DOI · Paper page
  14. Januszewski, Michal , and Viren Jain . 2019. “Segmentation-Enhanced CycleGAN.” bioRxiv. https://doi.org/10.1101/548081. · DOI
  15. 2018, High-Precision Automated Reconstruction of Neurons with Flood-Filling Networks, Nature Methods, 15, 605, 10.1038/s41592-018-0049-4 · DOI · Paper page
  16. 2018, A Topological Representation of Branching Neuronal Morphologies, Neuroinformatics, 16, 3, 10.1007/s12021-017-9341-1 · DOI
  17. 10.1016/j.cell.2015.06.054 · DOI · Paper page
  18. Khairy, Khaled , Gennady Denisov , and Stephan Saalfeld . 2018. “Joint Deformable Registration of Large EM Image Volumes: A Matrix Solver Approach.” arXiv [cs.CV]. arXiv. http://arxiv.org/abs/1804.10019. · arXiv
  19. 10.1038/nature13240 · DOI · Paper page
  20. Kornfeld, Jörgen , Sam E. Benezra , Rajeevan T. Narayanan , Fabian Svara , Robert Egger , Marcel Oberlaender , Winfried Denk , and Michael A. Long . 2017. “EM Connectomics Reveals Axonal Target Variation in a Sequence-Generating Network.” eLife 6 (March). https://doi.org/10.7554/eLife.24364. · DOI
  21. 10.1016/j.conb.2018.04.030 · DOI
  22. Kreshuk, Anna , Jan Funke , Albert Cardona , and Fred A. Hamprecht . 2015. “Who Is Talking to Whom: Synaptic Partner Detection in Anisotropic Volumes of Insect Brain.” In Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015, 661–68. Springer International Publishing. · DOI · Paper page
  23. Lee, Kisuk , Jonathan Zung , Peter Li , Viren Jain , and H. Sebastian Seung . 2017. “Superhuman Accuracy on the SNEMI3D Connectomics Challenge.” arXiv [cs.CV]. arXiv. http://arxiv.org/abs/1706.00120. · arXiv
  24. 10.1038/nature17192 · DOI
  25. Lewis, J. P. 1995. “Fast Template Matching.” Vision Interface 95 (May). http://dx.doi.org/.
  26. Rangamani, Padmini , Miriam Bell , Andrea Cugno , Donya Ohadi , Thomas M. Bartol , Ravi Iyengar , and Terrence J. Sejnowski . 2019. “Dendritic Spine Geometry and Ultrastructure Dictate the Spatiotemporal Dynamics of Second Messengers.” Biophysical Journal. https://doi.org/10.1016/j.bpj.2018.11.1651. · DOI
  27. 10.1093/bioinformatics/btp266 · DOI · Paper page
  28. 2012, Elastic Volume Reconstruction from Series of Ultra-Thin Microscopy Sections, Nature Methods, 9, 717, 10.1038/nmeth.2072 · DOI · Paper page
  29. Sato, M. , I. Bitter , M. A. Bender , A. E. Kaufman , and M. Nakajima . 2000. “TEASAR: Tree-Structure Extraction Algorithm for Accurate and Robust Skeletons.” In Proceedings the Eighth Pacific Conference on Computer Graphics and Applications, 281–449. ieeexplore.ieee.org.
  30. 10.1038/nature24005 · DOI · Paper page
  31. Shan Xu, C. , Michal Januszewski , Zhiyuan Lu , Shin-Ya Takemura , Kenneth J. Hayworth , Gary Huang , Kazunori Shinomiya , et al. 2020. “A Connectome of the Adult Drosophila Central Brain.” bioRxiv. https://doi.org/10.1101/2020.01.21.911859. · DOI
  32. Shepherd, Gordon M. , and Sten Grillner . 2018. Handbook of Brain Microcircuits. Oxford University Press. · DOI
  33. 2014, A Genetic and Computational Approach to Structurally Classify Neuronal Types, Nature Communications, 5, 3512, 10.1038/ncomms4512 · DOI
  34. Takemura, Shin-Ya , Aljoscha Nern , Dmitri B. Chklovskii , Louis K. Scheffer , Gerald M. Rubin , and Ian A. Meinertzhagen . 2017. “The Comprehensive Connectome of a Neural Substrate for ‘ON’ Motion Detection in Drosophila.” eLife 6 (April). https://doi.org/10.7554/eLife.24394. · DOI · Paper page
  35. 10.1073/pnas.1509820112 · DOI · Paper page
  36. 10.1038/nn.4290 · DOI
  37. Wetzel, A. W. , J. Bakal , M. Dittrich , D. G. C. Hildebrand , J. L. Morgan , and J. W. Lichtman . 2016. “Registering Large Volume Serial-Section Electron Microscopy Image Sets for Neural Circuit Reconstruction Using FFT Signal Whitening.” In 2016 IEEE Applied Imagery Pattern Recognition Workshop (AIPR), 1–10.
  38. Zhao, Ting , and Stephen M. Plaza . 2014. “Automatic Neuron Type Identification by Neurite Localization in the Drosophila Medulla.” arXiv [q-bio.NC]. arXiv. http://arxiv.org/abs/1409.1892. · arXiv
  39. 10.1016/j.cell.2018.06.019 · DOI · Paper page
  40. Zheng, Zhihao , Feng Li , Corey Fisher , Iqbal J. Ali , Nadiya Sharifi , Steven Calle-Schuler , Joseph Hsu , et al. 2020. “Structured Sampling of Olfactory Input by the Fly Mushroom Body.” bioRxiv. https://doi.org/10.1101/2020.04.17.047167. · DOI · Paper page
  41. Zuiderveld, Karel . 1994. “Contrast Limited Adaptive Histogram Equalization.” In Graphics Gems IV, edited by Paul S. Heckbert , 474–85. San Diego, CA, USA: Academic Press Professional, Inc. · DOI · Paper page

Cited by

Sorted by citations

Metadata

Authors
Li, P. H., Lindsey, L. F., Januszewski, M., Zheng, Z., Bates, A. S., Taisz, I., Tyka, M., Nichols, M., Li, F., Perlman, E., Maitin-Shepard, J., Blakely, T., Leavitt, L., Jefferis, G. S. X. E., Bock, D., & Jain, V.
ORCID
Publisher
openRxiv
Published
2019-04-11
DOI
10.1101/605634
Cited by
55
References
10
Type
Preprint

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review