CoChat
Sign Up
Cited by 85Cell Image Analysis TechniquesAdvanced Fluorescence Microscopy TechniquesSingle-cell and spatial transcriptomics

Tracking cell lineages in 3D by incremental deep learning

Open access
Sugawara, Ko, Çevrim, Çağrı, Averof, Michalis·2022·eLife
DOI 10.7554/elife.69380PMC PMC8741210

Published version of a preprint

Open this paper in CoChat — free account

Abstract

Deep learning is emerging as a powerful approach for bioimage analysis. Its use in cell tracking is limited by the scarcity of annotated data for the training of deep-learning models. Moreover, annotation, training, prediction, and proofreading currently lack a unified user interface. We present ELEPHANT, an interactive platform for 3D cell tracking that addresses these challenges by taking an incremental approach to deep learning. ELEPHANT provides an interface that seamlessly integrates cell track annotation, deep learning, prediction, and proofreading. This enables users to implement cycles of incremental learning starting from a few annotated nuclei. Successive prediction-validation cycles enrich the training data, leading to rapid improvements in tracking performance. We test the software’s performance against state-of-the-art methods and track lineages spanning the entire course of leg regeneration in a crustacean over 1 week (504 timepoints). ELEPHANT yields accurate, fully-validated cell lineages with a modest investment in time and effort.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.7554/elife.69380

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The human-in-the-loop approach has been demonstrated for simple ROI such as nuclei and round cells, but not for cells with complex morphologies that require full cytoplasmic segmentation.

    “This approach has been demonstrated for simple ROI such as nuclei and round cells, which allow for weak annotations such as clicks and squiggles18,19, but not for cells with complex morphologies that require full cytoplasmic segmentation.”

    From Cellpose 2.0: how to train your own model (2022) · cites this paper as [19]

    The Results and discussion section describes human-in-the-loop detection and tracking of nuclei using sparse ellipsoid annotations and user proofreading. It does not demonstrate round-cell annotation with clicks or squiggles, nor establish the broader claim that this approach has not been demonstrated for complex cell morphologies requiring full cytoplasmic segmentation. No page number is available in our copy. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • Using an iterative approach, a 3D dataset of nuclei was segmented in approximately one month.

    “For example, using an iterative approach19, a 3D dataset of nuclei was segmented in approximately one month.”

    From Cellpose 2.0: how to train your own model (2022) · cites this paper as [19]

    The Results and discussion report that an interactive ELEPHANT workflow completed cell-lineage tracking in approximately one month, and Materials and methods describe iterative annotation of nuclei in the 3D dataset. The month-long result was reconstruction and validation of cell lineages, not specifically segmentation of a dataset of nuclei. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 40

  1. Alwes F Enjolras C Averof M 2016 Live imaging reveals the progenitors and cell dynamics of limb regeneration eLife 5 e19766 10.7554/eLife.19766 27776632 PMC5079749 · DOI
  2. Amat F Myers EW Keller PJ 2013 Fast and robust optical flow for time-lapse microscopy using super-voxels Bioinformatics 29 373 380 10.1093/bioinformatics/bts706 23242263 PMC3562071 · DOI
  3. Castilla C Maska M Sorokin DV Meijering E Ortiz-de-Solorzano C 2019 3-D Quantification of Filopodia in Motile Cancer Cells IEEE Transactions on Medical Imaging 38 862 872 10.1109/TMI.2018.2873842 30296215 · DOI
  4. Cicek O Abdulkadir A Lienkamp SS Brox T Ronneberger O 2016 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation Ourselin S Joskowicz L Sabuncu MR Unal G Wells W Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 Springer International Publishing 10.1007/978-3-319-46723-8 · DOI
  5. Crocker JC Grier DG 1996 Methods of Digital Video Microscopy for Colloidal Studies Journal of Colloid and Interface Science 179 298 310 10.1006/jcis.1996.0217 · DOI
  6. de Medeiros G 2021 Multiscale light-sheet organoid imaging framework European Light Microscopy Initiative 2021 10.22443/rms.elmi2021.90 · DOI
  7. Dzyubachyk O van Cappellen WA Essers J Niessen WJ Meijering E 2010 Advanced level-set-based cell tracking in time-lapse fluorescence microscopy IEEE Transactions on Medical Imaging 29 852 867 10.1109/TMI.2009.2038693 20199920 · DOI
  8. Hayashida J Bise R 2019 Cell Tracking with Deep Learning for Cell Detection and Motion Estimation in Low-Frame-Rate Shen D Cal Image Computing and Computer Assisted Intervention – MICCAI 2019 Springer International Publishing 397 405
  9. Hayashida J Nishimura K Bise R 2020 MPM: Joint Representation of Motion and Position Map for Cell Tracking 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition 3823 3832 10.1109/CVPR42600.2020.00388 · DOI
  10. He K Zhang X Ren S Sun J 2015a Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification IEEE International Conference on Computer Vision 1026 1034 10.1109/ICCV.2015.123 · DOI · Paper page
  11. Horn BKP Schunck BG 1981 Determining optical flow Artificial Intelligence 17 185 203 10.1016/0004-3702(81)90024-2 · DOI · Paper page
  12. Ilg E Mayer N Saikia T Keuper M Dosovitskiy A Brox T 2017 FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks 2017 IEEE Conference on Computer Vision and Pattern Recognition 1647 1655 10.1109/CVPR.2017.179 · DOI
  13. Kingma DP Ba J 2014 Adam: A Method for Stochastic Optimization arXiv https://arxiv.org/abs/1412.6980 · arXiv
  14. Kok RNU Hebert L Huelsz-Prince G Goos YJ Zheng X Bozek K Stephens GJ Tans SJ van Zon JS 2020 OrganoidTracker: Efficient cell tracking using machine learning and manual error correction PLOS ONE 15 e0240802 10.1371/journal.pone.0240802 33091031 PMC7580893 · DOI
  15. Lucas BD Kanade T 1981 An iterative image registration technique with an application to stereo vision Proceedings of the 7th international joint conference on Artificial intelligence - Volume 2 IJC 674 679
  16. Maas AL Hannun AY Ng AY 2013 Rectifier nonlinearities improve neural network acoustic models CML Workshop on Deep Learning for Audio, Speech and Language Processing
  17. Magnusson KEG Jalden J Gilbert PM Blau HM 2015 Global linking of cell tracks using the Viterbi algorithm IEEE Transactions on Medical Imaging 34 911 929 10.1109/TMI.2014.2370951 25415983 PMC4765504 · DOI
  18. Maška M Daněk O Garasa S Rouzaut A Muñoz-Barrutia A Ortiz-de-Solorzano C 2013 Segmentation and shape tracking of whole fluorescent cells based on the Chan-Vese model IEEE Transactions on Medical Imaging 32 995 1006 10.1109/TMI.2013.2243463 23372077 · DOI
  19. Maška M Ulman V Svoboda D Matula P Matula P Ederra C Urbiola A España T Venkatesan S Balak DMW Karas P Bolcková T Streitová M Carthel C Coraluppi S Harder N Rohr K Magnusson KEG Jaldén J Blau HM Dzyubachyk O Křížek P Hagen GM Pastor-Escuredo D Jimenez-Carretero D Ledesma-Carbayo MJ Muñoz-Barrutia A Meijering E Kozubek M Ortiz-de-Solorzano C 2014 A benchmark for comparison of cell tracking algorithms Bioinformatics 30 1609 1617 10.1093/bioinformatics/btu080 24526711 PMC4029039 · DOI · Paper page
  20. Matula P Maška M Sorokin DV Matula P Ortiz-de-Solórzano C Kozubek M 2015 Cell Tracking Accuracy Measurement Based on Comparison of Acyclic Oriented Graphs PLOS ONE 10 e0144959 10.1371/journal.pone.0144959 26683608 PMC4686175 · DOI
  21. Moen E Bannon D Kudo T Graf W Covert M Van Valen D 2019 Deep learning for cellular image analysis Nature Methods 16 1233 1246 10.1038/s41592-019-0403-1 31133758 PMC8759575 · DOI · Paper page
  22. Moore J Allan C Besson S Burel JM Diel E Gault D Kozlowski K Lindner D Linkert M Manz T Moore W Pape C Tischer C 2021 OME-NGFF: a next-generation file format for expanding bioimaging data-access strategies Nature Methods 18 1496 1498 10.1038/s41592-021-01326-w 34845388 PMC8648559 · DOI
  23. Murray JI Bao Z Boyle TJ Boeck ME Mericle BL Nicholas TJ Zhao Z Sandel MJ Waterston RH 2008 Automated analysis of embryonic gene expression with cellular resolution in C. elegans Nature Methods 5 703 709 10.1038/nmeth.1228 18587405 PMC2553703 · DOI
  24. Nair V Hinton GE 2010 Rectified linear units improve Restricted Boltzmann machines ICML 2010 - Proceedings, 27th International Conference on Machine Learning
  25. Ouyang W Aristov A Lelek M Hao X Zimmer C 2018 Deep learning massively accelerates super-resolution localization microscopy Nature Biotechnology 36 460 468 10.1038/nbt.4106 29658943 · DOI
  26. Paszke A Gross S Massa F Lerer A Bradbury J Chanan G Killeen T Lin Z Gimelshein N Antiga L Desmaison A Köpf A Yang E DeVito Z Raison M Tejani A Chilamkurthy S Steiner B Fang L Bai J 2019 PyTorch: An Imperative Style, High-Performance Deep Learning Library arXiv https://arxiv.org/abs/1912.01703 · arXiv
  27. Pietzsch T Saalfeld S Preibisch S Tomancak P 2015 BigDataViewer: visualization and processing for large image data sets Nature Methods 12 481 483 10.1038/nmeth.3392 26020499 · DOI
  28. Scherr T Löffler K Böhland M Mikut R 2020 Cell segmentation and tracking using CNN-based distance predictions and a graph-based matching strategy PLOS ONE 15 e0243219 10.1371/journal.pone.0243219 33290432 PMC7723299 · DOI
  29. Schindelin J Arganda-Carreras I Frise E Kaynig V Longair M Pietzsch T Preibisch S Rueden C Saalfeld S Schmid B Tinevez JY White DJ Hartenstein V Eliceiri K Tomancak P Cardona A 2012 Fiji: an open-source platform for biological-image analysis Nature Methods 9 676 682 10.1038/nmeth.2019 22743772 PMC3855844 · DOI · Paper page
  30. Schneider CA Rasband WS Eliceiri KW 2012 NIH Image to ImageJ: 25 years of image analysis Nature Methods 9 671 675 10.1038/nmeth.2089 22930834 PMC5554542 · DOI · Paper page
  31. Sugawara K 2021c ELEPHANT: Tracking cell lineages in 3D by incremental deep learning swh:1:rev:449f9ff8ad17ce75f355e18f815653ec0aa4bbb8 SoftwareHeritage https://archive.softwareheritage.org/swh:1:dir:e69da53d731182d6c6ffcb97588396e59a472e4f;origin=https://github.com/elephant-track/elephant-client;visit=swh:1:snp:f7f13f47ba9af8edaef97291b89bc4825a63a1b9;anchor=swh:1:rev:449f9ff8ad17ce75f355e18f815653ec0aa4bbb8 PMC8741210 34989675
  32. Tseng Q Wang I Duchemin-Pelletier E Azioune A Carpi N Gao J Filhol O Piel M Théry M Balland M 2011 A new micropatterning method of soft substrates reveals that different tumorigenic signals can promote or reduce cell contraction levels Lab on a Chip 11 2231 2240 10.1039/c0lc00641f 21523273 · DOI
  33. Ulman V Maška M Magnusson KEG Ronneberger O Haubold C Harder N Matula P Matula P Svoboda D Radojevic M Smal I Rohr K Jaldén J Blau HM Dzyubachyk O Lelieveldt B Xiao P Li Y Cho S-Y Dufour AC Olivo-Marin J-C Reyes-Aldasoro CC Solis-Lemus JA Bensch R Brox T Stegmaier J Mikut R Wolf S Hamprecht FA Esteves T Quelhas P Demirel Ö Malmström L Jug F Tomancak P Meijering E Muñoz-Barrutia A Kozubek M Ortiz-de-Solorzano C 2017 An objective comparison of cell-tracking algorithms Nature Methods 14 1141 1152 10.1038/nmeth.4473 29083403 PMC5777536 · DOI
  34. van der Walt S Schönberger JL Nunez-Iglesias J Boulogne F Warner JD Yager N Gouillart E Yu T scikit-image contributors 2014 scikit-image: image processing in Python PeerJ 2 e453 10.7717/peerj.453 25024921 PMC4081273 · DOI
  35. Virtanen P Gommers R Oliphant TE Haberland M Reddy T Cournapeau D Burovski E Peterson P Weckesser W Bright J van der Walt SJ Brett M Wilson J Millman KJ Mayorov N Nelson ARJ Jones E Kern R Larson E Carey CJ Polat İ Feng Y Moore EW VanderPlas J Laxalde D Perktold J Cimrman R Henriksen I Quintero EA Harris CR Archibald AM Ribeiro AH Pedregosa F van Mulbregt P SciPy 1.0 Contributors 2020 SciPy 1.0: fundamental algorithms for scientific computing in Python Nature Methods 17 261 272 10.1038/s41592-019-0686-2 32015543 PMC7056644 · DOI · Paper page
  36. Weigert M Schmidt U Boothe T Müller A Dibrov A Jain A Wilhelm B Schmidt D Broaddus C Culley S Rocha-Martins M Segovia-Miranda F Norden C Henriques R Zerial M Solimena M Rink J Tomancak P Royer L Jug F Myers EW 2018 Content-aware image restoration: pushing the limits of fluorescence microscopy Nature Methods 15 1090 1097 10.1038/s41592-018-0216-7 30478326 · DOI
  37. Weigert M Schmidt U Haase R Sugawara K Myers G 2020 Star-convex Polyhedra for 3D Object Detection and Segmentation in Microscopy 2020 IEEE Winter Conference on Applications of Computer Vision 3655 3662 10.1109/WACV45572.2020.9093435 · DOI
  38. Wen C Miura T Voleti V Yamaguchi K Tsutsumi M Yamamoto K Otomo K Fujie Y Teramoto T Ishihara T Aoki K Nemoto T Hillman EMC Kimura KD 2021 3DeeCellTracker, a deep learning-based pipeline for segmenting and tracking cells in 3D time lapse images eLife 10 e59187 10.7554/eLife.59187 33781383 PMC8009680 · DOI
  39. Wolff C Tinevez JY Pietzsch T Stamataki E Harich B Guignard L Preibisch S Shorte S Keller PJ Tomancak P Pavlopoulos A 2018 Multi-view light-sheet imaging and tracking with the MaMuT software reveals the cell lineage of a direct developing arthropod limb eLife 7 e34410 10.7554/eLife.34410 29595475 PMC5929908 · DOI
  40. Wu Y He K 2020 Group Normalization International Journal of Computer Vision 128 742 755 10.1007/s11263-019-01198-w · DOI · Paper page

Cited by

Sorted by citations

Authors
Sugawara, K., Çevrim, Ç., & Averof, M.
Affiliations
École Normale Supérieure de Lyon (FR)
Corresponding author
  • Sugawara, Ko
  • Averof, Michalis
ORCID
Venue
eLife
Publisher
eLife Sciences Publications, Ltd
Published
2022-01-06
Volume
11
ISSN
2050-084X (ISSN-L)
DOI
10.7554/elife.69380
PMC
PMC8741210
OpenAlex
W4205616442
Cited by
85
References
4
Citations by year
2026: 8 · 2025: 22 · 2024: 16 · 2023: 20 · 2022: 11 · 2021: 7 · 2019: 1
FWCI
6.07
Type
journal article
Language
en
Open access
Licence
Locations
  • eLife (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • HAL (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • H2020 European Research Council (ERC-2015-AdG #694918)
  • Boehringer Ingelheim Fonds
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Cell Image Analysis Techniques
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Advanced Fluorescence Microscopy Techniques
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
Keywords
3D cell tracking, lineage tracing, elephants, training data annotation, proofreading, leg regeneration, Crustacea
MeSH terms
Deep Learning, Humans, Image Processing, Computer-Assisted, Cell Lineage, Cell Tracking, Data Curation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review