Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 521

The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.

Democratising deep learning for microscopy with ZeroCostDL4Mic

Open access
von Chamier, Lucas, Laine, Romain F., Jukkala, Johanna, Spahn, Christoph, Krentzel, Daniel, Nehme, Elias, Lerche, Martina, Hernández-Pérez, Sara, Mattila, Pieta K., Karinou, Eleni, Holden, Séamus, Solak, Ahmet Can, Krull, Alexander, Buchholz, Tim-Oliver, Jones, Martin L., Royer, Loïc A., Leterrier, Christophe, Shechtman, Yoav, Jug, Florian, Heilemann, Mike, Jacquemet, Guillaume, Henriques, Ricardo·2021·Nature Communications
DOI 10.1038/s41467-021-22518-0

Published version of a preprint

Open this paper in CoChat — free account

Abstract

Deep Learning (DL) methods are powerful analytical tools for microscopy and can outperform conventional image processing pipelines. Despite the enthusiasm and innovations fuelled by DL technology, the need to access powerful and compatible resources to train DL networks leads to an accessibility barrier that novice users often find difficult to overcome. Here, we present ZeroCostDL4Mic, an entry-level platform simplifying DL access by leveraging the free, cloud-based computational resources of Google Colab. ZeroCostDL4Mic allows researchers with no coding expertise to train and apply key DL networks to perform tasks including segmentation (using U-Net and StarDist), object detection (using YOLOv2), denoising (using CARE and Noise2Void), super-resolution microscopy (using Deep-STORM), and image-to-image translation (using Label-free prediction - fnet, pix2pix and CycleGAN). Importantly, we provide suitable quantitative tools for each network to evaluate model performance, allowing model optimisation. We demonstrate the application of the platform to study multiple biological processes.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41467-021-22518-0

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 18 references

Research works printed but not linked: 67

  1. Redmon, J. & Farhadi, A. YOLO9000: better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 7263–7271 (2017). · DOI · Paper page
  2. Schmidt, U., Weigert, M., Broaddus, C. & Myers, G. Cell detection with Star-Convex polygons. in Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 (eds. Frangi, A. F. et al.) Vol. 11071, 265–273 (Springer International Publishing, 2018). · DOI · Paper page
  3. Weigert, M., Schmidt, U., Haase, R., Sugawara, K. & Myers, G. Star-convex polyhedra for 3d object detection and segmentation in microscopy. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 3666–3673 (2020). · DOI · Paper page
  4. Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, 1090–1097 (2018). · DOI · Paper page
  5. Krull, A., Buchholz, T. O. & Jug, F. Noise2void-learning denoising from single noisy images. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2129–2137 (2019). · DOI · Paper page
  6. Araújo, T. et al. Classification of breast cancer histology images using convolutional neural networks. PLoS ONE 12, e0177544 (2017). · DOI · Paper page
  7. Haberl, M. G. et al. CDeep3M—Plug-and-Play cloud-based deep learning for image segmentation. Nat. Methods 15, 677–680 (2018). · DOI · Paper page
  8. Buchholz, T. O., Prakash, M., Schmidt, D., Krull, A. & Jug, F. DenoiSeg: joint denoising and segmentation. European Conference on Computer Vision. pp. 324-337 (Springer, Cham, 2020). · DOI
  9. Falk, T. et al. U-Net: deep learning for cell counting, detection, and morphometry. Nat. Methods 16, 67–70 (2019). · DOI · Paper page
  10. Bannon, D. et al. DeepCell Kiosk: scaling deep learning-enabled cellular image analysis with Kubernetes. Nat. Methods 18, 43–5 (2021). · DOI
  11. Belevich, I. & Jokitalo, E. DeepMIB: user-friendly and open-source software for training of deep learning network for biological image segmentation. PLoS Comput Biol. 17, e1008374 (2021). · DOI
  12. Hollandi, R., Szkalisity, A. & Toth, T. nucleAIzer: a parameter-free deep learning framework for nucleus segmentation using image style transfer. Cell Syst. 10, 453–458 (2020). · DOI · Paper page
  13. Ouyang, W., Mueller, F., Hjelmare, M., Lundberg, E. & Zimmer, C. ImJoy: an open-source computational platform for the deep learning era. Nat. Methods 16, 1199–1200 (2019). · DOI
  14. Berg, S. et al. ilastik: interactive machine learning for (bio)image analysis. Nat. Methods 16, 1226–1232 (2019). · DOI · Paper page
  15. McQuin, C. et al. CellProfiler 3.0: Next-generation image processing for biology. PLoS Biol. 16, e2005970 (2018). · DOI · Paper page
  16. Gómez-de-Mariscal, E. et al. DeepImageJ: a user-friendly plugin to run deep learning models in ImageJ. Preprint at http://biorxiv.org/lookup/doi/10.1101/799270 (2019). · DOI
  17. Antun, V., Renna, F., Poon, C., Adcock, B. & Hansen, A. C. On instabilities of deep learning in image reconstruction and the potential costs of AI. Proc. Natl Acad. Sci. USA 117, 30088–30095 (2020). · DOI · Paper page
  18. Möckl, L., Roy, A. R. & Moerner, W. E. Deep learning in single-molecule microscopy: fundamentals, caveats, and recent developments [Invited]. Biomed. Opt. Express 11, 1633 (2020). · DOI
  19. Ounkomol, C., Seshamani, S., Maleckar, M. M., Collman, F. & Johnson, G. R. Label-free prediction of three-dimensional fluorescence images from transmitted-light microscopy. Nat. Methods 15, 917–920 (2018). · DOI · Paper page
  20. Moen, E. et al. Deep learning for cellular image analysis. Nat. Methods 16, 1233–1246 (2019). · DOI · Paper page
  21. Nehme, E., Weiss, L. E., Michaeli, T. & Shechtman, Y. Deep-STORM: super-resolution single-molecule microscopy by deep learning. Optica 5, 458 (2018). · DOI · Paper page
  22. Bloice, M. D., Roth, P. M. & Holzinger, A. Biomedical image augmentation using Augmentor. Bioinformatics 35, 4522–4524 (2019). · DOI · Paper page
  23. Carneiro, T. et al. Performance analysis of google colaboratory as a tool for accelerating deep learning applications. IEEE Access 6, 61677–61685 (2018). · DOI · Paper page
  24. Melsted, P. et al. Modular and efficient pre-processing of single-cell RNA-seq. Preprint at http://biorxiv.org/lookup/doi/10.1101/673285 (2019). · DOI
  25. Spiers, H. et al. Citizen science, cells and CNNs—deep learning for automatic segmentation of the nuclear envelope in electron microscopy data, trained with volunteer segmentations. Preprint at http://biorxiv.org/lookup/doi/10.1101/2020.07.28.223024 (2020). · doi:10.1101/2020.07.28.223024
  26. Tinevez, J.-Y. et al. TrackMate: an open and extensible platform for single-particle tracking. Methods 115, 80–90 (2017). · DOI · Paper page
  27. Quinn, J. A. et al. Deep convolutional neural networks for microscopy-based point of care diagnostics. in Machine Learning for Healthcare Conference (eds. Doshi-Velez, F. et al.) 271–281 (PMLR, 2016).
  28. Tosheva, K. L., Yuan, Y., Matos Pereira, P., Culley, S. & Henriques, R. Between life and death: strategies to reduce phototoxicity in super-resolution microscopy. J. Phys. Appl. Phys. 53, 163001 (2020). · DOI
  29. Strack, R. Hessian structured illumination microscopy. Nat. Methods 15, 407–407 (2018). · doi:10.1038/s41592-018-0023-1
  30. Jin, L. et al. Deep learning enables structured illumination microscopy with low light levels and enhanced speed. Nat. Commun. 11, 1934 (2020). · DOI · Paper page
  31. Jacquemet, G., Carisey, A. F., Hamidi, H., Henriques, R. & Leterrier, C. The cell biologist’s guide to super-resolution microscopy. J. Cell Sci. 133, jcs240713 (2020). · DOI · Paper page
  32. Nelson, A. J. & Hess, S. T. Molecular imaging with neural training of identification algorithm (neural network localization identification). Microsc. Res. Tech. 81, 966–972 (2018). · DOI
  33. Manor, U. et al. Deep learning‐based point‐scanning super‐resolution imaging. FASEB J. 34, 1–1 (2020). · doi:10.1096/fasebj.2020.34.s1.02496
  34. Owen, D. M. et al. PALM imaging and cluster analysis of protein heterogeneity at the cell surface. J. Biophotonics 3, 446–454 (2010). · DOI · Paper page
  35. Sengupta, P. et al. Probing protein heterogeneity in the plasma membrane using PALM and pair correlation analysis. Nat. Methods 8, 969–975 (2011). · DOI · Paper page
  36. Levet, F. et al. SR-Tesseler: a method to segment and quantify localization-based super-resolution microscopy data. Nat. Methods 12, 1065–1071 (2015). · DOI · Paper page
  37. Heilemann, M. et al. Subdiffraction-resolution fluorescence imaging with conventional fluorescent probes. Angew. Chem. Int. Ed. 47, 6172–6176 (2008). · DOI · Paper page
  38. Jungmann, R. et al. Multiplexed 3D cellular super-resolution imaging with DNA-PAINT and exchange-PAINT. Nat. Methods 11, 313–318 (2014). · DOI · Paper page
  39. Ovesný, M., Křížek, P., Borkovec, J., Švindrych, Z. & Hagen, G. M. ThunderSTORM: a comprehensive ImageJ plug-in for PALM and STORM data analysis and super-resolution imaging. Bioinformatics 30, 2389–2390 (2014). · DOI · Paper page
  40. Culley, S. et al. Quantitative mapping and minimization of super-resolution optical imaging artifacts. Nat. Methods 15, 263–266 (2018). · DOI · Paper page
  41. Goodfellow, I. et al. Generative adversarial nets. in Advances in Neural Information Processing Systems 27 (eds. Ghahramani, Z. et al.) 2672–2680 (Curran Associates, Inc., 2014).
  42. Gustafsson, N. et al. Fast live-cell conventional fluorophore nanoscopy with ImageJ through super-resolution radial fluctuations. Nat. Commun. 7, 12471 (2016). · DOI · Paper page
  43. Laine, R. F. et al. NanoJ: a high-performance open-source super-resolution microscopy toolbox. J. Phys. Appl. Phys. 52, 163001 (2019). · DOI · Paper page
  44. Fazeli, E. et al. Automated cell tracking using StarDist and TrackMate. F1000Research 9, 1279 (2020). · DOI
  45. Rueden, C. T. et al. ImageJ2: ImageJ for the next generation of scientific image data. BMC Bioinforma. 18, 529 (2017). · DOI · Paper page
  46. LaChance, J. & Cohen, D. J. Practical fluorescence reconstruction microscopy for large samples and low-magnification imaging. PLoS Comput Biol. 16, e1008443 (2020). · DOI
  47. Moen, E. et al. Accurate cell tracking and lineage construction in live-cell imaging experiments with deep learning. Preprint at http://biorxiv.org/lookup/doi/10.1101/803205 (2019). · DOI
  48. von Chamier, L., Laine, R. F. & Henriques, R. Artificial intelligence for microscopy: what you should know. Biochem. Soc. Trans. 47, 1029–1040 (2019). · DOI
  49. Belthangady, C. & Royer, L. A. Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction. Nat. Methods 16, 1215–1225 (2019). · DOI · Paper page
  50. Nichols, J. A., Herbert Chan, H. W. & Baker, M. A. B. Machine learning: applications of artificial intelligence to imaging and diagnosis. Biophys. Rev. 11, 111–118 (2019). · DOI · Paper page
  51. Kirillov, A., He, K., Girshick, R., Rother, C. & Dollár, P. Panoptic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9404–9413 (2019). · DOI · Paper page
  52. Everingham, M. & Winn, J. The Pascal Visual Object Classes Challenge 2012 (VOC2012) Development Kit. Pattern Analysis, Statistical Modelling and Computational Learning, Technical Report. vol. 8 (VOC2012, 2011).
  53. Lavoie-Cardinal, F. et al. MICRA-Net: MICRoscopy Analysis Neural Network to solve detection, classification, and segmentation from a single simple auxiliary task. https://www.researchsquare.com/article/rs-95613/v1 (2020). · DOI
  54. Lavoie-Cardinal, F. et al. Neuronal activity remodels the F-actin based submembrane lattice in dendrites but not axons of hippocampal neurons. Sci. Rep. 10, 11960 (2020). · DOI
  55. Hollandi, R., Diósdi, Á., Hollandi, G., Moshkov, N. & Horváth, P. AnnotatorJ: an ImageJ plugin to ease hand annotation of cellular compartments. Mol. Biol. Cell 31, 2179–2186 (2020). · DOI
  56. Speiser, A. et al. Deep learning enables fast and dense single-molecule localization with high accuracy. Preprint at http://biorxiv.org/lookup/doi/10.1101/2020.10.26.355164 (2020). · doi:10.1101/2020.10.26.355164
  57. Khadangi, A., Boudier, T. & Rajagopal, V. EM-stellar: benchmarking deep learning for electron microscopy image segmentation. Bioinformatics. https://doi.org/10.1093/bioinformatics/btaa1094, btaa1094 (2021). · DOI
  58. Stubb, A. et al. Fluctuation-based super-resolution traction force microscopy. Nano Lett. 20, 2230–2245 (2020). · DOI
  59. Jacquemet, G. et al. FiloQuant reveals increased filopodia density during breast cancer progression. J. Cell Biol. 216, 3387–3403 (2017). · DOI
  60. Legant, W. R. et al. High-density three-dimensional localization microscopy across large volumes. Nat. Methods 13, 359–365 (2016). · DOI · Paper page
  61. Vassilopoulos, S., Gibaud, S., Jimenez, A., Caillol, G. & Leterrier, C. Ultrastructure of the axonal periodic scaffold reveals a braid-like organization of actin rings. Nat. Commun. 10, 5803 (2019). · DOI · Paper page
  62. Arganda-Carreras, I. et al. Crowdsourcing the creation of image segmentation algorithms for connectomics. Front. Neuroanat. 9, 142 (2015). · DOI · Paper page
  63. Wortel, I. M. N., Dannenberg, K., Berry, J. C., Miller, M. J. & Textor, J. CelltrackR: an R package for fast and flexible analysis of immune cell migration data. Preprint at http://biorxiv.org/lookup/doi/10.1101/670505 (2019). · doi:10.1101/670505
  64. Kaukonen, R., Jacquemet, G., Hamidi, H. & Ivaska, J. Cell-derived matrices for studying cell proliferation and directional migration in a complex 3D microenvironment. Nat. Protoc. 12, 2376–2390 (2017). · DOI
  65. Jimenez, A., Friedl, K. & Leterrier, C. About samples, giving examples: optimized single molecule localization microscopy. Methods 174, 100–114 (2020). · DOI
  66. Mlodzianoski, M. J. et al. Sample drift correction in 3D fluorescence photoactivation localization microscopy. Opt. Express 19, 15009 (2011). · DOI · Paper page
  67. Postma, M. & Goedhart, J. PlotsOfData—a web app for visualizing data together with their summaries. PLoS Biol. 17, e3000202 (2019). · DOI · Paper page

Cited by

See all 521

Metadata

Authors
von Chamier, L., Laine, R. F., Jukkala, J., Spahn, C., Krentzel, D., Nehme, E., Lerche, M., Hernández-Pérez, S., Mattila, P. K., Karinou, E., Holden, S., Solak, A. C., Krull, A., Buchholz, T.-O., Jones, M. L., Royer, L. A., Leterrier, C., Shechtman, Y., Jug, F., Heilemann, M., Jacquemet, G., & Henriques, R.
Venue
Nature Communications
Publisher
Springer Science and Business Media LLC
Published
2021-04-15
Volume
12
Issue
1
Pages
2276-2276
ISSN
2041-1723 (ISSN-L)
DOI
10.1038/s41467-021-22518-0
OpenAlex
W3156859314
Cited by
521
References
18
FWCI
38.74
Type
journal article
Language
en
Open access
gold
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Democratising deep learning for microscopy with ZeroCostDL4Mic. https://app.cochat.ai/papers/10.1038/s41467-021-22518-0
The paper
von Chamier, L., Laine, R. F., Jukkala, J., Spahn, C., Krentzel, D., Nehme, E., Lerche, M., Hernández-Pérez, S., Mattila, P. K., Karinou, E., Holden, S., Solak, A. C., Krull, A., Buchholz, T.-O., Jones, M. L., Royer, L. A., Leterrier, C., Shechtman, Y., Jug, F., . . . Henriques, R. (2021). Democratising deep learning for microscopy with ZeroCostDL4Mic. Nature Communications. https://doi.org/10.1038/s41467-021-22518-0

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.