CoChat
Sign Up
Cited by 42Adaptive optics and wavefront sensingAdvanced Fluorescence Microscopy TechniquesDigital Holography and Microscopy

Deep learning-driven adaptive optics for single-molecule localization microscopy

Open access
Zhang, Peiyi, Ma, Donghan, Cheng, Xi, Tsai, Andy P., Tang, Yu, Gao, Hao-Cheng, Fang, Li, Bi, Cheng +3 more·2023·Nature Methods
DOI 10.1038/s41592-023-02029-0PMC PMC10630144

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

The inhomogeneous refractive indices of biological tissues blur and distort single-molecule emission patterns generating image artifacts and decreasing the achievable resolution of single-molecule localization microscopy (SMLM). Conventional sensorless adaptive optics methods rely on iterative mirror changes and image-quality metrics. However, these metrics result in inconsistent metric responses and thus fundamentally limit their efficacy for aberration correction in tissues. To bypass iterative trial-then-evaluate processes, we developed deep learning-driven adaptive optics for SMLM to allow direct inference of wavefront distortion and near real-time compensation. Our trained deep neural network monitors the individual emission patterns from single-molecule experiments, infers their shared wavefront distortion, feeds the estimates through a dynamic filter and drives a deformable mirror to compensate sample-induced aberrations. We demonstrated that our method simultaneously estimates and compensates 28 wavefront deformation shapes and improves the resolution and fidelity of three-dimensional SMLM through >130-µm-thick brain tissue specimens.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41592-023-02029-0

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • AI-driven medical imaging analysis significantly improves the accuracy and efficiency of diagnostic processes.

    “AI-driven medical imaging analysis has emerged as a pivotal tool in modern healthcare, significantly improving the accuracy and efficiency of diagnostic processes.66,74,76,77,78,79,80,81,82,83”

    From A foundational architecture for AI agents in healthcare (2025) · cites this paper as [82]

    The Main section (page not available) reports that deep learning-driven adaptive optics improves the precision and fidelity of single-molecule microscopy and provides near real-time correction. The paper studies research imaging of cells and tissues, not medical-image analysis in diagnostic processes; it does not establish improved diagnostic accuracy or efficiency. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 74

  1. [1]Hell SW Far-field optical nanoscopy Science 2007 316 1153 1158 17525330
  2. [2]Hell SW Wichmann J Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy Opt. Lett. 1994 19 780 782 19844443
  3. [3]Wu Y Shroff H Faster, sharper, and deeper: structured illumination microscopy for biological imaging Nat. Methods 2018 15 1011 1019 30478322
  4. [4]Betzig E Imaging intracellular fluorescent proteins at nanometer resolution Science 2006 313 1642 1645 16902090
  5. [5]Hess ST Gould TJ Gunewardene M Bewersdorf J Mason MD Ultrahigh resolution imaging of biomolecules by fluorescence photoactivation localization microscopy Methods Mol. Biol. 2009 544 483 522 19488720 PMC6141240
  6. [6]Rust MJ Bates M Zhuang X Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM) Nat. Methods 2006 3 793 795 16896339 PMC2700296
  7. [7]Schmidt R Spherical nanosized focal spot unravels the interior of cells Nat. Methods 2008 5 539 544 18488034
  8. [8]Huang B Wang W Bates M Zhuang X Three-dimensional super-resolution imaging by stochastic optical reconstruction microscopy Science 2008 319 810 813 18174397 PMC2633023
  9. [9]Xu K Zhong G Zhuang X Actin, spectrin, and associated proteins form a periodic cytoskeletal structure in axons Science 2013 339 452 456 23239625 PMC3815867
  10. [11]Moerner WE Shechtman Y Wang Q Single-molecule spectroscopy and imaging over the decades Faraday Discuss. 2015 184 9 36 26616210 PMC4782608
  11. [12]Von Diezmann, A., Shechtman, Y. & Moerner, W. E. Three-dimensional localization of single molecules for super-resolution imaging and single-particle tracking. Chem. Rev.117, 7244–7275 (2017). PMC5471132 28151646
  12. [13]Huang F Ultra-high resolution 3D imaging of whole cells Cell 2016 166 1028 1040 27397506 PMC5005454
  13. [14]Liu S Huang F Enhanced 4Pi single-molecule localization microscopy with coherent pupil based localization Commun. Biol. 2020 3 220 32385402 PMC7210115
  14. [15]Bates M Optimal precision and accuracy in 4Pi-STORM using dynamic spline PSF models Nat. Methods 2022 19 603 612 35577958 PMC9119851
  15. [16]Oi C LIVE-PAINT allows super-resolution microscopy inside living cells using reversible peptide–protein interactions Commun. Biol. 2020 3 458 32820217 PMC7441314
  16. [17]Smith CS Joseph N Rieger B Lidke KA Fast, single-molecule localization that achieves theoretically minimum uncertainty Nat. Methods 2010 7 373 375 20364146 PMC2862147
  17. [18]Liu S Huh H Lee S-H Huang F Three-dimensional single-molecule localization microscopy in whole-cell and tissue specimens Annu. Rev. Biomed. Eng. 2020 22 155 184 32243765 PMC7430714
  18. [19]Tyson, R. Principles of Adaptive Optics (Academic Press, 1991).
  19. [20]Hardy, J. W. Adaptive Optics for Astronomical Telescopes (Oxford University Press, 1998).
  20. [21]Booth M Andrade D Burke D Patton B Zurauskas M Aberrations and adaptive optics in super-resolution microscopy Microscopy 2015 64 251 261 26124194 PMC4711293
  21. [22]Ji N Adaptive optical fluorescence microscopy Nat. Methods 2017 14 374 380 28362438
  22. [23]Hampson KM Adaptive optics for high-resolution imaging Nat. Rev. Methods Primers 2021 1 68 35252878 PMC8892592
  23. [24]Wang K Rapid adaptive optical recovery of optimal resolution over large volumes Nat. Methods 2014 11 625 628 24727653 PMC4069208
  24. [25]Liu R Li Z Marvin JS Kleinfeld D Direct wavefront sensing enables functional imaging of infragranular axons and spines Nat. Methods 2019 16 615 618 31209383 PMC6763395
  25. [26]Hu L Hu S Gong W Si K Deep learning assisted Shack–Hartmann wavefront sensor for direct wavefront detection Opt. Lett. 2020 45 3741–3744 32630943
  26. [27]Turcotte R Dynamic super-resolution structured illumination imaging in the living brain Proc. Natl Acad. Sci. USA 2019 116 9586 9591 31028150 PMC6511017
  27. [28]Li Z Fast widefield imaging of neuronal structure and function with optical sectioning in vivo Sci. Adv. 2020 6 eaaz3870 32494711 PMC7209992
  28. [30]Tehrani KF Xu J Zhang Y Shen P Kner P Adaptive optics stochastic optical reconstruction microscopy (AO-STORM) using a genetic algorithm Opt. Express 2015 23 13677 13692 26074617
  29. [31]Tehrani KF Zhang Y Shen P Kner P Adaptive optics stochastic optical reconstruction microscopy (AO-STORM) by particle swarm optimization Biomed. Opt. Express 2017 8 5087 5097 29188105 PMC5695955
  30. [32]Mlodzianoski MJ Active PSF shaping and adaptive optics enable volumetric localization microscopy through brain sections Nat. Methods 2018 15 583 586 30013047 PMC6071422
  31. [33]Siemons ME Hanemaaijer NAK Kole MHP Kapitein LC Robust adaptive optics for localization microscopy deep in complex tissue Nat. Commun. 2021 12 3407 34099685 PMC8184833
  32. [34]Booth MJ Neil MAA Juskaitis R Wilson T Adaptive aberration correction in a confocal microscope Proc. Natl Acad. Sci. USA 2002 99 5788 5792 11959908 PMC122854
  33. [35]Débarre D Booth MJ Wilson T Image based adaptive optics through optimisation of low spatial frequencies Opt. Express 1991 15 5131 5139 19547145
  34. [36]Žurauskas M IsoSense: frequency enhanced sensorless adaptive optics through structured illumination Optica 2019 6 370–379 31417942 PMC6683765
  35. [37]Thomas B Wolstenholme A Chaudhari SN Kipreos ET Kner P Enhanced resolution through thick tissue with structured illumination and adaptive optics J. Biomed. Opt. 2015 20 26006 25714992
  36. [38]Débarre D Botcherby EJ Booth MJ Wilson T Adaptive optics for structured illumination microscopy Opt. Exp. 2008 16 9290 9305 18575493
  37. [39]Gould TJ Burke D Bewersdorf J Booth MJ Adaptive optics enables 3D STED microscopy in aberrating specimens Opt. Express 2012 20 20998 23037223 PMC3635694
  38. [40]Xu F Three-dimensional nanoscopy of whole cells and tissues with in situ point spread function retrieval Nat. Methods 2020 17 531 540 32371980 PMC7289454
  39. [41]Zhang P Analyzing complex single-molecule emission patterns with deep learning Nat. Methods 2018 15 913 916 30377349 PMC6624853
  40. [43]LeCun Y Bottou L Bengio Y Haffner P Gradient-based learning applied to document recognition Proc. IEEE 1998 86 2278 2323
  41. [45]He, K., Zhang, X., Ren, S. & Sun, J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. in Proceedings of the IEEE International Conference on Computer Vision 1026–1034 (2015).
  42. [46]Ioffe, S. & Szegedy, C. Batch normalization: accelerating deep network training by reducing internal covariate shift. in Proceedings of the32nd International Conference on Machine Learning 448–456 (2015).
  43. [48]Wyant, J. C. & Creath, K. Basic wavefront aberration theory for optical metrology. in Applied Optics and Optical Engineering Vol. 11 (Academic Press, 1992).
  44. [49]Kay, S. M. Fundamentals of Statistical Signal Processing, Volume I: Estimation Theory (Prentice Hall, 1993).
  45. [50]Hanser BM Gustafsson MGL Agard DA Sedat JW Phase-retrieved pupil functions in wide-field fluorescence microscopy J. Microsc. 2004 216 32 48 15369481
  46. [51]Haber A Bifano T General approach to precise deformable mirror control Opt. Express 2021 29 33741 33759 34809180
  47. [53]Salloway S Amyloid-related imaging abnormalities in 2 phase 3 studies evaluating aducanumab in patients with early Alzheimer disease JAMA Neurol. 2022 79 13 21 34807243 PMC8609465
  48. [54]Sevigny J The antibody aducanumab reduces Aβ plaques in Alzheimer’s disease Nature 2016 537 50 56 27582220
  49. [56]Berry KP Nedivi E Spine dynamics: are they all the same? Neuron 2017 96 43 55 28957675 PMC5661952
  50. [57]Pchitskaya E Bezprozvanny I Dendritic spines shape analysis—classification or clusterization? Perspective Front. Synaptic Neurosci. 2020 12 31 33117142 PMC7561369
  51. [58]Tønnesen J Katona G Rózsa B Nägerl UV Spine neck plasticity regulates compartmentalization of synapses Nat. Neurosci. 2014 17 678 685 24657968
  52. [59]Runge K Cardoso C de Chevigny A Dendritic spine plasticity: function and mechanisms Front. Synaptic Neurosci. 2020 12 36 32982715 PMC7484486
  53. [60]Kissinger ST Visual experience-dependent oscillations and underlying circuit connectivity changes are impaired in Fmr1 KO mice Cell Rep. 2020 31 107486 32268079 PMC7201849
  54. [61]Arenkiel BR In vivo light-induced activation of neural circuitry in transgenic mice expressing channelrhodopsin-2 Neuron 2007 54 205 218 17442243 PMC3634585
  55. [62]Kubota Y Conserved properties of dendritic trees in four cortical interneuron subtypes Sci. Rep. 2011 1 89 22355608 PMC3216575
  56. [63]Behabadi BF Mel BW J4 at Sweet 16: a new wrinkle? Neural Comput. 2007 19 2865 2870 17883343
  57. [64]Lührs T 3D structure of Alzheimer’s amyloid-β(1–42) fibrils Proc. Natl Acad. Sci. USA 2005 102 17342 17347 16293696 PMC1297669
  58. [65]Pfeiffer T Chronic 2P-STED imaging reveals high turnover of dendritic spines in the hippocampus in vivo Elife 2018 7 e34700 29932052 PMC6014725
  59. [66]Yuste R Bonhoeffer T Genesis of dendritic spines: insights from ultrastructural and imaging studies Nat. Rev. Neurosci. 2004 5 24 34 14708001
  60. [67]Descloux A Grußmayer KS Radenovic A Parameter-free image resolution estimation based on decorrelation analysis Nat. Methods 2019 16 918 924 31451766
  61. [68]Diederich B Then P Jugler A Forster R Heintzmann R cellSTORM—cost-effective super-resolution on a cellphone using dSTORM PLoS ONE 2019 14 e0209827 30625170 PMC6326471
  62. [69]Gu L Molecular-scale axial localization by repetitive optical selective exposure Nat. Methods 2021 18 369 373 33795876
  63. [70]Dong D Super-resolution fluorescence-assisted diffraction computational tomography reveals the three-dimensional landscape of the cellular organelle interactome Light. Sci. Appl. 2020 9 11 32025294 PMC6987131
  64. [71]Zhao W Sparse deconvolution improves the resolution of live-cell super-resolution fluorescence microscopy Nat. Biotechnol. 2022 40 606 617 34782739
  65. [72]Park J-H Kong L Zhou Y Cui M Large-field-of-view imaging by multi-pupil adaptive optics Nat. Methods 2017 14 581 583 28481364 PMC5482233
  66. [73]Legant WR High-density three-dimensional localization microscopy across large volumes Nat. Methods 2016 13 359 365 26950745 PMC4889433
  67. [74]Power RM Huisken J A guide to light-sheet fluorescence microscopy for multiscale imaging Nat. Methods 2017 14 360 373 28362435
  68. [75]Gradinaru V Treweek J Overton K Deisseroth K Hydrogel-tissue chemistry: principles and applications Annu. Rev. Biophys. 2018 47 355 376 29792820 PMC6359929
  69. [76]Chen F Tillberg PW Boyden ES Expansion microscopy Science 2015 347 543 548 25592419 PMC4312537
  70. [78]Tsai AP INPP5D expression is associated with risk for Alzheimer’s disease and induced by plaque-associated microglia Neurobiol. Dis. 2021 153 105303 33631273 PMC8082515
  71. [79]Olivier N Keller D Gönczy P Manley S Resolution doubling in 3D-STORM imaging through improved buffers PLoS ONE 2013 8 e69004 23874848 PMC3714239
  72. [80]Parthasarathy R Rapid, accurate particle tracking by calculation of radial symmetry centers Nat. Methods 2012 9 724 726 22688415
  73. [81]Mcgorty R Kamiyama D Huang B Active microscope stabilization in three dimensions using image correlation Opt. Nano. 2013 2 3 PMC3874277 24380058
  74. [82]Petrov PN Moerner WE Addressing systematic errors in axial distance measurements in single-emitter localization microscopy Opt. Express 2020 28 18616 18632 32672159 PMC7340385

Cited by

Sorted by citations

Authors
Zhang, P., Ma, D., Cheng, X., Tsai, A. P., Tang, Y., Gao, H.-C., Fang, L., Bi, C., Landreth, G. E., Chubykin, A. A., & Huang, F.
Affiliations
  • Purdue University West Lafayette (US)
  • Indiana University School of Medicine (US)
  • Indiana University Indianapolis (US)
Corresponding author
  • Landreth, Gary E.
  • Chubykin, Alexander A.
  • Huang, Fang
ORCID
Venue
Nature Methods
Publisher
Springer Science and Business Media LLC
Published
2023-09-28
Volume
20
Issue
11
Pages
1748-1758
ISSN
  • 1548-7091 (ISSN-L)
  • 1548-7105
DOI
10.1038/s41592-023-02029-0
PMC
PMC10630144
OpenAlex
W4387130407
Cited by
42
References
5
Citations by year
2026: 19 · 2025: 14 · 2024: 9
FWCI
4.88
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Methods (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • IUScholarWorks (Indiana University) (submittedVersion, cc-by)
Funders
  • U.S. Department of Health & Human Services | NIH | National Institute of General Medical Sciences (GM119785)
  • U.S. Department of Health & Human Services | NIH | National Institute of Mental Health (MH123401)
  • U.S. Department of Health & Human Services | NIH | National Institute on Aging (AG074566)
Topics
  • Physical Sciences › Physics and Astronomy › Atomic and Molecular Physics, and Optics › Adaptive optics and wavefront sensing
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Advanced Fluorescence Microscopy Techniques
  • Physical Sciences › Physics and Astronomy › Atomic and Molecular Physics, and Optics › Digital Holography and Microscopy
Keywords
adaptive optics, single-molecule localization microscopy, deformable mirror, aberration correction, deep neural network, inhomogeneous refractive index, brain tissue imaging
MeSH terms
Deep Learning, Brain, Microscopy, Optics and Photonics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review