CoChat
Sign Up
Cited by 7Cell Image Analysis TechniquesAdvanced Neural Network ApplicationsMedical Image Segmentation Techniques

The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.

SwinCell: a 3D transformer and flow-based framework for improved cell segmentation

Open access
Zhang, Xiao, Lin, Zihan, Wang, Liguo, Chu, Yong S., Yang, Yang, Xiao, Xianghui, Lin, Yuewei, Liu, Qun·2025·Communications Biology
DOI 10.1038/s42003-025-08397-xPMC PMC12218999

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Abstract Segmentation of three-dimensional (3D) cellular images is fundamental for studying and understanding cell structure and function. However, 3D cellular segmentation is challenging, particularly for dense cells and tissues. This challenge arises mainly from the complex contextual information within 3D images, anisotropic properties, and the sensitivity to internal cellular structures, which often lead to miss-segmentation. In this work, we introduce SwinCell, a 3D transformer-based framework that leverages Swin-transformer for flow prediction and effectively distinguishes individual cell instances in 3D. We demonstrate the broad utility of the SwinCell in the segmentation of nuclei, colon tissue cells, and dense cultured cells. SwinCell strikes a balance between maintaining detailed local feature recognition and understanding broader contextual information. Tested extensively with both public and in-house 3D cell imaging datasets, SwinCell shows superior performance in segmenting dense cells in 3D, making it a powerful 3D segmentation tool for cellular analysis that could expedite research in cell biology and tissue engineering.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s42003-025-08397-x

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 SwinCell workflow. a Image labeling and annotation. Raw 3D cellular images are manually labeled by separating individual cells for subsequent processing. The semantic cell labels are converted to instance labels, which are then transformed into gradient flows using the algorithm described in the Method section. The RGB components of the color-encoded flow map represent gradient flows in the Z, X, and Y directions, respectively. Scalebar is 15 µm. b Model training. The training process involves randomly cropping 3D image patches embedded with their corresponding labels. The model inputs are embedded in raw images, and the outputs are four-channel predictions. The first three channels are gradient flows in Z, X, and Y directions, and the fourth channel is the cell probability map. During model training, a loss function is computed from the model predictions and the ground truth, which is then used to back-propagate errors and update the model’s parameters. c Model architecture. SwinCell utilizes a U-Net-style architecture with four levels. Each level in the encoder section contains a Swin-Unit, and the decoder section utilizes convolutional upsampling. d Model inference. The model takes raw images as input and produces four-channel predictions. Predicted image patches are stitched together to reconstruct the complete cellular image. Gradient tracking is then applied to convert the predicted channels into a cell instance mask. Scalebar is 15 µm.
Fig. 1 SwinCell workflow. a Image labeling and annotation. Raw 3D cellular images are manually labeled by separating individual cells for subsequent processing. The semantic cell labels are converted to instance labels, which are then transformed into gradient flows using the algorithm described in the Method section. The RGB components of the color-encoded flow map represent gradient flows in the Z, X, and Y directions, respectively. Scalebar is 15 µm. b Model training. The training process involves randomly cropping 3D image patches embedded with their corresponding labels. The model inputs are embedded in raw images, and the outputs are four-channel predictions. The first three channels are gradient flows in Z, X, and Y directions, and the fourth channel is the cell probability map. During model training, a loss function is computed from the model predictions and the ground truth, which is then used to back-propagate errors and update the model’s parameters. c Model architecture. SwinCell utilizes a U-Net-style architecture with four levels. Each level in the encoder section contains a Swin-Unit, and the decoder section utilizes convolutional upsampling. d Model inference. The model takes raw images as input and produces four-channel predictions. Predicted image patches are stitched together to reconstruct the complete cellular image. Gradient tracking is then applied to convert the predicted channels into a cell instance mask. Scalebar is 15 µm.CC BY-NC-ND · PubMed Central
Fig. 2 Segmentation performance of SwinCell on the Allen dataset. Representative slices of raw images (a), ground truth masks (b), segmentation using SwinCell (c), segmentation using pre-trained Cellpose model (d) segmentation using fine-tuned Cellpose model (e). The positions of the YZ, XZ slices are indicated by white dashed lines. f Model performance measured by F1 score at different IoU (Intersection over Union) matching thresholds. Two different Cellpose models were selected for comparison, and they are labeled as v1 and v2, respectively. Scalebar is 20 µm.
Fig. 2 Segmentation performance of SwinCell on the Allen dataset. Representative slices of raw images (a), ground truth masks (b), segmentation using SwinCell (c), segmentation using pre-trained Cellpose model (d) segmentation using fine-tuned Cellpose model (e). The positions of the YZ, XZ slices are indicated by white dashed lines. f Model performance measured by F1 score at different IoU (Intersection over Union) matching thresholds. Two different Cellpose models were selected for comparison, and they are labeled as v1 and v2, respectively. Scalebar is 20 µm.CC BY-NC-ND · PubMed Central
Fig. 3 Segmentation performance of SwinCell on the colon dataset. a Representative orthogonal slice and their zoomed-in views. Blue and red boxes highlight Regions of Interest (ROIs) in XY plane and XZ plane, respectively. Scalebar is 20 µm. b Zoomed-in view of segmented masks of SwinCell, Unet-Transformer, Cellpose 2D and 2.5D. Two orthogonal slice views of the segmented cell masks are shown. Different colors were used to define cell boundaries. Scalebar is 5 µm. c Comparison of segmentation performance using F1 score, mean average precision (mAP) and recall. Error bar is the standard deviation (S.D.) of n = 6. d SwinCell segmentation performance against the other four algorithms using the volumes of segmented cells. GT, ground truth. The p-value on top of each violin plot shows a significant level of difference as compared with ground truth. SwinCell and Unet-Transformer predicted cell volumes and diameters are not significant from ground truth, while Cellpose 2D and 2.5D predicted volumes and diameters are significantly different (p < 0.005) from GT. The p-values are derived from n = 459 (GT), 439 (SwinCell), 449 (UNet-Transformer), 1057 (Cellpose 2D), and 859 (Cellpose 2.5D) segmented masks. e SwinCell segmentation performance against other algorithms using the diameters of segmented cells as metric. The width of each violin plot indicates the frequency of diameters within the dataset, with wider sections corresponding to higher number of cell masks with diameters around that value. The p-values are derived from the same data sizes as in (d).
Fig. 3 Segmentation performance of SwinCell on the colon dataset. a Representative orthogonal slice and their zoomed-in views. Blue and red boxes highlight Regions of Interest (ROIs) in XY plane and XZ plane, respectively. Scalebar is 20 µm. b Zoomed-in view of segmented masks of SwinCell, Unet-Transformer, Cellpose 2D and 2.5D. Two orthogonal slice views of the segmented cell masks are shown. Different colors were used to define cell boundaries. Scalebar is 5 µm. c Comparison of segmentation performance using F1 score, mean average precision (mAP) and recall. Error bar is the standard deviation (S.D.) of n = 6. d SwinCell segmentation performance against the other four algorithms using the volumes of segmented cells. GT, ground truth. The p-value on top of each violin plot shows a significant level of difference as compared with ground truth. SwinCell and Unet-Transformer predicted cell volumes and diameters are not significant from ground truth, while Cellpose 2D and 2.5D predicted volumes and diameters are significantly different (p < 0.005) from GT. The p-values are derived from n = 459 (GT), 439 (SwinCell), 449 (UNet-Transformer), 1057 (Cellpose 2D), and 859 (Cellpose 2.5D) segmented masks. e SwinCell segmentation performance against other algorithms using the diameters of segmented cells as metric. The width of each violin plot indicates the frequency of diameters within the dataset, with wider sections corresponding to higher number of cell masks with diameters around that value. The p-values are derived from the same data sizes as in (d).CC BY-NC-ND · PubMed Central
Fig. 4 Segmentation performance of SwinCell on the in-house Nanolive dataset. a Representative views of two 3D cell-image slices of three orthogonal views. The positions of the slices are indicated by dash lines. b Ground truth masks. c Segmented cells masked by SwinCell. d Segmented cells masked by Cellpose. The matched F1 scores were displayed at the bottom right corner of each segmented image. e–g Comparison of the segmentation performance of SwinCell (blue) against Cellpose (red) using F1 score at 0.5 threshold (e) accuracy (f) and recall (g). h Mean averaged precision (mAP) of SwinCell and Cellpose at different matching thresholds.
Fig. 4 Segmentation performance of SwinCell on the in-house Nanolive dataset. a Representative views of two 3D cell-image slices of three orthogonal views. The positions of the slices are indicated by dash lines. b Ground truth masks. c Segmented cells masked by SwinCell. d Segmented cells masked by Cellpose. The matched F1 scores were displayed at the bottom right corner of each segmented image. e–g Comparison of the segmentation performance of SwinCell (blue) against Cellpose (red) using F1 score at 0.5 threshold (e) accuracy (f) and recall (g). h Mean averaged precision (mAP) of SwinCell and Cellpose at different matching thresholds.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 4 references

Research works printed but not linked: 34

  1. Pérez-Montero, A., Zaragoza, O., Luque, A., Hortelano, S. & Acebo, P. Visualization of lipid droplets in the alveolar macrophage cell line MH-S with live-cell imaging by 3D holotomographic microscopy (Nanolive). Bio-Protoc. 13, e4629 (2023). · DOI
  2. Lin, Z. et al. Correlative single-cell hard X-ray computed tomography and X-ray fluorescence imaging. Commun. Biol. 7, 1–9 (2024). · DOI
  3. Hou, Z., Nightingale, F., Zhu, Y., MacGregor-Chatwin, C. & Zhang, P. Structure of native chromatin fibres revealed by Cryo-ET in situ. Nat. Commun. 14, 6324 (2023). · DOI
  4. Chen, M. et al. A complete data processing workflow for cryo-ET and subtomogram averaging. Nat. Methods 16, 1161–1168 (2019). · DOI
  5. Xu, C. S. et al. An open-access volume electron microscopy atlas of whole cells and tissues. Nature 599, 147–151 (2021). · DOI
  6. Lu, M. et al. ERnet: a tool for the semantic segmentation and quantitative analysis of endoplasmic reticulum topology. Nat. Methods 20, 569–579 (2023). · DOI
  7. Jiménez, J., Doerr, S., Martínez-Rosell, G., Rose, A. S. & De Fabritiis, G. DeepSite: protein-binding site predictor using 3D-convolutional neural networks. Bioinforma. Oxf. Engl. 33, 3036–3042 (2017). · DOI
  8. Heinrich, L. et al. Whole-cell organelle segmentation in volume electron microscopy. Nature 599, 141–146 (2021). · DOI
  9. Greenwald, N. F. et al. Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning. Nat. Biotechnol. 40, 555–565 (2022). · DOI · Paper page
  10. Stringer, C., Wang, T., Michaelos, M. & Pachitariu, M. Cellpose: a generalist algorithm for cellular segmentation. Nat. Methods 18, 100–106 (2021). · DOI · Paper page
  11. He, K., Gkioxari, G., Dollár, P. & Girshick, R. Mask R-CNN. Proc. IEEE Int. Conf. Comput. Vis. 2017, 2980–2988 https://doi.ieeecomputersociety.org/10.1109/ICCV.2017.322. (2017). · DOI · Paper page
  12. Li, G. et al. Segmentation of touching cell nuclei using gradient flow tracking. J. Microsc. 231, 47–58 (2008). · DOI
  13. Weigert, M., Schmidt, U., Haase, R., Sugawara, K. & Myers, G. Star-convex polyhedra for 3D object detection and segmentation in microscopy. Proc. IEEE Winter Conf. Appl. Comput. Vis. 2020, 3655–3662 (2020).
  14. Schmidt, U., Weigert, M., Broaddus, C. & Myers, G. Cell detection with star-convex polygons. Lect. Notes Comput. Sci. 11071, 265–273 (2018). · DOI · Paper page
  15. Eisenstein, M. AI under the microscope: the algorithms powering the search for cells. Nature 623, 1095–1097 (2023). · DOI
  16. Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30, 5998–6008 (2017).
  17. Hatamizadeh, A. et al. Swin UNETR: Swin transformers for semantic segmentation of brain tumors in MRI images. Lect. Notes Comput. Sci. 12962, 272–284 (2022). · DOI · Paper page
  18. Çiçek, Ö, Abdulkadir, A., Lienkamp, S. S., Brox, T. & Ronneberger, O. 3D U-Net: Learning dense volumetric segmentation from sparse annotation. Lect. Notes Comput. Sci. 9901, 424–432 (2016). · DOI · Paper page
  19. Ounkomol, C., Seshamani, S., Maleckar, M. M., Collman, F. & Johnson, G. R. Label-free prediction of three-dimensional fluorescence images from transmitted-light microscopy. Nat. Methods 15, 917–920 (2018). · DOI · Paper page
  20. Wagner, F. R. et al. Preparing samples from whole cells using focused-ion-beam milling for cryo-electron tomography. Nat. Protoc. 15, 2041–2070 (2020). · DOI
  21. He, J., Li, T. & Huang, S.-Y. Improvement of cryo-EM maps by simultaneous local and non-local deep learning. Nat. Commun. 14, 3217 (2023). · DOI
  22. Paul, S. & Chen, P.-Y. Vision Transformers Are Robust Learners. Proc. AAAI Conf. Artif. Intell. 36, 2071–2081 (2022).
  23. Tang, Y. et al. Self-supervised pre-training of Swin transformers for 3D medical image analysis. Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. 2022, 20730–20740 (2022).
  24. Wagner, S. J. et al. Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study. Cancer Cell 41, 1650–1661.e4 (2023). · DOI · Paper page
  25. Svoboda, D., Homola, O. & Stejskal, S. Generation of 3D digital phantoms of colon tissue. Lect. Notes Comput. Sci. 6754, 31–39 (2011). · DOI
  26. Paszke, A. et al. PyTorch: An imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst. 32, 8024–8035 (2019).
  27. Kluyver, T. et al. Jupyter Notebooks—a publishing format for reproducible computational workflows. in: (eds Loizides, F. & Scmidt, B.) 87–90 (IOS Press, 2016). https://doi.org/10.3233/978-1-61499-649-1-87. · DOI
  28. Hunter, J. D. Matplotlib: a 2D Graphics Environment. Comput. Sci. Eng. 9, 90–95 (2007). · DOI · Paper page
  29. Waskom, M. L. seaborn: statistical data visualization. J. Open Source Softw. 6, 3021 (2021). · DOI · Paper page
  30. Yushkevich, P. A. et al. User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. NeuroImage 31, 1116–1128 (2006). · DOI · Paper page
  31. Cardoso, M. J. et al. MONAI: an open-source framework for deep learning in healthcare. Preprint at arXiv http://arxiv.org/abs/2211.02701 (2022). · arXiv
  32. Harris, C. R. et al. Array programming with NumPy. Nature 585, 357–362 (2020). · DOI · Paper page
  33. Virtanen, P. et al. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods 17, 261–272 (2020). · DOI · Paper page
  34. Eschweiler, D., Smith, R. S. & Stegmaier, J. Robust 3D cell segmentation: extending the view of Cellpose. Proc. IEEE Int. Conf. Image Process 2022, 9897942 (2022).

Metadata

Authors
Zhang, X., Lin, Z., Wang, L., Chu, Y. S., Yang, Y., Xiao, X., Lin, Y., & Liu, Q.
Affiliations
Brookhaven National Laboratory (US)
Corresponding author
  • Zhang, Xiao
  • Lin, Yuewei
  • Liu, Qun
ORCID
Venue
Communications Biology
Publisher
Springer Science and Business Media LLC
Published
2025-07-01
Volume
8
Issue
1
Pages
962-962
ISSN
2399-3642 (ISSN-L)
DOI
10.1038/s42003-025-08397-x
PMC
PMC12218999
OpenAlex
W4411885557
Cited by
7
References
4
Citations by year
2026: 5 · 2025: 2
FWCI
2.64
Type
journal article
Language
en
Open access
Licence
Locations
  • Communications Biology (publishedVersion, cc-by-nc-nd)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • DOE | SC | Biological and Environmental Research (KP1601011)
  • DOE | LDRD | Brookhaven National Laboratory (LDRD21-013)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Cell Image Analysis Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
Keywords
3D cell segmentation, Swin Transformer, nuclei segmentation, attention mechanism, tissue engineering
MeSH terms
Algorithms, Colon, Humans, Software, Imaging, Three-Dimensional

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: SwinCell: a 3D transformer and flow-based framework for improved cell segmentation. https://app.cochat.ai/papers/10.1038/s42003-025-08397-x
The paper
Zhang, X., Lin, Z., Wang, L., Chu, Y. S., Yang, Y., Xiao, X., Lin, Y., & Liu, Q. (2025). SwinCell: a 3D transformer and flow-based framework for improved cell segmentation. Communications Biology. https://doi.org/10.1038/s42003-025-08397-x