CoChat
Sign Up
Cited by 811Cell Image Analysis TechniquesDigital Imaging for Blood DiseasesAdvanced Neural Network Applications

The DOI matches its Crossref record; 48 of the 49 references it cites were found in a registry or an index.

Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl

Open access
Caicedo, Juan C., Goodman, Allen, Karhohs, Kyle W., Cimini, Beth A., Ackerman, Jeanelle, Haghighi, Marzieh, Heng, CherKeng, Becker, Tim, Doan, Minh, McQuin, Claire, Rohban, Mohammad, Singh, Shantanu, Carpenter, Anne E.·2019·Nature Methods
DOI 10.1038/s41592-019-0612-7PMC PMC6919559
Open this paper in CoChat — free accountOpen PDF

Abstract

Segmenting the nuclei of cells in microscopy images is often the first step in the quantitative analysis of imaging data for biological and biomedical applications. Many bioimage analysis tools can segment nuclei in images but need to be selected and configured for every experiment. The 2018 Data Science Bowl attracted 3,891 teams worldwide to make the first attempt to build a segmentation method that could be applied to any two-dimensional light microscopy image of stained nuclei across experiments, with no human interaction. Top participants in the challenge succeeded in this task, developing deep-learning-based models that identified cell nuclei across many image types and experimental conditions without the need to manually adjust segmentation parameters. This represents an important step toward configuration-free bioimage analysis software tools.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41592-019-0612-7

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Accuracy and usability of segmentation strategies in the second-stage holdout sets. a, The histogram counts participant teams (n = 739) according to the official competition score of their best submission. The top five competitors are labeled in the distribution, as is the reference segmentation obtained by an expert analyst using CellProfiler. b, Accuracy of the top three solutions measured as the F1 score at multiple IoU thresholds. The scale of the x axis of the histogram in panel a (competition score) is correlated with the area under the curve of the F1 score versus IoU thresholds. The top three models had a similar performance with slight differences at the tails of the curves. c, Breakdown of accuracy in the second-stage evaluation set for the top performing model and three reference solutions. The distribution of F1-scores at a single IoU threshold (IoU = 0.7) shows points (n = 106) that each represented the segmentation accuracy of one image in the set of 106 annotated images of the second-stage evaluation (Methods). The color of single-image points corresponds to the group of images defined for reference evaluations (Methods and Fig. 2). The average of the distribution is marked with a larger point labeled with the corresponding average accuracy value. d, Estimated time required to configure the segmentation tools evaluated in c (Supplementary Note 4).
Fig. 1 Accuracy and usability of segmentation strategies in the second-stage holdout sets. a, The histogram counts participant teams (n = 739) according to the official competition score of their best submission. The top five competitors are labeled in the distribution, as is the reference segmentation obtained by an expert analyst using CellProfiler. b, Accuracy of the top three solutions measured as the F1 score at multiple IoU thresholds. The scale of the x axis of the histogram in panel a (competition score) is correlated with the area under the curve of the F1 score versus IoU thresholds. The top three models had a similar performance with slight differences at the tails of the curves. c, Breakdown of accuracy in the second-stage evaluation set for the top performing model and three reference solutions. The distribution of F1-scores at a single IoU threshold (IoU = 0.7) shows points (n = 106) that each represented the segmentation accuracy of one image in the set of 106 annotated images of the second-stage evaluation (Methods). The color of single-image points corresponds to the group of images defined for reference evaluations (Methods and Fig. 2). The average of the distribution is marked with a larger point labeled with the corresponding average accuracy value. d, Estimated time required to configure the segmentation tools evaluated in c (Supplementary Note 4).CC BY · PubMed Central
Fig. 2 Performance of submitted solutions across varying, imbalanced image types. a, Example images of the five visually grouped image types (Methods) are shown across the bottom and the chart shows the spread of F1 scores (Methods) across all second-stage submissions. F1 scores were measured at a threshold of 0.7 IoU (Methods). Box plot: center line, median; box limits, upper and lower quartiles; whiskers, 1.5× interquartile range; points, outliers; colored points, top three participants. b, The distribution of the various image types is shown, color-coded as in a. The top competitors segmented all image types with high accuracy despite the imbalance of examples in the training set. c, Detail of accuracy results by image types and object coverage (IoU) thresholds. The x axis displays IoU thresholds and the y axis represents accuracy measured with F1 scores. For each participant, the plot displays five curves showing the trend of segmentation accuracy at different object coverage thresholds.
Fig. 2 Performance of submitted solutions across varying, imbalanced image types. a, Example images of the five visually grouped image types (Methods) are shown across the bottom and the chart shows the spread of F1 scores (Methods) across all second-stage submissions. F1 scores were measured at a threshold of 0.7 IoU (Methods). Box plot: center line, median; box limits, upper and lower quartiles; whiskers, 1.5× interquartile range; points, outliers; colored points, top three participants. b, The distribution of the various image types is shown, color-coded as in a. The top competitors segmented all image types with high accuracy despite the imbalance of examples in the training set. c, Detail of accuracy results by image types and object coverage (IoU) thresholds. The x axis displays IoU thresholds and the y axis represents accuracy measured with F1 scores. For each participant, the plot displays five curves showing the trend of segmentation accuracy at different object coverage thresholds.CC BY · PubMed Central
Fig. 3 Example segmentation maps for various images obtained by the top three participants and the CellProfiler reference. The segmentation maps show pixel-wise alignments between target segmentation masks and predicted segmentations. If the masks align correctly, pixels in the boundaries are colored white. If the target mask or part of it is missing, pixels in the boundaries are colored blue. If the predicted segmentation is introduced in a region without real object, the boundary pixels are red.
Fig. 3 Example segmentation maps for various images obtained by the top three participants and the CellProfiler reference. The segmentation maps show pixel-wise alignments between target segmentation masks and predicted segmentations. If the masks align correctly, pixels in the boundaries are colored white. If the target mask or part of it is missing, pixels in the boundaries are colored blue. If the predicted segmentation is introduced in a region without real object, the boundary pixels are red.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 3

  1. Hollandi, R. et al. A deep learning framework for nucleus segmentation using image style transfer. Preprint at bioRxiv https://doi.org/10.1101/580605 (2019). · DOI
  2. Bolukbasi, T., Chang, K.-W., Zou, J. Y., Saligrama, V. & Kalai, A. T. in Advances in Neural Information Processing Systems (eds Lee, D. D. et al.) 4349–4357 (papers.nips.cc, 2016).
  3. Ren, S., He, K., Girshick, R. & Sun, J. in Advances in Neural Information Processing Systems 28 (eds. Cortes, C. et al.) 91–99 (Curran Associates, 2015).

Cited by

Sorted by citations

See all 811

Related

See all

Metadata

Authors
Caicedo, J. C., Goodman, A., Karhohs, K. W., Cimini, B. A., Ackerman, J., Haghighi, M., Heng, C., Becker, T., Doan, M., McQuin, C., Rohban, M., Singh, S., & Carpenter, A. E.
Affiliations
Broad Institute (US)
Corresponding author
Carpenter, Anne E.
ORCID
Venue
Nature Methods
Publisher
Springer Science and Business Media LLC
Published
2019-10-21
Volume
16
Issue
12
Pages
1247-1253
ISSN
  • 1548-7091 (ISSN-L)
  • 1548-7105
DOI
10.1038/s41592-019-0612-7
PMC
PMC6919559
OpenAlex
W2980998394
Cited by
809
References
49
Citations by year
2026: 86 · 2025: 142 · 2024: 150 · 2023: 144 · 2022: 124 · 2021: 114 · 2020: 44 · 2019: 4
FWCI
34.63
Type
journal article
Language
en
Update notices
correction (2020-01-22)
Open access
Licence
Locations
  • Nature Methods (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Funders
  • Eli Lilly and Company
  • Nvidia
  • Broad Institute
  • National Institutes of Health
  • National Institute of General Medical Sciences (R35GM122547)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Cell Image Analysis Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Imaging for Blood Diseases
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
nuclei segmentation, bioimage analysis, deep learning, microscopy image analysis, automated image analysis
MeSH terms
Data Science, Cell Nucleus, Humans, Image Processing, Computer-Assisted, Microscopy, Fluorescence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review