Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images
Open accessThe DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Abstract
Nuclear segmentation and classification within Haematoxylin & Eosin stained histology images is a fundamental prerequisite in the digital pathology work-flow. The development of automated methods for nuclear segmentation and classification enables the quantitative analysis of tens of thousands of nuclei within a whole-slide pathology image, opening up possibilities of further analysis of large-scale nuclear morphometry. However, automated nuclear segmentation and classification is faced with a major challenge in that there are several different types of nuclei, some of them exhibiting large intra-class variability such as the nuclei of tumour cells. Additionally, some of the nuclei are often clustered together. To address these challenges, we present a novel convolutional neural network for simultaneous nuclear segmentation and classification that leverages the instance-rich information encoded within the vertical and horizontal distances of nuclear pixels to their centres of mass. These distances are then utilised to separate clustered nuclei, resulting in an accurate segmentation, particularly in areas with overlapping instances. Then, for each segmented instance the network predicts the type of nucleus via a devoted up-sampling branch. We demonstrate state-of-the-art performance compared to other methods on multiple independent multi-tissue histology image datasets. As part of this work, we introduce a new dataset of Haematoxylin & Eosin stained colorectal adenocarcinoma image tiles, containing 24,319 exhaustively annotated nuclei with associated class labels.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
QuPath: Open source software for digital pathology image analysis
Bankhead, P., Loughrey, M. B., Fernández, J. A., Dombrowski, Y., McArt, D. G., Dunne, P. D., McQuaid, S., Gray, R. T., Murray, L. J., Coleman, H. G., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-17204-5 · Cited by 8,908 · cited 6×
“We also compared the performance of our model to two open source software applications: Cell Profiler [42] and QuPath [43].”
BackgroundMethodInfluentialCellProfiler: image analysis software for identifying and quantifying cell phenotypes
Carpenter, A. E., Jones, T. R., Lamprecht, M. R., Clarke, C., Kang, I. H., Friman, O., Guertin, D. A., Chang, J. H., Lindquist, R. A., Moffat, J., et al. (2006) · Genome Biology · doi:10.1186/gb-2006-7-10-r100 · Cited by 5,157
Deep learning in medical image analysis
Shen, D., Wu, G., & Suk, H.-I. (2017) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-071516-044442 · Cited by 4,968
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2844175 · Cited by 3,457
Histopathological Image Analysis: A Review
Gurcan, M. N., Boucheron, L. E., Can, A., Madabhushi, A., Rajpoot, N. M., & Yener, B. (2009) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2009.2034865 · Cited by 2,052
Self-citationKirillov, A., He, K., Girshick, R., Rother, C., & Dollár, P. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00963 · Cited by 1,659
Mask R-CNNPreprint
He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017) · arXiv (Cornell University) · Cited by 1,252
Sirinukunwattana, K., Raza, S. E. A., Tsang, Y.-W., Snead, D. R. J., Cree, I. A., & Rajpoot, N. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2525803 · Cited by 1,224 · cited 3×
“Note, because SC-CNN does not produce a segmentation mask, we do not report the PQ for this method.”
BackgroundInfluentialSelf-citationA Dataset and a Technique for Generalized Nuclear Segmentation for Computational Pathology
Kumar, N., Verma, R., Sharma, S., Bhargava, S., Vahadane, A., & Sethi, A. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2677499 · Cited by 1,126
Image analysis and machine learning in digital pathology: challenges and opportunities
Madabhushi, A., & Lee, G. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.06.037 · Cited by 1,093
Diagnostic Concordance Among Pathologists Interpreting Breast Biopsy Specimens
Elmore, J. G., Longton, G. M., Carney, P. A., Geller, B. M., Onega, T., Tosteson, A. N. A., Nelson, H. D., Pepe, M. S., Allison, K. H., Schnitt, S. J., et al. (2015) · JAMA · doi:10.1001/jama.2015.1405 · Cited by 751
Segmentation of Nuclei in Histopathology Images by Deep Regression of the Distance Map
Naylor, P., Laé, M., Reyal, F., & Walter, T. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2865709 · Cited by 685
DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation
Chen, H., Qi, X., Yu, L., & Heng, P.-A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.273 · Cited by 623
Quantitative Image Analysis of Cellular Heterogeneity in Breast Tumors Complements Genomic Profiling
Yuan, Y., Failmezger, H., Rueda, O. M., Ali, H. R., Gräf, S., Chin, S.-F., Schwarz, R. F., Curtis, C., Dunning, M. J., Bardwell, H., et al. (2012) · Science Translational Medicine · doi:10.1126/scitranslmed.3004330 · Cited by 426
He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06870 · Cited by 406
Yang, X., Li, H., & Zhou, X. (2006) · IEEE Transactions on Circuits and Systems I: Regular Papers · doi:10.1109/tcsi.2006.884469 · Cited by 404
Automatic Nuclei Segmentation in H&E Stained Breast Cancer Histopathology Images
Veta, M., van Diest, P. J., Kornegoor, R., Huisman, A., Viergever, M. A., & Pluim, J. P. W. (2013) · PLoS ONE · doi:10.1371/journal.pone.0070221 · Cited by 369
Methods for Segmentation and Classification of Digital Microscopy Tissue Images
Vu, Q. D., Graham, S., Kurc, T., To, M. N. N., Shaban, M., Qaiser, T., Koohbanani, N. A., Khurram, S. A., Kalpathy-Cramer, J., Zhao, T., et al. (2019) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2019.00053 · Cited by 336
Self-citationOn the Robustness of Semantic Segmentation Models to Adversarial Attacks
Arnab, A., Miksik, O., & Torr, P. H. S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00099 · Cited by 319
Segmentation of Clustered Nuclei With Shape Markers and Marking Function
Cheng, J., & Rajapakse, J. C. (2009) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2008.2008635 · Cited by 302
Automatic cell nuclei segmentation and classification of breast cancer histopathology images
Wang, P., Hu, X., Li, Y., Liu, Q., & Zhu, X. (2016) · Signal Processing · doi:10.1016/j.sigpro.2015.11.011 · Cited by 270
Micro-Net: A unified model for segmentation of various objects in microscopy images
Raza, S. E. A., Cheung, L., Shaban, M., Graham, S., Epstein, D., Pelengaris, S., Khan, M., & Rajpoot, N. M. (2019) · Medical Image Analysis · doi:10.1016/j.media.2018.12.003 · Cited by 259
Self-citationCorredor, G., Wang, X., Zhou, Y., Lu, C., Fu, P., Syrigos, K., Rimm, D. L., Yang, M., Romero, E., Schalper, K. A., et al. (2019) · Clinical Cancer Research · doi:10.1158/1078-0432.ccr-18-2013 · Cited by 247
Ali, S., & Madabhushi, A. (2012) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2012.2190089 · Cited by 243
Detection and Segmentation of Cell Nuclei in Virtual Microscopy Images: A Minimum-Model Approach
Wienert, S., Heim, D., Saeger, K., Stenzinger, A., Beil, M., Hufnagl, P., Dietel, M., Denkert, C., & Klauschen, F. (2012) · Scientific Reports · doi:10.1038/srep00503 · Cited by 236
Nuclei segmentation in histopathology images using deep neural networks
Naylor, P., Lae, M., Reyal, F., & Walter, T. (2017) · 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) · doi:10.1109/isbi.2017.7950669 · Cited by 202
CIA-Net: Robust Nuclei Instance Segmentation with Contour-Aware Information Aggregation
Zhou, Y., Onder, O. F., Dou, Q., Tsougenis, E., Chen, H., & Heng, P.-A. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-20351-1_53 · Cited by 196
Zwerger, M., Ho, C. Y., & Lammerding, J. (2011) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-071910-124736 · Cited by 163
Lu, C., Romo-Bucheli, D., Wang, X., Janowczyk, A., Ganesan, S., Gilmore, H., Rimm, D., & Madabhushi, A. (2018) · Laboratory Investigation · doi:10.1038/s41374-018-0095-7 · Cited by 158
A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images
Cui, Y., Zhang, G., Liu, Z., Xiong, Z., & Hu, J. (2019) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-019-02008-8 · Cited by 138
Prostate cancer detection: Fusion of cytological and textural features
Nguyen, K., Jain, A. K., & Sabata, B. (2012) · Journal of Pathology Informatics · doi:10.4103/2153-3539.92030 · Cited by 73
SAMS-NET: Stain-aware multi-scale network for instance-based nuclei segmentation in histology images
Graham, S., & Rajpoot, N. (2018) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2018.8363645 · Cited by 65
Self-citationSharma, H., Zerbe, N., Heim, D., Wienert, S., Behrens, H., Hellwich, O., & Hufnagl, P. (2015) · International Conference on Computer Vision Theory and Applications · doi:10.5220/0005247900370046 · Cited by 61
Novel digital signatures of tissue phenotypes for predicting distant metastasis in colorectal cancer
Sirinukunwattana, K., Snead, D. R. J., Epstein, D. B. A., Aftab, Z., Mujeeb, I., Tsang, Y. W., Cree, I. A., & Rajpoot, N. (2018) · Scientific Reports · doi:10.1038/s41598-018-31799-3 · Cited by 54
Self-citationSegmentation of neuronal nuclei based on clump splitting and a two-step binarization of images
LaTorre, A., Alonso‐Nanclares, L., Muelas, S., Peña, J. M., & DeFelipe, J. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.06.010 · Cited by 51
A bottom-up approach for tumour differentiation in whole slide images of lung adenocarcinoma
Alsubaie, N. M., Sirinukunwattana, K., Raza, S. E. A., Snead, D. R. J., & Rajpoot, N. (2018) · SPIE Medical Imaging: Digital and Computational Pathology · doi:10.1117/12.2293316 · Cited by 31
Self-citationCellular Community Detection for Tissue Phenotyping in Histology Images
Javed, S., Fraz, M. M., Epstein, D., Snead, D. R. J., & Rajpoot, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00949-6_15 · Cited by 21
Self-citationDeep Learning Models Delineates Multiple Nuclear Phenotypes in H&E Stained Histology SectionsPreprint
Khoshdeli, M., & Parvin, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04427 · Cited by 11
Nucleus detection using gradient orientation information and linear least squares regression
Kwak, J. T., Hewitt, S. M., Xu, S., Pinto, P. A., & Wood, B. J. (2015) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2081413 · Cited by 8
Self-citation
Research works printed but not linked: 1
- Liao, 2016, Automatic segmentation for cell images based on bottleneck detection and ellipse fitting, Neurocomputing, 173, 615, 10.1016/j.neucom.2015.08.006 · doi:10.1016/j.neucom.2015.08.006
Cited by
Data-efficient and weakly supervised computational pathology on whole-slide images
Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Chen, R. J., Barbieri, M., & Mahmood, F. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-020-00682-w · Cited by 2,282
The Tabula Sapiens: A multiple-organ, single-cell transcriptomic atlas of humans
The Tabula Sapiens Consortium*, Jones, R. C., Karkanias, J., Krasnow, M. A., Pisco, A. O., Quake, S. R., Salzman, J., Yosef, N., Bulthaup, B., Brown, P., et al. (2022) · Science · doi:10.1126/science.abl4896 · Cited by 1,056
Greenwald, N. F., Miller, G., Moen, E., Kong, A., Kagel, A., Dougherty, T., Fullaway, C. C., McIntosh, B. J., Leow, K. X., Schwartz, M. S., et al. (2022) · Nature Biotechnology · doi:10.1038/s41587-021-01094-0 · Cited by 953
A visual-language foundation model for computational pathology
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L. P., Gerber, G., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02856-4 · Cited by 966
Deep learning in histopathology: the path to the clinic
van der Laak, J., Litjens, G., & Ciompi, F. (2021) · Nature Medicine · doi:10.1038/s41591-021-01343-4 · Cited by 902
Metadata
- Authors
- Graham, S., Vu, Q. D., Raza, S. E. A., Azam, A., Tsang, Y. W., Kwak, J. T., & Rajpoot, N.
- Venue
- Medical Image Analysis
- Publisher
- Elsevier BV
- Published
- 2019-12
- Volume
- 58
- Pages
- 101563
- ISSN
- 1361-8415 (ISSN-L)
- OpenAlex
- W2953567336
- Cited by
- 1,423
- References
- 51
- FWCI
- 38.68
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- Funders
- Ministry of Science ICT and Future Planning
- National Research Foundation of Korea
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.