The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index.
Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases
Open accessAbstract
BACKGROUND: Deep learning (DL) is a representation learning approach ideally suited for image analysis challenges in digital pathology (DP). The variety of image analysis tasks in the context of DP includes detection and counting (e.g., mitotic events), segmentation (e.g., nuclei), and tissue classification (e.g., cancerous vs. non-cancerous). Unfortunately, issues with slide preparation, variations in staining and scanning across sites, and vendor platforms, as well as biological variance, such as the presentation of different grades of disease, make these image analysis tasks particularly challenging. Traditional approaches, wherein domain-specific cues are manually identified and developed into task-specific "handcrafted" features, can require extensive tuning to accommodate these variances. However, DL takes a more domain agnostic approach combining both feature discovery and implementation to maximally discriminate between the classes of interest. While DL approaches have performed well in a few DP related image analysis tasks, such as detection and tissue classification, the currently available open source tools and tutorials do not provide guidance on challenges such as (a) selecting appropriate magnification, (b) managing errors in annotations in the training (or learning) dataset, and (c) identifying a suitable training set containing information rich exemplars. These foundational concepts, which are needed to successfully translate the DL paradigm to DP tasks, are non-trivial for (i) DL experts with minimal digital histology experience, and (ii) DP and image processing experts with minimal DL experience, to derive on their own, thus meriting a dedicated tutorial. AIMS: This paper investigates these concepts through seven unique DP tasks as use cases to elucidate techniques needed to produce comparable, and in many cases, superior to results from the state-of-the-art hand-crafted feature-based classification approaches. RESULTS: Specifically, in this tutorial on DL for DP image analysis, we show how an open source framework (Caffe), with a singular network architecture, can be used to address: (a) nuclei segmentation (F-score of 0.83 across 12,000 nuclei), (b) epithelium segmentation (F-score of 0.84 across 1735 regions), (c) tubule segmentation (F-score of 0.83 from 795 tubules), (d) lymphocyte detection (F-score of 0.90 across 3064 lymphocytes), (e) mitosis detection (F-score of 0.53 across 550 mitotic events), (f) invasive ductal carcinoma detection (F-score of 0.7648 on 50 k testing patches), and (g) lymphoma classification (classification accuracy of 0.97 across 374 images). CONCLUSION: This paper represents the largest comprehensive study of DL approaches in DP to date, with over 1200 DP images used during evaluation. The supplemental online material that accompanies this paper consists of step-by-step instructions for the usage of the supplied source code, trained models, and input data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
(2014) · Cited by 33,419
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
(2010) · International Conference on Machine Learning · Cited by 13,995
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
(2010) · Cited by 8,261
(2011) · Cited by 6,993
Lee, H., Grosse, R., Ranganath, R., & Ng, A. Y. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553453 · Cited by 2,412
Convolutional networks and applications in vision
LeCun, Y., Kavukcuoglu, K., & Farabet, C. (2010) · Proceedings of 2010 IEEE International Symposium on Circuits and Systems · doi:10.1109/iscas.2010.5537907 · Cited by 2,213 · cited 1×
“In past approaches, [34] a sigmoid function was typically used, but more recent implementations [35,36] have shown that a rectified linear (ReLu) activation has more favorable properties.”
MethodHistopathological Image Analysis: A Review
Gurcan, M. N., Boucheron, L. E., Can, A., Madabhushi, A., Rajpoot, N. M., & Yener, B. (2009) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2009.2034865 · Cited by 2,052
Self-citationQuantification of histochemical staining by color deconvolution.
Ruifrok, A. C. C., & Johnston, D. A. (2001) · PubMed · Cited by 2,029
Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks
Cireşan, D. C., Giusti, A., Gambardella, L. M., & Schmidhuber, J. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40763-5_51 · Cited by 1,536 · cited 7×
“In particular, detection of invasive ductal carcinomas (IDCs), [9] mitosis detection, [8] neuron segmentation, [15] colon gland segmentation, [16] nuclei segmentation [17-19] and detection, [20] brain tumor classification, [21] epithelial tumor nuclei identification, [22] epithelium segmentation,…”
BackgroundMethodInfluentialLeCun, Y., Bottou, L., Orr, G., & Müller, K. (1998) · neural information processing systems · Cited by 1,333
Improving deep neural networks for LVCSR using rectified linear units and dropout
Dahl, G. E., Sainath, T. N., & Hinton, G. E. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639346 · Cited by 1,310
(2012) · neural information processing systems · Cited by 1,291
Sirinukunwattana, K., Raza, S. E. A., Tsang, Y.-W., Snead, D. R. J., Cree, I. A., & Rajpoot, N. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2525803 · Cited by 1,224 · cited 1×
“In particular, detection of invasive ductal carcinomas (IDCs),[9] mitosis detection,[8] neuron segmentation,[15] colon gland segmentation,[16] nuclei segmentation[17-19] and detection,[20] brain tumor classification,[21] epithelial tumor nuclei identification,[22] epithelium segmentation,[23] and glioma grading[24] ha…”
MethodTheano: new features and speed improvements
Bastien, F., Lamblin, P., Pascanu, R., Bergstra, J., Goodfellow, I., Bergeron, A., Bouchard, N., Warde-Farley, D., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5590 · Cited by 1,005 · cited 1×
“…identifying a suitable training set containing information rich exemplars, have not been specifically addressed by existing open source tools [11,12] or by the numerous tutorials for DL. [13,14] The previous DL work in DP performed very well in their respective tasks though each required a…”
BackgroundLeCun, Y., Bottou, L., Orr, G. B., & Müller, K. R. (1998) · Lecture Notes in Computer Science · doi:10.1007/3-540-49430-8_2 · Cited by 919
Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images
Xu, J., Xiang, L., Liu, Q., Gilmore, H., Wu, J., Tang, J., & Madabhushi, A. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2015.2458702 · Cited by 854
Self-citationSystematic Analysis of Breast Cancer Morphology Uncovers Stromal Features Associated with Survival
Beck, A. H., Sangoi, A. R., Leung, S., Marinelli, R. J., Nielsen, T. O., van de Vijver, M. J., West, R. B., van de Rijn, M., & Koller, D. (2011) · Science Translational Medicine · doi:10.1126/scitranslmed.3002564 · Cited by 763
Breast Cancer Histopathology Image Analysis: A Review
Veta, M., Pluim, J. P. W., van Diest, P. J., & Viergever, M. A. (2014) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2014.2303852 · Cited by 696
Irshad, H., Veillard, A., Roux, L., & Racoceanu, D. (2014) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2013.2295804 · Cited by 637
Cruz-Roa, A., Basavanhally, A., González, F., Gilmore, H., Feldman, M., Ganesan, S., Shih, N., Tomaszewski, J., & Madabhushi, A. (2014) · SPIE Proceedings · doi:10.1117/12.2043872 · Cited by 573
Self-citationGleason grading and prognostic factors in carcinoma of the prostate
Humphrey, P. A. (2004) · Modern Pathology · doi:10.1038/modpathol.3800054 · Cited by 524
Assessment of algorithms for mitosis detection in breast cancer histopathology images
Veta, M., van Diest, P. J., Willems, S. M., Wang, H., Madabhushi, A., Cruz-Roa, A., Gonzalez, F., Larsen, A. B. L., Vestergaard, J. S., Dahl, A. B., et al. (2015) · Medical Image Analysis · doi:10.1016/j.media.2014.11.010 · Cited by 480
Self-citationXu, J., Luo, X., Wang, G., Gilmore, H., & Madabhushi, A. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.01.034 · Cited by 457
Self-citationWang, H., Cruz-Roa, A., Basavanhally, A., Gilmore, H., Shih, N., Feldman, M., Tomaszewski, J., Gonzalez, F., & Madabhushi, A. (2014) · Journal of Medical Imaging · doi:10.1117/1.jmi.1.3.034003 · Cited by 349
Self-citationMitosis detection in breast cancer histological images An ICPR 2012 contest
Ludovic, R., Daniel, R., Nicolas, L., Maria, K., Humayun, I., Jacques, K., Frédérique, C., Catherine, G., Gilles, L. N., & Metin N, G. (2013) · Journal of Pathology Informatics · doi:10.4103/2153-3539.112693 · Cited by 302
Ertosun, M. G., & Rubin, D. L. (2015) · PubMed · Cited by 269
WND-CHARM: Multi-purpose image classification using compound image transforms
Orlov, N., Shamir, L., Macura, T., Johnston, J., Eckley, D. M., & Goldberg, I. G. (2008) · Pattern Recognition Letters · doi:10.1016/j.patrec.2008.04.013 · Cited by 265
A Stochastic Polygons Model for Glandular Structures in Colon Histology Images
Sirinukunwattana, K., Snead, D. R. J., & Rajpoot, N. M. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2015.2433900 · Cited by 257
Doyle, S., Feldman, M., Tomaszewski, J., & Madabhushi, A. (2012) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2010.2053540 · Cited by 253
Self-citationProceedings of the 24th Annual International Conference on Machine Learning (ICML 2007)
Jaeger, M. (2007) · Cited by 218
Fatakdawala, H., Xu, J., Basavanhally, A., Bhanot, G., Ganesan, S., Feldman, M., Tomaszewski, J. E., & Madabhushi, A. (2010) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2010.2041232 · Cited by 194
Self-citationXu, Y., Jia, Z., Ai, Y., Zhang, F., Lai, M., & Chang, E. I.-C. (2015) · 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2015.7178109 · Cited by 187
Pylearn2: a machine learning research library
Goodfellow, I., Warde-Farley, D., Lamblin, P., Dumoulin, V., Mirza, M., Pascanu, R., Bergstra, J., Bastien, F., & Bengio, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1308.4214 · Cited by 180
Learning to Detect Cells Using Non-overlapping Extremal Regions
Arteta, C., Lempitsky, V., Noble, J. A., & Zisserman, A. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33415-3_43 · Cited by 177
Emerging Themes in Image Informatics and Molecular Analysis for Digital Pathology
Bhargava, R., & Madabhushi, A. (2016) · Annual Review of Biomedical Engineering · doi:10.1146/annurev-bioeng-112415-114722 · Cited by 161
Self-citationGenestie, C., Sigal‐Zafrani, B., Asselain, B. J. R., Fourquet, A., Rozan, S., Validire, P., Vincent‐Salomon, A., & Sastre‐Garau, X. (1998) · PubMed · Cited by 151
Kainz, P., Pfeiffer, M., & Urschler, M. (2017) · PeerJ · doi:10.7717/peerj.3874 · Cited by 148
Deep Learning Based Automatic Immune Cell Detection for Immunohistochemistry Images
Chen, T., & Chefd’hotel, C. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10581-9_3 · Cited by 133
International Symposium on Biomedical Imaging
(2009) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2032059 · Cited by 118
Lewis, J. S., Ali, S., Luo, J. R., Thorstad, W. L., & Madabhushi, A. (2013) · The American Journal of Surgical Pathology · doi:10.1097/pas.0000000000000086 · Cited by 86
Self-citationJanowczyk, A. R., Doyle, S. T., Gilmore, H. L., & Madabhushi, A. (2016) · Computer Methods in Biomechanics and Biomedical Engineering Imaging & Visualization · doi:10.1080/21681163.2016.1141063 · Cited by 66
Self-citationBasavanhally, A. N., Yu, E. Y., Xu, J., et al. (2011) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.878092 · Cited by 65
Basavanhally, A. N., Feldman, M. D., Shih, N. N. C., Mies, C., Tomaszewski, J., Ganesan, S., & Madabhushi, A. (2012) · Journal of Pathology Informatics · doi:10.4103/2153-3539.92027 · Cited by 61
Self-citationKainz, P., Pfeiffer, M., & Urschler, M. (2015) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-121723 · Cited by 54
Fatakdawala, H., Basavanhally, A. N., Xu, J., Bhanot, G., Ganesan, S., Feldman, M. D., Tomaszewski, J., & Madabhushi, A. (2009) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe.2009.75 · Cited by 43
Self-citationSirinukunwattana, K., Raza, S. E. A., Tsang, Y., Snead, D. R. J., Cree, I. A., & Rajpoot, N. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-28194-0_19 · Cited by 36
Nuclei segmentation via sparsity constrained convolutional regression
Zhou, Y., Chang, H., Barner, K. E., & Parvin, B. (2015) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2015.7164109 · Cited by 26
Pattern Recognition in Histopathological Images: An ICPR 2010 Contest
Gürcan, M. N., Madabhushi, A., & Rajpoot, N. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-17711-8_23 · Cited by 21
Self-citationA random polygons model of glandular structures in colon histology images
Sirinukunwattana, K., Snead, D. R. J., & Rajpoot, N. (2015) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2015.7164168 · Cited by 4
Cited by
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Artificial intelligence in radiology
Hosny, A., Parmar, C., Quackenbush, J., Schwartz, L. H., & Aerts, H. J. W. L. (2018) · Nature Reviews Cancer · doi:10.1038/s41568-018-0016-5 · Cited by 3,803
Applications of machine learning in drug discovery and development
Vamathevan, J., Clark, D., Czodrowski, P., Dunham, I., Ferran, E., Lee, G., Li, B., Madabhushi, A., Shah, P., Spitzer, M., et al. (2019) · Nature Reviews Drug Discovery · doi:10.1038/s41573-019-0024-5 · Cited by 3,228
Self-citationA systematic study of the class imbalance problem in convolutional neural networks
Buda, M., Maki, A., & Mazurowski, M. A. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.07.011 · Cited by 3,088
“One of the most commonly used method in deep learning [16, 30, 31, 32].”
MethodClinical-grade computational pathology using weakly supervised deep learning on whole slide images
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., & Fuchs, T. J. (2019) · Nature Medicine · doi:10.1038/s41591-019-0508-1 · Cited by 2,799
Metadata
- Authors
- Janowczyk, A., & Madabhushi, A.
- Venue
- Journal of Pathology Informatics
- Publisher
- Elsevier BV
- Published
- 2016-01
- Volume
- 7
- Issue
- 1
- Pages
- 29
- ISSN
- 2153-3539 (ISSN-L)
- OpenAlex
- W2504150216
- Cited by
- 1,348
- References
- 55
- FWCI
- 84.37
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.