Fine-Tuning and training of densenet for histopathology image representation using TCGA diagnostic slides
Open accessAbstract
Feature vectors provided by pre-trained deep artificial neural networks have become a dominant source for image representation in recent literature. Their contribution to the performance of image analysis can be improved through fine-tuning. As an ultimate solution, one might even train a deep network from scratch with the domain-relevant images, a highly desirable option which is generally impeded in pathology by lack of labeled images and the computational expense. In this study, we propose a new network, namely KimiaNet, that employs the topology of the DenseNet with four dense blocks, fine-tuned and trained with histopathology images in different configurations. We used more than 240,000 image patches with 1000×1000 pixels acquired at 20× magnification through our proposed "high-cellularity mosaic" approach to enable the usage of weak labels of 7126 whole slide images of formalin-fixed paraffin-embedded human pathology samples publicly available through The Cancer Genome Atlas (TCGA) repository. We tested KimiaNet using three public datasets, namely TCGA, endometrial cancer images, and colorectal cancer images by evaluating the performance of search and classification when corresponding features of different networks are used for image representation. As well, we designed and trained multiple convolutional batch-normalized ReLU (CBR) networks. The results show that KimiaNet provides superior results compared to the original DenseNet and smaller CBR networks when used as feature extractor to represent histopathology images.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528162 · Cited by 5,759 · see citing works
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · see citing works
Digital pathology and artificial intelligence
Niazi, M. K. K., Parwani, A. V., & Gurcan, M. N. (2019) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30154-8 · Cited by 1,232 · see citing works
CNN Features off-the-shelf: an Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · arXiv · doi:10.48550/arxiv.1403.6382 · Cited by 607 · see citing works
Detecting Cancer Metastases on Gigapixel Pathology Images
Liu, Y., Gadepalli, K., Norouzi, M., Dahl, G. E., Kohlberger, T., Boyko, A., Venugopalan, S., Timofeev, A., Nelson, P. Q., Corrado, G. S., et al. (2017) · arXiv · doi:10.48550/arxiv.1703.02442 · Cited by 506 · see citing works
Flagged first, then by citations
Research works printed but not linked: 33
- Armenteros, J.J. , Sønderby, C.K. , Sønderby, S.K. , Nielsen, H. , Winther, O. , 2017. Deeploc: prediction of protein subcellular localization using deep learn- ing. Bioinformatics 33 (21), 3387-3395 .
- Bilaloglu, S. , Wu, J. , Fierro, E. , Sanchez, R.D. , Ocampo, P.S. , Razavian, N. , Coudray, N. , Tsirigos, A. , 2019. Efficient pan-cancer whole-slide image classification and out- lier detection using convolutional neural networks. bioRxiv 633123 .
- Burt, J.R. , Torosdagli, N. , Khosravan, N. , RaviPrakash, H. , Mortazi, A. , Tissavirasing- ham, F. , Hussein, S. , Bagci, U. , 2018. Deep learning beyond cats and dogs: recent advances in diagnosing breast cancer with deep neural networks. Br. J. Radiol. 91 (1089), 20170545 .
- Campanella, G. , Hanna, M.G. , Geneslaw, L. , Miraflor, A. , Silva, V.W.K. , Busam, K. , Brogi, E. , Reuter, V.E. , Klimstra, D.S. , Fuchs, T.J. , 2019. Clinical-grade computa- tional pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25 (8), 1301-1309 .
- Cooper, L.A. , Demicco, E.G. , Saltz, J.H. , Powell, R.T. , Rao, A. , Lazar, A.J. , 2018. Pan- cancer insights from the cancer genome atlas: the pathologist's perspective. J. Pathol. 244 (5), 512-524 .
- Coudray, N. , Ocampo, P.S. , Sakellaropoulos, T. , Narula, N. , Snuderl, M. , Fenyö, D. , Mor- eira, A.L. , Razavian, N. , Tsirigos, A. , 2018. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24 (10), 1559 .
- Deng, J. , Dong, W. , Socher, R. , Li, L.-J. , Li, K. , Fei-Fei, L. , 2009. Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. Ieee, pp. 248-255 .
- Faust, K. , Bala, S. , van Ommeren, R. , Portante, A. , Al Qawahmed, R. , Djuric, U. , Dia- mandis, P. , 2019. Intelligent feature engineering and ontological mapping of brain tumour histomorphologies by deep learning. Nature Machine Intelligence 1 (7), 316-321 .
- Faust, K. , Xie, Q. , Han, D. , Goyle, K. , Volynskaya, Z. , Djuric, U. , Diamandis, P. , 2018. Visualizing histopathologic deep learning classification and anomaly detection using nonlinear feature space dimensionality reduction. BMC Bioinformatics 19 (1), 173 .
- Fu, Y. , Jung, A.W. , Torne, R.V. , Gonzalez, S. , Vohringer, H. , Jimenez-Linan, M. , Moore, L. , Gerstung, M. , 2019. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. bioRxiv 813543 .
- Glorot, X. , Bengio, Y. , 2010. Understanding the difficulty of training deep feedfor- ward neural networks. In: Proceedings of the thirteenth international confer- ence on artificial intelligence and statistics, pp. 249-256 .
- Gurcan, M.N. , Boucheron, L. , Can, A. , Madabhushi, A. , Rajpoot, N. , Yener, B. , 2009. Histopathological image analysis: a review. IEEE Rev Biomed Eng 2, 147 .
- Gutman, D.A. , Cobb, J. , Somanna, D. , Park, Y. , Wang, F. , Kurc, T. , Saltz, J.H. , Brat, D.J. , Cooper, L.A. , Kong, J. , 2013. Cancer digital slide archive: an informatics resource to support integrated in silico analysis of tcga pathology data. Journal of the American Medical Informatics Association 20 (6), 1091-1098 .
- Huang, G. , Liu, Z. , Van Der Maaten, L. , Weinberger, K.Q. , 2017. Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vi- sion and pattern recognition, pp. 4700-4708 .
- Janowczyk, A . , Madabhushi, A . , 2016. Deep learning for digital pathology image analysis: acomprehensive tutorial with selected use cases. J. Pathol. Inform. 7 . Jegou, H. , Perronnin, F. , Douze, M. , Sánchez, J. , Perez, P. , Schmid, C. , 2011. Aggregat- ing local image descriptors into compact codes. IEEE Trans. Pattern Anal. Mach. Intell. 34 (9), 1704-1716 .
- Kieffer, B. , Babaie, M. , Kalra, S. , Tizhoosh, H.R. , 2017. Convolutional neural networks for histopathology image classification: Training vs. using pre-trained networks. In: 2017 Seventh International Conference on Image Processing Theory, Tools and Applications (IPTA). IEEE, pp. 1-6 .
- Kumar, M.D., Babaie, M., Tizhoosh, H.R., 2018. Deep barcodes for fast retrieval of histopathology scans. In: 2018 International Joint Conference on Neural Net- works (IJCNN), pp. 1-8. doi: 10.1109/IJCNN.2018.8489574 . · DOI
- Kumar, M.D. , Babaie, M. , Zhu, S. , Kalra, S. , Tizhoosh, H.R. , 2017. A comparative study of cnn, bovw and lbp for classification of histopathological images. In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, pp. 1-7 .
- Mormont, R. , Geurts, P. , Marée, R. , 2018. Comparison of deep transfer learning strategies for digital pathology. In: Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition Workshops, pp. 2262-2271 .
- Najafabadi, M.M. , Villanustre, F. , Khoshgoftaar, T.M. , Seliya, N. , Wald, R. , Muharemagic, E. , 2015. Deep learning applications and challenges in big data analytics. J. Big Data 2 (1), 1 .
- Niazi, M.K.K. , Parwani, A.V. , Gurcan, M.N. , 2019. Digital pathology and artificial in- telligence. The Lancet Oncology 20 (5), e253-e261 .
- Onder, D. , Zengin, S. , Sarioglu, S. , 2014. A review on color normalization and color deconvolution methods in histopathology. Applied Immunohistochemistry & Molecular Morphology 22 (10), 713-719 .
- Phan, H.T.H. , Kumar, A. , Kim, J. , Feng, D. , 2016. Transfer learning of a convolutional neural network for hep-2 cell image classification. In: 2016 IEEE 13th Interna- tional Symposium on Biomedical Imaging (ISBI). IEEE, pp. 1208-1211 .
- Raghu, M. , Zhang, C. , Kleinberg, J. , Bengio, S. , 2019. Transfusion: Understanding transfer learning for medical imaging. In: Advances in Neural Information Pro- cessing Systems, pp. 3342-3352 .
- Spanhol, F.A., Oliveira, L.S., Cavalin, P.R., Petitjean, C., Heutte, L., 2017. Deep fea- tures for breast cancer histopathological image classification. In: 2017 IEEE In- ternational Conference on Systems, Man, and Cybernetics (SMC), pp. 1868-1873. doi: 10.1109/SMC.2017.8122889 . · DOI
- Srivastava, R.K. , Greff, K. , Schmidhuber, J. , 2015. Training very deep networks. In: Advances in neural information processing systems, pp. 2377-2385 .
- Sun, H., Zeng, X., Xu, T., Peng, G., Ma, Y., 2019. Computer-aided diagnosis in histopathological images of the endometrium using a convolutional neural net- work and attention mechanisms. IEEE J. Biomed. Health Inform. doi: 10.1109/ JBHI.2019.2944977 . 1-1 · DOI
- Tizhoosh, H.R. , Zhu, S. , Lo, H. , Chaudhari, V. , Mehdi, T. , 2016. Minmax radon bar- codes for medical image retrieval. In: International Symposium on Visual Com- puting. Springer, pp. 617-627 .
- Travis, W.D. , 2014. Pathology and diagnosis of neuroendocrine tumors: lung neu- roendocrine. Thorac. Surg. Clin. 24 (3), 257-266 .
- Watanabe, S. , Hori, T. , Le Roux, J. , Hershey, J.R. , 2017. Student-teacher network learn- ing with enhanced features. In: 2017 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP). IEEE, pp. 5275-5279 .
- Wei, J.W. , Tafe, L.J. , Linnik, Y.A. , Vaickus, L.J. , Tomita, N. , Hassanpour, S. , 2019. Pathol- ogist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks. Sci. Rep. 9 (1), 3358 .
- Zhang, J. , Lu, C. , Li, X. , Kim, H.-J. , Wang, J. , 2019. A full convolutional network based on densenet for remote sensing scene classification. Math. Biosci. Eng 16 (5), 3345-3367 .
- Zhang, K. , Guo, Y. , Wang, X. , Yuan, J. , Ding, Q. , 2019. Multiple feature reweight densenet for image classification. IEEE Access 7, 9872-9880 .
Cited by
Sorted by citations
Towards a general-purpose foundation model for computational pathology
Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., Chen, B., Zhang, A., Shao, D., Shaban, M., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02857-3 · Cited by 1,560 · see citing works
A visual-language foundation model for computational pathology
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L. P., Gerber, G., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02856-4 · Cited by 966 · see citing works
Transformer-based unsupervised contrastive learning for histopathological image classification
Wang, X., Yang, S., Zhang, J., Wang, M., Zhang, J., Yang, W., Huang, J., & Han, X. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102559 · Cited by 751 · see citing works
A multimodal whole-slide foundation model for pathology
Ding, T., Wagner, S. J., Song, A. H., Chen, R. J., Lu, M. Y., Zhang, A., Vaidya, A. J., Jaume, G., Shaban, M., Kim, A., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03982-3 · Cited by 171 · see citing works
“Developed using self-supervised learning (SSL) on millions of histology image patches (or regions of interests), these models capture morphological patterns in histology patch embeddings, such as tissue organization and cellular structure 2–17 .”
Fast and scalable search of whole-slide images via self-supervised deep learning
Chen, C., Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Schaumberg, A. J., & Mahmood, F. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00929-8 · Cited by 130 · see citing works
BreastScreening-AI: Evaluating medical intelligent agents for human-AI interactions
Calisto, F. M., Santiago, C., Nunes, N., & Nascimento, J. C. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102285 · Cited by 125 · see citing works
Biased data, biased AI: deep networks predict the acquisition site of TCGA images
Dehkharghanian, T., Bidgoli, A. A., Riasatian, A., Mazaheri, P., Campbell, C. J. V., Pantanowitz, L., Tizhoosh, H. R., & Rahnamayan, S. (2023) · Diagnostic Pathology · doi:10.1186/s13000-023-01355-3 · Cited by 67 · see citing works
Self-citationHistopathology Slide Indexing and Search — Are We There Yet?
Shang, H. H., Nasr, M. S., Veerla, J. P., Saurav, J. R., Hajighasemi, A., Malidarreh, P., Huber, M., Moleta, C., Makker, J., & Luber, J. M. (2024) · NEJM AI · doi:10.1056/aics2300019 · Cited by 5 · see citing works
- Authors
- Riasatian, A., Babaie, M., Maleki, D., Kalra, S., Valipour, M., Hemati, S., Zaveri, M., Safarpoor, A., Shafiei, S., Afshari, M., Rasoolijaberi, M., Sikaroudi, M., Adnan, M., Shah, S., Choi, C., Damaskinos, S., Campbell, C. J., Diamandis, P., Pantanowitz, L., Kashani, H., Ghodsi, A., & Tizhoosh, H. R.
- Affiliations
- University of Waterloo (CA)
- McMaster University (CA)
- University of Toronto (CA)
- University of Pittsburgh Medical Center (US)
- Corresponding author
- Babaie, Morteza
- ORCID
- Venue
- Medical Image Analysis
- Publisher
- Elsevier BV
- Published
- 2021-05
- Volume
- 70
- Pages
- 102032
- ISSN
- 1361-8415 (ISSN-L)
- OpenAlex
- W3138973186
- Cited by
- 221
- References
- 5
- Citations by year
- 2026: 30 · 2025: 60 · 2024: 49 · 2023: 42 · 2022: 27 · 2021: 13
- FWCI
- 11.58
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Medical Image Analysis (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Government of Ontario
- Natural Sciences and Engineering Research Council of Canada
- Ontario Research Foundation
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- DenseNet, fine-tuning, whole slide images, TCGA, weak labels, feature extraction, endometrial cancer, colorectal cancer
- MeSH terms
- Humans, Image Processing, Computer-Assisted, Neoplasms, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
