CoChat
Sign Up
Cited by 20

Unsupervised Learning Based on Multiple Descriptors for WSIs Diagnosis

Sheikh, Taimoor Shakeel, Kim, Jee-Yeon, Shim, Jaesool, Cho, Migyung·2022·Diagnostics
DOI 10.3390/diagnostics12061480PMC PMC9222016
Open this paper in CoChat — free account

Abstract

An automatic pathological diagnosis is a challenging task because histopathological images with different cellular heterogeneity representations are sometimes limited. To overcome this, we investigated how the holistic and local appearance features with limited information can be fused to enhance the analysis performance. We propose an unsupervised deep learning model for whole-slide image diagnosis, which uses stacked autoencoders simultaneously feeding multiple-image descriptors such as the histogram of oriented gradients and local binary patterns along with the original image to fuse the heterogeneous features. The pre-trained latent vectors are extracted from each autoencoder, and these fused feature representations are utilized for classification. We observed that training with additional descriptors helps the model to overcome the limitations of multiple variants and the intricate cellular structure of histopathology data by various experiments. Our model outperforms existing state-of-the-art approaches by achieving the highest accuracies of 87.2 for ICIAR2018, 94.6 for Dartmouth, and other significant metrics for public benchmark datasets. Our model does not rely on a specific set of pre-trained features based on classifiers to achieve high performance. Unsupervised spaces are learned from the number of independent multiple descriptors and can be used with different variants of classifiers to classify cancer diseases from whole-slide images. Furthermore, we found that the proposed model classifies the types of breast and lung cancer similar to the viewpoint of pathologists by visualization. We also designed our whole-slide image processing toolbox to extract and process the patches from whole-slide images.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/diagnostics12061480

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Several investigations show that deep learning techniques can assist in detailed histologic classification.

    “Several investigations underline how deep learning techniques can assist in detailed histologic classification [55,56].”

    From Artificial Intelligence in Oncologic Thoracic Surgery: Clinical Decision Support and Emerging Applications (2026) · cites this paper as [55]

  • These studies addressed a five-category classification task-distinguishing lepidic, acinar, papillary, micropapillary, and solid growth patterns-and showed that AI can support accurate subtype identification and deeper characterization of tumor morphology.

    “These studies addressed a complex five-category classification task—distinguishing lepidic, acinar, papillary, micropapillary, and solid growth patterns—and showed that AI can support both accurate subtype identification and a deeper characterization of tumor morphology.”

    From Artificial Intelligence in Oncologic Thoracic Surgery: Clinical Decision Support and Emerging Applications (2026) · cites this paper as [55]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 48

  1. [1]Shamshirband S. Fathi M. Dehzangi A. Chronopoulos A.T. Alinejad-Rokny H. A review on deep learning approaches in healthcare systems: Taxonomies, challenges, and open issues J. Biomed. Inform. 2021 113 103627 10.1016/j.jbi.2020.103627 33259944 · DOI
  2. [2]Zhang Y. Wei Y. Wu Q. Zhao P. Niu S. Huang J. Tan M. Collaborative unsupervised domain adaptation for medical image diagnosis IEEE Trans. Image Process. 2020 29 7834 7844 10.1109/TIP.2020.3006377 · DOI
  3. [3]Lee J. Warner E. Shaikhouni S. Bitzer M. Kretzler M. Gipson D. Pennathur S. Bellovich K. Bhat Z. Gadegbeku C. Unsupervised machine learning for identifying important visual features through bag-of-words using histopathology data from chronic kidney disease Sci. Rep. 2022 12 1 13 10.1038/s41598-022-08974-8 35318420 PMC8941143 · DOI
  4. [4]Yan J. Chen H. Li X. Yao J. Deep contrastive learning based tissue clustering for annotation-free histopathology image analysis Comput. Med. Imaging Graph. 2022 97 102053 10.1016/j.compmedimag.2022.102053 35306442 · DOI
  5. [5]Chikontwe P. Kim M. Nam S.J. Go H. Park S.H. Multiple instance learning with center embeddings for histopathology classification Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention Lima, Peru 4–8 October 2020 Springer Berlin/Heidelberg, Germany 2020 519 528
  6. [6]Wang X. Han T.X. Yan S. An HOG-LBP human detector with partial occlusion handling Proceedings of the 2009 IEEE 12th International Conference on Computer Vision Kyoto, Japan 27 September–4 October 2009 IEEE Piscataway, NJ, USA 2009 32 39
  7. [8]Ahonen T. Hadid A. Pietikainen M. Face description with local binary patterns: Application to face recognition IEEE Trans. Pattern Anal. Mach. Intell. 2006 28 2037 2041 10.1109/TPAMI.2006.244 17108377 · DOI
  8. [9]Yu K.H. Zhang C. Berry G.J. Altman R.B. Ré C. Rubin D.L. Snyder M. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features Nat. Commun. 2016 7 12474 10.1038/ncomms12474 27527408 PMC4990706 · DOI · Paper page
  9. [10]Balazsi M. Blanco P. Zoroquiain P. Levine M.D. Burnier M.N. Jr. Invasive ductal breast carcinoma detector that is robust to image magnification in whole digital slides J. Med. Imaging 2016 3 027501 10.1117/1.JMI.3.2.027501 PMC4870400 27226977 · DOI
  10. [11]Barker J. Hoogi A. Depeursinge A. Rubin D.L. Automated classification of brain tumor type in whole-slide digital pathology images using local representative tiles Med. Image Anal. 2016 30 60 71 10.1016/j.media.2015.12.002 26854941 · DOI
  11. [12]Spanhol F.A. Oliveira L.S. Petitjean C. Heutte L. Breast cancer histopathological image classification using convolutional neural networks Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN) Vancouver, BC, Canada 24–29 July 2016 IEEE Piscataway, NJ, USA 2016 2560 2567
  12. [13]Zhu X. Yao J. Zhu F. Huang J. Wsisa: Making survival prediction from whole slide histopathological images Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Honolulu, HI, USA 21–26 July 2017 7234 7242
  13. [14]Whitney J. Corredor G. Janowczyk A. Ganesan S. Doyle S. Tomaszewski J. Feldman M. Gilmore H. Madabhushi A. Quantitative nuclear histomorphometry predicts oncotype DX risk categories for early stage ER+ breast cancer BMC Cancer 2018 18 610 10.1186/s12885-018-4448-9 29848291 PMC5977541 · DOI
  14. [15]Bahlmann C. Patel A. Johnson J. Ni J. Chekkoury A. Khurd P. Kamen A. Grady L. Krupinski E. Graham A. Automated detection of diagnostically relevant regions in H&E stained digital pathology slides Proceedings of the Medical Imaging 2012: Computer-Aided Diagnosis San Diego, CA, USA 4–9 February 2012 International Society for Optics and Photonics Bellingham, WA, USA 2012 Volume 8315 831504
  15. [16]Bejnordi B.E. Litjens G. Hermsen M. Karssemeijer N. van der Laak J.A. A multi-scale superpixel classification approach to the detection of regions of interest in whole slide histopathology images Proceedings of the Medical Imaging 2015: Digital Pathology Orlando, FL, USA 25–26 February 2015 International Society for Optics and Photonics Bellingham, WA, USA 2015 Volume 9420 94200H
  16. [18]Bejnordi B.E. Zuidhof G. Balkenhol M. Hermsen M. Bult P. van Ginneken B. Karssemeijer N. Litjens G. van der Laak J. Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images J. Med. Imaging 2017 4 044504 10.1117/1.JMI.4.4.044504 29285517 PMC5729919 · DOI
  17. [19]Lin H. Chen H. Dou Q. Wang L. Qin J. Heng P.A. Scannet: A fast and dense scanning framework for metastastic breast cancer detection from whole-slide image Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) Lake Tahoe, NV, USA 12–15 March 2018 IEEE Piscataway, NJ, USA 2018 539 546
  18. [20]Cruz-Roa A. Gilmore H. Basavanhally A. Feldman M. Ganesan S. Shih N. Tomaszewski J. Madabhushi A. González F. High-throughput adaptive sampling for whole-slide histopathology image analysis (HASHI) via convolutional neural networks: Application to invasive breast cancer detection PLoS ONE 2018 13 e0196828 10.1371/journal.pone.0196828 29795581 PMC5967747 · DOI
  19. [21]Attallah O. MB-AI-His: Histopathological diagnosis of pediatric medulloblastoma and its subtypes via AI Diagnostics 2021 11 359 10.3390/diagnostics11020359 33672752 PMC7924641 · DOI
  20. [22]Attallah O. Zaghlool S. AI-Based Pipeline for Classifying Pediatric Medulloblastoma Using Histopathological and Textural Images Life 2022 12 232 10.3390/life12020232 35207519 PMC8879027 · DOI
  21. [23]Anwar F. Attallah O. Ghanem N. Ismail M.A. Automatic breast cancer classification from histopathological images Proceedings of the 2019 International Conference on Advances in the Emerging Computing Technologies (AECT), Al Madinah Al Munawwarah Saudi Arabia 10 February 2020 IEEE Piscataway, NJ, USA 2020 1 6
  22. [24]Attallah O. Anwar F. Ghanem N.M. Ismail M.A. Histo-CADx: Duo cascaded fusion stages for breast cancer diagnosis from histopathological images PeerJ Comput. Sci. 2021 7 e493 10.7717/peerj-cs.493 PMC8093954 33987459 · DOI
  23. [25]Dundar M.M. Badve S. Bilgin G. Raykar V. Jain R. Sertel O. Gurcan M.N. Computerized classification of intraductal breast lesions using histopathological images IEEE Trans. Biomed. Eng. 2011 58 1977 1984 10.1109/TBME.2011.2110648 21296703 PMC3328096 · DOI
  24. [26]Sudharshan P. Petitjean C. Spanhol F. Oliveira L.E. Heutte L. Honeine P. Multiple instance learning for histopathological breast cancer image classification Expert Syst. Appl. 2019 117 103 111 10.1016/j.eswa.2018.09.049 · DOI · Paper page
  25. [27]Mercan C. Aksoy S. Mercan E. Shapiro L.G. Weaver D.L. Elmore J.G. Multi-instance multi-label learning for multi-class classification of whole slide breast histopathology images IEEE Trans. Med. Imaging 2017 37 316 325 10.1109/TMI.2017.2758580 28981408 PMC5774338 · DOI
  26. [28]Xia Y. Nie L. Zhang L. Yang Y. Hong R. Li X. Weakly supervised multilabel clustering and its applications in computer vision IEEE Trans. Cybern. 2016 46 3220 3232 10.1109/TCYB.2015.2501385 27046858 · DOI
  27. [29]Doyle S. Feldman M. Tomaszewski J. Madabhushi A. A boosted Bayesian multiresolution classifier for prostate cancer detection from digitized needle biopsies IEEE Trans. Biomed. Eng. 2010 59 1205 1218 10.1109/TBME.2010.2053540 20570758 · DOI
  28. [30]Basavanhally A. Ganesan S. Feldman M. Shih N. Mies C. Tomaszewski J. Madabhushi A. Multi-field-of-view framework for distinguishing tumor grade in ER+ breast cancer from entire histopathology slides IEEE Trans. Biomed. Eng. 2013 60 2089 2099 10.1109/TBME.2013.2245129 23392336 PMC5778451 · DOI
  29. [31]Liu Y. Gadepalli K. Norouzi M. Dahl G.E. Kohlberger T. Boyko A. Venugopalan S. Timofeev A. Nelson P.Q. Corrado G.S. Detecting cancer metastases on gigapixel pathology images arXiv 2017 1703.02442
  30. [32]Hou L. Samaras D. Kurc T.M. Gao Y. Davis J.E. Saltz J.H. Patch-based convolutional neural network for whole slide tissue image classification Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Las Vegas, NV, USA 27–30 June 2016 2424 2433 PMC5085270 27795661
  31. [33]Mahmood T. Kim S.G. Koo J.H. Park K.R. Artificial Intelligence-Based Tissue Phenotyping in Colorectal Cancer Histopathology Using Visual and Semantic Features Aggregation Mathematics 2022 10 1909 10.3390/math10111909 · DOI
  32. [34]Mobadersany P. Yousefi S. Amgad M. Gutman D.A. Barnholtz-Sloan J.S. Vega J.E.V. Brat D.J. Cooper L.A. Predicting cancer outcomes from histology and genomics using convolutional networks Proc. Natl. Acad. Sci. USA 2018 115 E2970 E2979 10.1073/pnas.1717139115 29531073 PMC5879673 · DOI · Paper page
  33. [35]Guo W. Liang W. Deng Q. Zou X. A Multimodal Affinity Fusion Network for Predicting the Survival of Breast Cancer Patients Front. Genet. 2021 1323 10.3389/fgene.2021.709027 PMC8417828 34490038 · DOI
  34. [36]Tong L. Sha Y. Wang M.D. Improving classification of breast cancer by utilizing the image pyramids of whole-slide imaging and multi-scale convolutional neural networks Proceedings of the 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC) Milwaukee, WI, USA 15–19 July 2019 IEEE Piscataway, NJ, USA 2019 Volume 1 696 703 PMC7302109 32558827
  35. [37]Vincent P. Larochelle H. Bengio Y. Manzagol P.A. Extracting and composing robust features with denoising autoencoders Proceedings of the 25th International Conference on Machine Learning Helsinki, Finland 5–9 July 2008 1096 1103
  36. [38]Roy A.G. Conjeti S. Carlier S.G. Houissa K. König A. Dutta P.K. Laine A.F. Navab N. Katouzian A. Sheet D. Multiscale distribution preserving autoencoders for plaque detection in intravascular optical coherence tomography Proceedings of the 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI) Prague, Czech Republic 13–16 April 2016 IEEE Piscataway, NJ, USA 2016 1359 1362
  37. [39]Goodfellow I. Bengio Y. Courville A. Bengio Y. Deep Learning MIT Press Cambridge Cambridge, MA, USA 2016 Volume 1
  38. [40]Kingma D.P. Welling M. Auto-encoding variational bayes arXiv 2013 1312.6114
  39. [41]Reinhard E. Adhikhmin M. Gooch B. Shirley P. Color transfer between images IEEE Comput. Graph. Appl. 2001 21 34 41 10.1109/38.946629 · DOI
  40. [42]Aresta G. Araújo T. Kwok S. Chennamsetty S.S. Safwan M. Alex V. Marami B. Prastawa M. Chan M. Donovan M. Bach: Grand challenge on breast cancer histology images Med. Image Anal. 2019 56 122 139 10.1016/j.media.2019.05.010 31226662 · DOI
  41. [43]Wei J.W. Tafe L.J. Linnik Y.A. Vaickus L.J. Tomita N. Hassanpour S. Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks Sci. Rep. 2019 9 1 8 10.1038/s41598-019-40041-7 30833650 PMC6399447 · DOI
  42. [44]He K. Zhang X. Ren S. Sun J. Deep residual learning for image recognition Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Las Vegas, NV, USA 26 June–1 July 2016 770 778
  43. [45]Szegedy C. Vanhoucke V. Ioffe S. Shlens J. Wojna Z. Rethinking the inception architecture for computer vision Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Las Vegas, NV, USA 27–30 June 2016 2818 2826
  44. [46]Huang G. Liu Z. Van Der Maaten L. Weinberger K.Q. Densely connected convolutional networks Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Honolulu, HI, USA 21–26 July 2017 4700 4708
  45. [47]Howard A.G. Zhu M. Chen B. Kalenichenko D. Wang W. Weyand T. Andreetto M. Adam H. Mobilenets: Efficient convolutional neural networks for mobile vision applications arXiv 2017 1704.04861
  46. [49]Toğaçar M. Özkurt K.B. Ergen B. Cömert Z. BreastNet: A novel convolutional neural network model through histopathological images for the diagnosis of breast cancer Phys. A Stat. Mech. Appl. 2020 545 123592 10.1016/j.physa.2019.123592 · DOI
  47. [50]Aatresh A.A. Alabhya K. Lal S. Kini J. Saxena P. LiverNet: Efficient and robust deep learning model for automatic diagnosis of sub-types of liver hepatocellular carcinoma cancer from H&E stained liver histopathology images Int. J. Comput. Assist. Radiol. Surg. 2021 16 1549 1563 34053009
  48. [51]Burçak K.C. Baykan Ö.K. Uğuz H. A new deep convolutional neural network model for classifying breast cancer histopathological images and the hyperparameter optimisation of the proposed model J. Supercomput. 2021 77 973 989 10.1007/s11227-020-03321-y · DOI

Cited by

Sorted by citations

Authors
Sheikh, T. S., Kim, J.-Y., Shim, J., & Cho, M.
Affiliations
  • Department of Computer & Media Engineering, Tongmyong University, Busan 48520, Korea
  • Department of Pathology, Pusan National University Yangsan Hospital, School of Medicine, Pusan National University, Yangsan-si 50612, Korea
  • School of Mechanical Engineering, Yeungnam University, Gyeongsan 38541, Korea
ORCID
Venue
Diagnostics
Publisher
MDPI AG
Published
2022-06-16
Volume
12
Issue
6
Pages
1480
ISSN
2075-4418
DOI
10.3390/diagnostics12061480
PMC
PMC9222016
Cited by
20
References
2
Language
en
Licence
creativecommons.org/licenses/by/4.0/ (vor)
Funders
National Research Foundation of Korea (NRF) (2021R1A2B5B01001789)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review