CoChat
Sign Up
Cited by 44AI in cancer detectionArtificial Intelligence in Healthcare and EducationRadiomics and Machine Learning in Medical Imaging

Multi-modality artificial intelligence in digital pathology

Open access
Qiao, Yixuan, Zhao, Lianhe, Luo, Chunlong, Luo, Yufan, Wu, Yang, Li, Shengtong, Bu, Dechao, Zhao, Yi·2022·Briefings in Bioinformatics
DOI 10.1093/bib/bbac367PMC PMC9677480
Open this paper in CoChat — free account

Abstract

In common medical procedures, the time-consuming and expensive nature of obtaining test results plagues doctors and patients. Digital pathology research allows using computational technologies to manage data, presenting an opportunity to improve the efficiency of diagnosis and treatment. Artificial intelligence (AI) has a great advantage in the data analytics phase. Extensive research has shown that AI algorithms can produce more up-to-date and standardized conclusions for whole slide images. In conjunction with the development of high-throughput sequencing technologies, algorithms can integrate and analyze data from multiple modalities to explore the correspondence between morphological features and gene expression. This review investigates using the most popular image data, hematoxylin–eosin stained tissue slide images, to find a strategic solution for the imbalance of healthcare resources. The article focuses on the role that the development of deep learning technology has in assisting doctors’ work and discusses the opportunities and challenges of AI.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/bib/bbac367

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In digital pathology applications, whole slide images have been used for predicting genomic features.

    “In digital pathology applications, whole slide images have been used for predicting genomic features [19]”

    From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [19]

  • The multimodal learning of WSIs and genomic datasets is a challenge and opportunity for benefiting the clinical diagnosis and treatment stratification in clinical applications.

    “Besides, the multimodal learning of WSIs and genomic datasets is a challenge and opportunity for benefiting the clinical diagnosis and treatment stratification in clinical applications [19].”

    From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [19]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 188

  1. [1]Konig IR, Fuchs O, Hansen G, et al. What is precision medicine? Eur Respir J 2017; 50(4): 1700391. 29051268
  2. [2]Echle A, Rindtorff NT, Brinker TJ, et al. Deep learning in cancer pathology: a new generation of clinical biomarkers. Br J Cancer 2021; 124(4): 686– 96. 33204028 PMC7884739
  3. [3]Cardiff RD, Miller CH, Munn RJ. Manual hematoxylin and eosin staining of mouse tissue sections. Cold Spring Harb Protoc 2014; 2014(6): 655– 8. 24890205
  4. [4]Fischer AH, Jacobson KA, Rose J, et al. Hematoxylin and eosin staining of tissue and cell sections. CSH Protoc 2008; 2008: pdb.prot4986. 21356829
  5. [5]Pantanowitz L, Sharma A, Carter AB, et al. Twenty years of digital pathology: an overview of the road travelled, what is on the horizon, and the emergence of vendor-neutral archives. J Pathol Inform 2018; 9: 40. 30607307 PMC6289005
  6. [6]LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521(7553): 436– 44. 26017442
  7. [7]Chang J, Lan Z, Cheng C, et al. Data uncertainty learning in face recognition. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (Cvpr), p. 5709– 18. IEEE, 2020.
  8. [9]Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42: 60– 88. 28778026
  9. [10]Jiang Y, Xie J, Huang W, et al. Tumor immune microenvironment and chemosensitivity signature for predicting response to chemotherapy in gastric cancer. Cancer Immunol Res 2019; 7(12): 2065– 73. 31615816
  10. [11]Ongsulee P. Artificial intelligence, machine learning and deep learning. In: 2017 15th International Conference on ICT and Knowledge Engineering (ICT&KE), pp. 1– 6. IEEE, 2017.
  11. [12]Bishop CM, Nasrabadi NM. Pattern Recognition and Machine Learning, Vol. 4, No. 4, p. 738. New York: Springer, 2006.
  12. [13]Friedman N, Geiger D, Goldszmidt M. Bayesian network classifiers. Mach Learn 1997; 29(2): 131– 63.
  13. [14]Quinlan JR. Induction of decision trees. Mach Learn 1986; 1(1): 81– 106.
  14. [15]Cortes C, Vapnik V. Support-vector networks. Mach Learn 1995; 20(3): 273– 97.
  15. [16]Eraslan G, Avsec Ž, Gagneur J, et al. Deep learning: new computational modelling techniques for genomics. Nat Rev Genet 2019; 20(7): 389– 403. 30971806
  16. [17]Dias R, Torkamani A. Artificial intelligence in clinical and genomic diagnostics. Genome Med 2019; 11(1): 70. 31744524 PMC6865045
  17. [18]Angenent-Mari NM, Garruss AS, Soenksen LR, et al. A deep learning approach to programmable RNA switches. Nat Commun 2020; 11(1): 5057. 33028812 PMC7541447
  18. [19]LeCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition. Proc IEEE 1998; 86(11): 2278– 324.
  19. [21]Hosny A, Parmar C, Quackenbush J, et al. Artificial intelligence in radiology. Nat Rev Cancer 2018; 18(8): 500– 10. 29777175 PMC6268174
  20. [22]Lundervold AS, Lundervold A. An overview of deep learning in medical imaging focusing on MRI. Z Med Phys 2019; 29(2): 102– 27. 30553609
  21. [23]Liu Q, Fang L, Yu G, et al. Detection of DNA base modifications by deep recurrent neural network on Oxford Nanopore sequencing data. Nat Commun 2019; 10(1): 2449. 31164644 PMC6547721
  22. [24]Singh J, Hanson J, Paliwal K, et al. RNA secondary structure prediction using an ensemble of two-dimensional deep neural networks and transfer learning. Nat Commun 2019; 10(1): 5407. 31776342 PMC6881452
  23. [25]Zhang Z, Chen P, McGough M, et al. Pathologist-level interpretable whole-slide cancer diagnosis with deep learning. Nat Mach Intell 2019; 1(5): 236– 45.
  24. [26]Amgad M, Atteya LA, Hussein H, et al. Explainable nucleus classification using decision tree approximation of learned embeddings. Bioinformatics 2022; 38: 513– 9. PMC10142876 34586355
  25. [27]Holmstrom O, Linder N, Kaingu H, et al. Point-of-care digital cytology with artificial intelligence for cervical cancer screening in a resource-limited setting. JAMA Netw Open 2021; 4(3): e211740. 33729503 PMC7970338
  26. [28]Schneider L, Laiouar-Pedari S, Kuntz S, et al. Integration of deep learning-based image analysis and genomic data in cancer pathology: a systematic review. Eur J Cancer 2022; 160: 80– 91. 34810047
  27. [29]Kuntz S, Krieghoff-Henning E, Kather JN, et al. Gastrointestinal cancer classification and prognostication from histology using deep learning: systematic review. Eur J Cancer 2021; 155: 200– 15. 34391053
  28. [30]Calderaro J, Kather JN. Artificial intelligence-based pathology for gastrointestinal and hepatobiliary cancers. Gut 2021; 70(6): 1183– 93. 33214163
  29. [31]Couture HD, Williams LA, Geradts J, et al. Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype. NPJ Breast Cancer 2018; 4: 30. 30182055 PMC6120869
  30. [32]Yang P, Zhai Y, Li L, et al. A deep metric learning approach for histopathological image retrieval. Methods 2020; 179: 14– 25. 32439386
  31. [33]Shi X, Sapkota M, Xing F, et al. Pairwise based deep ranking hashing for histopathology image classification and retrieval. Patt Recogn 2018; 81: 14– 22.
  32. [34]Vahadane A, Peng T, Sethi A, et al. Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans Med Imaging 2016; 35(8): 1962– 71. 27164577
  33. [35]Vasiljević J, Feuerhake F, Wemmert C, et al. Towards histopathological stain invariance by unsupervised domain augmentation using generative adversarial networks. Neurocomputing 2021; 460: 277– 91.
  34. [36]Tellez D, Litjens G, van der Laak J, et al. Neural image compression for gigapixel histopathology image analysis. IEEE Trans Pattern Anal Mach Intell 2019; 43(2): 567– 78. 31442971
  35. [37]Yang P, Hong Z, Yin X, et al. Self-supervised visual representation learning for histopathological images. International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 47– 57. Springer, Cham, 2021.
  36. [38]Sirinukunwattana K, Raza SEA, Tsang YW, et al. Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imaging 2016; 35(5): 1196– 206. 26863654
  37. [39]Qaiser T, Tsang YW, Taniyama D, et al. Fast and accurate tumor segmentation of histology images using persistent homology and deep convolutional features. Med Image Anal 2019; 55: 1– 14. 30991188
  38. [40]Xie Y, Xing F, Shi X, et al. Efficient and robust cell detection: a structured regression approach. Med Image Anal 2018; 44: 245– 54. 28797548 PMC6051760
  39. [41]Kumar N, Verma R, Sharma S, et al. A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans Med Imaging 2017; 36(7): 1550– 60. 28287963
  40. [42]Marsh JN, Matlock MK, Kudose S, et al. Deep learning global glomerulosclerosis in transplant kidney frozen sections. IEEE Trans Med Imaging 2018; 37(12): 2718– 28. 29994669 PMC6296264
  41. [43]D'Alfonso TM, Ho DJ, Hanna MG, et al. Multi-magnification-based machine learning as an ancillary tool for the pathologic assessment of shaved margins for breast carcinoma lumpectomy specimens. Mod Pathol 2021; 34(8): 1487– 94. 33903728 PMC9906995
  42. [44]Bouteldja N, Klinkhammer BM, Bülow RD, et al. Deep learning-based segmentation and quantification in experimental kidney histopathology. J Am Soc Nephrol 2021; 32(1): 52– 68. 33154175 PMC7894663
  43. [45]Lu Z, Zhan X, Wu Y, et al. BrcaSeg: a deep learning approach for tissue quantification and genomic correlations of histopathological images. Genom Proteom Bioinformatics 2021; 19(6):1032–42. PMC9403022 34280546
  44. [46]Graham S, Chen H, Gamper J, et al. MILD-net: minimal information loss dilated network for gland instance segmentation in colon histology images. Med Image Anal 2019; 52: 199– 211. 30594772
  45. [47]Jang HJ, Song IH, Lee SH. Deep learning for automatic subclassification of gastric carcinoma using whole-slide histopathology images. Cancers 2021; 13(15):3811. 10.3390/cancers13153811. PMC8345042 34359712 · DOI
  46. [48]Naylor P, Lae M, Reyal F, et al. Segmentation of nuclei in histopathology images by deep regression of the distance map. IEEE Trans Med Imaging 2019; 38(2): 448– 59. 30716022
  47. [49]Zormpas-Petridis K, Noguera R, Ivankovic DK, et al. SuperHistopath: a deep learning pipeline for mapping tumor heterogeneity on low-resolution whole-slide digital histopathology images. Front Oncol 2021; 10: 586292. 33552964 PMC7855703
  48. [50]Zhang J, Hua Z, Yan K, et al. Joint fully convolutional and graph convolutional networks for weakly-supervised segmentation of pathology images. Med Image Anal 2021; 73: 102183. 34340108
  49. [51]Lee S, Amgad M, Mobadersany P, et al. Interactive classification of whole-slide imaging data for cancer researchers. Cancer Res 2021; 81(4): 1171– 7. 33355190 PMC8026494
  50. [52]Rivenson Y, Liu T, Wei Z, et al. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light Sci Appl 2019; 8: 23. 30728961 PMC6363787
  51. [53]de Haan K, Zhang Y, Zuckerman JE, et al. Deep learning-based transformation of H&E stained tissues into special stains. Nat Commun 2021; 12(1): 4884. 34385460 PMC8361203
  52. [54]Fujitani M, Mochizuki Y, Iizuka S, et al. Re-staining pathology images by FCNN. In: 2019 16th International Conference on Machine Vision Applications (MVA), pp. 1– 6. IEEE, 2019.
  53. [55]Levy JJ, Azizgolshani N, Andersen MJ, Jr, et al. A large-scale internal validation study of unsupervised virtual trichrome staining technologies on nonalcoholic steatohepatitis liver biopsies. Mod Pathol 2021; 34(4): 808– 22. 33299110 PMC7985027
  54. [56]Gadermayr M, Gupta L, Appel V, et al. Generative adversarial networks for facilitating stain-independent supervised and unsupervised segmentation: a study on kidney histology. IEEE Trans Med Imaging 2019; 38(10): 2293– 302. 30762541
  55. [57]Zhang Y, de Haan K, Rivenson Y, et al. Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue. Light Sci Appl 2020; 9: 78. 32411363 PMC7203145
  56. [58]Kwak MS, Lee HH, Yang JM, et al. Deep convolutional neural network-based lymph node metastasis prediction for colon cancer using histopathological images. Front Oncol 2021; 10: 619803. 33520727 PMC7838556
  57. [59]Klimov S, Xue Y, Gertych A, et al. Predicting metastasis risk in pancreatic neuroendocrine tumors using deep learning image analysis. Front Oncol 2021; 10: 593211. 33718106 PMC7946991
  58. [60]Hu Y, Su F, Dong K, et al. Deep learning system for lymph node quantification and metastatic cancer identification from whole-slide pathology images. Gastric Cancer 2021; 24(4): 868– 77. 33484355
  59. [61]Lin H, Chen H, Graham S, et al. Fast ScanNet: fast and dense analysis of multi-gigapixel whole-slide images for cancer metastasis detection. IEEE Trans Med Imaging 2019; 38(8): 1948– 58. 30624213
  60. [62]Woerl AC, Eckstein M, Geiger J, et al. Deep learning predicts molecular subtype of muscle-invasive bladder cancer from conventional histopathological slides. Eur Urol 2020; 78(2): 256– 64. 32354610
  61. [63]Klein S, Quaas A, Quantius J, et al. Deep learning predicts HPV association in oropharyngeal squamous cell carcinomas and identifies patients with a favorable prognosis using regular H&E stains. Clin Cancer Res 2021; 27(4): 1131– 8. 33262137
  62. [64]Cao R, Yang F, Ma SC, et al. Development and interpretation of a pathomics-based model for the prediction of microsatellite instability in colorectal cancer. Theranostics 2020; 10(24): 11080– 91. 33042271 PMC7532670
  63. [65]Kather JN, Pearson AT, Halama N, et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat Med 2019; 25(7): 1054– 6. 31160815 PMC7423299
  64. [67]Yamashita R, Long J, Longacre T, et al. Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study. Lancet Oncol 2021; 22(1): 132– 41. 33387492
  65. [68]Wang Y, Coudray N, Zhao Y, et al. HEAL: an automated deep learning framework for cancer histopathology image analysis. Bioinformatics 2021; 37(22):4291–95. 10.1093/bioinformatics/btab380. 34009289 · DOI
  66. [69]Fu H, Mi W, Pan B, et al. Automatic pancreatic ductal adenocarcinoma detection in whole slide images using deep convolutional neural networks. Front Oncol 2021; 11: 665929. 34249702 PMC8267174
  67. [70]Yao H, Zhang X, Zhou X, et al. Parallel structure deep neural network using CNN and RNN with an attention mechanism for breast cancer histology image classification. Cancers (Basel) 2019; 11(12):1901. 10.3390/cancers11121901. PMC6966545 31795390 · DOI
  68. [71]Syrykh C, Abreu A, Amara N, et al. Accurate diagnosis of lymphoma on whole-slide histopathology images using deep learning. NPJ Digit Med 2020; 3: 63. 32377574 PMC7195401
  69. [72]Park J, Jang BG, Kim YW, et al. A prospective validation and observer performance study of a deep learning algorithm for pathologic diagnosis of gastric tumors in endoscopic biopsies. Clin Cancer Res 2021; 27(3): 719– 28. 33172897
  70. [73]Yang H, Chen L, Cheng Z, et al. Deep learning-based six-type classifier for lung cancer and mimics from histopathological whole slide images: a retrospective study. BMC Med 2021; 19(1): 80. 33775248 PMC8006383
  71. [75]Ström P, Kartasalo K, Olsson H, et al. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. Lancet Oncol 2020; 21(2): 222– 32. 31926806
  72. [76]Chuang WY, Chang SH, Yu WH, et al. Successful identification of nasopharyngeal carcinoma in nasopharyngeal biopsies using deep learning. Cancers 2020; 12(2):507. 10.3390/cancers12020507. PMC7072217 32098314 · DOI
  73. [77]Sheikh TS, Lee Y, Cho M. Histopathological classification of breast cancer images using a multi-scale input and multi-feature network. Cancers 2020; 12(8):2031. 10.3390/cancers12082031. PMC7465368 32722111 · DOI
  74. [78]Steinbuss G, Kriegsmann M, Zgorzelski C, et al. Deep learning for the classification of non-Hodgkin lymphoma on histopathological images. Cancers 2021; 13(10):2419. 10.3390/cancers13102419. PMC8156071 34067726 · DOI
  75. [79]Bulten W, Balkenhol M, Belinga JJA, et al. Artificial intelligence assistance significantly improves Gleason grading of prostate biopsies by pathologists. Mod Pathol 2021; 34(3): 660– 71. 32759979 PMC7897578
  76. [80]Perincheri S, Levi AW, Celli R, et al. An independent assessment of an artificial intelligence system for prostate cancer detection shows strong diagnostic accuracy. Mod Pathol 2021; 34(8): 1588– 95. 33782551 PMC8295034
  77. [81]Nir G, Hor S, Karimi D, et al. Automatic grading of prostate cancer in digitized histopathology images: learning from multiple experts. Med Image Anal 2018; 50: 167– 80. 30340027
  78. [82]Lu MY, Williamson DFK, Chen TY, et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat Biomed Eng 2021; 5(6): 555– 70. 33649564 PMC8711640
  79. [83]Wang S, Zhu Y, Yu L, et al. RMDL: recalibrated multi-instance deep learning for whole slide gastric image classification. Med Image Anal 2019; 58: 101549. 31499320
  80. [84]Dov D, Kovalsky SZ, Assaad S, et al. Weakly supervised instance learning for thyroid malignancy prediction from whole slide cytopathology images. Med Image Anal 2021; 67: 101814. 33049578 PMC7726041
  81. [85]Wang KS, Yu G, Xu C, et al. Accurate diagnosis of colorectal cancer based on histopathology images using artificial intelligence. BMC Med 2021; 19(1): 76. 33752648 PMC7986569
  82. [86]Yoshida H, Shimazu T, Kiyuna T, et al. Automated histological classification of whole-slide images of gastric biopsy specimens. Gastric Cancer 2018; 21(2): 249– 57. 28577229
  83. [87]Mercan C, Aksoy S, Mercan E, et al. Multi-instance multi-label learning for multi-class classification of whole slide breast histopathology images. IEEE Trans Med Imaging 2018; 37(1): 316– 25. 28981408 PMC5774338
  84. [88]Wang X, Chen H, Gan C, et al. Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE Trans Cybern 2020; 50(9): 3950– 62. 31484154
  85. [89]Pirovano A, Almeida LG, Ladjal S, et al. Computer-aided diagnosis tool for cervical cancer screening with weakly supervised localization and detection of abnormalities using adaptable and explainable classifier. Med Image Anal 2021; 73: 102167. 34333217
  86. [90]Pinckaers H, Bulten W, van der Laak J, et al. Detection of prostate cancer in whole-slide images through end-to-end training with image-level labels. IEEE Trans Med Imaging 2021; 40(7): 1817– 26. 33729928
  87. [91]Campanella G, Hanna MG, Geneslaw L, et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med 2019; 25(8): 1301– 9. 31308507 PMC7418463
  88. [92]Li D, Bledsoe JR, Zeng Y, et al. A deep learning diagnostic platform for diffuse large B-cell lymphoma with high accuracy across multiple hospitals. Nat Commun 2020; 11(1): 6004. 33244018 PMC7691991
  89. [93]BenTaieb A, Li-Chang H, Huntsman D, et al. A structured latent model for ovarian carcinoma subtyping from histopathology slides. Med Image Anal 2017; 39: 194– 205. 28521242
  90. [94]Yu KH, Zhang C, Berry GJ, et al. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nat Commun 2016; 7: 12474. 27527408 PMC4990706
  91. [95]Kulkarni PM, Robinson EJ, Sarin Pradhan J, et al. Deep learning based on standard H&E images of primary melanoma tumors identifies patients at risk for visceral recurrence and death. Clin Cancer Res 2020; 26(5): 1126– 34. 31636101 PMC8142811
  92. [97]Koyuncu CF, Lu C, Bera K, et al. Computerized tumor multinucleation index (MuNI) is prognostic in p16+ oropharyngeal carcinoma. J Clin Invest 2021; 131(8). 10.1172/JCI145488. PMC8075166 33651718 · DOI
  93. [99]Courtiol P, Maussion C, Moarii M, et al. Deep learning-based classification of mesothelioma improves prediction of patient outcome. Nat Med 2019; 25(10): 1519– 25. 31591589
  94. [100]Sun M, Zhou W, Qi X, et al. Prediction of BAP1 expression in uveal melanoma using densely-connected deep classification networks. Cancers 2019; 11(10):1579. 10.3390/cancers11101579. PMC6826369 31623293 · DOI
  95. [101]Yao J, Zhu X, Jonnagaddala J, et al. Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Med Image Anal 2020; 65: 101789. 32739769
  96. [102]Skrede O-J, de Raedt S, Kleppe A, et al. Deep learning for prediction of colorectal cancer outcome: a discovery and validation study. The Lancet 2020; 395(10221): 350– 60. 32007170
  97. [103]Shi JY, Wang X, Ding GY, et al. Exploring prognostic indicators in the pathological images of hepatocellular carcinoma based on deep learning. Gut 2021; 70(5): 951– 61. 32998878
  98. [104]Ghosh A, Sirinukunwattana K, Khalid Alham N, et al. The potential of artificial intelligence to detect lymphovascular invasion in testicular cancer. Cancers 2021; 13(6):1325. 10.3390/cancers13061325. PMC7998792 33809521 · DOI
  99. [105]Corredor G, Wang X, Zhou Y, et al. Spatial architecture and arrangement of tumor-infiltrating lymphocytes for predicting likelihood of recurrence in early-stage non-small cell lung cancer. Clin Cancer Res 2019; 25(5): 1526– 34. 30201760 PMC6397708
  100. [106]Saito A, Toyoda H, Kobayashi M, et al. Prediction of early recurrence of hepatocellular carcinoma after resection using digital pathology images assessed by machine learning. Mod Pathol 2021; 34(2): 417– 25. 32948835 PMC7817520
  101. [107]Shim WS, Yim K, Kim TJ, et al. DeepRePath: identifying the prognostic features of early-stage lung adenocarcinoma using multi-scale pathology images and deep convolutional neural networks. Cancers 2021; 13(13):3308. 10.3390/cancers13133308. PMC8268823 34282757 · DOI
  102. [108]Foersch S, Eckstein M, Wagner DC, et al. Deep learning for diagnosis and survival prediction in soft tissue sarcoma. Ann Oncol 2021; 32(9): 1178– 87. 34139273
  103. [109]Wang Y, Acs B, Robertson S, et al. Improved breast cancer histological grading using deep learning. Ann Oncol 2022; 33(1): 89– 98. 34756513
  104. [110]Zadeh Shirazi A, McDonnell MD, Fornaciari E, et al. A deep convolutional neural network for segmentation of whole-slide pathology images identifies novel tumor cell-perivascular niche interactions that are associated with poor survival in glioblastoma. Br J Cancer 2021; 125(3): 337– 50. 33927352 PMC8329064
  105. [111]Yu KH, Hu V, Wang F, et al. Deciphering serous ovarian carcinoma histopathology and platinum response by convolutional neural networks. BMC Med 2020; 18(1): 236. 32807164 PMC7433108
  106. [112]Nearchou IP, Ueno H, Kajiwara Y, et al. Automated detection and classification of desmoplastic reaction at the colorectal tumour front using deep learning. Cancers 2021; 13(7):1615. 10.3390/cancers13071615. PMC8036363 33807394 · DOI
  107. [113]Wang X, Chen Y, Gao Y, et al. Predicting gastric cancer outcome from resected lymph node histopathology images using deep learning. Nat Commun 2021; 12(1): 1637. 33712598 PMC7954798
  108. [114]Chou M, Illa-Bochaca I, Minxi B, et al. Optimization of an automated tumor-infiltrating lymphocyte algorithm for improved prognostication in primary melanoma. Mod Pathol 2021; 34(3): 562– 71. 33005020 PMC7983061
  109. [115]Narayanan PL, Raza SEA, Hall AH, et al. Unmasking the immune microecology of ductal carcinoma in situ with deep learning. NPJ Breast Cancer 2021; 7(1): 19. 33649333 PMC7921670
  110. [116]Acs B, Ahmed FS, Gupta S, et al. An open source automated tumor infiltrating lymphocyte algorithm for prognosis in melanoma. Nat Commun 2019; 10(1): 5440. 31784511 PMC6884485
  111. [117]Heindl A, Sestak I, Naidoo K, et al. Relevance of spatial heterogeneity of immune infiltration for predicting risk of recurrence after endocrine therapy of ER+ breast cancer. J Natl Cancer Inst 2018; 110(2): 166– 75. PMC6298573 28859291
  112. [118]Bhargava HK, Leo P, Elliott R, et al. Computationally derived image signature of stromal morphology is prognostic of prostate cancer recurrence following prostatectomy in African American patients. Clin Cancer Res 2020; 26(8): 1915– 23. 32139401 PMC7165025
  113. [119]Zhao L, Dong Q, Luo C, et al. DeepOmix: A scalable and interpretable multi-omics deep learning framework and application in cancer survival analysis. Computational and structural biotechnology journal 2021; 19:2719–25. PMC8131983 34093987
  114. [120]Chen X, Chen S, Song S, et al. Cell type annotation of single-cell chromatin accessibility data via supervised Bayesian embedding. Nat Mach Intell 2022; 4(2): 116– 26.
  115. [121]Liu Q, Chen S, Jiang R, et al. Simultaneous deep generative modeling and clustering of single cell genomic data. Nat Mach Intell 2021; 3(6): 536– 44. 34179690 PMC8223760
  116. [122]Chen SQ, Yan G, Zhang W, et al. RA3 is a reference-guided approach for epigenetic characterization of single cells. Nat Commun 2021; 12(1):2177. 10.1038/s41467-021-22495-4. PMC8041798 33846355 · DOI
  117. [123]Liu Q, Xu J, Jiang R, et al. Density estimation using deep generative neural networks. Proc Natl Acad Sci U S A 2021; 118(15):e2101344118. PMC8054014 33833061
  118. [124]Chelebian E, Avenel C, Kartasalo K, et al. Morphological features extracted by AI associated with spatial transcriptomics in prostate cancer. Cancers 2021; 13(19):4837. 10.3390/cancers13194837. PMC8507756 34638322 · DOI
  119. [126]Diao JA, Wang JK, Chui WF, et al. Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes. Nat Commun 2021; 12(1): 1613. 33712588 PMC7955068
  120. [127]Jin L, Shi F, Chun Q, et al. Artificial intelligence neuropathologist for glioma classification using deep learning on hematoxylin and eosin stained slide images and molecular markers. Neuro Oncol 2021; 23(1): 44– 52. 32663285 PMC7850049
  121. [128]Qu H, Zhou M, Yan Z, et al. Genetic mutation and biological pathway prediction based on whole slide images in breast carcinoma using deep learning. NPJ Precis Oncol 2021; 5(1): 87. 34556802 PMC8460699
  122. [129]Valieris R, Amaro L, Osório CABT, et al. Deep learning predicts underlying features on pathology images with therapeutic relevance for breast and gastric cancer. Cancers 2020; 12(12):3687. 10.3390/cancers12123687. PMC7763049 33316873 · DOI
  123. [130]Zheng H, Momeni A, Cedoz PL, et al. Whole slide images reflect DNA methylation patterns of human tumors. NPJ Genom Med 2020; 5: 11. 32194984 PMC7064513
  124. [131]Tavolara TE, Niazi MKK, Gower AC, et al. Deep learning predicts gene expression as an intermediate data modality to identify susceptibility patterns in Mycobacterium tuberculosis infected diversity outbred mice. EBioMedicine 2021; 67: 103388. 34000621 PMC8138606
  125. [132]Chen M, Zhang B, Topatana W, et al. Classification and mutation prediction based on histopathology H&E images in liver cancer using deep learning. NPJ Precis Oncol 2020; 4: 14. 32550270 PMC7280520
  126. [133]Huang K, Mo Z, Zhu W, et al. Prediction of target-drug therapy by identifying gene mutations in lung cancer with histopathological stained image and deep learning techniques. Front Oncol 2021; 11: 642945. 33928031 PMC8076857
  127. [134]Coudray N, Ocampo PS, Sakellaropoulos T, et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat Med 2018; 24(10): 1559– 67. 30224757 PMC9847512
  128. [135]Noorbakhsh J, Farahmand S, Foroughi pour A, et al. Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images. Nat Commun 2020; 11(1): 6367. 33311458 PMC7733499
  129. [136]Bian C, Wang Y, Lu Z, et al. ImmunoAIzer: a deep learning-based computational framework to characterize cell distribution and gene mutation in tumor microenvironment. Cancers 2021; 13(7):1659. 10.3390/cancers13071659. PMC8036970 33916145 · DOI
  130. [137]Mobadersany P, Yousefi S, Amgad M, et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc Natl Acad Sci U S A 2018; 115(13): E2970– 9. 29531073 PMC5879673
  131. [138]Chen RJ, Lu MY, Weng WH, et al. Multimodal co-attention transformer for survival prediction in gigapixel whole slide images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4015– 25. IEEE, 2021.
  132. [139]Zhang F, Zhong L-Z, Zhao X, et al. A deep-learning-based prognostic nomogram integrating microscopic digital pathology and macroscopic magnetic resonance images in nasopharyngeal carcinoma: a multi-cohort study. Ther Adv Med Oncol 2020; 12: 1758835920971416. 33403013 PMC7739087
  133. [140]Naik N, Madani A, Esteva A, et al. Deep learning-enabled breast cancer hormonal receptor status determination from base-level H&E stains. Nat Commun 2020; 11(1): 5727. 33199723 PMC7670411
  134. [141]Lepine C, Klein P, Voron T, et al. Histological severity risk factors identification in juvenile-onset recurrent respiratory papillomatosis: how immunohistochemistry and AI algorithms can help? Front Oncol 2021; 11: 596499. 33763347 PMC7982831
  135. [142]Chatrian A, Colling RT, Browning L, et al. Artificial intelligence for advance requesting of immunohistochemistry in diagnostically uncertain prostate biopsies. Mod Pathol 2021; 34(9): 1780– 94. 34017063 PMC8376647
  136. [143]Martino F, Varricchio S, Russo D, et al. A machine-learning approach for the assessment of the proliferative compartment of solid tumors on hematoxylin-eosin-stained sections. Cancers (Basel) 2020; 12(5): 1344. PMC7281627 32466184
  137. [144]Schmauch B, Romagnoni A, Pronier E, et al. A deep learning model to predict RNA-Seq expression of tumours from whole slide images. Nat Commun 2020; 11(1): 3877. 32747659 PMC7400514
  138. [145]AbdulJabbar K, Ahmed Raza SE, Rosenthal R, et al. Geospatial immune variability illuminates differential evolution of lung adenocarcinoma. Nat Med 2020; 26(7): 1054– 62. 32461698 PMC7610840
  139. [146]Rawat RR, Ruderman D, Macklin P, et al. Correlating nuclear morphometric patterns with estrogen receptor status in breast cancer pathologic specimens. NPJ Breast Cancer 2018; 4: 32. 30211313 PMC6123433
  140. [147]Van Eycke YR, Balsat C, Verset L, et al. Segmentation of glandular epithelium in colorectal tumours to automatically compartmentalise IHC biomarker quantification: a deep learning approach. Med Image Anal 2018; 49: 35– 45. 30081241
  141. [148]Su F, Sun Y, Hu Y, et al. Development and validation of a deep learning system for ascites cytopathology interpretation. Gastric Cancer 2020; 23(6): 1041– 50. 32500456
  142. [149]Bao G, Wang X, Xu R, et al. PathoFusion: an open-source AI framework for recognition of pathomorphological features and mapping of immunohistochemical data. Cancers 2021; 13(4):617. 10.3390/cancers13040617. PMC7913958 33557152 · DOI
  143. [150]Race AM, Sutton D, Hamm G, et al. Deep learning-based annotation transfer between molecular imaging modalities: an automated workflow for multimodal data integration. Anal Chem 2021; 93(6): 3061– 71. 33534548
  144. [151]Carneiro G, Peng T, Bayer C, et al. Automatic quantification of tumour hypoxia from multi-modal microscopy images using weakly-supervised learning methods. IEEE Trans Med Imaging 2017; 36(7): 1405– 17. 28278461
  145. [152]Jayapandian CP, Chen Y, Janowczyk AR, et al. Development and evaluation of deep learning-based segmentation of histologic structures in the kidney cortex with multiple histologic stains. Kidney Int 2021; 99(1): 86– 101. 32835732 PMC8414393
  146. [153]Gehrung M, Crispin-Ortuzar M, Berman AG, et al. Triage-driven diagnosis of Barrett's esophagus for early detection of esophageal adenocarcinoma using deep learning. Nat Med 2021; 27(5): 833– 41. 33859411
  147. [154]Rathore S, Niazi T, Iftikhar MA, et al. Glioma grading via analysis of digital pathology images using machine learning. Cancers 2020; 12(3):578. 10.3390/cancers12030578. PMC7139732 32131409 · DOI
  148. [155]Chen S, Zhang N, Jiang L, et al. Clinical use of a machine learning histopathological image signature in diagnosis and survival prediction of clear cell renal cell carcinoma. Int J Cancer 2021; 148(3): 780– 90. 32895914
  149. [156]Brinker TJ, Kiehl L, Schmitt M, et al. Deep learning approach to predict sentinel lymph node status directly from routine histology of primary melanoma tumours. Eur J Cancer 2021; 154: 227– 34. 34298373
  150. [157]Pei L, Jones KA, Shboul ZA, et al. Deep neural network analysis of pathology images with integrated molecular data for enhanced glioma classification and grading. Front Oncol 2021; 11: 668694. 34277415 PMC8282424
  151. [158]Yu KH, Berry GJ, Rubin DL, et al. Association of omics features with histopathology patterns in lung adenocarcinoma. Cell Syst 2017; 5(6): 620– 627.e3. 29153840 PMC5746468
  152. [159]Graham S, Vu QD, Raza SEA, et al. Hover-net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med Image Anal 2019; 58: 101563. 31561183
  153. [160]Wang X, Fang Y, Yang S, et al. A hybrid network for automatic hepatocellular carcinoma segmentation in H&E-stained whole slide images. Med Image Anal 2021; 68: 101914. 33285479
  154. [161]Wang H, Jiang Y, Li B, et al. Single-cell spatial analysis of tumor and immune microenvironment on whole-slide image reveals hepatocellular carcinoma subtypes. Cancers 2020; 12(12):3562. 10.3390/cancers12123562. PMC7761227 33260561 · DOI
  155. [162]Ciga O, Martel AL. Learning to segment images with classification labels. Med Image Anal 2021; 68: 101912. 33260115
  156. [163]Schrammen PL, Ghaffari Laleh N, Echle A, et al. Weakly supervised annotation-free cancer detection and prediction of genotype in routine histopathology. J Pathol 2022; 256(1): 50– 60. 34561876
  157. [164]Lu MY, Chen TY, Williamson DFK, et al. AI-based pathology predicts origins for cancers of unknown primary. Nature 2021; 594(7861): 106– 10. 33953404
  158. [165]Riasatian A, Babaie M, Maleki D, et al. Fine-tuning and training of densenet for histopathology image representation using TCGA diagnostic slides. Med Image Anal 2021; 70: 102032. 33773296
  159. [167]Sobhani F, Robinson R, Hamidinekoo A, et al. Artificial intelligence and digital pathology: opportunities and implications for immuno-oncology. Biochim Biophys Acta Rev Cancer 2021; 1875(2): 188520. 33561505 PMC9062980
  160. [168]Duggento A, Conti A, Mauriello A, et al. Deep computational pathology in breast cancer. Semin Cancer Biol 2021; 72: 226– 37. 32818626
  161. [169]Network, C.G.A.R. Comprehensive genomic characterization defines human glioblastoma genes and core pathways. Nature 2008; 455(7216): 1061– 8. 18772890 PMC2671642
  162. [170]Roux L, Racoceanu D, Loménie N, et al. Mitosis detection in breast cancer histological images an ICPR 2012 contest. J Pathol Inform 2013; 4: 8. 23858383 PMC3709417
  163. [172]Sirinukunwattana K, Pluim JPW, Chen H, et al. Gland segmentation in colon histology images: the glas challenge contest. Med Image Anal 2017; 35: 489– 502. 27614792
  164. [174]Veta M, Heng YJ, Stathonikos N, et al. Predicting breast tumor proliferation from whole-slide images: the TUPAC16 challenge. Med Image Anal 2019; 54: 111– 21. 30861443
  165. [175]Litjens G, Bandi P, Ehteshami Bejnordi B, et al. 1399 H&E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset. Gigascience 2018; 7(6):giy065. 10.1093/gigascience/giy065. PMC6007545 29860392 · DOI
  166. [176]Aresta G, Araújo T, Kwok S, et al. BACH: grand challenge on breast cancer histology images. Med Image Anal 2019; 56: 122– 39. 31226662
  167. [177]Veeling BS, Linmans J, Winkens J, et al. Rotation equivariant CNNs for digital pathology. Medical Image Computing and Computer Assisted Intervention - Miccai 2018, Pt Ii, 2018. 11071: p. 210– 8. Springer, Cham.
  168. [178]Borovec J, Kybic J, Arganda-Carreras I, et al. ANHIR: automatic non-rigid histological image registration challenge. IEEE Trans Med Imaging 2020; 39(10): 3042– 52. 32275587 PMC7584382
  169. [180]Li JH, Yang S, Huang X, et al. Signet ring cell detection with a semi-supervised learning framework. Information Processing in Medical Imaging, IPMI 2019, 11492. p. 842– 54. Springer, Cham.
  170. [181]Bertram CA, Aubreville M, Marzahl C, et al. A large-scale dataset for mitotic figure assessment on whole slide images of canine cutaneous mast cell tumor. Sci Data 2019; 6: 274. 31754105 PMC6872565
  171. [182]Kim YJ, Jang H, Lee K, et al. PAIP 2019: liver cancer segmentation challenge. Med Image Anal 2021; 67: 101854. 33091742
  172. [183]Conde-Sousa E, Vale J, Feng M, et al. HEROHE challenge: assessing HER2 status in breast cancer without immunohistochemistry or in situ hybridization. arXiv preprint arXiv:2111.04738. 2021. · arXiv
  173. [184]Aubreville M, Bertram CA, Donovan TA, et al. A completely annotated whole slide image dataset of canine breast cancer to aid human breast cancer research. Sci Data 2020; 7(1): 417. 33247116 PMC7699627
  174. [185]Verma R, Kumar N, Patil A, et al. MoNuSAC2020: a multi-organ nuclei segmentation and classification challenge. IEEE Trans Med Imaging 2021; 40(12): 3413– 23. 34086562
  175. [186]Bulten W, Kartasalo K, Chen PHC, et al. Artificial intelligence for diagnosis and Gleason grading of prostate cancer: the PANDA challenge. Nature medicine 2020; 28(1): 154– 63. 10.5281/zenodo. PMC8799467 35027755 · DOI · Paper page
  176. [187]Xu F, Zhu C, Tang W, et al. Predicting axillary lymph node metastasis in early breast cancer using deep learning on primary tumor biopsy slides. Front Oncol 2021; 11: 759007. 34722313 PMC8551965
  177. [188]Aubreville M, Bertram C, Veta M, et al. Quantifying the scanner-induced domain gap in mitosis detection. arXiv preprint arXiv:2103.16515. 2021. · arXiv
  178. [189]Amgad M, Atteya LA, Hussein H, et al. NuCLS: a scalable crowdsourcing, deep learning approach and dataset for nucleus classification, localization and segmentation. arXiv preprint arXiv:2102.09099. 2021. PMC9112766 35579553 · arXiv
  179. [190]Cuadrado-Godia E, Dwivedi P, Sharma S, et al. Cerebral small vessel disease: a review focusing on pathophysiology, biomarkers, and machine learning strategies. J Stroke 2018; 20(3): 302– 20. 30309226 PMC6186915
  180. [191]Han C, Lin J, Mai J, et al. Multi-layer pseudo-supervision for histopathology tissue semantic segmentation using patch-level classification labels. Medical Image Analysis. 2021;102487. 35671591
  181. [192]Graham S, Jahanifar M, Dang VQ, et al. CoNIC: colon nuclei identification and counting challenge 2022. arXiv preprint arXiv:2111.14485. 2021. · arXiv
  182. [193]Bera K, Schalper KA, Rimm DL, et al. Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology. Nat Rev Clin Oncol 2019; 16(11): 703– 15. 31399699 PMC6880861
  183. [194]Howard FM, Dolezal J, Kochanny S, et al. The impact of site-specific digital histology signatures on deep learning model accuracy and bias. Nat Commun 2021; 12(1): 4423. 34285218 PMC8292530
  184. [195]Boschman J, Farahani H, Darbandsari A, et al. The utility of color normalization for AI-based diagnosis of hematoxylin and eosin-stained pathology images. J Pathol 2022; 256(1): 15– 24. 34543435
  185. [197]McCombe KD, Craig SG, Viratham Pulsawatdi A, et al. HistoClean: open-source software for histological image pre-processing and augmentation to improve development of robust convolutional neural networks. Comput Struct Biotechnol J 2021; 19: 4840– 53. 34522291 PMC8426467
  186. [199]Chen CL, Chen CC, Yu WH, et al. An annotation-free whole-slide training approach to pathological classification of lung cancer types using deep learning. Nat Commun 2021; 12(1): 1193. 33608558 PMC7896045
  187. [202]Hanna MG, Reuter VE, Ardon O, et al. Validation of a digital pathology system including remote review during the COVID-19 pandemic. Mod Pathol 2020; 33(11): 2115– 27. 32572154 PMC7306935
  188. [204]Deng J, Dong W, Socher R, et al. Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, 2009, 248–55.

Cited by

Sorted by citations

Authors
Qiao, Y., Zhao, L., Luo, C., Luo, Y., Wu, Y., Li, S., Bu, D., & Zhao, Y.
Affiliations
  • Chinese Academy of Sciences (CN)
  • Massachusetts Institute of Technology (US)
Corresponding author
  • Zhao, Lianhe
  • Zhao, Yi
ORCID
Venue
Briefings in Bioinformatics
Publisher
Oxford University Press (OUP)
Published
2022-09-16
Volume
23
Issue
6
ISSN
  • 1467-5463 (ISSN-L)
  • 1477-4054
DOI
10.1093/bib/bbac367
PMC
PMC9677480
OpenAlex
W4296777402
Cited by
44
References
15
Citations by year
2026: 12 · 2025: 9 · 2024: 15 · 2023: 7 · 2022: 1
FWCI
0.56
Type
journal article
Language
en
Open access
Licence
Locations
  • Briefings in Bioinformatics (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
Funders
  • National Key Research and Development Program of China (2021YFC2500203)
  • Strategic Priority Research Program of the Chinese Academy of Sciences (XDA16021400)
  • Zhejiang Provincial Natural Science Foundation of China (LY20C060001)
  • Innovation Fund of Institute of Computing and Technology (E161080)
  • Zhejiang Provincial Research Center for Cancer Intelligent Diagnosis and Molecular Technology (JBZX-202003)
  • Shandong First Medical University & Shandong Academy of Medical Sciences (922-001003130 185RC)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Keywords
digital pathology, whole slide images, H&E staining, deep learning, morphological features, gene expression, next-generation sequencing, AI-assisted diagnosis
MeSH terms
Algorithms, Artificial Intelligence, Eosine Yellowish-(YS), Humans

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review