CoChat
Sign Up
Cited by 40AI in cancer detectionSingle-cell and spatial transcriptomicsRadiomics and Machine Learning in Medical Imaging

Learning to predict RNA sequence expressions from whole slide images with applications for search and classification

Open access
Alsaafin, Areej, Safarpoor, Amir, Sikaroudi, Milad, Hipp, Jason D., Tizhoosh, H. R.·2023·Communications Biology
DOI 10.1038/s42003-023-04583-xPMC PMC10033650
Open this paper in CoChat — free accountOpen PDF

Abstract

Deep learning methods are widely applied in digital pathology to address clinical challenges such as prognosis and diagnosis. As one of the most recent applications, deep models have also been used to extract molecular features from whole slide images. Although molecular tests carry rich information, they are often expensive, time-consuming, and require additional tissue to sample. In this paper, we propose tRNAsformer, an attention-based topology that can learn both to predict the bulk RNA-seq from an image and represent the whole slide image of a glass slide simultaneously. The tRNAsformer uses multiple instance learning to solve a weakly supervised problem while the pixel-level annotation is not available for an image. We conducted several experiments and achieved better performance and faster convergence in comparison to the state-of-the-art algorithms. The proposed tRNAsformer can assist as a computational pathology tool to facilitate a new generation of search and classification methods by combining the tissue morphology and the molecular fingerprint of the biopsy samples.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s42003-023-04583-x

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Some deep learning approaches have focused on detecting molecular alterations in image data, while most existing studies have centered on inferring spatial transcriptome or bulk transcriptome information from histopathology images.

    “While some deep learning approaches have focused on detecting molecular alterations in image data [132], most existing studies have centered on inferring spatial transcriptome or bulk transcriptome information from histopathology images [133–136].”

    From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [136]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 31 · Cited by 40

Research works printed but not linked: 32

  1. [1]Hou, L. et al. Patch-based convolutional neural network for whole slide tissue image classification. in Proceedings of the IEEE Conference on Computervision and Pattern Recognition 2424–2433 (2016). PMC5085270 27795661
  2. [2]Kalra S Pan-cancer diagnostic consensus through searching archival histopathology images using artificial intelligence NPJ Digit. Med. 2020 3 1 15 10.1038/s41746-020-0238-2 32195366 PMC7064517 · DOI
  3. [3]Wang H Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features J. Med. Imaging 2014 1 034003 10.1117/1.JMI.1.3.034003 PMC4479031 26158062 · DOI
  4. [5]Schmauch B A deep learning model to predict rna-seq expression of tumours from whole slide images Nat. Commun. 2020 11 1 15 10.1038/s41467-020-17678-4 32747659 PMC7400514 · DOI · Paper page
  5. [6]Levy-Jurgenson A Tekpli X Kristensen VN Yakhini Z Spatial transcriptomics inferred from pathology whole-slide images links tumor heterogeneity to survival in breast and lung cancer Sci. Rep. 2020 10 1 11 10.1038/s41598-020-75708-z 33139755 PMC7606448 · DOI · Paper page
  6. [7]He B Integrating spatial gene expression and breast tumour morphology via deep learning Nat. Biomed. Eng. 2020 4 827 834 10.1038/s41551-020-0578-x 32572199 · DOI · Paper page
  7. [8]Tavolara TE Deep learning predicts gene expression as an intermediate data modality to identify susceptibility patterns in mycobacterium tuberculosis infected diversity outbred mice EBioMedicine 2021 67 103388 10.1016/j.ebiom.2021.103388 34000621 PMC8138606 · DOI
  8. [9]Coudray N Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning Nat. Med. 2018 24 1559 1567 10.1038/s41591-018-0177-5 30224757 PMC9847512 · DOI · Paper page
  9. [10]Schaumberg, A. J., Rubin, M. A. & Fuchs, T. J. H&E-stained whole slide image deep learning predicts spop mutation state in prostate cancer. Preprint at BioRxiv10.1101/064279 (2017).
  10. [11]Segal E Friedman N Kaminski N Regev A Koller D From signatures to models: understanding cancer using microarrays Nat. Genet. 2005 37 S38 S45 10.1038/ng1561 15920529 · DOI
  11. [12]Kamps R Next-generation sequencing in oncology: genetic diagnosis, risk prediction and cancer classification Int. J. Mol. Sci. 2017 18 308 10.3390/ijms18020308 28146134 PMC5343844 · DOI
  12. [13]Sung H Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries CA: Cancer J. Clin. 2021 71 209 249 33538338
  13. [14]Shuch B Understanding pathologic variants of renal cell carcinoma: distilling therapeutic opportunities from biologic complexity Eur. Urol. 2015 67 85 97 10.1016/j.eururo.2014.04.029 24857407 · DOI
  14. [15]Tabibu S Vinod P Jawahar C Pan-renal cell carcinoma classification and survival prediction from histopathology images using deep learning Sci. Rep. 2019 9 10509 10.1038/s41598-019-46718-3 31324828 PMC6642160 · DOI
  15. [16]Dietterich TG Lathrop RH Lozano-Pérez T Solving the multiple instance problem with axis-parallel rectangles Artif. Intell. 1997 89 31 71 10.1016/S0004-3702(96)00034-3 · DOI · Paper page
  16. [17]Hubbard T The ensembl genome database project Nucleic Acids Res. 2002 30 38 41 10.1093/nar/30.1.38 11752248 PMC99161 · DOI
  17. [18]Bartha Á Győrffy B Tnmplot. com: a web tool for the comparison of gene expression in normal, tumor and metastatic tissues Int. J. Mol. Sci. 2021 22 2622 10.3390/ijms22052622 33807717 PMC7961455 · DOI
  18. [19]Luo M-S Huang G-J Liu B-X Immune infiltration in nasopharyngeal carcinoma based on gene expression Medicine 2019 98 e17311 10.1097/MD.0000000000017311 31574860 PMC6775332 · DOI
  19. [20]Hoffman GE Schadt E E. variancepartition: interpreting drivers of variation in complex gene expression studies BMC Bioinforma. 2016 17 1 13 10.1186/s12859-016-1323-z PMC5123296 27884101 · DOI · Paper page
  20. [21]Campbell-Staton SC Velotta JP Winchell KM Selection on adaptive and maladaptive gene expression plasticity during thermal adaptation to urban heat islands Nat. Commun. 2021 12 1 14 10.1038/s41467-021-26334-4 34702827 PMC8548502 · DOI
  21. [22]Avsec Ž Effective gene expression prediction from sequence by integrating long-range interactions Nat. Methods 2021 18 1196 1203 10.1038/s41592-021-01252-x 34608324 PMC8490152 · DOI · Paper page
  22. [25]Spyromitros-Xioufis E Tsoumakas G Groves W Vlahavas I Multi-target regression via input space expansion: treating targets as inputs Mach. Learn. 2016 104 55 98 10.1007/s10994-016-5546-z · DOI
  23. [26]Stacke, K., Eilertsen, G., Unger, J. & Lundström, C. A closer look at domain shift for deep learning in histopathology. Preprint at https://arxiv.org/abs/1909.11575 (2019). 33085623 · arXiv
  24. [27]Asilian Bidgoli A Rahnamayan S Dehkharghanian T Grami A Tizhoosh H Bias reduction in representation of histopathology images using deep feature selection Sci. Rep. 2022 12 1 12 10.1038/s41598-022-24317-z 36411301 PMC9678861 · DOI
  25. [28]Dehkharghanian, T. et al. Biased data, biased AI: deep networks predict the acquisition site of TCGA images. BMC Diagnostic Pathology (2023). PMC10189924 37198691
  26. [29]Safarpoor, A., Shafiei, S., Gonzalez, R., Parwani, A. & Tizhoosh, H. Renal cell carcinoma whole-slide image classification and search using deep learning. Research Square10.21203/rs.3.rs-971708/v1 (2021).
  27. [31]Dehghani, M., Zamani, H., Severyn, A., Kamps, J. & Croft, W. B. Neural ranking models with weak supervision. in Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval 65–74 (2017).
  28. [32]Sikaroudi, M. et al. Supervision and source domain impact on representation learning: a histopathology case study. in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) 1400–1403 (IEEE, 2020). 33018251
  29. [33]Gildenblat, J. & Klaiman, E. Self-supervised similarity learning for digital pathology. Preprint at https://arxiv.org/abs/1905.08139 (2019). · arXiv
  30. [34]Dosovitskiy, A. et al. An image is worth 16 × 16 words: Transformers for image recognition at scale. Preprint at https://arxiv.org/abs/2010.11929 (2020). · arXiv
  31. [35]Hendrycks, D. & Gimpel, K. Gaussian error linear units (gelus). Preprint at https://arxiv.org/abs/1606.08415 (2016). · arXiv
  32. [36]Loshchilov, I. & Hutter, F. Decoupled weight decay regularization. Preprint at https://arxiv.org/abs/1711.05101 (2017). · arXiv

Authors
Alsaafin, A., Safarpoor, A., Sikaroudi, M., Hipp, J. D., & Tizhoosh, H. R.
Affiliations
  • Mayo Clinic (US)
  • University of Waterloo (CA)
Corresponding author
Tizhoosh, H. R.
ORCID
Venue
Communications Biology
Publisher
Springer Science and Business Media LLC
Published
2023-03-22
Volume
6
Issue
1
Pages
304-304
ISSN
2399-3642 (ISSN-L)
DOI
10.1038/s42003-023-04583-x
PMC
PMC10033650
OpenAlex
W4353065773
Cited by
40
References
31
Citations by year
2026: 6 · 2025: 14 · 2024: 11 · 2023: 8 · 2022: 1
FWCI
3.68
Type
journal article
Language
en
Open access
Licence
Locations
  • Communications Biology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Government of Ontario
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Single-cell and spatial transcriptomics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Keywords
whole slide images, multiple instance learning, weakly supervised learning, digital pathology, computational pathology, tissue morphology, molecular fingerprints, image representation, image search
MeSH terms
RNA-Seq, Algorithms, Base Sequence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review