SetQuence & SetOmic: Deep Set Transformer-based Representations of Cancer Multi-Omics
Abstract
In oncology, Deep Learning has shown great potential to personalise tasks such as tumour type classification, based on per-patient omics data-sets. Being high dimensional, incorporation of such data in one model is a challenge, often leading to one-dimensional studies and, therefore, information loss. Instead, we first propose relying on non-fixed sets of mutated genome sequences, which can be used for supervised learning of oncology-relevant tasks by our Transformer-based Deep Neural Network, SETQUENCE. Second, we extend the model to incorporate these representations as well as multiple sources of omics data in a flexible way with SETOMIC. Evaluation, using these representations, shows improved robustness and reduced information loss compared to previous approaches, while still being computationally tractable. By means of Explainable Artificial Intelligence methods, our models are shown to be able to recapitulate the biological contribution of several features in cancer, such as individual expression loci. This validation opens the door to novel directions in multi-faceted genome-wide biomarker discovery and personalised treatment among other presently clinically relevant tasks.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Jurenaite et al. reported that genes with highest attribution scores all carried known biological significance in cancer occurrence, providing confirmatory evidence on reliability of the AI system.
“For the genetic data, the authors reported that the genes with the highest attribution scores all carried known biological significance in cancer occurrence, which provides valuable confirmatory evidence on the reliability of the AI system.”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [35]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Lecture Notes in Computer Science 1205
(1999) · Industrial Robot the international journal of robotics research and application · doi:10.1108/ir.1999.04926fae.001 · Cited by 39,228 · see citing works
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · see citing works
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881 · see citing works
The cancer genome atlas pan-cancer analysis project
The Cancer Genome Atlas Research Network, Weinstein, J. N., Collisson, E. A., Mills, G. B., Shaw, K. R. M., Ozenberger, B. A., Ellrott, K., Shmulevich, I., Sander, C., & Stuart, J. M. (2013) · Nature Genetics · doi:10.1038/ng.2764 · Cited by 9,092 · see citing works
The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups
METABRIC Group, Curtis, C., Shah, S. P., Chin, S.-F., Turashvili, G., Rueda, O. M., Dunning, M. J., Speed, D., Lynch, A. G., Samarajiwa, S., et al. (2012) · Nature · doi:10.1038/nature10983 · Cited by 6,097 · see citing works
The GeneCards suite: from gene data mining to disease genome sequence analyses
Stelzer, G., Rosen, N., Plaschkes, I., Zimmerman, S., Twik, M., Fishilevich, S., Stein, T. I., Nudel, R., Lieder, I., Mazor, Y., et al. (2016) · Current Protocols in Bioinformatics · doi:10.1002/cpbi.5 · Cited by 5,550 · see citing works
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · see citing works
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087 · see citing works
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · see citing works
Deep learning for healthcare: review, opportunities and challenges
Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2018) · Briefings in Bioinformatics · doi:10.1093/bib/bbx044 · Cited by 3,076 · see citing works
Multi-omics approaches to disease
Hasin, Y., Seldin, M., & Lusis, A. (2017) · Genome Biology · doi:10.1186/s13059-017-1215-1 · Cited by 2,953 · see citing works
Learning Important Features Through Propagating Activation Differences
Shrikumar, A., Greenside, P., & Kundaje, A. (2017) · arXiv · doi:10.48550/arxiv.1704.02685 · Cited by 2,761 · see citing works
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524 · see citing works
Advances in Intelligent Systems and Computing
Шаховська, Н. Б. (2016) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-45991-2 · Cited by 1,822 · see citing works
g:Profiler—a web-based toolset for functional profiling of gene lists from large-scale experiments
Reimand, J., Kull, M., Peterson, H., Hansen, J., & Vilo, J. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkm226 · Cited by 1,525 · see citing works
Ji, Y., Zhou, Z., Liu, H., & Davuluri, R. V. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab083 · Cited by 1,303 · see citing works
Extracellular Matrix in the Tumor Microenvironment and Its Impact on Cancer Therapy
Henke, E., Nandigama, R., & Ergün, S. (2020) · Frontiers in Molecular Biosciences · doi:10.3389/fmolb.2019.00160 · Cited by 1,142 · see citing works
An estimation of the number of cells in the human body
Bianconi, E., Piovesan, A., Facchin, F., Beraudi, A., Casadei, R., Frabetti, F., Vitale, L., Pelleri, M. C., Tassani, S., Piva, F., et al. (2013) · Annals of Human Biology · doi:10.3109/03014460.2013.807878 · Cited by 1,012 · see citing works
Integration strategies of multi-omics data for machine learning analysis
Picard, M., Scott-Boyer, M.-P., Bodein, A., Périn, O., & Droit, A. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.06.030 · Cited by 696 · see citing works
Tumor Microenvironment and Differential Responses to Therapy
Hirata, E., & Sahai, E. (2017) · Cold Spring Harbor Perspectives in Medicine · doi:10.1101/cshperspect.a026781 · Cited by 422 · see citing works
MOLI: multi-omics late integration with deep neural networks for drug response prediction
Sharifi-Noghabi, H., Zolotareva, O., Collins, C. C., & Ester, M. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz318 · Cited by 421 · see citing works
Frequency and spectrum of PIK3CA somatic mutations in breast cancer
Martínez‐Sáez, O., Chic, N., Pascual, T., Adamo, B., Vidal, M. J., González‐Farré, B., Sanfeliu, E., Schettini, F., Conte, B., Brasó‐Maristany, F., et al. (2020) · Breast Cancer Research · doi:10.1186/s13058-020-01284-9 · Cited by 387 · see citing works
Sox‐4, an Sry‐like HMG box protein, is a transcriptional activator in lymphocytes.
van de Wetering, M., Oosterwegel, M. A., van Norren, K., & Clevers, H. C. (1993) · The EMBO Journal · doi:10.1002/j.1460-2075.1993.tb06063.x · Cited by 348 · see citing works
Systems biology in cardiovascular disease: a multiomics approach
Joshi, A., Rienks, M., Theofilatos, K., & Mayr, M. (2020) · Nature Reviews Cardiology · doi:10.1038/s41569-020-00477-1 · Cited by 305 · see citing works
Elnaggar, A., Heinzinger, M., Dallago, C., Rihawi, G., Wang, Y., Jones, L., Gibbs, T., Fehér, T., Angerer, C. M., Steinegger, M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06225 · Cited by 305 · see citing works
Integrative multi-omics approaches in cancer research: from biological networks to clinical subtypes
Heo, Y. J., Hwa, C., Lee, G.-H., Park, J.-M., & An, J.-Y. (2021) · Molecules and Cells · doi:10.14348/molcells.2021.0042 · Cited by 294 · see citing works
Machine learning and statistical methods for clustering single-cell RNA-sequencing data
Petegrosso, R., Li, Z., & Kuang, R. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz063 · Cited by 277 · see citing works
Murphy, B. J., Andrews, G. K., Bittel, D. C., Discher, D. J., McCue, J. M., Green, C. J., Yanovsky, M., Giaccia, A. J., Sutherland, R. M., Laderoute, K. R., et al. (1999) · PubMed · Cited by 166 · see citing works
To Embed or Not: Network Embedding as a Paradigm in Computational Biology
Nelson, W., Žitnik, M., Wang, B., Leskovec, J., Goldenberg, A., & Sharan, R. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.00381 · Cited by 159 · see citing works
Vargas, A. C., Reed, A. E. M., Waddell, N., Lane, A., Reid, L., Smart, C. E., Cocciardi, S., Silva, L. M. D., Song, S., Chenevix‐Trench, G., et al. (2012) · Breast Cancer Research and Treatment · doi:10.1007/s10549-012-2123-4 · Cited by 147 · see citing works
Variational autoencoders for cancer data integration: design principles and computational practice
Simidjievski, N., Bodnar, C., Tariq, I., Scherer, P., Andres Terre, H., Shams, Z., Jamnik, M., & Liò, P. (2019) · Frontiers in Genetics · doi:10.3389/fgene.2019.01205 · Cited by 145 · see citing works
DeepGene: an advanced cancer type classifier based on deep learning and somatic point mutations
Yuan, Y., Shi, Y. Y., Li, C., Kim, J., Cai, W., Han, Z., & Feng, D. D. (2016) · BMC Bioinformatics · doi:10.1186/s12859-016-1334-9 · Cited by 138 · see citing works
OmiEmbed: A Unified Multi-Task Deep Learning Framework for Multi-Omics Data
Zhang, X., Xing, Y., Sun, K., & Guo, Y. (2021) · Cancers · doi:10.3390/cancers13123047 · Cited by 112 · see citing works
(1935) · The Lancet · doi:10.1016/s0140-6736(00)57102-5 · Cited by 112 · see citing works
Tang, Z., Shen, Q., Xie, H., Zhou, X., Li, J., Feng, J., Liu, H., Wang, W., Zhang, S., & Ni, S. (2016) · Oncotarget · doi:10.18632/oncotarget.10086 · Cited by 110 · see citing works
Withnell, E., Zhang, X., Sun, K., & Guo, Y. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbab315 · Cited by 106 · see citing works
Xu, H., Yan, M., Patra, J., Natrajan, R., Yan, Y., Swagemakers, S. M. A., Tomaszewski, J. M., Verschoor, S., Millar, E. K. A., van der Spek, P. J., et al. (2011) · Breast Cancer Research · doi:10.1186/bcr2814 · Cited by 102 · see citing works
LOXL2 Inhibitors and Breast Cancer Progression
Ferreira, S., Saraiva, N., Rijo, P. M., & Fernandes, A. S. (2021) · Antioxidants · doi:10.3390/antiox10020312 · Cited by 101 · see citing works
Saux, C. J., Tronecker, H., Bogić, L., Bryant‐Greenwood, G. D., Boyd, C. D., & Csiszár, K. (1999) · Journal of Biological Chemistry · doi:10.1074/jbc.274.18.12939 · Cited by 94 · see citing works
Distinct prognostic values of S100 mRNA expression in breast cancer
Zhang, S., Wang, Z., Liu, W., Lei, R., Shan, J., Li, L., & Wang, X. (2017) · Scientific Reports · doi:10.1038/srep39786 · Cited by 90 · see citing works
Erin, N., Podnos, A., Tanriover, G., Duymuş, Ö., Côté, É., Khatri, I. A., & Gorczynski, R. M. (2014) · Oncogene · doi:10.1038/onc.2014.317 · Cited by 81 · see citing works
PKA signaling drives mammary tumorigenesis through Src
Beristain, A. G., Molyneux, S. D., Joshi, P. A., Pomroy, N. C., Grappa, M. A. D., Chang, M. C., Kirschner, L. S., Privé, G. G., Pujana, M. Á., & Khokha, R. (2014) · Oncogene · doi:10.1038/onc.2014.41 · Cited by 80 · see citing works
Pignatelli, M., Cocca, C., Santos, Á., & Pérez‐Castillo, A. (2003) · Oncogene · doi:10.1038/sj.onc.1206824 · Cited by 61 · see citing works
Xue, F., Shi, Z., Wei, F., Lou, Y., Liu, Y., & You, Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20858 · Cited by 61 · see citing works
MYADM regulates Rac1 targeting to ordered membranes required for cell spreading and migration
Aranda, J. F., Reglero-Real, N., Kremer, L., Marcos‐Ramiro, B., Ruiz-Sáenz, A., Calvo, M., Enrich, C., Correas, I., Millán, J., & Alonso, M. A. (2011) · Molecular Biology of the Cell · doi:10.1091/mbc.e10-11-0910 · Cited by 55 · see citing works
Clinical and misdiagnosed analysis of primary pulmonary lymphoma: a retrospective study
Yao, D., Zhang, L., Wu, P. L., Gu, X., Chen, Y., Wang, L. X., & Huang, X. (2018) · BMC Cancer · doi:10.1186/s12885-018-4184-1 · Cited by 52 · see citing works
Mut2Vec: distributed representation of cancerous mutations
Kim, S., Lee, H., Kim, K., & Kang, J. (2018) · BMC Medical Genomics · doi:10.1186/s12920-018-0349-7 · Cited by 51 · see citing works
Systematic identification of the key candidate genes in breast cancer stroma
Wang, Y., Xu, H., Zhu, B., Qiu, Z., & Lin, Z. (2018) · Cellular & Molecular Biology Letters · doi:10.1186/s11658-018-0110-4 · Cited by 50 · see citing works
Learning Activation Functions: A new paradigm for understanding Neural NetworksPreprint
Goyal, M., Goyal, R., & Lall, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.09529 · Cited by 50 · see citing works
The Difficulty of Training Sparse Neural NetworksPreprint
Evci, U., Pedregosa, F., Gomez, A. N., & Elsen, E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.10732 · Cited by 47 · see citing works
Mazlan, A. U., Sahabudin, N. A., Remli, M. A., Ismail, N. S. N., Mohamad, M. S., Nies, H. W., & Warif, N. B. A. (2021) · Processes · doi:10.3390/pr9081466 · Cited by 46 · see citing works
Tang, Y., & Ha, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.02869 · Cited by 41 · see citing works
SLC39A6: a potential target for diagnosis and therapy of esophageal carcinoma
Cui, X., Shen, Y., Jin, T., Su, L., Li, T., Zhang, S., Peng, H., Liu, C., Li, S., Yang, L., et al. (2015) · Journal of Translational Medicine · doi:10.1186/s12967-015-0681-z · Cited by 39 · see citing works
Gal, J., Bailleux, C., Chardin, D., Pourcher, T., Gilhodes, J., Jing, L., Guigonis, J., Ferrero, J.-., Milano, G. A., Mograbi, B., et al. (2020) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2020.05.021 · Cited by 39 · see citing works
Yuan, B., Yang, D., Rothberg, B. E. G., Chang, H., & Xu, T. (2020) · Scientific Reports · doi:10.1038/s41598-020-75715-0 · Cited by 36 · see citing works
DNA Methylation Predicts the Response of Triple-Negative Breast Cancers to All-Trans Retinoic Acid
Coyle, K. M., Dean, C. A., Thomas, M. L., Vidovic, D., Giacomantonio, C. A., Helyer, L. K., & Marcato, P. (2018) · Cancers · doi:10.3390/cancers10110397 · Cited by 27 · see citing works
Mining mutation contexts across the cancer genome to map tumor site of origin
Chakraborty, S., Martin, A. S., Guan, Z., Begg, C. B., & Shen, R. (2021) · Nature Communications · doi:10.1038/s41467-021-23094-z · Cited by 22 · see citing works
Ming, W., Xie, H., Hu, Z., Chen, Y., Zhu, Y., Bai, Y., Liu, H., Sun, X. S., Liu, Y., & Gu, W. (2019) · Frontiers in Oncology · doi:10.3389/fonc.2019.00985 · Cited by 21 · see citing works
Tao, Y., Cai, C., Cohen, W. W., & Lu, X. (2019) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789811215636_0008 · Cited by 19 · see citing works
K-Means Clustering with Infinite Feature Selection for Classification Tasks in Gene Expression Data
Remli, M. A., Daud, K. M., Nies, H. W., Mohamad, M. S., Deris, S., Omatu, S., Kasim, S. B., & Sulong, G. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-60816-7_7 · Cited by 14 · see citing works
OmniOpt – A Tool for Hyperparameter Optimization on HPC
Winkler, P., Koch, N., Hornig, A., & Gerritzen, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-90539-2_19 · Cited by 9 · see citing works
Primary Pulmonary Round Cell Sarcomas: Multiple Potential Pitfalls for the Pathologist
Richards, R., Jour, G., Tafe, L. J., Pinto, A., Brčić, I., Linos, K. D., & Kerr, D. A. (2022) · International Journal of Surgical Pathology · doi:10.1177/10668969221091586 · Cited by 4 · see citing works
Hagenah, J., Kühl, K., Scharfschwerdt, M., & Ernst, F. (2018) · Cited by 3 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
1 of the 70 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Pahud de Mortanges, A., Luo, H., Shu, S. Z., Kamath, A., Suter, Y., Shelan, M., Pöllinger, A., & Reyes, M. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01190-w · Cited by 93 · see citing works
“For example, Jurenaite et al. used non-fi xed sets of mutated genome sequences (mutomes) and transcriptomes in a transformer-based deep neural network, aiming to predict seven common tumor types 35 .”
A review on the applications of Transformer-based language models for nucleotide sequence analysis
Ghosh, N., Santoni, D., Saha, I., & Felici, G. (2025) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2025.03.024 · Cited by 31 · see citing works
“In this regard, Jure-naite et al. [88] have proposed SETQUENCE which is a Transformer-based deep neural network for supervised learning of oncology related tasks.”
Sabry, Z., Arora, H. S., Chandrasekaran, S., & Wang, Z. (2025) · Exploration of Medicine · doi:10.37349/emed.2025.1001340 · Cited by 17 · see citing works
“To better understand the biological contribution of several features in cancer, researchers recently developed a transformer XAI that takes mutated genetic sequences and combines it with additional omics data, such as transcriptomics to determine oncology-relevant outcomes such as tumor type [41].”
XAI Unveiled: Revealing the Potential of Explainable AI in Medicine - A Systematic Review
Scarpato, N., Ferroni, P., & Guadagni, F. (2024) · IEEE Access · doi:10.1109/access.2024.3514197 · Cited by 7 · see citing works
Păduraru, D. N., Palcău, A. C., Gorecki, G.-P., Dinulescu, A., & Băean, M.-L. (2026) · International Journal of Molecular Sciences · doi:10.3390/ijms27093951 · Cited by 3 · see citing works
“Transformer-based architectures, originally developed for natural language processing, are being adapted for genomic sequence modeling, offering potential for simultaneous analysis of mutation, methylation, and fragmentation features within a unified model [83,84].”
- Authors
- Jurenaite, N., León-Periñán, D., Donath, V., Torge, S., & Jäkel, R.
- Affiliations
- Center for Information Services and High Performance Computing (DE)
- Venue
- 2022 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB)
- Publisher
- IEEE
- Published
- 2022-08-15
- Pages
- 1-9
- OpenAlex
- W4293518013
- Cited by
- 5
- References
- 70
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 2
- FWCI
- 18.24
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) (publishedVersion)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Cancer Research › Cancer Genomics and Diagnostics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Keywords
- multi-omics integration, cancer genomics, explainable AI, personalised cancer treatment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
