UNI-RNA: UNIVERSAL PRE-TRAINED MODELS REVOLUTIONIZE RNA RESEARCH
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
A bstract: RNA molecules play a crucial role as intermediaries in diverse biological processes. Attaining a profound understanding of their function can substantially enhance our comprehension of life’s activities and facilitate drug development for numerous diseases. The advent of high-throughput sequencing technologies makes vast amounts of RNA sequence data accessible, which contains invaluable information and knowledge. However, deriving insights for further application from such an immense volume of data poses a significant challenge. Fortunately, recent advancements in pre-trained models have surfaced as a revolutionary solution for addressing such challenges owing to their exceptional ability to automatically mine and extract hidden knowledge from massive datasets. Inspired by the past successes, we developed a novel context-aware deep learning model named Uni-RNA that performs pre-training on the largest dataset of RNA sequences at the unprecedented scale to date. During this process, our model autonomously unraveled the obscured evolutionary and structural information embedded within the RNA sequences. As a result, through fine-tuning, our model achieved the state-of-the-art (SOTA) performances in a spectrum of downstream tasks, including both structural and functional predictions. Overall, Uni-RNA established a new research paradigm empowered by the large pre-trained model in the field of RNA, enabling the community to unlock the power of AI at a whole new level to significantly expedite the pace of research and foster groundbreaking discoveries.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,990
MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets
Steinegger, M., & Söding, J. (2017) · Nature Biotechnology · doi:10.1038/nbt.3988 · Cited by 4,840 · cited 1×
“To further refine our database and reduce redundancy, we employed the mmseqs2 clustering algorithm [71].”
Pan, Q., Shai, O., Lee, L. J., Frey, B. J., & Blencowe, B. J. (2008) · Nature Genetics · doi:10.1038/ng.259 · Cited by 3,951 · cited 1×
“For instance, the study of mRNA requires intricate techniques to analyze processes such as alternative splicing, RNA editing, and mRNA decay[8–10].”
BackgroundLong Noncoding RNAs: Cellular Address Codes in Development and Disease
Batista, P. J., & Chang, H. Y. (2013) · Cell · doi:10.1016/j.cell.2013.02.012 · Cited by 2,531 · cited 1×
“Similarly, the more versatile yet less understood long non-coding RNAs (lncRNAs) also necessitate in-depth examination to decipher their complex mechanisms[11, 12].”
BackgroundRoFormer: Enhanced transformer with rotary position embedding
Su, J., Ahmed, M., Lu, Y., Pan, S., Bo, W., & Liu, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2023.127063 · Cited by 1,957
RoFormer: Enhanced Transformer with Rotary Position Embedding
Su, J., Lu, Y., Pan, S., Murtadha, A., Wen, B., & Liu, Y. (2021) · arXiv · doi:10.48550/arxiv.2104.09864 · Cited by 1,952 · cited 1×
“Owing to the position-agnostic nature of the transformer architecture, we integrate rotary embedding[72] to encapsulate positional information into the model.”
MethodLong noncoding RNAs: past, present, and future
Kung, J. T. Y., Colognori, D., & Lee, J. T. (2013) · Genetics · doi:10.1534/genetics.112.146704 · Cited by 1,874
Therapeutic siRNA: state of the art
Hu, B., Zhong, L., Weng, Y., Peng, L., Huang, Y., Zhao, Y., & Liang, X.-J. (2020) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-020-0207-x · Cited by 1,586 · cited 1×
“With future studies, we envision the Uni-RNA can help enabling efficient RNA therapy development, including ASO[63], SiRNA[64], Aptamer[65] and RNA targeting small molecules[66], given more corresponding experimental data and further training.”
mRNA vaccines for infectious diseases: principles, delivery and clinical translation
Chaudhary, N., Weissman, D., & Whitehead, K. A. (2021) · Nature Reviews Drug Discovery · doi:10.1038/s41573-021-00283-5 · Cited by 1,555
Ji, Y., Zhou, Z., Liu, H., & Davuluri, R. V. (2021) · Bioinformatics · doi:10.1093/bioinformatics/btab083 · Cited by 1,303
The highways and byways of mRNA decay
Garneau, N. L., Wilusz, J., & Wilusz, C. J. (2007) · Nature Reviews Molecular Cell Biology · doi:10.1038/nrm2104 · Cited by 1,294 · cited 1×
“For instance, the study of mRNA requires intricate techniques to analyze processes such as alternative splicing, RNA editing, and mRNA decay[8–10].”
BackgroundFlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Dao, T., Fu, D. Y., Ermon, S., Rudra, A., & Ré, C. (2022) · arXiv · doi:10.48550/arxiv.2205.14135 · Cited by 1,214 · cited 1×
“Implementation Details In the pursuit of enhancing the efficacy of our proposed model, we have employed the IO-aware Flash Attention mechanism[73] in place of the traditional MultiHeadAttention.”
MethodUnlocking the promise of mRNA therapeutics
Rohner, E., Yang, R., Foo, K. S., Goedel, A., & Chien, K. R. (2022) · Nature Biotechnology · doi:10.1038/s41587-022-01491-z · Cited by 612 · cited 1×
“In the past decade, there has been a rising trend in the research and clinical development to manipulate mRNA to achieve diverse physiological functions[46, 47].”
BackgroundHuman 5′ UTR design and variant effect prediction from a massively parallel translation assay
Sample, P. J., Wang, B., Reid, D. W., Presnyak, V., McFadyen, I. J., Morris, D. R., & Seelig, G. (2019) · Nature Biotechnology · doi:10.1038/s41587-019-0164-5 · Cited by 442 · cited 2×
“We conduct experiments on benchmarks used in previous work[48].”
MethodThe accuracy of ribosomal RNA comparative structure models
Gutell, R. R., Lee, J. C., & Cannone, J. J. (2002) · Current Opinion in Structural Biology · doi:10.1016/s0959-440x(02)00339-1 · Cited by 364 · cited 1×
“To tackle these problems, numerous computational methods have been proposed and extensively investigated[24–30].”
MethodChen, J., Hu, Z., Sun, S., Tan, Q., Wang, Y., Yu, Q., Zong, L., Hong, L., Xiao, J., Shen, T., et al. (2022) · doi:10.1101/2022.08.06.503062 · Cited by 247 · cited 7×
“Besides, we also utilized the datasets employed by RNA-FM[33], which include random data (Random 7600) and human data (Human 7600).”
BackgroundMethodInfluentialSelf-citationLinearFold: linear-time approximate RNA folding by 5'-to-3' dynamic programming and beam search
Huang, L., Zhang, H., Deng, D., Zhao, K., Liu, K., Hendrix, D. A., & Mathews, D. H. (2019) · Bioinformatics · doi:10.1093/bioinformatics/btz375 · Cited by 207
The four ingredients of single-sequence RNA secondary structure prediction. A unifying perspective
Rivas, E. (2013) · RNA Biology · doi:10.4161/rna.24971 · Cited by 57
Cited by
ERNIE-RNA: an RNA language model with structure-enhanced representations
Yin, W., Zhang, Z., Zhang, S., He, L., Zhang, R., Jiang, R., Liu, G., Wang, J., Zhang, X., Qin, T., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-64972-0 · Cited by 55
Predicting RNA structures and functions by artificial intelligence.
Zhang, J., Lang, M., Zhou, Y., & Zhang, Y. (2024) · Trends in Genetics · doi:10.1016/j.tig.2023.10.001 · Cited by 48
BiRNA-BERT allows efficient RNA language modeling with adaptive tokenization
Tahmid, M. T., Shahgir, H. S., Mahbub, S., Dong, Y., & Bayzid, M. S. (2025) · Communications Biology · doi:10.1038/s42003-025-08982-0 · Cited by 16
RNA-TorsionBERT: leveraging language models for RNA 3D torsion angles prediction
Bernard, C., Postic, G., Ghannay, S., & Tahi, F. (2024) · doi:10.1101/2024.06.06.597803 · Cited by 16
RNA-TorsionBERT: leveraging language models for RNA 3D torsion angles prediction
Bernard, C., Postic, G., Ghannay, S., & Tahi, F. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btaf004 · Cited by 15
Metadata
- Authors
- Wang, X., Gu, R., Chen, Z., Li, Y., Ji, X., Ke, G., & Wen, H.
- Venue
- bioRxiv (Cold Spring Harbor Laboratory)
- Publisher
- openRxiv
- Published
- 2023-07-12
- OpenAlex
- W4384405439
- Cited by
- 55
- References
- 19
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.