UNI-RNA: UNIVERSAL PRE-TRAINED MODELS REVOLUTIONIZE RNA RESEARCH
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
A bstract: RNA molecules play a crucial role as intermediaries in diverse biological processes. Attaining a profound understanding of their function can substantially enhance our comprehension of life’s activities and facilitate drug development for numerous diseases. The advent of high-throughput sequencing technologies makes vast amounts of RNA sequence data accessible, which contains invaluable information and knowledge. However, deriving insights for further application from such an immense volume of data poses a significant challenge. Fortunately, recent advancements in pre-trained models have surfaced as a revolutionary solution for addressing such challenges owing to their exceptional ability to automatically mine and extract hidden knowledge from massive datasets. Inspired by the past successes, we developed a novel context-aware deep learning model named Uni-RNA that performs pre-training on the largest dataset of RNA sequences at the unprecedented scale to date. During this process, our model autonomously unraveled the obscured evolutionary and structural information embedded within the RNA sequences. As a result, through fine-tuning, our model achieved the state-of-the-art (SOTA) performances in a spectrum of downstream tasks, including both structural and functional predictions. Overall, Uni-RNA established a new research paradigm empowered by the large pre-trained model in the field of RNA, enabling the community to unlock the power of AI at a whole new level to significantly expedite the pace of research and foster groundbreaking discoveries.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,990
MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets
Steinegger, M., & Söding, J. (2017) · Nature Biotechnology · doi:10.1038/nbt.3988 · Cited by 4,840 · cited 1×
“To further refine our database and reduce redundancy, we employed the mmseqs2 clustering algorithm [71].”
Pan, Q., Shai, O., Lee, L. J., Frey, B. J., & Blencowe, B. J. (2008) · Nature Genetics · doi:10.1038/ng.259 · Cited by 3,951 · cited 1×
“For instance, the study of mRNA requires intricate techniques to analyze processes such as alternative splicing, RNA editing, and mRNA decay[8–10].”
BackgroundLong Noncoding RNAs: Cellular Address Codes in Development and Disease
Batista, P. J., & Chang, H. Y. (2013) · Cell · doi:10.1016/j.cell.2013.02.012 · Cited by 2,531 · cited 1×
“Similarly, the more versatile yet less understood long non-coding RNAs (lncRNAs) also necessitate in-depth examination to decipher their complex mechanisms[11, 12].”
BackgroundRoFormer: Enhanced transformer with rotary position embedding
Su, J., Ahmed, M., Lu, Y., Pan, S., Bo, W., & Liu, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2023.127063 · Cited by 1,957
RoFormer: Enhanced Transformer with Rotary Position Embedding
Su, J., Lu, Y., Pan, S., Murtadha, A., Wen, B., & Liu, Y. (2021) · arXiv · doi:10.48550/arxiv.2104.09864 · Cited by 1,952 · cited 1×
“Owing to the position-agnostic nature of the transformer architecture, we integrate rotary embedding[72] to encapsulate positional information into the model.”
MethodTherapeutic siRNA: state of the art
Hu, B., Zhong, L., Weng, Y., Peng, L., Huang, Y., Zhao, Y., & Liang, X.-J. (2020) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-020-0207-x · Cited by 1,586 · cited 1×
“With future studies, we envision the Uni-RNA can help enabling efficient RNA therapy development, including ASO[63], SiRNA[64], Aptamer[65] and RNA targeting small molecules[66], given more corresponding experimental data and further training.”
mRNA vaccines for infectious diseases: principles, delivery and clinical translation
Chaudhary, N., Weissman, D., & Whitehead, K. A. (2021) · Nature Reviews Drug Discovery · doi:10.1038/s41573-021-00283-5 · Cited by 1,555
The highways and byways of mRNA decay
Garneau, N. L., Wilusz, J., & Wilusz, C. J. (2007) · Nature Reviews Molecular Cell Biology · doi:10.1038/nrm2104 · Cited by 1,294 · cited 1×
“For instance, the study of mRNA requires intricate techniques to analyze processes such as alternative splicing, RNA editing, and mRNA decay[8–10].”
Background
Cited by
ERNIE-RNA: an RNA language model with structure-enhanced representations
Yin, W., Zhang, Z., Zhang, S., He, L., Zhang, R., Jiang, R., Liu, G., Wang, J., Zhang, X., Qin, T., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-64972-0 · Cited by 55
Predicting RNA structures and functions by artificial intelligence.
Zhang, J., Lang, M., Zhou, Y., & Zhang, Y. (2024) · Trends in Genetics · doi:10.1016/j.tig.2023.10.001 · Cited by 48
BiRNA-BERT allows efficient RNA language modeling with adaptive tokenization
Tahmid, M. T., Shahgir, H. S., Mahbub, S., Dong, Y., & Bayzid, M. S. (2025) · Communications Biology · doi:10.1038/s42003-025-08982-0 · Cited by 16
RNA-TorsionBERT: leveraging language models for RNA 3D torsion angles prediction
Bernard, C., Postic, G., Ghannay, S., & Tahi, F. (2024) · doi:10.1101/2024.06.06.597803 · Cited by 16
RNA-TorsionBERT: leveraging language models for RNA 3D torsion angles prediction
Bernard, C., Postic, G., Ghannay, S., & Tahi, F. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btaf004 · Cited by 15
Metadata
- Authors
- Wang, X., Gu, R., Chen, Z., Li, Y., Ji, X., Ke, G., & Wen, H.
- Venue
- bioRxiv (Cold Spring Harbor Laboratory)
- Publisher
- openRxiv
- Published
- 2023-07-12
- OpenAlex
- W4384405439
- Cited by
- 55
- References
- 17
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: UNI-RNA: UNIVERSAL PRE-TRAINED MODELS REVOLUTIONIZE RNA RESEARCH. https://app.cochat.ai/papers/10.1101/2023.07.11.548588
- The paper
- Wang, X., Gu, R., Chen, Z., Li, Y., Ji, X., Ke, G., & Wen, H. (2023). UNI-RNA: UNIVERSAL PRE-TRAINED MODELS REVOLUTIONIZE RNA RESEARCH. bioRxiv (Cold Spring Harbor Laboratory). https://doi.org/10.1101/2023.07.11.548588
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.