CoChat

RiNALMo: general-purpose RNA language models can generalize well on structure prediction tasks

Open access
Penić, Rafael Josip, Vlašić, Tin, Huber, Roland G., Wan, Yue, Šikić, Mile · 2025 · Nature Communications
Cited by 122

The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.

Abstract

While RNA has recently been recognized as an interesting small-molecule drug target, many challenges remain to be addressed before we take full advantage of it. This emphasizes the necessity to improve our understanding of its structures and functions. Over the years, sequencing technologies have produced an enormous amount of unlabeled RNA data, which hides a huge potential. Motivated by the successes of protein language models, we introduce RiboNucleic Acid Language Model (RiNALMo) to unveil the hidden code of RNA. RiNALMo is the largest RNA language model to date, with 650M parameters pre-trained on 36M non-coding RNA sequences from several databases. It can extract hidden knowledge and capture the underlying structure information implicitly embedded within the RNA sequences. RiNALMo achieves state-of-the-art results on several downstream tasks. Notably, we show that its generalization capabilities overcome the inability of other deep learning methods for secondary structure prediction to generalize on unseen RNA families. RiNALMo, a large-scale RNA language model trained on non-coding RNA sequences, captures structural information and achieves state-of-the-art performance on multiple tasks, notably generalizing to unseen RNA families in secondary structure prediction.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41467-025-60872-5

Cites

25 of 25 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 25 references

Research works printed but not linked: 47

  1. Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), (Minneapolis, Minnesota, USA, 2019).
  2. Nambiar, A. et al. Transforming the language of life: transformer neural networks for protein prediction tasks. In: Proc. 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, 1–8 (ACM, 2020). · DOI · Paper page
  3. Childs-Disney, J. L. et al. Targeting RNA structures with small molecules. Nat. Rev. Drug Discov. 21, 736–762 (2022). · DOI · Paper page
  4. Garner, A. L. Contemporary progress and opportunities in RNA-targeted drug discovery. ACS Med. Chem. Lett. 14, 251–259 (2023). · DOI
  5. Schneider, B. et al. When will RNA get its AlphaFold moment? Nucleic Acids Res. 51, 9522–9532 (2023). · DOI · Paper page
  6. Wang, X. et al. Uni-RNA Universal pre-trained models revolutionize RNA research. bioRxiv https://doi.org/10.1101/2023.07.11.548588 (2023). · DOI · Paper page
  7. Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017).
  8. RNAcentral Consortium RNAcentral 2021: secondary structure integration, improved sequence search and new member databases. Nucleic Acids Res. 49 212–220 (2020). · DOI · Paper page
  9. Zhang, Y. et al. Multiple sequence alignment-based RNA language model and its application to structural inference. Nucleic Acids Res. 52, 3–3 (2024). · DOI · Paper page
  10. Chen, K. et al. Self-supervised learning on millions of primary RNA sequences from 72 vertebrates improves sequence-based RNA splicing prediction, Brief. Bioinform. 25, bbae163 (2024). · DOI
  11. Ji, Y., Zhou, Z., Liu, H. & Davuluri, R. V. DNABERT pre-trained bidirectional encoder representations from transformers model for DNA-language in genome. Bioinformatics 37, 2112–2120 (2021). · DOI · Paper page
  12. Celaj, A. et al. An RNA foundation model enables discovery of disease mechanisms and candidate therapeutics. bioRxiv https://doi.org/10.1101/2023.09.20.558508 (2023). · DOI
  13. Chu, Y. et al. A 5’ UTR language model for decoding untranslated regions of mRNA and function predictions. Nat. Mach. Intell. 6, 449–460 (2024). · DOI · Paper page
  14. Reuter, J. S. & Mathews, D. H. RNAstructure software for RNA secondary structure prediction and analysis. BMC Bioinform. 11, 129 (2010). · DOI · Paper page
  15. Do, C. B., Woods, D. A. & Batzoglou, S. CONTRAfold: RNA secondary structure prediction without physics-based models. Bioinformatics 22, 90–98 (2006). · DOI · Paper page
  16. Singh, J., Hanson, J., Paliwal, K. & Zhou, Y. RNA secondary structure prediction using an ensemble of two-dimensional deep neural networks and transfer learning. Nat. Commun. 10, 5407 (2019). · DOI · Paper page
  17. Fu, L. et al. UFold: fast and accurate RNA secondary structure prediction with deep learning. Nucleic Acids Res. 50, 14–14 (2021). · DOI · Paper page
  18. Sato, K., Akiyama, M. & Sakakibara, Y. RNA secondary structure prediction using deep learning with thermodynamic integration. Nat. Commun. 12, 941 (2021). · DOI · Paper page
  19. Danaee, P. et al. bpRNA large-scale automated annotation and analysis of RNA secondary structure. Nucleic Acids Res. 46, 5381–5394 (2018). · DOI · Paper page
  20. Berman, H. M. et al. The Protein Data Bank. Nucleic Acids Res. 28, 235–242 (2000). · DOI · Paper page
  21. Yang, H. et al. Tools for the automatic identification and classification of RNA base pairs. Nucleic Acids Res. 31, 3450–3460 (2003). · DOI · Paper page
  22. Andronescu, M., Condon, A., Hoos, H. H., Mathews, D. H. & Murphy, K. P. Efficient parameter estimation for RNA secondary structure prediction. Bioinformatics 23, 19–28 (2007). · DOI · Paper page
  23. Andronescu, M., Condon, A., Hoos, H. H., Mathews, D. H. & Murphy, K. P. Computational approaches for RNA energy parameter estimation. RNA 16, 2304–2318 (2010). · DOI · Paper page
  24. Kalvari, I. et al. Rfam 14: expanded coverage of metagenomic, viral and microRNA families. Nucleic Acids Res. 49, 192–200 (2020). · DOI · Paper page
  25. Wayment-Steele, H. K. et al. RNA secondary structure packages evaluated and improved by high-throughput experiments. Nat. Methods 19, 1234–1242 (2022). · DOI · Paper page
  26. Szikszai, M., Wise, M., Datta, A., Ward, M. & Mathews, D. H. Deep learning models for RNA secondary structure prediction (probably) do not generalize across families. Bioinformatics 38, 3892–3899 (2022). · DOI · Paper page
  27. Justyna, M., Antczak, M. & Szachniuk, M. Machine learning for RNA 2D structure prediction benchmarked on experimental data. Brief. Bioinforma. 24, 153 (2023). · DOI · Paper page
  28. Sloma, MichaelFandMathews,DavidH. Exact calculation of loop formation probability identifies folding motifs in RNA secondary structures. RNA 22, 1808–1818 (2016). · DOI · Paper page
  29. Mathews, D. H. How to benchmark RNA secondary structure prediction accuracy. Methods 162, 60–67 (2019). · DOI · Paper page
  30. Hamada, M., Sato, K., Kiryu, H., Mituyama, T. & Asai, K. Predictions of RNA secondary structure by combining homologous sequence information. Bioinformatics 25, 330–338 (2009). · DOI
  31. Will, S., Reiche, K., Hofacker, I. L., Stadler, P. F. & Backofen, R. Inferring noncoding RNA families and classes by means of genome-scale structure-based clustering. PLoS Comput. Biol. 3, 65 (2007). · DOI · Paper page
  32. Hamada, M., Sato, K. & Asai, K. Improving the accuracy of predicting secondary structure for aligned RNA sequences. Nucleic Acids Res. 39, 393–402 (2010). · DOI
  33. Puton, T., Kozlowski, L. P., Rother, K. M. & Bujnicki, J. M. CompaRNA: a server for continuous benchmarking of automated methods for RNA secondary structure prediction. Nucleic Acids Res. 41, 4307–4323 (2013). · DOI
  34. Nawrocki, E. P. & Eddy, S. R. Infernal 1.1: 100-fold faster RNA homology searches. Bioinformatics 29, 2933–2935 (2013). · DOI · Paper page
  35. Scalzitti, N. et al. Spliceator: multi-species splice site prediction using convolutional neural networks. BMC Bioinforma. 22, 1–26 (2021). · DOI · Paper page
  36. Scalzitti, N., Jeannin-Girardon, A., Collet, P., Poch, O. & Thompson, J. D. A benchmark study of ab initio gene prediction methods in diverse eukaryotic organisms. BMC Genom. 21, 1–20 (2020). · DOI · Paper page
  37. Kalvari, I. et al. Rfam 13.0: shifting to a genome-centric resource for non-coding RNA families. Nucleic Acids Res. 46, 335–342 (2018). · DOI · Paper page
  38. Noviello, T. M. R., Ceccarelli, F., Ceccarelli, M. & Cerulo, L. Deep learning predicts short non-coding RNA functions from only raw sequence data. PLoS Comput. Biol. 16, 1008415 (2020). · DOI
  39. Cao, J. et al. High-throughput 5’ UTR engineering for enhanced protein production in non-viral gene therapies. Nat. Commun. 12, 4138 (2021). · DOI · Paper page
  40. Anand, R. ett al. RNA-FrameFlow: flow matching for de novo 3D RNA backbone design. https://doi.org/10.48550/arXiv.2406.13839 (2024). · DOI
  41. Churkin, A., Barash, D. (eds.) RNA design: methods and protocols. https://doi.org/10.1007/978-1-0716-4079-1 (Springer, 2024). · doi:10.1007/978-1-0716-4079-1
  42. Xiong, R. et al. On layer normalization in the transformer architecture. In: Proc. International Conference on Machine Learning, 10524–10533 (PMLR, 2020).
  43. Parisien, M., Cruz, J. A., Westhof, E. & Major, F. New metrics for comparing and assessing discrepancies between RNA 3D structures and models. RNA 15, 1875–1885 (2009). · DOI
  44. Kerpedjiev, P., Hammer, S. & Hofacker, I. L. Forna (force-directed rna): Simple and effective online rna secondary structure diagrams. Bioinformatics 31, 3377–3379 (2015). · DOI · Paper page
  45. Paszke, A. et al. Pytorch: an imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst. 32, 8024–8035 (2019).
  46. Sayers, E. W. et al. Database resources of the National Center for Biotechnology Information in 2023. Nucleic Acids Res. 51, 29–38 (2022). · DOI · Paper page
  47. Martin, F. J. et al. Ensembl 2023. Nucleic Acids Res. 51, 933–941 (2022). · DOI · Paper page

Cited by

See all 122

Metadata

Authors
Penić, R. J., Vlašić, T., Huber, R. G., Wan, Y., & Šikić, M.
Venue
Nature Communications
Published
2025-07-01
Volume
16
Issue
1
Pages
5671-5671
ISSN
2041-1723 (ISSN-L)
OpenAlex
W4411886797
Cited by
122
References
25
FWCI
29.40
Type
journal article
Language
en
Open access
gold
Licence
Funders
  • National Research Foundation Singapore (NRF-CRP27-2021RS-0001)
  • Agency for Science, Technology and Research (I23D1AG079)
  • National Supercomputing Centre (NSCC), Singapore

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: RiNALMo: general-purpose RNA language models can generalize well on structure prediction tasks. https://app.cochat.ai/papers/10.1038/s41467-025-60872-5
The paper
Penić, R. J., Vlašić, T., Huber, R. G., Wan, Y., & Šikić, M. (2025). RiNALMo: general-purpose RNA language models can generalize well on structure prediction tasks. Nature Communications. https://doi.org/10.1038/s41467-025-60872-5

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.