Simulating 500 million years of evolution with a language model
Open accessPublished version of a preprint
Abstract
More than 3 billion years of evolution have produced an image of biology encoded into the space of natural proteins. Here, we show that language models trained at scale on evolutionary data can generate functional proteins that are far away from known proteins. We present ESM3, a frontier multimodal generative language model that reasons over the sequence, structure, and function of proteins. ESM3 can follow complex prompts combining its modalities and is highly responsive to alignment to improve its fidelity. We have prompted ESM3 to generate fluorescent proteins. Among the generations that we synthesized, we found a bright fluorescent protein at a far distance (58% sequence identity) from known fluorescent proteins, which we estimate is equivalent to simulating 500 million years of evolution.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Language-model-created proteins such as esmGFP or OpenCrispr1 are different from any of their naturally occurring counterparts.
“This type of evaluation is already considered within the molecular design space because language-model-created proteins, such as esmGFP52 or OpenCrispr1,53 highlight how different they are from any of their naturally occurring counterparts.”
From How to build the virtual cell with artificial intelligence: Priorities and opportunities (2024) · cites this paper as [52]
Cites
MAFFT Multiple Sequence Alignment Software Version 7: Improvements in Performance and Usability
Katoh, K., & Standley, D. M. (2013) · Molecular Biology and Evolution · doi:10.1093/molbev/mst010 · Cited by 46,209 · see citing works
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · see citing works
Accurate structure prediction of biomolecular interactions with AlphaFold 3
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A. J., Bambrick, J., et al. (2024) · Nature · doi:10.1038/s41586-024-07487-w · Cited by 15,861 · see citing works
Self-citationUniProt: the Universal Protein Knowledgebase in 2023
The UniProt Consortium, Bateman, A., Martin, M.-J., Orchard, S., Magrane, M., Ahmad, S., Alpi, E., Bowler-Barnett, E. H., Britto, R., Bye-A-Jee, H., et al. (2023) · Nucleic Acids Research · doi:10.1093/nar/gkac1052 · Cited by 7,210 · see citing works
Evolutionary-scale prediction of atomic level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al. (2022) · doi:10.1101/2022.07.20.500902 · Cited by 5,523 · see citing works
Self-citationRives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2016239118 · Cited by 3,286 · see citing works
Self-citationRives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., et al. (2019) · doi:10.1101/622803 · Cited by 3,280 · see citing works
Self-citationFast and accurate protein structure search with Foldseek
van Kempen, M., Kim, S. S., Tumescheit, C., Mirdita, M., Lee, J., Gilchrist, C. L. M., Söding, J., & Steinegger, M. (2022) · doi:10.1101/2022.02.07.479398 · Cited by 2,598 · see citing works
Fast and accurate protein structure search with Foldseek
van Kempen, M., Kim, S. S., Tumescheit, C., Mirdita, M., Lee, J., Gilchrist, C. L. M., Söding, J., & Steinegger, M. (2024) · Nature Biotechnology · doi:10.1038/s41587-023-01773-0 · Cited by 2,596 · see citing works
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 2,523 · see citing works
Watson, J. L., Juergens, D., Bennett, N. R., Trippe, B. L., Yim, J., Eisenach, H. E., Ahern, W., Borst, A. J., Ragotte, R. J., Milles, L. F., et al. (2022) · doi:10.1101/2022.12.09.519842 · Cited by 2,369 · see citing works
ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., et al. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3095381 · Cited by 2,329 · see citing works
De novo design of protein structure and function with RFdiffusion
Watson, J. L., Juergens, D., Bennett, N. R., Trippe, B. L., Yim, J., Eisenach, H. E., Ahern, W., Borst, A. J., Ragotte, R. J., Milles, L. F., et al. (2023) · Nature · doi:10.1038/s41586-023-06415-8 · Cited by 2,325 · see citing works
Varadi, M., Bertoni, D., Magana, P., Paramval, U., Pidruchna, I., Radhakrishnan, M., Tsenkov, M., Nair, S., Mirdita, M., Yeo, J., et al. (2024) · Nucleic Acids Research · doi:10.1093/nar/gkad1011 · Cited by 2,052 · see citing works
Unified rational protein engineering with sequence-based deep representation learning
Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., & Church, G. M. (2019) · Nature Methods · doi:10.1038/s41592-019-0598-1 · Cited by 1,131 · see citing works
Classifier-Free Diffusion Guidance
Ho, J., & Salimans, T. (2022) · arXiv · doi:10.48550/arxiv.2207.12598 · Cited by 1,126 · see citing works
Large language models generate functional protein sequences across diverse families
Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos, J. L., Xiong, C., Sun, Z. Z., Socher, R., et al. (2023) · Nature Biotechnology · doi:10.1038/s41587-022-01618-2 · Cited by 1,078 · see citing works
Mutation effects predicted from sequence co-variation
Hopf, T. A., Ingraham, J. B., Poelwijk, F. J., Schärfe, C. P. I., Springer, M., Sander, C., & Marks, D. S. (2017) · Nature Biotechnology · doi:10.1038/nbt.3769 · Cited by 775 · see citing works
Modeling the language of life – Deep Learning Protein Sequences
Heinzinger, M., Elnaggar, A., Wang, Y., Dallago, C., Nechaev, D., Matthes, F., & Rost, B. (2019) · doi:10.1101/614313 · Cited by 606 · see citing works
Local fitness landscape of the green fluorescent protein
Sarkisyan, K. S., Bolotin, D. A., Meer, M. V., Usmanova, D. R., Mishin, A. S., Sharonov, G. V., Ivankov, D. N., Bozhanova, N. G., Baranov, M. S., Soylemez, O., et al. (2016) · Nature · doi:10.1038/nature17995 · Cited by 591 · see citing works
ProGen2: Exploring the boundaries of protein language models
Nijkamp, E., Ruffolo, J. A., Weinstein, E. N., Naik, N., & Madani, A. (2023) · Cell Systems · doi:10.1016/j.cels.2023.10.002 · Cited by 480 · see citing works
MaskGIT: Masked Generative Image Transformer
Chang, H., Zhang, H., Jiang, L., Liu, C., & Freeman, W. T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01103 · Cited by 446 · see citing works
Illuminating protein space with a programmable generative model
Ingraham, J. B., Baranov, M., Costello, Z., Barber, K. W., Wang, W., Ismail, A., Frappier, V., Lord, D. M., Ng-Thow-Hing, C., Van Vlack, E. R., et al. (2023) · Nature · doi:10.1038/s41586-023-06728-8 · Cited by 444 · see citing works
Fine-Tuning Language Models from Human Preferences
Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., & Irving, G. (2019) · arXiv · doi:10.48550/arxiv.1909.08593 · Cited by 413 · see citing works
Transformer protein language models are unsupervised structure learners
Rao, R., Meier, J., Sercu, T., Ovchinnikov, S., & Rives, A. (2020) · doi:10.1101/2020.12.15.422761 · Cited by 341 · see citing works
Self-citationLanguage models generalize beyond natural proteins
Verkuil, R., Kabeli, O., Du, Y., Wicky, B. I. M., Milles, L. F., Dauparas, J., Baker, D., Ovchinnikov, S., Sercu, T., & Rives, A. (2022) · doi:10.1101/2022.12.21.521521 · Cited by 181 · see citing works
Self-citationHaas, J., Barbato, A., Behringer, D., Studer, G., Roth, S., Bertoni, M., Mostaguir, K., Gumienny, R., & Schwede, T. (2018) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.25431 · Cited by 168 · see citing works
New prediction categories in <scp>CASP15</scp>
Kryshtafovych, A., Antczak, M., Szachniuk, M., Zok, T., Kretsch, R. C., Rangan, R., Pham, P., Das, R., Robin, X., Studer, G., et al. (2023) · Proteins: Structure, Function, and Bioinformatics · doi:10.1002/prot.26515 · Cited by 54 · see citing works
Small-scale proxies for large-scale Transformer training instabilities
Wortsman, M., Liu, P. J., Xiao, L., Everett, K., Alemi, A., Adlam, B., Co-Reyes, J. D., Gur, I., Kumar, A., Novak, R., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.14322 · Cited by 5
Flagged first, then by citations
Cited by
Sorted by citations
How to build the virtual cell with artificial intelligence: Priorities and opportunities
Bunne, C., Roohani, Y., Rosen, Y., Gupta, A., Zhang, X., Roed, M., Alexandrov, T., AlQuraishi, M., Brennan, P., Burkhardt, D. B., et al. (2024) · Cell · doi:10.1016/j.cell.2024.11.015 · Cited by 393 · see citing works
Self-citationPredicting cellular responses to perturbation across diverse contexts with State
Adduri, A. K., Gautam, D., Bevilacqua, B., Naghipourfar, M., Imran, A., Shah, R., Teyssier, N., Verma, R., Carpenter, C., Eraslan, B., et al. (2026) · Cell · doi:10.1016/j.cell.2026.07.052 · Cited by 100 · see citing works
Self-citationTowards end-to-end automation of AI research
Lu, C., Lu, C., Lange, R. T., Yamada, Y., Hu, S., Foerster, J., Ha, D., & Clune, J. (2026) · Nature · doi:10.1038/s41586-026-10265-5 · Cited by 116 · see citing works
Predicting the conformational flexibility of antibody and T-cell receptor CDRs
Spoendlin, F. C., Fernández-Quintero, M. L., Raghavan, S. S. R., Turner, H. L., Gharpure, A., Loeffler, J. R., Wong, W. K., Bujotzek, A., Georges, G., Ward, A. B., et al. (2025) · doi:10.1101/2025.03.19.644119 · Cited by 20 · see citing works
Reliable prediction of protein–protein binding affinity changes upon mutations with Pythia-PPI
Tao, F., Sun, J., Gao, P., Gao, G. F., & Wu, B. (2025) · National Science Review · doi:10.1093/nsr/nwaf231 · Cited by 10 · see citing works
Efficient protein structure generation with sparse denoising models
Jendrusch, M., & Korbel, J. O. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01100-z · Cited by 10 · see citing works
Liu, L., Liu, Y., Ye, X., Zhang, S., Li, Y., Zhu, K., Xue, Y., Zhou, J., Zhang, X., & Fang, X. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01264-2 · Cited by 1 · see citing works
Li, J., Shringarpure, S., Wong, E., & Ho, A. (2026) · doi:10.64898/2026.06.29.735386
- Authors
- Hayes, T., Rao, R., Akin, H., Sofroniew, N. J., Oktay, D., Lin, Z., Verkuil, R., Tran, V. Q., Deaton, J., Wiggert, M., Badkundri, R., Shafkat, I., Gong, J., Derry, A., Molina, R. S., Thomas, N., Khan, Y. A., Mishra, C., Kim, C., Bartie, L. J., Nemeth, M., Hsu, P. D., Sercu, T., Candido, S., & Rives, A.
- Affiliations
- EvolutionaryScale, PBC, New York, NY, USA.
- Arc Research Institute (US)
- Corresponding author
- Rives, Alexander
- ORCID
- Hayes, Thomas
- Rao, Roshan
- Akin, Halil
- Sofroniew, Nicholas J.
- Oktay, Deniz
- Lin, Zeming
- Verkuil, Robert
- Tran, Vincent Q.
- Deaton, Jonathan
- Wiggert, Marius
- Badkundri, Rohil
- Derry, Alexander
- Molina, Raul S.
- Thomas, Neil
- Khan, Yousuf A.
- Bartie, Liam J.
- Nemeth, Matthew
- Hsu, Patrick D.
- Sercu, Tom
- Candido, Salvatore
- Rives, Alexander
- Venue
- Science
- Published
- 2025-02-21
- Volume
- 387
- Issue
- 6736
- Pages
- 850-858
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W4406440058
- Cited by
- 937
- References
- 29
- Citations by year
- 2026: 486 · 2025: 382 · 2024: 65 · 2023: 2 · 2022: 1 · 2020: 1
- FWCI
- 261.83
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Science (publishedVersion, public-domain)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
- Keywords
- protein evolution, generative language models, multimodal language models, protein design, fluorescent proteins, protein structure prediction, protein function, evolutionary simulation
- MeSH terms
- Deep Learning, Computer Simulation, Language, Luminescent Proteins, Proteins, Sequence Alignment, Evolution, Molecular
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
