The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index; 9 of the 16 sources its claims cite were checked for support.
Evolutionary-scale prediction of atomic level protein structure with a language model
Open accessAbstract
Artificial intelligence has the potential to open insight into the structure of proteins at the scale of evolution. It has only recently been possible to extend protein structure prediction to two hundred million cataloged proteins. Characterizing the structures of the exponentially growing billions of protein sequences revealed by large scale gene sequencing experiments would necessitate a break-through in the speed of folding. Here we show that direct inference of structure from primary sequence using a large language model enables an order of magnitude speed-up in high resolution structure prediction. Leveraging the insight that language models learn evolutionary patterns across millions of sequences, we train models up to 15B parameters, the largest language model of proteins to date. As the language models are scaled they learn information that enables prediction of the three-dimensional structure of a protein at the resolution of individual atoms. This results in prediction that is up to 60x faster than state-of-the-art while maintaining resolution and accuracy. Building on this, we present the ESM Metage-nomic Atlas. This is the first large-scale structural characterization of metagenomic proteins, with more than 617 million structures. The atlas reveals more than 225 million high confidence predictions, including millions whose structures are novel in comparison with experimentally determined structures, giving an unprecedented view into the vast breadth and diversity of the structures of some of the least understood proteins on earth.
Overview
CoChat Summary
TL;DR
The paper introduces ESM-2 protein language models up to 15B parameters and ESMFold, an end-to-end single-sequence structure predictor that reaches atomic-resolution accuracy while being up to 60x faster than state-of-the-art pipelines. It uses ESMFold to build the ESM Metagenomic Atlas, folding over 617 million MGnify90 sequences and identifying more than 225 million high-confidence structures, many novel relative to known sequences and structures.
Key findings
- Scaling ESM-2 from 8M to 15B parameters improves language modeling and structural information: validation perplexity falls from 10.45 to 6.37, unsupervised long-range contact precision (LR P@L) rises from 0.16 to 0.54, and structure-module-only LDDT rises from 0.48 to 0.72. The 15B model reaches TM-score 0.71 on CAMEO and 0.54 on CASP14, 0.064 higher than the 150M ESM-2 model on both.
- ESMFold achieves average TM-score 0.83 on CAMEO and 0.68 on CASP14, compared with AlphaFold2 at 0.88 and 0.85 and RosettaFold at 0.82 on CAMEO. On a single NVIDIA V100 GPU, ESMFold predicts a 384-residue protein in 14.2 seconds, 6x faster than a single AlphaFold2 model, with up to ~60x speedup on shorter sequences; MSA search alone can take over 10 minutes.
- The ESM Metagenomic Atlas folds over 617 million MGnify90 sequences in 2 weeks on ~2,000 GPUs, yielding ~365 million good-confidence predictions (mean pLDDT > 0.5 and pTM > 0.5) and ~225 million high-confidence predictions (mean pLDDT > 0.7 and pTM > 0.7). In a 1-million high-confidence sample, 76.8% have <90% sequence identity to UniRef90, 12.6% have no PDB structure with TM-score > 0.5, and 10.4% have both low structural similarity (TM-score ≤ 0.5) and no close sequence homolog (<30% identity).
- Confidence is well calibrated: for high-confidence predictions (pLDDT > 0.7), CAMEO accuracy is comparable to AlphaFold2 (ESMFold LDDT = 0.83, AlphaFold2 LDDT = 0.85); for very high confidence (pLDDT > 0.9), median all-atom RMSD95 is 1.42 Å and backbone RMSD95 is 0.94 Å.
Methods
- ESM-2 is a BERT-style transformer masked language model trained on UniRef50/UniRef90 sequences, scaled from 8M to 15B parameters, with models trained for up to 500K updates (15B for 270K updates) and evaluated on held-out UniRef50 clusters.
- ESMFold adds a 48-block folding head to a frozen 3B-parameter ESM-2, trained on ~25K PDB clusters covering ~325K experimentally determined chains plus ~12M AlphaFold2-predicted structures; evaluation uses temporally held-out CAMEO (194 proteins) and CASP14 (51 targets) test sets.
- Metagenomic analysis folded MGnify90 sequences of length 20–1024, covering 99% of the database, and used Foldseek, MMseqs2, blastp, and UMAP for structural and sequence similarity analyses.
Limitations
- The paper states that ESMFold was not trained on protein complexes, so complex prediction is out of distribution; it also notes that ablating MSAs for AlphaFold and RosettaFold is an artificial single-sequence setting because those models were not explicitly trained for single sequences.
- The paper states that calibration becomes critical as structure prediction scales, and that current models are far from the limit of scale in parameters, sequence data, and compute.
- Evident from the text: this is a bioRxiv preprint not certified by peer review; ESMFold is less accurate than AlphaFold2 overall on CAMEO (0.83 vs 0.88 TM-score) and CASP14 (0.68 vs 0.85), with a larger gap on hard CAMEO targets (0.45 vs 0.62).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 3 Not valid · 6 Not checked · 115 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“the vast majority (76.8%) of high confidence predictions being separate from UniRef90 (32) by at least 90% sequence identity”
What the sources say
- Supportedcites [32] source
The citation is attached to UniRef90, identifying the reference database used for the citing paper’s comparison. The Introduction describes UniRef90 as clustering sequences at the 90% identity level; the 76.8% figure is the citing paper’s own result, not a result attributed to this source. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Very high confidence predictions in the metagenomic atlas are expected to often be reliable at a resolution sufficient for insight similar to experimentally determined structures, such as into the biochemistry of active sites (61);”
What the sources say
- Partly supportedcites [61] sourceExperimental — accuracy not yet measured
The High-resolution models section says predicted models can meet demanding structural requirements and sometimes support ligand-binding studies; the Medium-resolution models section says predictions can help locate active sites. The paper does not assess predictions in the metagenomic atlas or establish that its very high-confidence predictions are often reliable enough to provide experimentally comparable insight into active-site biochemistry. Page numbers are unavailable in our copy. (LLM, conf=0.85)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“We fold over 617 million sequences from the MGnify90 database (25).”
What the sources say
- Partly supportedcites [25] sourceExperimental — accuracy not yet measured
The 'UPDATES TO THE MGnify PROTEIN DATABASE' section describes MGnify protein sequences clustered at 90% identity, supporting the identification of the database. It reports over 1.1 billion sequences in the full database, but does not give a count of 617 million for the 90%-clustered set. The citing authors' folding of those sequences is their own work, not a finding attributed to this source. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“In a comparison with AlphaFold-Multimer (45) on a dataset of 2,978 recent multimeric complexes deposited in the PDB, ESMFold achieves the same qualitative DockQ (39) categorization for 53.2% of chain pairs, despite not being trained on protein complexes (Fig. S6).”
What the sources say
- Supportedcites [45] Richard Evans , Michael O’Neill , Alexander Pritzel , Natasha Antropova , Andrew Senior , Tim Green , Augustin Žídek , Russ Bates , Sam Blackwell , Jason Yim , Olaf Ronneberger , Sebastian Bodenstein , Michal Zielinski , Alex Bridgland , Anna Potapenko , Andrew Cowie , Kathryn Tunyasuvunakool , Rishub Jain , Ellen Clancy , Pushmeet Kohli , John Jumper , and Demis Hassabis . Protein complex prediction with AlphaFold-Multimer, March 2022. URL https://www.biorxiv.org/content/10.1101/2021.10.04.463034v2. Pages: 2021.10.04.463034 Section: New Results. source
The citation identifies AlphaFold-Multimer as the comparator. The Results (p. 5) describe this paper's AlphaFold-Multimer predictions and benchmarks. The 2,978-complex comparison and 53.2% figure are presented as the citing authors' own results, not findings attributed to this source. (LLM, conf=0.92)
- Supportedcites [39] source
Citation (39) is attached to “DockQ” and identifies the quality measure used for the categorization. The Conclusions describe DockQ as a protein–protein docking model quality score that closely recapitulates the four CAPRI quality classes; the citation is not offered as the source of the ESMFold comparison or its 53.2% result. (LLM, conf=0.93)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This training also results in the emergence of structure in the models.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
69 more claims
Create a free account to see all 131 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 32
- Probably not a substantial claim. 27
- The quote only partly states this claim. 1
- Borderline: one of the tests for a substantial claim is unclear. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This difficulty is amplified by the fact that the repository of publicly available structures, consisting of the combined databases of AFDB and the ESMatlas38, is approaching a billion entries.”
From Clustering predicted structures at the scale of the known protein universe (2023) · cites this paper as [38]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Evo also reaches competitive performance with leading protein-specific language models (41, 46–48) (Fig. 2B).”
From Sequence modeling and design from molecular to genome scale with Evo (2024) · cites this paper as [47]
The cited source says what the claim attributes to it.
In the citing paper
“are the best and most accurate answer to the structure prediction problem to date”
From The power and pitfalls of AlphaFold2 for structure prediction beyond rigid globular proteins (2024) · cites this paper as [19]
The cited source says what the claim attributes to it.
In the citing paper
“Such modeling paradigms have been used successfully in the biomolecular realm, for example, to predict three-dimensional (3D) molecular structures from sequences32–34 and interactions between different molecular components.35–38”
From How to build the virtual cell with artificial intelligence: Priorities and opportunities (2024) · cites this paper as [34]
The cited source says what the claim attributes to it.
In the citing paper
“The unprecedented accuracy of AlphaFold2,127 RosettaFold,128 and others129,130 offers, at least in principle, the opportunity to virtually screen any protein of interest with the structure-based approach, irrespective of its a priori ligandability (after AlphaFold2, the structural coverage of the human proteome increased from 48% to 76% of the total residues).131”
From Advancing Targeted Protein Degradation via Multiomics Profiling and Artificial Intelligence (2023) · cites this paper as [129]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 4. Evolutionary-scale structural characterization of metagenomics · p. 7
“PDB 3HR4 A, TM-score 0.67”
§ Evolutionary-scale prediction of atomic level protein structure with a language model · p. 9
“The predicted structure of MGYP000936678158 aligns to an experimental structure from a bacterial nuclease (light brown, PDB: 3H4R)”
Citations
Cites
Basic local alignment search tool
Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990) · Journal of Molecular Biology · doi:10.1006/jmbi.1990.9999 · Cited by 98,669
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
A Programmable Dual-RNA–Guided DNA Endonuclease in Adaptive Bacterial Immunity
Jinek, M., Chylinski, K., Fonfara, I., Hauer, M., Doudna, J. A., & Charpentier, E. (2012) · Science · doi:10.1126/science.1225829 · Cited by 17,116
ColabFold - Making protein folding accessible to all
Mirdita, M., Schütze, K., Moriwaki, Y., Heo, L., Ovchinnikov, S., & Steinegger, M. (2021) · doi:10.1101/2021.08.15.456425 · Cited by 9,391
SCANPY: large-scale single-cell gene expression data analysis
Wolf, F. A., Angerer, P., & Theis, F. J. (2018) · Genome Biology · doi:10.1186/s13059-017-1382-0 · Cited by 8,881
Varadi, M., Anyango, S., Deshpande, M., Nair, S., Natassia, C., Yordanova, G., Yuan, D., Stroe, O., Wood, G., Laydon, A., et al. (2022) · Nucleic Acids Research · doi:10.1093/nar/gkab1061 · Cited by 7,850
Research works printed but not linked: 1
- Ashish Vaswani , Noam Shazeer , Niki Parmar , Jakob Uszkoreit , Llion Jones , Aidan N Gomez , Łukasz Kaiser , and Illia Polosukhin . Attention Is All You Need. In Advances in Neural Information Processing Systems, pages 5998–6008, 2017. URL https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf.
Cited by
Accurate structure prediction of biomolecular interactions with AlphaFold 3
Abramson, J., Adler, J., Dunger, J., Evans, R., Green, T., Pritzel, A., Ronneberger, O., Willmore, L., Ballard, A. J., Bambrick, J., et al. (2024) · Nature · doi:10.1038/s41586-024-07487-w · Cited by 15,861
UCSF ChimeraX: Tools for structure building and analysis
Meng, E. C., Goddard, T. D., Pettersen, E. F., Couch, G. S., Pearson, Z. J., Morris, J. H., & Ferrin, T. E. (2023) · Protein Science · doi:10.1002/pro.4792 · Cited by 4,442
Fast and accurate protein structure search with Foldseek
van Kempen, M., Kim, S. S., Tumescheit, C., Mirdita, M., Lee, J., Gilchrist, C. L. M., Söding, J., & Steinegger, M. (2022) · doi:10.1101/2022.02.07.479398 · Cited by 2,598
Fast and accurate protein structure search with Foldseek
van Kempen, M., Kim, S. S., Tumescheit, C., Mirdita, M., Lee, J., Gilchrist, C. L. M., Söding, J., & Steinegger, M. (2024) · Nature Biotechnology · doi:10.1038/s41587-023-01773-0 · Cited by 2,596
“3), and the ESM Atlas contains over 617 million metagenomic structures predicted by ESMFold 4 .”
BackgroundWatson, J. L., Juergens, D., Bennett, N. R., Trippe, B. L., Yim, J., Eisenach, H. E., Ahern, W., Borst, A. J., Ragotte, R. J., Milles, L. F., et al. (2022) · doi:10.1101/2022.12.09.519842 · Cited by 2,369
Metadata
- Authors
- Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., dos Santos Costa, A., Fazel-Zarandi, M., Sercu, T., Candido, S., & Rives, A.
- Affiliations
- New York University (US)
- Stanford University (US)
- Corresponding author
- Rives, Alexander
- ORCID
- Venue
- Science
- Publisher
- openRxiv
- Published
- 2022-07-21
- Volume
- 379
- Issue
- 6637
- Pages
- 1123-1130
- ISSN
- 0036-8075 (ISSN-L)
- OpenAlex
- W4327550249
- Cited by
- 5,990
- References
- 71
- Citations by year
- 2026: 2468 · 2025: 1837 · 2024: 1170 · 2023: 485 · 2022: 23
- FWCI
- 460.33
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Science (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (submittedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Structural Biology › Enzyme Structure and Function
- Keywords
- protein structure prediction, large language models, protein evolution, atomic-scale structure, protein folding, structural diversity
- MeSH terms
- Machine Learning, Amino Acid Sequence, Protein Conformation, Proteins, Evolution, Molecular, Sequence Analysis, Protein
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evolutionary-scale prediction of atomic level protein structure with a language model. https://app.cochat.ai/papers/10.1101/2022.07.20.500902
- The paper
- Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., dos Santos Costa, A., Fazel-Zarandi, M., Sercu, T., Candido, S., & Rives, A. (2022). Evolutionary-scale prediction of atomic level protein structure with a language model. Science. https://doi.org/10.1101/2022.07.20.500902
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.