The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index; 2 of the 23 sources its claims cite were checked for support.
Large language models generate functional protein sequences across diverse families
Open accessAbstract
Deep-learning language models have shown promise in various biotechnological applications, including protein design and engineering. Here we describe ProGen, a language model that can generate protein sequences with a predictable function across large protein families, akin to generating grammatically and semantically correct natural language sentences on diverse topics. The model was trained on 280 million protein sequences from >19,000 families and is augmented with control tags specifying protein properties. ProGen can be further fine-tuned to curated sequences and tags to improve controllable generation performance of proteins from families with sufficient homologous samples. Artificial proteins fine-tuned to five distinct lysozyme families showed similar catalytic efficiencies as natural lysozymes, with sequence identity to natural proteins as low as 31.4%. ProGen is readily adapted to diverse protein families, as we demonstrate with chorismate mutase and malate dehydrogenase.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
32 Own result · 14 Own interpretation · 11 Not checked yet · 8 Own method · 4 No basis given · 1 Couldn’t confirm · accuracy not yet measured
None of the sources cited for this claim have been checked yet.
In the paper
“This indicates that the model has captured evolutionary conservation patterns without training on explicit alignment information such as with Potts models51, as implemented in direct coupling analysis7,52–55.”
§ Large language models generate functional protein sequences across diverse families > Abstract
What the sources say
- Not checked yetcites [51] Mchaourab, H. S., Oh, K. J., Fang, C. J. & Hubbell, W. L. Conformation of T4 lysozyme in solution. Hinge-bending motion and the substrate-induced conformational transition studied by site-directed spin labeling. Biochemistry 36, 307–316 (1997). source
- Not checked yetcites [7] Russ, W. P. et al. An evolution-based model for designing chorismate mutase enzymes. Science 369, 440–445 (2020). source
- Could not verifycites [52] Kim, J.-K. et al. BetaCavityWeb: a webserver for molecular voids and channels. Nucleic Acids Res. 43, W413–W418 (2015). sourceExperimental — accuracy not yet measured
The Methods and Materials passages describe Voronoi-based identification of molecular voids and channels, not a model learning evolutionary conservation. Searches for alignment information, Potts models, and direct coupling analysis found no passage in our copy addressing the attributed claim. (LLM, conf=0.78)
- Not checked yetcites [53] Rost, B. Twilight zone of protein sequence alignments. Protein Eng. 12, 85–94 (1999). source
- Not checked yetcites [54] Pearson, W. R. An introduction to sequence similarity (‘homology’) searching. Curr. Protoc. Bioinforma. 3, 3.1 (2013). ChapterUnit.
- Not checked yetcites [55] Repecka, D. et al. Expanding functional protein sequence spaces using generative adversarial networks. Nat. Mach. Intell. 3, 324–333 (2021). source
hedgeddirectn = 5- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“As in the crystal structure of L056, the predicted artificial structures roughly match known structures found in nature, Fig S11, including for low identity (<40%) artificial sequences.”
§ Large language models generate functional protein sequences across diverse families > Abstract
What the sources say
- Could not verifycites [14] Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021). sourceExperimental — accuracy not yet measured
The Abstract and Main section describe AlphaFold’s structure-prediction accuracy, including cases without a known similar structure. Our copy does not address whether predicted structures of functional artificial sequences resemble natural structures, including at below 40% sequence identity (searched: artificial sequences, L056, and 40% identity). (LLM, conf=0.77)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our protein language model is trained by simply learning to predict the next amino acid for over 280 million protein sequences from thousands of protein families, without explicit biophysical modeling.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“After training, ProGen can be prompted to generate full-length protein sequences for any protein family from scratch, with a varying degree of similarity to natural proteins.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Our artificial lysozymes span the sequence landscape of natural lysozymes, Fig. 2a, across five families which contain diverse protein folds, active site architectures, and enzymatic mechanisms49,50.”
§ Large language models generate functional protein sequences across diverse families > Abstract
What the sources say
- Not checked yetcites [49] Baase, W. A., Liu, L., Tronrud, D. E. & Matthews, B. W. Lessons from the lysozyme of phage T4. Protein Sci. 19, 631–641 (2010). source
- Not checked yetcites [50] Kuroki, R., Weaver, L. H. & Matthews, B. W. A covalent enzyme–substrate intermediate with saccharide distortion in a mutant T4 lysozyme. Science 262, 2030–2033 (1993). source
plainn = 5- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The five lysozyme families chosen because they lack disulfide bonds can be expressed in high-throughput cell-free systems.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“they demonstrate similar residue position entropies when forming separate multiple sequence alignments of natural and artificial proteins within each family, Fig. 2c.”
§ Large language models generate functional protein sequences across diverse families > Abstract
boostedn = 5- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the positive control set of 100 natural proteins, 72% were well-expressed based on chromatography peaks and band visualization.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our artificial proteins match activity levels of natural proteins even at lower levels of sequence identity to any known natural protein, (Fig 3b, Fig. S5).”
§ Large language models generate functional protein sequences across diverse families > Abstract
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The representative natural proteins exhibited similar levels of functionality with 59% (53/90) of total proteins considered functional.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“None of the bmDCA artificial proteins exhibited a detectable level of functionality, Fig. S10”
§ Large language models generate functional protein sequences across diverse families > Abstract
What the sources say
- Not checked yetcites [7] Russ, W. P. et al. An evolution-based model for designing chorismate mutase enzymes. Science 369, 440–445 (2020). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 29
- Probably not a substantial claim. 16
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ProGen is a 1.2 billion-parameter neural network trained using a publicly available dataset of 280 million protein sequences.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we could readily measure the KM values for the purified apparent lower molecular weight species, where both L056 and L070 were highly active and had comparable Michaelis-Menten parameters to HEWL (Fig. 3e).”
§ Large language models generate functional protein sequences across diverse families > Abstract
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Among our artificial proteins, 73% (66/90) were functional and exhibited high levels of functionality across families, Fig. 3c.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Notably a small number of proteins, both within the natural and artificial proteins, were within an order of magnitude of HEWL, which was substantially more active than all negative controls.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Two proteins, L056 (max ID 69.6%) and L070 (max ID 89.2%) expressed well and incurred bactericidal activities towards the E. coli BL21(DE3) strain used during overnight induction at 16C.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The ProGen-generated proteins express equally well (72/100 total) across all bins of sequence identity to any known natural protein (maxID 40–90%), Fig 2e.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Despite having comparable enzymatic activities, L070 and L056 only share 17.9% sequence ID.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also found that there is no bias to location or structural element to the mutations that diverge L056 and L070 from their respective nearest sequence homolog in nature. Differing residues are instead uniformly distributed.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The KM values of both monomers were too weak to be measured using a heterogeneous, fluorescein-labeled Micrococcus lysodeikticus cell wall substrate (Molecular Probes EnzChek Lysozyme kit); however, both monomers were active using a pseudo-first order kinetic assay (Fig. S7).”
§ Large language models generate functional protein sequences across diverse families > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Some mutations are even found within the active site cleft and within regions that influence conformational state (for L056).”
§ Large language models generate functional protein sequences across diverse families > Abstract
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“We determined a 2.5 Å resolution crystal of L056 (see Methods, Figure 3e). The global fold was similar to predictions, with a Cα RMSD of 2.9 Å from the backbone structure predicted by trRosetta and 2.3 Å RMSD from a WT T4 Lysozyme structure59,60.”
§ Large language models generate functional protein sequences across diverse families > Abstract
What the sources say
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“bmDCA was unable to fit three out of the five lysozyme families, and exhibited 60% detectable expression (30/50 proteins) for the remaining two protein families.”
§ Large language models generate functional protein sequences across diverse families > Abstract
What the sources say
- Not checked yetcites [7] Russ, W. P. et al. An evolution-based model for designing chorismate mutase enzymes. Science 369, 440–445 (2020). source
plainn = 5- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Alignment with a structure featuring a covalently trapped substrate (PDB 148L) reveals that the active site cleft is well formed with the key catalytic residue Glu15 (Glu 11 in T4L) and key substrate binding residue Thr30 (Thr26 in T4L) correctly positioned.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These likelihoods showed a significant drop in AUC of 0.18 for CM (p<0.0001, two-tailed test, n=1617) and 0.08 for MDH (p<0.1, two-tailed test, n=56), as compared to fine-tuned ProGen when predicting if an artificial sequence should function.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plaindirectn = 1,617 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“On MDH function data, ProGen log-likelihoods had an AUC of 0.94 (Fig. 4f), which was better than ProteinGAN discriminator scores, with an AUC of 0.87 (p<0.1, two-tailed test, n=56).”
§ Large language models generate functional protein sequences across diverse families > Abstract
What the sources say
- Not checked yetcites [66] Vig, J. et al. BERTology meets biology: Interpreting attention in protein language models. In International Conference on Learning Representations (ICLR, 2020). source
plainn = 56 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“On CM function data, ProGen log-likelihoods had an area under the curve (AUC) of 0.85, significantly better (p<0.0001, two-tailed test, n=1617) than bmDCA, which had an AUC of 0.78 (Fig. 4c).”
§ Large language models generate functional protein sequences across diverse families > Abstract
What the sources say
- Not checked yetcites [7] Russ, W. P. et al. An evolution-based model for designing chorismate mutase enzymes. Science 369, 440–445 (2020). source
plainn = 1,617- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We purified 6 highly expressed proteins and found that they were all active, but with significantly lower Michaelis-Menten activities than HEWL or the previously generated artificial proteins L056 and L070 (Fig 3f).”
§ Large language models generate functional protein sequences across diverse families > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of the selected sequences, 78 out of the 89 (88%) expressed well and 24 out of the 78 (31%) were soluble (Supplemental Figure 8).”
§ Large language models generate functional protein sequences across diverse families > Abstract
plainn = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“L013 and L038 expressed highly to inclusion bodies.”
§ Large language models generate functional protein sequences across diverse families > Methods > E. coli Expression of Lysozyme Variants
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For lowID lysozyme A5 the above protocol was altered slightly to accommodate lower catalytic activity of these variants: reaction volumes were increased to 20 μL, plate was covered with an optically transparent seal (Microseal ® ‘B’ seal; BioRad) to mitigate sample evaporation, fluorescent reads were taken every 5 mins for 16 hours with 5 s of linear plate shaking prior to each measurement to min”
§ Large language models generate functional protein sequences across diverse families > Methods > Michaelis-Menten kinetics of lysozyme variants using fluorescein-labeled M. lysodeikticus cell wall
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The MDH functional labels are binary, so no thresholding was needed to compute AUCs.”
§ Large language models generate functional protein sequences across diverse families > Methods > Evaluating ProGen on Other Protein Systems
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The resultant data were not described by a single exponential model but were described by a double exponential model (Eq. 6), likely due to the heterogeneity of the substrate, and all data were fit in IgorPro7.”
§ Large language models generate functional protein sequences across diverse families > Methods > Lysozyme kcat/KM extrapolation from pseudo first-order kinetic data
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For induced cultures of L056 and L070, significant turbidity was observed in the spent media after cells were pelleted at 3,500 r.c.f. for 30 mins at 4°C.”
§ Large language models generate functional protein sequences across diverse families > Methods > E. coli Expression of Lysozyme Variants
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Two peaks were resolved for each variant and that harbored lysozyme activity against the fluorescein-labeled M. Lysodeikticus cell wall substrate (EnzChek kit; ThermoFisher).”
- Also stated in: § Purification of L056 and L070 from spent media · plain
“Two peaks were resolved for each variant and that harbored lysozyme activity against the fluorescein-labeled M. Lysodeikticus cell wall substrate (EnzChek kit; ThermoFisher) that corresponded to peaks observed in the first pool purification.”
§ Large language models generate functional protein sequences across diverse families > Methods > Purification of L056 and L070 from spent media
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Also stated in: § Purification of L056 and L070 from spent media · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Significant translational non-crystallographic symmetry and differences with the search model resulted in maps that were initially hard to interpret.”
§ Large language models generate functional protein sequences across diverse families > Methods > Crystallization and structure determination of L056
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These results indicate that ProGen generates protein sequences that not only can express well but also maintain enzymatic function for diverse sequence landscapes across protein families.”
§ Large language models generate functional protein sequences across diverse families > Abstract
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Well-expression and solubility in the cell-free system indicate that the artificial proteins are structurally well-folded.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Taken together, L056 and L070 harbor potent catalytic activity and bactericidal capabilities that are comparable to HEWL, while diverging from their nearest known natural sequence by 53 and 18 amino acids respectively.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: HEWL is a suitable reference for native lysozyme catalytic activity.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The perplexity on a held-out test set was monitored and did not exceed training set perplexity throughout model training.”
§ Large language models generate functional protein sequences across diverse families > Methods > ProGen Training
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While we do not expect our language model to generate proteins that belong to a completely different distribution or domain (for example creating a new fold that catalyzes an unnatural reaction), it can significantly expand the space of protein sequences from those sampled by evolution.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The 280 million-sequence training set is diverse enough to learn a universal protein representation transferable across families.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The aggregated dataset was split into a training set of size 280 million and two test sets, an out-of-distribution test set (OOD-test) of size 100k from 20 protein families and a randomly sampled in-domain test set (ID-test) of size 1 million, that were held-out for training and used for evaluation.”
§ Large language models generate functional protein sequences across diverse families > Methods > Training Data Curation
plainn = 20- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“In sum, ProGen’s model likelihoods are better aligned with experimentally-measured assay data on two diverse protein datasets—chorismate mutase and malate dehydrogenase—than the sequence generation methods from original studies specifically tailored for these families.”
§ Large language models generate functional protein sequences across diverse families > Abstract
What the sources say
- Not checked yetcites [7] Russ, W. P. et al. An evolution-based model for designing chorismate mutase enzymes. Science 369, 440–445 (2020). source
- Not checked yetcites [66] Vig, J. et al. BERTology meets biology: Interpreting attention in protein language models. In International Conference on Learning Representations (ICLR, 2020). source
plaincorrelationaln = 1,617- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These results indicate that both components of our training strategy—initial training on the universal sequence dataset and fine-tuning on the protein family of interest—contribute significantly to final model performance.”
§ Large language models generate functional protein sequences across diverse families > Abstract
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“For robustness, we trained three discriminators using different random seeds.”
§ Large language models generate functional protein sequences across diverse families > Methods > Lysozyme Sequence Selection
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Proteins that generated fluorescence one standard deviation above the maximum fluorescence of any negative control were considered functional.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The resultant data were not described by a single exponential model but were described by a double exponential model (Eq. 6), likely due to the heterogeneity of the substrate, and all data were fit in IgorPro7.”
§ Large language models generate functional protein sequences across diverse families > Methods > Lysozyme kcat/KM extrapolation from pseudo first-order kinetic data
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: The average molecular weight of the fluorescein-labeled M. lysodeikticus cell wall substrate is unknown and the substrate is heterogeneous.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We also ensured that no two generated samples share over 80% identity with each other.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“After measuring the AUC of each model for each dataset, we used bootstrapping to compute the statistical significance of the difference in AUC of fine-tuned ProGen vs. the reference method (bmDCA and ProGen ablations for CM, ProteinGAN and ProGen ablations for MDH).”
§ Large language models generate functional protein sequences across diverse families > Methods > Evaluating ProGen on Other Protein Systems
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The fine-tuning dataset was clustered at 80% sequence identity and 10% of the clusters were held-out as a validation set for hyper-parameter optimization and stopping criteria.”
§ Large language models generate functional protein sequences across diverse families > Methods > Lysozyme Generation
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“We also applied thresholding at 0.42 norm relative enrichment (r.e.) to obtain binary labels for CM function, which roughly corresponds to the cut-off point between two modes that exist in CM function data, to be used for ROC curves, following the original study7.”
§ Large language models generate functional protein sequences across diverse families > Methods > Evaluating ProGen on Other Protein Systems
What the sources say
- Not checked yetcites [7] Russ, W. P. et al. An evolution-based model for designing chorismate mutase enzymes. Science 369, 440–445 (2020). source
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We performed structure prediction without an MSA as input so as to not heavily bias the structure prediction toward a known natural mode.”
§ Large language models generate functional protein sequences across diverse families > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper relies on this without support.
The five lysozyme families chosen because they lack disulfide bonds can be expressed in high-throughput cell-free systems.
“Within the lysozyme clan, we chose to work with five families that lack disulfide bonds as to be amenable to high-throughput cell-free expression: Phage lysozyme, Pesticin, Glucosaminidase, Glycoside hydrolase, and Transglycosylase.”
The paper relies on this without support.
Well-expression and solubility in the cell-free system indicate that the artificial proteins are structurally well-folded.
“These results indicate that ProGen can generate artificial proteins that are structurally well-folded for proper expression as compared to a batch of natural proteins”
The paper relies on this without support.
The 280 million-sequence training set is diverse enough to learn a universal protein representation transferable across families.
“Trained in this unsupervised manner from a large, varied protein sequence database, ProGen learns a universal, domain-independent representation of proteins that subsumes local and global structure motifs”
The paper relies on this without support.
HEWL is a suitable reference for native lysozyme catalytic activity.
“Hen egg white lysozyme (HEWL), a naturally-evolved exemplar protein, was measured as positive control”
The paper relies on this without support.
Model likelihoods for GANs are intractable.
“Since model likelihoods for GANs are intractable, we used discriminator scores corresponding to the probability at which the ProteinGAN discriminator predicted each sample was real as a ProteinGAN model score for each MDH sequence.”
The paper relies on this without support.
The average molecular weight of the fluorescein-labeled M. lysodeikticus cell wall substrate is unknown and the substrate is heterogeneous.
“(the average molecular weight of the fluorescein-labeled M. Lysodeikticus cell wall substrate was unknown and likely heterogeneous).”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“synthetic proteins optimized for specific families of lysozymes using ProGen have exhibited catalytic efficiencies comparable to their naturally occurring counterparts with low sequence identity.”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Madani et al. 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“AI models have already been used to design new CRISPR enzymes,53 functional proteins,128 and even entire prokaryotic genomes.51”
From How to build the virtual cell with artificial intelligence: Priorities and opportunities (2024) · cites this paper as [128]
The cited source says what the claim attributes to it.
In the citing paper
“The success of LLMs has revolutionized AI learning in natural language and image processing [75] and has been extended into the learning of biological sequence datasets [80].”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [80]
The cited source says what the claim attributes to it.
In the citing paper
“the ProGen (Madani et al. 2023) language model employs deep learning techniques to produce protein sequences with predefined functionalities.”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Madani et al. 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Large language models generate functional protein sequences across diverse families > Abstract
“a state-of-the-art transformer-based conditional language model trained only with evolutionary sequence data”
§ Large language models generate functional protein sequences across diverse families > Methods > ProGen Training
“The model was initialized with pretrained weights of CTRL28, which was trained on an English language corpus.”
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Gene Ontology: tool for the unification of biology
Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M., Davis, A. P., Dolinski, K., Dwight, S. S., Eppig, J. T., et al. (2000) · Nature Genetics · doi:10.1038/75556 · Cited by 43,206
Features and development of Coot
Emsley, P., Lohkamp, B., Scott, W. G., & Cowtan, K. (2010) · Acta Crystallographica Section D Biological Crystallography · doi:10.1107/s0907444910007493 · Cited by 29,915
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Phaser crystallographic software
McCoy, A. J., Grosse-Kunstleve, R. W., Adams, P. D., Winn, M. D., Storoni, L. C., & Read, R. J. (2007) · Journal of Applied Crystallography · doi:10.1107/s0021889807021206 · Cited by 20,612
Kabsch, W. (2010) · Acta Crystallographica Section D Biological Crystallography · doi:10.1107/s0907444909047337 · Cited by 16,463
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Research works printed but not linked: 13
- Vaswani, A. et al. Attention is all you need. In 31st Conference on Neural Information Processing Systems (NIPS, 2017).
- Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT, 2019).
- Zellers, R. et al. Defending against neural fake news. In 33rd Conference on Neural Information Processing Systems (NeurIPS, 2019).
- Radford, A., Narasimhan, K., Salimans, T. & Sutskever, I. Improving language understanding by generative pre-training. Preprint at https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf (2018).
- Dathathri, S. et al. Plug and play language models: a simple approach to controlled text generation. In 8th International Conference on Learning Representations (ICLR, 2020).
- Pettit, L. D. The IUPAC stability constants database. Chem. Int. 28, 14–15 (2006).
- Bengio, Y., Ducharme, R., Vincent, P. & Janvin, C. A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003).
- Goyal, K., Dyer, C. & Berg-Kirkpatrick, T. Exposing the implicit energy networks behind masked language models via metropolis–hastings. In 10th International Conference on Learning Representations (ICLR, 2022).
- Alley, E., Khimulya, G., Biswas, S., AlQuraishi, M. & Church, G. M. Unified rational protein engineering with sequence-based deep representation learning. Nat. Methods 16, 1315–1322 (2019). · DOI · Paper page
- Pascanu, R., Mikolov, T. & Bengio, Y. On the difficulty of training recurrent neural networks. In Proc. 30th International Conference on Machine Learning (eds. Dasgupta, S. & McAllester, D.) 1310–1318 (PMLR, 2013).
- Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014).
- Holtzman, A., Buys, J., Du, L., Forbes, M. & Choi, Y. The curious case of neural text degeneration. In 8th International Conference on Learning Representations (ICLR, 2020).
- Goodfellow, I. J. et al. Generative adversarial networks. In 28th Conference on Neural Information Processing Systems (NIPS, 2014).
Cited by
Sorted by citations
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944
Simulating 500 million years of evolution with a language model
Hayes, T., Rao, R., Akin, H., Sofroniew, N. J., Oktay, D., Lin, Z., Verkuil, R., Tran, V. Q., Deaton, J., Wiggert, M., et al. (2025) · Science · doi:10.1126/science.ads0018 · Cited by 937
Sequence modeling and design from molecular to genome scale with Evo
Nguyen, E., Poli, M., Durrant, M. G., Kang, B., Katrekar, D., Li, D. B., Bartie, L. J., Thomas, A. W., King, S. H., Brixi, G., et al. (2024) · Science · doi:10.1126/science.ado9336 · Cited by 580
From nature to industry: Harnessing enzymes for biocatalysis
Buller, R., Lutz, S., Kazlauskas, R. J., Šnajdrová, R., Moore, J. C., & Bornscheuer, U. T. (2023) · Science · doi:10.1126/science.adh8615 · Cited by 759
Alqahtani, T., Badreldin, H. A., Alrashed, M., Alshaya, A. I., Alghamdi, S. S., bin Saleh, K., Alowais, S. A., Alshaya, O. A., Rahman, I., Al Yami, M. S., et al. (2023) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2023.05.016 · Cited by 565
Algorithm for optimized mRNA design improves stability and immunogenicity
Zhang, H., Zhang, L., Lin, A., Xu, C., Li, Z., Liu, K., Liu, B., Ma, X., Zhao, F., Jiang, H., et al. (2023) · Nature · doi:10.1038/s41586-023-06127-z · Cited by 531
Scaffolding protein functional sites using deep learning
Wang, J., Lisanza, S., Juergens, D., Tischer, D., Watson, J. L., Castro, K. M., Ragotte, R., Saragovi, A., Milles, L. F., Baek, M., et al. (2022) · Science · doi:10.1126/science.abn2100 · Cited by 487
ProGen2: Exploring the boundaries of protein language models
Nijkamp, E., Ruffolo, J. A., Weinstein, E. N., Naik, N., & Madani, A. (2023) · Cell Systems · doi:10.1016/j.cels.2023.10.002 · Cited by 480
Self-citationGat, I., Remez, T., Shaul, N., Kreuk, F., Chen, R. T. Q., Synnaeve, G., Adi, Y., & Lipman, Y. (2024) · Neural Information Processing Systems · doi:10.48550/arxiv.2407.15595 · Cited by 411
“…to other domains such as, code (Roziere et al., 2023; Li et al., 2023; Chen et al., 2021), biology (Zhang et al., 2024; Ferruz and Höcker, 2022; Madani et al., 2023), math (Romera-Paredes et al., 2024; Imani et al., 2023; Ahn et al., 2024), audio (Kreuk et al., 2022; Copet et al., 2024; Hassid…”
BackgroundLarge language models (LLMs): survey, technical frameworks, and future challenges
Kumar, P. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10888-y · Cited by 395
How to build the virtual cell with artificial intelligence: Priorities and opportunities
Bunne, C., Roohani, Y., Rosen, Y., Gupta, A., Zhang, X., Roed, M., Alexandrov, T., AlQuraishi, M., Brennan, P., Burkhardt, D. B., et al. (2024) · Cell · doi:10.1016/j.cell.2024.11.015 · Cited by 393
Welcome to the Era of ChatGPT et al.: The Prospects of Large Language Models
Teubner, T., Flath, C. M., Weinhardt, C., van der Aalst, W., & Hinz, O. (2023) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00795-x · Cited by 384
Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook.
AlSaad, R., Abd-alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.-A., Damseh, R., & Sheikh, J. (2024) · Journal of medical Internet research · doi:10.2196/59505 · Cited by 349
“…proteomics, protein language models are used for secondary structure and contact prediction (ProtTrans [127]), protein sequence generation (ProGen [128]), protein function prediction (ProtST [129]), major posttranslational modification prediction (ProteinBERT [130]), biophysical property…”
De novo protein design—From new structures to programmable functions
Kortemme, T. (2024) · Cell · doi:10.1016/j.cell.2023.12.028 · Cited by 383
“One class of machine learning models that has been successfully applied to protein design are large language models (examples include ProtGPT2 20 , ESM-2 7 and ProGen 21 ).”
InfluentialMachine learning for functional protein design
Notin, P., Rollins, N. J., Gal, Y., Sander, C., & Marks, D. S. (2024) · Nature Biotechnology · doi:10.1038/s41587-024-02127-0 · Cited by 304
“Explicitly modeling this information provides finer control over generated sequences, for instance, by conditioning on broad taxonomic labels 20,153 and, increasingly, on more granular property values 154,155 .”
InfluentialBengesi, S., El-Sayed, H., Sarker, M. K., Houkpati, Y., Irungu, J., & Oladunni, T. (2024) · IEEE Access · doi:10.1109/access.2024.3397775 · Cited by 470
“Through the generation of molecular structures [245] and predictive modeling, it expedites the identification of novel therapeutic compounds.”
BackgroundMachine learning for antimicrobial peptide identification and design
Wan, F., Wong, F. J., Collins, J. J., & de la Fuente‐Núñez, C. (2024) · Nature Reviews Bioengineering · doi:10.1038/s44222-024-00152-x · Cited by 317
“These methods have demonstrated strong predictive ability in computer vision 73 , natural language processing 21,74,75 and computational biology 39,76–79 .”
BackgroundMethodCaduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling
Schiff, Y., Kao, C.-H., Gokaslan, A., Dao, T., Gu, A., & Kuleshov, V. (2024) · International Conference on Machine Learning · doi:10.48550/arxiv.2403.03234 · Cited by 275
“In proteomics, these models have enabled predicting protein structures from sequences (Jumper et al., 2021; Lin et al., 2023), deciphering the functions and interactions of amino Sequence models are also standard tools in genomics (Zhou & Troyanskaya, 2015; Avsec et al., 2021).”
MethodThe Gene Ontology knowledgebase in 2026
Aleksander, S., Balhoff, J. P., Carbon, S., Cherry, J. M., Ebert, D., Feuermann, M., Gaudet, P., Harris, N. L., Hill, D. P., Kalita, P., et al. (2025) · Nucleic Acids Research · doi:10.1093/nar/gkaf1292 · Cited by 281
“One of the key transformative developments in AI has been the development of the Transformer architecture (https://arxiv.org/ abs/1706.03762), which underpins both text-based large language models (LLMs) exemplified by ChatGPT, as well as genome and protein language models [52] (gLMs and pLMs).”
InfluentialRapid in silico directed evolution by a protein language model with EVOLVEpro
Jiang, K., Yan, Z., Bernardo, M. D., Sgrizzi, S. R., Villiger, L., Kayabölen, A., Kim, B., Carscadden, J. K., Hiraizumi, M., Nishimasu, H., et al. (2024) · Science · doi:10.1126/science.adr6006 · Cited by 234
Fine-tuning protein language models boosts predictions across diverse tasks
Schmirler, R., Heinzinger, M., & Rost, B. (2024) · Nature Communications · doi:10.1038/s41467-024-51844-2 · Cited by 229
Opportunities and Challenges for Machine Learning-Assisted Enzyme Engineering
Yang, J., Li, F.-Z., & Arnold, F. H. (2024) · ACS Central Science · doi:10.1021/acscentsci.3c01275 · Cited by 268
How to Build the Virtual Cell with Artificial Intelligence: Priorities and Opportunities
Bunne, C., Roohani, Y. H., Rosen, Y., Gupta, A., Zhang, X.-K., Roed, M., Alexandrov, T., AlQuraishi, M., Brennan, P., Burkhardt, D. B., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2409.11654 · Cited by 210
“AI models have already been used to design new CRISPR enzymes 67 , functional proteins 158 , and even entire prokaryotic genomes 65 .”
Schneider, J., Meske, C., & Kuss, P. (2024) · Business & Information Systems Engineering · doi:10.1007/s12599-024-00851-0 · Cited by 145
“…abilities to comprehend, generate, and adapt content across diverse domains, including creative generations (Chen et al. 2023), software debugging (Sobania et al. 2023), protein sequencing (Madani et al. 2023), or cross-modality outputs such as text-to-image creations (Ramesh et al. 2021).”
Machine learning in preclinical drug discovery
Catacutan, D. B., Alexander, J., Arnold, A., & Stokes, J. M. (2024) · Nature Chemical Biology · doi:10.1038/s41589-024-01679-1 · Cited by 227
“Protein language models (PLMs) may overcome this challenge by pretraining transformer-based models on protein sequences, which can learn patterns reflective of evolution while capturing secondary tertiary structures 72,74,75 .”
RiNALMo: general-purpose RNA language models can generalize well on structure prediction tasks
Penić, R. J., Vlašić, T., Huber, R. G., Wan, Y., & Šikić, M. (2025) · Nature Communications · doi:10.1038/s41467-025-60872-5 · Cited by 169
“In recent years, the exploration of language models (LMs) has gone beyond the domain of natural language processing (NLP), reaching into the realms of biology and its data.”
Foundation Model for Advancing Healthcare: Challenges, Opportunities and Future Directions
He, Y., Huang, F., Jiang, X., Nie, Y., Wang, M., Wang, J., & Chen, H. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3496744 · Cited by 162
“The NTP in BFM learns to predict the next token based on previous tokens, achieving great success on sequence data, i.e., DNA, RNA, and protein [47], [151], [152], [153].”
A Comprehensive Survey of Foundation Models in Medicine
Khan, W., Leem, S., See, K. B., Wong, J. K., Zhang, S., & Fang, R. (2025) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2025.3531360 · Cited by 137
“Additionally, we refer readers to several articles for detailed works concerned with the biology and omics domain: [91], [197], [198], [199], [200].”
Zhang, Y., Chen, X., Jin, B., Wang, S., Ji, S., Wang, W., & Han, J.-W. (2024) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2406.10833 · Cited by 158
“Besides CLIP, other general-domain vision-language architectures, such as LLaVA (Li et al., 2023a), PaLM-E (Tu et al., 2024), and Gem-ini (Saab et al., 2024 (Ji et al., 2021), and RNABERT (Akiyama and Sakakibara, 2022), adopt BERT-like architectures and MLM as the pre-training task ( i.e. , predicting masked amino aci…”
MethodxTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of ProteinPreprint
Chen, B., Cheng, X., Li, P., Geng, Y.-A., Gong, J., Li, S., Bei, Z.-L., Tan, X., Wang, B., Zeng, X., et al. (2024) · bioRxiv · doi:10.1101/2023.07.05.547496 · Cited by 157
“BERT[25] ProGen[9], ProGen2[13], ProtGPT2[26] ProtTrans[8], Ankh[27] xTrimoPGLM /”
BackgroundInfluentialGuiding questions to avoid data leakage in biological machine learning applications
Bernett, J., Blumenthal, D. B., Grimm, D. G., Haselbeck, F., Joeres, R., Kalinina, O. V., & List, M. (2024) · Nature Methods · doi:10.1038/s41592-024-02362-y · Cited by 184
Revolutionizing Health Care: The Transformative Impact of Large Language Models in Medicine
Zhang, K., Meng, X., Yan, X., Ji, J., Liu, J., Xu, H., Zhang, H., Liu, D., Wang, J., Wang, X., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/59069 · Cited by 148
“Their role in drug research and development is also emerging, aiding in new drug discoveries through detailed analysis of chemical and biological data [29,30].”
SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation
Cheng, S., Bian, Y., Liu, D.-W., Jiang, Y.-H., Liu, Y.-H., Zhang, L.-F., Yao, Q., Tian, Z.-B., Wang, W.-H., Guo, Q.-P., et al. (2025) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2510.06303 · Cited by 150
“Similarly, modeling protein sequences to resolve structural motifs and functional domains is inherently at odds with the unidirectional autoregressive (AR) process, a foundational challenge for generative models like the ProGen series [37, 42, 8].”
A Survey of Reasoning with Foundation Models: Concepts, Methodologies, and Outlook
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Qiu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2025) · ACM Computing Surveys · doi:10.1145/3729218 · Cited by 45
“Reasoning.”
GENA-LM: A Family of Open-Source Foundational DNA Language Models for Long Sequences
Fishman, V., Kuratov, Y., Shmelev, A., Petrov, M., Penzar, D., Shepelin, D., Chekanov, N., Kardymon, O., & Burtsev, M. (2023) · doi:10.1101/2023.06.12.544594 · Cited by 137
Robust enzyme discovery and engineering with deep learning using CataPro
Wang, Z., Xie, D., Wu, D., Luo, X., Wang, S., Li, Y., Yang, Y., Li, W., & Zheng, L. (2025) · Nature Communications · doi:10.1038/s41467-025-58038-4 · Cited by 123
Hassan, Y. M., Mohamed, A., Hassan, Y. M., & El-Sayed, W. (2025) · Clinical and Experimental Medicine (Testo stampato) · doi:10.1007/s10238-024-01540-8 · Cited by 126
“…LOD, target ctDNA activation cleaved unpaired ssDNA, leading to linked AuNP dissociation, which changed the solution’s color and enhanced the metal-induced fluorescence signal, despite the rare presence of the required 5 ′ TTTN-3 ′ PAM sequence in genomic DNA, indicating great sensitivity [35, 36].”
InfluentialDesign of highly functional genome editors by modelling CRISPR–Cas sequences
Ruffolo, J. A., Nayfach, S., Gallagher, J. P., Bhatnagar, A., Beazer, J., Hussain, R., Russ, J., Yip, J., Hill, E., Pačesa, M., et al. (2025) · Nature · doi:10.1038/s41586-025-09298-z · Cited by 233
Self-citationWang, J., Feng, J., Kang, Y., Pan, P., Ge, J., Wang, Y., Wang, M., Wu, Z., Zhang, X., Yu, J., et al. (2025) · Science Advances · doi:10.1126/sciadv.ads8932 · Cited by 125
Sparks of function by de novo protein design
Chu, A. E., Lu, T., & Huang, P. (2024) · Nature Biotechnology · doi:10.1038/s41587-024-02133-2 · Cited by 131
“These include approaches that directly model sequence, perhaps conditioned on or jointly with functional properties 111–114 .”
MethodFunctional annotation of enzyme-encoding genes using deep learning with transformer layers
Kim, G. B., Kim, J. Y., Lee, J. A., Norsigian, C. J., Palsson, B. Ø., & Lee, S. Y. (2023) · Nature Communications · doi:10.1038/s41467-023-43216-z · Cited by 122
Bushuiev, R., Bushuiev, A., Samusevich, R., Brungs, C., Šivic, J., & Pluskal, T. (2025) · Nature Biotechnology · doi:10.1038/s41587-025-02663-3 · Cited by 113
CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation
Li, J., Ma, W., Li, X., Lou, Y., Zhou, G., & Zhou, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01730 · Cited by 65
“These models have also found widespread applications in the real world [6, 12, 29, 36].”
An AI Agent for Fully Automated Multi‐Omic Analyses
Zhou, J.-X., Zhang, B., Li, G.-W., Chen, X., Li, H., Xu, X.-P., Chen, S., He, W.-J., Xu, C.-C., Liu, L.-W., et al. (2024) · bioRxiv · doi:10.1101/2023.09.08.556814 · Cited by 109
“…analysis de-69 mands additional training, posing challenges for many wet Over the past few months, the rapid advancement of Large Language Models (LLMs) [57] has raised substan-91 tial expectations for the enhancement of scientific research, particularly in the field of biology [58], [59], [60].”
BackgroundProLLaMA: A Protein Large Language Model for Multitask Protein Language Processing
Lv, L., Lin, Z., Li, H., Liu, Y., Cui, J., Chen, C. Y., Yuan, L., & Tian, Y. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3564914 · Cited by 93
“We also retain sequences with a length of less than 512, aligning with ProGen [19].”
Target sequence-conditioned design of peptide binders using masked language modeling
Chen, T., Quinn, Z., Dumas, M., Peng, C. Y., Hong, L., Lopez-Gonzalez, M., Mestre, A. A., Watson, R., Vincoff, S., Zhao, L., et al. (2025) · Nature Biotechnology · doi:10.1038/s41587-025-02761-2 · Cited by 90
“These models have been extended to powerful applications, including antibody design, the creation of novel proteins and structure prediction, offering a streamlined approach to embedding useful protein information 14,15,17,18 .”
Navigating the landscape of enzyme design: from molecular simulations to machine learning
Zhou, J., & Huang, M. (2024) · Chemical Society Reviews · doi:10.1039/d4cs00196f · Cited by 168
Integrating protein language models and automatic biofoundry for enhanced protein evolution
Zhang, Q., Chen, W., Qin, M., Wang, Y., Pu, Z., Ding, K., Liu, Y., Zhang, Q., Li, D., Li, X., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-56751-8 · Cited by 101
“PLMs have proven powerful in modeling functional proteins 23 , predicting the direction of natural evolution 24 and design-ingnovelproteins 25 .”
HoneyComb: A Flexible LLM-Based Agent System for Materials SciencePreprint
Zhang, H., Song, Y., Hou, Z.-Y., Miret, S., & Liu, B. (2024) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2409.00135 · Cited by 103
“…et al., 2024; Song et al., 2023a) along with the development of custom LLMs and LLM-based systems for scientific domains including chemistry (Bran et al., 2023; Boiko et al., 2023), biology (Madani et al., 2023) and materials science (Song et al., 2023b; Gupta et al., 2022; Walker et al., 2021).”
BackgroundBEND: Benchmarking DNA Language Models on biologically meaningful tasksPreprint
Marin, F. I., Teufel, F., Horlacher, M., Madsen, D., Pultz, D., Winther, O., & Boomsma, W. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.12570 · Cited by 54
“The success of masked language modeling (MLM) has extended to the biological domain Rao et al. (2019); Bepler & Berger (2021); Madani et al. (2023); Rives et al. (2019), with protein language models (pLMs) now being widely used for prediction tasks on protein sequences.”
BackgroundNicheformer: a foundation model for single-cell and spatial omics
Tejada-Lapuerta, A., Schaar, A. C., Gutgesell, R., Palla, G., Halle, L., Minaeva, M., Vornholz, L., Dony, L., Drummer, F., Richter, T., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02814-z · Cited by 100
“As the corpus includes human and mouse data, we constructed a shared vocabulary by concatenating orthologous protein-coding genes and species-specific ones, totaling 20,310 gene tokens (Fig.”
Koh, H. Y., Zheng, Y., Yang, M., Arora, R., Webb, G. I., Pan, S., Li, L., & Church, G. M. (2025) · Nature Reviews Bioengineering · doi:10.1038/s44222-025-00349-8 · Cited by 117
“T4c addresses this by generating sequences for a specified 3D backbone (N–Cα–”
Genomic Language Models: Opportunities and Challenges
Benegas, G., Ye, C.-Z., Albors, C., Li, J. C., & Song, Y. S. (2024) · arXiv.org · doi:10.48550/arxiv.2407.11435 · Cited by 97
Na, H. (2024) · International Conference on Language Resources and Evaluation · doi:10.48550/arxiv.2403.16008 · Cited by 97
“The advancement of pre-trained language models (PLMs) has profoundly impacted various domains, such as finance (Shah et al., 2022), biology (Madani et al., 2023), heralding new frontiers in applications and research.”
BackgroundGenomic language model predicts protein co-regulation and function
Hwang, Y., Cornman, A. L., Kellogg, E. H., Ovchinnikov, S., & Girguis, P. R. (2024) · Nature Communications · doi:10.1038/s41467-024-46947-9 · Cited by 99
Pretraining a foundation model for generalizable fluorescence microscopy-based image restoration
Ma, C., Tan, W., He, R., & Yan, B. (2024) · Nature Methods · doi:10.1038/s41592-024-02244-3 · Cited by 89
“Recently, the concept of foundation models has been utilized in diverse life science applications, and foundation models have also demonstrated their impressive capabilities for clinical cases 26–30 , biotechnology 31,32 and so on (Supplementary Note 1).”
Jiang, R., Yuan, S., Zhou, Y., Wei, Y., Li, F., Wang, M., Chen, B., & Yu, H. (2024) · Biotechnology Advances · doi:10.1016/j.biotechadv.2024.108417 · Cited by 96
Design of highly functional genome editors by modeling the universe of CRISPR-Cas sequences
Ruffolo, J. A., Nayfach, S., Gallagher, J., Bhatnagar, A., Beazer, J., Hussain, R., Russ, J., Yip, J., Hill, E., Pacesa, M., et al. (2024) · doi:10.1101/2024.04.22.590591 · Cited by 90
“This approach has been validated through the design of functional lysozymes (19), with sequence identities as low as 31% to any known natural protein, and demonstrated in silico across multiple families (20).”
BackgroundSelf-citationBhat, S. M., Palepu, K., Hong, L., Mao, J., Ye, T., Iyer, R., Zhao, L., Chen, T., Vincoff, S., Watson, R., et al. (2024) · bioRxiv · doi:10.1101/2023.06.26.546591 · Cited by 90
“22,23 Motivated by these findings, here, we develop a pipeline to design de novo target-binding peptide motifs, only requiring the amino acid sequence of the target protein.”
Democratizing protein language models with parameter-efficient fine-tuning
Sledzieski, S., Kshirsagar, M., Baek, M., Dodhia, R., Lavista Ferres, J., & Berger, B. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2405840121 · Cited by 86
Roche, R., Moussad, B., Shuvo, M. H., Tarafder, S., & Bhattacharya, D. (2024) · Nucleic Acids Research · doi:10.1093/nar/gkae039 · Cited by 85
He, Y., Zhou, X., Chang, C. H., Chen, G., Liu, W., Li, G., Fan, X., Sun, M., Miao, C., Huang, Q., et al. (2024) · Molecular Cell · doi:10.1016/j.molcel.2024.01.021 · Cited by 79
Qiu, X., Li, H., Steeg, G. V., & Godzik, A. (2024) · Biomolecules · doi:10.3390/biom14030339 · Cited by 103
“ProGen [10], ProteinMPNN [11], EvoDi ff [12], and RFdi ff usion [13] extend the AI capabilities to novel protein design, as does Dif-fDock [14] to molecular docking.”
BackgroundKnowledge Mechanisms in Large Language Models: A Survey and Perspective
Wang, M.-R., Yao, Y., Xu, Z., Qiao, S.-F., Deng, S.-M., Wang, P., Chen, X., Gu, J.-C., Jiang, Y., Xie, P.-J., et al. (2024) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2407.15017 · Cited by 82
“…two levels: 1) LLMs create new terms following the current world’s principles comprehended by LLMs, such as new proteins (Shin et al., 2021; Madani et al., 2023), molecules (Bagal et al., 2022; Fang et al., 2023; Edwards et al., 2022), code (DeLorenzo et al., 2024), video (Kondratyuk et…”
Progress and opportunities of foundation models in bioinformatics
Li, Q., Hu, Z., Wang, Y., Li, L., Fan, Y., King, I., Jia, G., Wang, S., Song, L., & Li, Y. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae548 · Cited by 72
“Biological FMs such as ProtST [35], HyenaDNA [41], and ProGen [43] can be used in this approach, often combined with few-shot learning, to allow for more controlled outputs that are aligned with the desired application.”
BackgroundInfluentialDas, U. (2025) · Medicine in Drug Discovery · doi:10.1016/j.medidd.2025.100213 · Cited by 91
Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis
Liu, J., Cen, X., Yi, C., Wang, F.-A., Ding, J., Cheng, J., Wu, Q., Gai, B., Zhou, Y., He, R., et al. (2025) · Genomics, Proteomics & Bioinformatics · doi:10.1093/gpbjnl/qzaf011 · Cited by 75
“The success of LLMs has revolutionized AI learning in nat-ural language and image processing [75] and has been extended into the learning of biological sequence datasets [80].”
Large scale paired antibody language models
Kenlay, H., Dreyer, F. A., Kovaltsuk, A., Miketa, D., Pires, D. E. V., & Deane, C. M. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1012646 · Cited by 70
“…architectures such as Evolutionary Scale Modeling (ESM) (Rives et al., 2021; Lin et al., 2023), ProtTrans (El-naggar et al., 2022), and ProGen (Madani et al., 2023) have been pivotal in protein structure prediction and function annotation, providing a solid foundation for generalised protein…”
Evaluating generalizability of artificial intelligence models for molecular datasets
Ektefaie, Y., Shen, A. N., Bykova, D., Marin, M. G., Žitnik, M., & Farhat, M. R. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00931-6 · Cited by 69
“Many machine learning models are pretrained on molecular sequencing datasets and then trained and tested on usually smaller task-specific datasets not encountered in pretraining 12,15,50 .”
PTM-Mamba: A PTM-Aware Protein Language Model with Bidirectional Gated Mamba Blocks
Peng, F. Z., Schussheim, B., & Chatterjee, P. (2024) · bioRxiv · doi:10.1145/3627673.3680276 · Cited by 74
“Autoregressive pLMs, such as Pro-Gen and ProtGPT2, have generated diverse protein sequences with validated functional capacities [Madani et al., 2023, Ferruz et al., 2022].”
BackgroundDe novo generation of SARS-CoV-2 antibody CDRH3 with a pre-trained generative large language model
He, H., He, B., Guan, L., Zhao, Y., Jiang, F., Chen, G., Zhu, Q.-G., Chen, C. Y., Li, T., & Yao, J. (2024) · Nature Communications · doi:10.1038/s41467-024-50903-y · Cited by 78
“Madani et al. proposed ProGen 20 , a method for generating protein sequences that are controlled by protein properties.”
CodonTransformer: a multispecies codon optimizer using context-aware neural networks
Fallahpour, A., Gureghian, V., Filion, G. J. P., Lindner, A. B., & Pandi, A. (2025) · Nature Communications · doi:10.1038/s41467-025-58588-7 · Cited by 70
“This is of particular interest in addressing a pronounced bottleneck in the expressibility of de novo-designed proteins 14,16 .”
The role and future prospects of artificial intelligence algorithms in peptide drug development
Chen, Z., Wang, R., Guo, J., & Wang, X. (2024) · Biomedicine & Pharmacotherapy · doi:10.1016/j.biopha.2024.116709 · Cited by 79
Continuous evolution of user-defined genes at 1 million times the genomic mutation rate
Rix, G., Williams, R. L., Hu, V. J., Spinner, A., Pisera, A. (., Marks, D. S., & Liu, C. C. (2024) · Science · doi:10.1126/science.adm9073 · Cited by 71
Continuous evolution of user-defined genes at 1-million-times the genomic mutation rate
Rix, G., Williams, R. L., Spinner, H., Hu, V. J., Marks, D. S., & Liu, C. C. (2023) · doi:10.1101/2023.11.13.566922 · Cited by 71
“…at extracting meaningful representations of biomolecular structure and function from the extensive diversity within and across gene families, as exemplified by their ability to predict functional effects of mutations 26–28 , design functional sequences 29–31 , and predict protein structures 32–34 .”
BackgroundBiophysics-based protein language models for protein engineering
Gelman, S., Johnson, B., Freschlin, C. R., Sharma, A., D’Costa, S., Peters, J., Gitter, A., & Romero, P. A. (2025) · Nature Methods · doi:10.1038/s41592-025-02776-2 · Cited by 77
“Predictive models can guide searches over the sequence–function landscape to enhance natural proteins or design new proteins 6,34,35 .”
Harnessing generative AI to decode enzyme catalysis and evolution for enhanced engineering
Xie, W. J., & Warshel, A. (2023) · National Science Review · doi:10.1093/nsr/nwad331 · Cited by 69
Revolutionizing CRISPR technology with artificial intelligence
Kim, M., Go, M.-J., Kang, S.-H., Jeong, S.-H., & Lim, K. (2025) · Experimental & Molecular Medicine · doi:10.1038/s12276-025-01462-9 · Cited by 87
“More-over, beyond simple data analysis, AI-driven models generate novel DNA, RNA and amino acid sequences 34,35 , bringing innovation to genome editing fi elds that were dif fi cult to access.”
Foundation models for bioinformatics
Chen, Z., Wei, L., & Gao, G. (2024) · Quantitative Biology · doi:10.1002/qub2.69 · Cited by 27
“Oppositely, GPT ‐ like models can be adapted for control tag ‐ based de novo protein generation or protein engineering tasks with high diversity and success rate [45–49].”
Chen, B., Cheng, X., Li, P., Geng, Y., Gong, J., Li, S., Bei, Z., Tan, X., Wang, B., Zeng, X., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02636-z · Cited by 216
“As protein sequences serve as the blueprint for protein structure and function 4 , pretrained techniques on sequences, known as protein language models (PLMs), for example, the family of ESM models 5,6 , ProtTrans 7 and PROGEN 8 , offer a powerful tool for characterizing the properties and distributions of general pro…”
InfluentialScaling Unlocks Broader Generation and Deeper Functional Understanding of Proteins
Bhatnagar, A., Jain, S., Beazer, J., Curran, S., Hoffnagle, A. M., Ching, K. S., Martyn, M., Nayfach, S., Ruffolo, J. A., & Madani, A. (2025) · bioRxiv · doi:10.1101/2025.04.15.649055 · Cited by 65
“…Models Protein language models (PLMs) encompass a wide range of approaches including causal decoders that generate novel proteins from scratch [63, 68, 80] or infill spans in the middle of a protein [15, 86], bidirectional encoders useful for protein understanding [40, 59, 78, 90], and inverse…”
InfluentialSelf-citationZhai, J., Gokaslan, A., Schiff, Y., Berthel, A., Liu, Z.-Y., Lai, W., Miller, Z. R., Scheben, A., Stitzer, M. C., Romay, M. C., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2421738122 · Cited by 97
Wang, W., Shuai, Y., Zeng, M., Fan, W., & Li, M. (2025) · Nature Communications · doi:10.1038/s41467-024-54816-8 · Cited by 68
A general temperature-guided language model to design proteins of enhanced stability and activity
Jiang, F., Li, M., Dong, J., Yu, Y., Sun, X., Wu, B., Huang, J., Kang, L., Pei, Y., Zhang, L., et al. (2024) · Science Advances · doi:10.1126/sciadv.adr2641 · Cited by 71
Large Language Models in Drug Discovery and Development: From Disease Mechanisms to Clinical Trials
Zheng, Y., Koh, H. Y., Yang, M., Li, L., May, L. T., Webb, G. I., Pan, S.-R., & Church, G. (2024) · arXiv.org · doi:10.48550/arxiv.2409.04481 · Cited by 63
“Specialized LLMs like ProGen (Madani et al., 2023) and PoET (Watson et al., 2023) are utilized in this context.”
MethodMedical large language models are susceptible to targeted misinformation attacks
Han, T., Nebelung, S., Khader, F., Wang, T., Müller-Franzes, G., Kuhl, C., Försch, S., Kleesiek, J., Haarburger, C., Bressem, K. K., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01282-7 · Cited by 62
“Once trained, these models can perform a variety of tasks in a zero-shot manner, often achieving state-of-the-art performance in areas suchas natural language processing, computervision, and protein design 9 – 15 .”
Foundation models in bioinformatics
Guo, F., Guan, R., Li, Y., Liu, Q., Wang, X., Yang, C., & Wang, J. (2025) · National Science Review · doi:10.1093/nsr/nwaf028 · Cited by 127
“A new algorithm, ProGen [57 ], integrates UniprotKB keywords into conditional tags and generates proteins with desired structural properties.”
The OMG dataset: An Open MetaGenomic corpus for mixed-modality genomic language modelingPreprint
Cornman, A., West-Roberts, J. A., Camargo, A. P., Roux, S., Beracochea, M., Mirdita, M., Ovchinnikov, S., & Hwang, Y. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.08.14.607850 · Cited by 52
“Protein language models (pLMs) (Lin et al., 2023; Madani et al., 2023; Elnaggar et al., 2022) are typically trained on high quality and curated publicly available datasets such as UniRef (Suzek et al., 2007).”
BackgroundMethodAdvancing microbial production through artificial intelligence-aided biology
Gong, X., Zhang, J., Gan, Q., Teng, Y., Hou, J., Lyu, Y., Liu, Z., Wu, Z., Dai, R., Zou, Y., et al. (2024) · Biotechnology Advances · doi:10.1016/j.biotechadv.2024.108399 · Cited by 85
Deep learning for protein structure prediction and design—progress and applications
Jänes, J., & Beltrão, P. (2024) · Molecular Systems Biology · doi:10.1038/s44320-024-00016-x · Cited by 76
“(Madani et al, 2023), pre-de fi ned oligomerization states with novel proteins (11.5%), novel ion binding proteins (40”
BackgroundInfluentialOn the Scalability of GNNs for Molecular Graphs
Sypetkowski, M., Wenkel, F., Poursafaei, F., Dickson, N., Suri, K., Fradkin, P., & Beaini, D. (2024) · Neural Information Processing Systems · doi:10.48550/arxiv.2404.11568 · Cited by 61
“Madani et al. (2023) learn to generate a family of functional proteins.”
Programmable synthetic receptors: the next-generation of cell and gene therapies
Teng, F., Cui, T., Zhou, L., Gao, Q., Zhou, Q., & Li, W. (2024) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-023-01680-5 · Cited by 70
Benchmarking DNA foundation models for genomic and genetic tasks
Feng, H., Wu, L., Zhao, B., Huff, C. D., Zhang, J. J., Wu, J., Lin, L., Wei, P., & Wu, C. (2025) · Nature Communications · doi:10.1038/s41467-025-65823-8 · Cited by 83
Buess, L., Keicher, M., Navab, N., Maier, A., & Arasteh, S. T. (2025) · Biomedical Engineering Letters · doi:10.1007/s13534-025-00497-1 · Cited by 79
“…DNABERT [56] Structure analysis Geneformer [57] Classification Hie et al. [58] Antibody design MSA Transformer [59] Structure analysis ProGen [60] Structure prediction ProtGPT2 [61] Protein design ProtTrans [62] Structure analysis scBERT [63] Classification ToxinPred 3.0 [64] Classification…”
InfluentialDrugReAlign: a multisource prompt framework for drug repurposing based on large language models
Wei, J., Zhuo, L., Fu, X., Zeng, X., Wang, L., Zou, Q., & Cao, D. (2024) · BMC Biology · doi:10.1186/s12915-024-02028-3 · Cited by 60
“This ability is especially crucial in fields dealing with heterogeneous, interdisciplinary data, such as genomics [13], protein engineering [14], material science [15] and drug development [16, 17].”
BackgroundEvaluating the representational power of pre-trained DNA language models for regulatory genomicsPreprint
Tang, Z., Somia, N., Yu, Y.-Y., & Koo, P. K. (2024) · bioRxiv · doi:10.1101/2024.02.29.582810 · Cited by 58
“For instance, representations from pre-trained protein language models have been used to predict protein structures 9–11 , predict non-synonymous variant effects 12,13 , design novel protein sequences 14–16 , and study protein evolution 17,18 .”
Kobiella, C., Flores López, Y. S., Waltenberger, F., Draxler, F., & Schmidt, A. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641964 · Cited by 57
“ChatGPT has been widely adopted in different fields of knowledge work such as consumer science [69], education [49, 65], nat-ural science [58, 84], and software engineering [57].”
MethodProtein language models are biased by unequal sequence sampling across the tree of life
Ding, F., & Steinhardt, J. (2024) · doi:10.1101/2024.03.07.584001 · Cited by 57
“…models (pLMs) have been trained to model the distribution of naturally occurring protein sequences (Rives et al., 2021; Elnaggar et al., 2021; Madani et al., 2023; Lin et al., 2023; Alamdari et al., 2023). pLMs have been successfully used to predict protein 3D structure (Lin et al., 2023),…”
BackgroundMethodWang, T., Xiang, G., He, S., Su, L., Wang, Y., Yan, X., & Lu, H. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae409 · Cited by 57
Revolutionizing peptide‐based drug discovery: Advances in the post‐AlphaFold era
Chang, L., Mondal, A., Singh, B., Martínez‐Noa, Y., & Perez, A. (2024) · WIREs Computational Molecular Science · doi:10.1002/wcms.1693 · Cited by 56
Metadata
- Authors
- Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos, J. L., Xiong, C., Sun, Z. Z., Socher, R., Fraser, J. S., & Naik, N.
- Affiliations
- Salesforce (United States) (US)
- University of California, San Francisco (US)
- Howard Hughes Medical Institute (US)
- Lawrence Berkeley National Laboratory (US)
- Corresponding author
- Madani, Ali
- Naik, Nikhil
- ORCID
- Venue
- Nature Biotechnology
- Published
- 2023-01-26
- Volume
- 41
- Issue
- 8
- Pages
- 1099-1106
- ISSN
- 0733-222X (ISSN-L)
- 1087-0156
- 1546-1696
- PMC
- PMC10400306
- OpenAlex
- W4318071656
- Cited by
- 1,078
- References
- 80
- Citations by year
- 2026: 314 · 2025: 320 · 2024: 302 · 2023: 141
- FWCI
- 95.97
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- Nature Biotechnology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- eScholarship (California Digital Library) (submittedVersion)
- OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion)
- Funders
- U.S. Department of Health & Human Services | National Institutes of Health (GM123159)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Protein Structure and Dynamics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Machine Learning in Bioinformatics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Enzyme Catalysis and Immobilization
- Keywords
- large language models, protein sequence generation, protein design, protein families, lysozyme, chorismate mutase, malate dehydrogenase, catalytic efficiency, controllable generation, protein engineering
- MeSH terms
- Amino Acid Sequence, Chorismate Mutase, Estrogens, Conjugated (USP), Language, Proteins
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
