Large language models generate functional protein sequences across diverse families

“Building on similar transformer principles, generative sequence models have successfully produced functional proteins with experimentally validated activity, illustrating that LLM systems can move from passive annotation toward hypothesis-driven molecular design [55].”

“…enabling potential continues to expand across diverse domains, ranging from protein structure prediction (Jumper et al., 2021) and molecular design (Madani et al., 2023) in the life science, to novel compound generation and material property prediction in chemistry and materials science (Duan et…”

“Subsequent work extended diffusion models to discrete biological sequences [ 32 , 33 ], protein sequence generation [ 34 , 35 ], and biologically constrained optimization [ 36 , 37 ], high-lighting their versatility across genomic and molecular design tasks.”

“…(ProtT5-XL; Elnaggar et al., 2021), bidirectional MLM variants beyond ESM-2 (ESM-1b, Rives et al., 2021; AMPLIFY-350M, Fournier et al., 2024), and a causal LM (ProGen2-xlarge; Madani et al., 2023); we introduce repr-CJ as the technical change that makes the cross-architecture comparison possible.”

“For instance, ProGen [7, 19] and ProLLaMA [48] enable conditional generation to create novel proteins across diverse families, while frontier multimodal foundation model like ESM3 [11] can jointly reason over sequence, structure, and function to perform complex, instruction-guided design tasks,…”

“These embeddings have been successfully repurposed for diverse downstream tasks, including protein structure prediction (Madani et al. 2023, Ruffolo and Madani 2024, Zhang et al. 2024b, Gomez-Uribe et al. 2025) or mutation effect analysis (Marquet et al. 2022, Marquet et al. 2024, Sun and Shen…”

“Central to many of these advances are protein language models (PLMs), which learn high-dimensional representations of amino acid sequences that capture structural, functional, and evolutionary relationships [18], enabling the design of proteins with improved stability or function [19–21].”

“…scores from masked pLMs have been widely used as zero-shot proxies for mutational effects, functional tolerance, viral escape, and generated-protein filtering (Salazar et al., 2020; Meier et al., 2021b; Hie et al., 2021; Verkuil et al., 2022; Madani et al., 2023; Notin et al., 2023).”

“S1b) 23,33,34 .”

“Sequence-level generative models learn distributions over protein sequences, either unconditionally or under functional, family, or structural conditions [9, 54].”

“…patterns, active-site motifs, and mutational tolerance, and can help estimate missense variant impact, including effects that correlate with stability or function, enhancing variant prioritization for disease causality, and supporting sequence design toward desired protein functions [18–20].”

“Recent protein language models, such as ProtGPT2, 48 ProGen, 49 and ProGen2, 46 demonstrated protein sequence generation capabilities but lack precise control mechanisms.”

“Recent advances in large-scale protein language models (pLMs) 39–42 have demonstrated that amino acid sequences can be modeled analogously to natural language.”

“Large protein language models trained on massive corpora learn rich sequence regularities and can be used for unconditional generation or prompted design (Rives et al., 2021; Elnaggar et al., 2022; Madani et al., 2023).”

“In the protein domain, conditional generation is further along, with family-conditional models [Madani et al., 2023] and discrete-diffusion approaches [Alamdari et al., 2023] demonstrating controllable single-chain generation; the analogous problem for DNA at the construct level has remained open.”

“Protein sequences are input into the pretrained protein language models ESMFold and ESM-2, producing predicted structural PDB files and 2,560-dimensional sequence embeddings, respectively.”

“Some research treats protein sequences as biological language and utilizes transformer architectures to model amino acid interactions [55, 56].”

“PLM-based peptide and protein design [8] optimizes for sequence naturalness, that is, sequences that look plausible to a model trained on the protein universe.”

“The most straightforward approach is supervised fine-tuning (SFT) [22, 25, 26], which adapts the PLM on a curated set of sequences pre-screened to satisfy the target objective.”

“…and therapeutic design (Jumper et al. (2021); Mak et al. (2024); Baek et al. (2021); Dauparas et al. (2022); Lin et al. (2023); Corso et al. (2023); Chen et al. (2022); Olsen et al. (2022); Shuai et al. (2021); Bagal et al. (2022); Ahmad et al. (2022); Madani et al. (2023); Ferruz et al. (2022)).”

“Although deep learning has advanced protein structure prediction and design [21–25], most approaches operate primarily at the level of sequences, structures or local functional motifs, rather than explicitly learning domain-level compatibility across natural multidomain architectures.”

“Furthermore, experimental testing confirmed that the synthesized lysozymes retained enzymatic activity despite their low sequence homology with natural ones [9].”

“For example, protein language models (pLMs) have emerged as a mainstream framework for enzyme function prediction and design [112].”

“Generative models for protein design (Madani et al., 2023; Lin et al., 2023) produce variants that retain function while drifting in primary sequence, and red-team studies have begun to evaluate whether language-model assistance offers operational uplift to non-state actors planning biological…”

“These embeddings have been successfully repurposed for diverse downstream tasks, including protein structure prediction (Gomez-Uribe et al., 2025; Madani et al., 2023; Ruffolo & Madani, 2024; Z. Zhang et al., 2024) or mutation effect analysis (Marquet et al., 2022; Marquet et al., 2024; Sun & Shen,…”

“Protein language models (PLMs) have seen significant advancements in recent years (Rives et al. 2021, Elnaggar et al. 2022, Madani et al. 2023, Su et al. 2024).”

“This ability is in contrast to the state of the art in functional fine-tuning of protein language models for enzymes, where ‘function’ is often reduced to a single E.C. class 65 or semi-automated database tags 60,68 .”

“Large language models have been trained on massive corpora of text, code, images, and proteins [12, 18, 19, 46, 50, 63, 79].”

“17 These generative results challenge prior assumptions about minimum sequence identity requirements for function and suggest that PLM-learned sequence spaces are substantially broader than evolutionary sampling alone would suggest.”

“Given that the MHCII binding peptides lack stable secondary structures, they cannot be effectively designed using conventional or AI-based methods which rely heavily on the structural information (Madani et al .”

“Each pathway used initial embedding and contact map predictions [23] derived from the powerful ESM-1b protein language model [24].”

“Concurrently, generative models like Pro-Gen (Madani et al. 2023) have enabled the design of novel, experimentally validated functional proteins.”

“…motifs, or module-level constraints, PLM-guided design supports rational diversification while maintaining consistency with biological priors learned from large-scale sequence databases, significantly narrowing the effective search space compared with traditional random mutagenesis[38, 58].”

“Similarly, the ProGen family focuses on controllable and scalable protein synthesis [ 42 – 44 ] .”

“To realise this, we simulate mutational pathways from a source to a target sequence, using the MSA-Transformer to iteratively introduce substitutions that steer the source sequence toward the target.”

“Recently, protein language models (pLMs) have emerged as an effective and scalable solution (Rives et al., 2021; Lin et al., 2023; Madani et al., 2023; Nijkamp et al., 2023; Fournier et al., 2024).”

“More recently, generative AI and language models have been applied to the field of protein designs [26, 27].”

“In parallel, large protein language models have enabled sequence-based representations with coevolutionary dependencies and residue-level constraints 30-35 .”

“Motivated by the success of protein language models [29, 18, 16], recent work has scaled masked and autoregressive architectures to long contexts and trained them on one or more whole genomes.”

“Autoregressive protein language models (Madani et al., 2023; Ferruz et al., 2022) can sample novel sequences, but left-to-right factorization is poorly suited for constrained loop redesign where residues are jointly constrained by both N-terminal and C-terminal context.”

“…phylogenetic trees that impose discrete branching, PLM-derived embedding spaces encode molecular similarity as continuous coordinates within a high-dimensional manifold, thereby reflecting gradual evolutionary transitions (LaTourrette and Garcia-Ruiz, 2022; Madani et al., 2023; Mao et al., 2019).”

“Building on these successes, researchers have been exploring the potential of LLMs in understanding specialized domains beyond natural language, ranging from healthcare (Singhal et al., 2023) and chemistry (Bran et al., 2024) to biology (Madani et al., 2023) and other professional fields.”

“…[10, 11], drug discovery [12], personalized patient communication [13], large-scale data annotation [14, 15], protein sequence analysis and prediction [16, 17, 18, 19, 20, 21], clinical text processing [22], rare disease concept normalization [23], and general biomedical NLP applications [24, 25].”

“It enables de novo generation of functional proteins by conditioning on family, function, or structural annotations, with validated applications in antibody and enzyme engineering [11].”

“7,8 Together, these tools accelerate candidate prioritization and substantially reduce the need for exhaustive experimental screening.”

“Recent advances in generative AI and large language models (LLMs) [14] research offer a promising avenue for toxicity prediction [13] through learning the pattern of different compositions of the protein [7], [11], whose length may be as large as 40k-60k and beyond.”

“Protein design methods can generally be categorized into three approaches: sequence-based meth-ods [15, 30, 49], structure-based methods [50, 55, 58], and sequence-structure co-design approaches [21, 25].”

“Examples include ProtBERT [84], ESM [85], and ProGen [86].”

“Protein representation learning with protein language models (PLMs) extends the Transformer to amino-acid strings, producing dense embeddings for property prediction (16, 3, 10, 17, 4, 6, 7) or generative design (32, 34, 29, 12).”

“Transformer-based biological language models (BLMs) have shown remarkable ability to capture complex, long-range patterns in nucleotide and protein sequences [3, 26, 31, 39, 42], which makes them well-suited to uncover potential biological signals beyond lo-cal codon usage [8].”

“This training process captures the structural and evolutionary constraints embedded in protein sequences, enabling the computational design of proteins with a high likelihood of proper folding and function (Ferruz et al., 2022; Madani et al., 2023).”

“Generative models such as ProtGPT2 [4], ProGen [11], and RFdiffusion [16] have shown that language and diffusion models can generate de novo proteins with realistic folds and functional activity.”

“…and architectures (i.e. autoencoder vs encoder-only vs decoder-only), alphabet/vocabulary discretization (e.g. decomposing sequences into individual amino-acids or longer subsequences), and training task (i.e. autoregressive vs masked prediction objectives) vary significantly [1–3,10,17,18].”

“Generative PLMs can make small structural and/or functional changes, for example, by infilling nature-adjacent variants into an enzyme class [Munsamy et al., 2022, Madani et al., 2023].”

“Recent models such as ProtGPT2, EvoDiff, and AlphaFold leverage large language architectures, diffusion processes, and hybrid approaches to generate novel protein sequences and structures with increasing accuracy and accessibility [1]–[3].”

“[10,11] Harnessing the potential of generative AI models, we have recently developed “DrugGen,” an online tool that utilizes an enhanced transformer‑based generative model to generate valid small molecules with superior binding affinities and docking performance.”

“…[4], (cid:129) for “summarizing laboratory reports (eg, key findings, abnormal findings, notes on uncommon pathogens, advice on antibiotic selection, and next steps in testing)” [11], (cid:129) for generating “protein sequences with predictable functions across large protein families” [22].”

“Stochastic context-free grammars were also used to capture long-distance dependencies in amino acid positions [9–12].”