Sequence modeling and design from molecular to genome scale with Evo

In the paper

“We compiled a large genome dataset called OpenGenome (Materials and methods) with more than 80,000 bacterial and archaeal genomes and millions of predicted phage and plasmid sequences, covering 300 billion nucleotide tokens”

What the sources say

• ?Could not verify

  The Introduction and Data Availability sections describe GTDB as a bacterial and archaeal genome taxonomy, not OpenGenome or Evo’s training data. Searches for OpenGenome, 80,000 genomes, phage and plasmid sequences, and 300 billion nucleotide tokens found no passage establishing the claimed dataset composition or size. (LLM, conf=0.76)

  cites [35] Parks DH, GTDB: An ongoing census of bacterial and archaeal diversity through a phylogenetically consistent, rank normalized and complete genome-based taxonomy. Nucleic Acids Res. 50, D785– D794 (2022). doi: 10.1093/nar/gkab776 34520557 PMC8728215 sourceExperimental — accuracy not yet measured
• Partly supported

  The 'Automatic virus identification from IMG datasets' section (p. 3) says the authors processed 83,858 bacterial and archaeal isolate genomes, and the Abstract (p. 2) reports more than 15 million viral genomes and fragments. The paper does not establish that those sequences formed OpenGenome or were used to train Evo, nor does it give a 300-billion-token total. The Conclusion (p. 11) describes a plasmid-focused database as still in development, rather than reporting millions of plasmid sequences in this collection. (LLM, conf=0.86)

  cites [36] Camargo AP, IMG/VR v4: An expanded database of uncultivated virus genomes within a framework of extensive functional, taxonomic, and ecological metadata. Nucleic Acids Res. 51, D733– D743 (2023). doi: 10.1093/nar/gkac1037 36399502 PMC9825611 sourceExperimental — accuracy not yet measured
• ?Not checked — no full text of the sourcecites [37] Camargo AP, IMG/PR: A database of plasmids from genomes and metagenomes with rich annotations and metadata. Nucleic Acids Res. 52, D164– D173 (2024). doi: 10.1093/nar/gkad964 37930866 PMC10767988 source

In the paper

“existing attempts to model DNA as a language (15–17) are limited by the prevailing dense Transformer architecture, which incurs high computational cost as input sequence lengths grow relative to model width (scaling quadratically) and generally underperforms at single-nucleotide or byte-level resolution compared with models trained at coarser resolutions (18).”

What the sources say

• Partly supported

  Split into 2 atoms — 0 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

  cites [15] Zvyagin M., GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics. Int. J. High Perform. Comput. Appl 37, 683– 705 (2023). doi: 10.1177/10943420231201154 sourceExperimental — accuracy not yet measured
• Partly supported

  Split into 2 atoms — 1 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

  cites [16] Dalla-Torre H., The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics. bioRxiv 2023.01.11.523679 [Preprint] (2023); 10.1101/2023.01.11.523679. PMC11810778 39609566 sourceExperimental — accuracy not yet measured
• Partly supported

  Split into 2 atoms — 1 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

  cites [17] Zhou Z, Ji Y, Li W, Dutta P, Davuluri R, Liu H, DNABERT-2: Efficient foundation model and benchmark for multi-species genome. arXiv:2306.15006 [q-bio.GN] (2023). sourceExperimental — accuracy not yet measured
• Partly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  cites [18] Tay Y., Charformer: Fast Character Transformers via Gradient-based Subword Tokenization. arXiv: 2106. 12672 [cs.CL] (2022). sourceExperimental — accuracy not yet measured

In the paper

“For a few genomes, the zero-shot performance with 66k context is notably strong, with an AUROC of 0.90 on lambda phage essentiality data (74) and an AUROC of 0.84 on Pseudomonas aeruginosa essentiality data (75)”

What the sources say

• Supported

  The citation identifies the lambda-phage essentiality dataset used for the citing authors’ zero-shot evaluation, not the source of the AUROC result. The Results section (‘Genome-wide CRISPRi to map gene essentiality in λ’) reports gene-level essentiality measurements for lambda phage. The Pseudomonas result has a separate citation. (LLM, conf=0.91)

  cites [74] Piya D., Systematic and scalable genome-wide essentiality mapping to identify nonessential genes in phages. PLOS Biol. 21, e3002416 (2023). doi: 10.1371/journal.pbio.3002416 PMC10695390 38048319 source
• ?Not checked — no full text of the sourcecites [75] Turner KH, Wessel AK, Palmer GC, Murray JL, Whiteley M, Essential genome of Pseudomonas aeruginosa in cystic fibrosis sputum. Proc. Natl. Acad. Sci. U.S.A 112, 4110– 4115 (2015). doi: 10.1073/pnas.1419677112 25775563 PMC4386324 source

In the paper

“Evo’s zero-shot correlation on promoter-RBS sequences is also higher than the GC content of the promoter-RBS sequences (Spearman r = 0.47), zero-shot GenSLM likelihoods (Spearman r = 0.11), and RBS Calculator (Spearman r = 0.39)”

What the sources say

• Supported

  The citation identifies RBS Calculator as a comparison tool. The paper’s Materials and Methods section (‘Software Implementation’; page unavailable) explicitly names its software the RBS Calculator and says it predicts translation initiation rates. The citation does not need to establish the citing paper’s Evo benchmark correlations. (LLM, conf=0.90)

  cites [57] Salis HM, Mirsky EA, Voigt CA, Automated design of synthetic ribosome binding sites to control protein expression. Nat. Biotechnol 27, 946– 950 (2009). doi: 10.1038/nbt.1568 19801975 PMC2782888 source
• Partly supported

  The section “A systematic comparison of translation initiation rate models” (page unavailable) identifies RBS Calculator as a model for predicting translation initiation rates. It does not report a Spearman correlation of 0.39 on promoter-RBS sequences or compare the model with Evo, GC content, or GenSLM; those performance claims are not supported by this source. (LLM, conf=0.86)

  cites [58] Reis AC, Salis HM, An automated model test system for systematic development and improvement of gene expression models. ACS Synth. Biol 9, 3145– 3156 (2020). doi: 10.1021/acssynbio.0c00394 33054181 sourceExperimental — accuracy not yet measured

In the paper

“As a result, Transformer-based DNA models are constrained to short context lengths and use schemes that aggregate nucleotides into the basic units of language models, called tokens, thereby sacrificing single-nucleotide resolution (15, 16, 21–23).”

What the sources say

• Partly supported

  Section 4.2 contrasts standard 1,024- or 2,048-token contexts with the longer sequences needed for genomes, and Section 5.2.1 says GenSLM groups three nucleotides into each codon-level input token. But the paper does not say this sacrifices the ability to model individual nucleotide changes; Section 5.2.1 says GenSLM models individual mutations at nucleotide scale. The claim therefore overstates the resolution consequence and treats a common context-length challenge as a constraint on Transformer-based DNA models generally. (LLM, conf=0.83)

  cites [15] Zvyagin M., GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics. Int. J. High Perform. Comput. Appl 37, 683– 705 (2023). doi: 10.1177/10943420231201154 sourceExperimental — accuracy not yet measured
• Supported

  The Discussion (page unavailable) says the models’ attention span is limited to 6 kb and explains that standard transformer attention makes much longer inputs intractable. Appendix A.3 (page unavailable) says the tokenizer normally represents six nucleotides as one token, supporting the claim about aggregation rather than single-nucleotide token resolution. (LLM, conf=0.88)

  cites [16] Dalla-Torre H., The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics. bioRxiv 2023.01.11.523679 [Preprint] (2023); 10.1101/2023.01.11.523679. PMC11810778 39609566 source
• Supported

  The Introduction says DNABERT’s 500-base-pair input is insufficient for applications requiring longer context. The Conclusions say even GENA-LM’s longer inputs remain insufficient for some tasks and that BPE tokenization limits prediction granularity to tokens. Section 3.1 describes tokens that aggregate multiple base pairs, supporting the single-nucleotide-resolution point. (LLM, conf=0.88)

  cites [21] Fishman V., GENA-LM: A family of open-source foundational DNA language models for long sequences. bioRxiv 2023.06.12.544594 [Preprint] (2024); 10.1101/2023.06.12.544594. PMC11734698 39817513 source
• Partly supported

  Methods §2.2 says DNABERT uses k-mer tokens rather than individual bases and accepts at most 512 tokens per input, supporting the tokenization and context-limit portions of the claim. But the same section describes a method for processing longer sequences, and the Results state that DNABERT enables visualization of nucleotide-level importance. The paper therefore does not support the unqualified claim that token aggregation sacrifices single-nucleotide resolution. (LLM, conf=0.88)

  cites [22] Ji Y, Zhou Z, Liu H, Davuluri RV, DNABERT: Pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome. Bioinformatics 37, 2112– 2120 (2021). doi: 10.1093/bioinformatics/btab083 33538820 PMC11025658 sourceExperimental — accuracy not yet measured
• Partly supported

  The Introduction (page unavailable) describes GenSLMs as using codon-level representations and says that learning gene-to-gene interactions requires longer genomic segments. That supports an example of nucleotide aggregation and a need for longer context, but it does not establish that Transformer-based DNA models generally have short context lengths or sacrifice single-nucleotide resolution. (LLM, conf=0.78)

  cites [23] Hwang Y, Cornman AL, Kellogg EH, Ovchinnikov S, Girguis PR, Genomic language model predicts protein co-regulation and function. Nat. Commun 15, 2880 (2024). doi: 10.1038/s41467-024-46947-9 38570504 PMC10991518 sourceExperimental — accuracy not yet measured

In the paper

“The DNA-targeting Cas9 nuclease is typically encoded within 3000 to 4800 base pairs (bp) of coding sequence and found in close genomic proximity to its cognate CRISPR array (60).”

What the sources say

• Supported

  In “Metagenomic, Structural, and Functional Diversity of Cas9” (page unavailable), the paper reports Cas9 protein lengths of roughly 900–1,600 amino acids, with most centered around 1,100 or 1,350 amino acids. That is consistent with the approximate 3,000–4,800 bp coding range. “CRISPR-Cas9: From Yogurt to Genome Editing” (page unavailable) says cas genes are typically adjacent to CRISPR repeat elements, supporting the proximity claim. (LLM, conf=0.87)

  cites [60] Hsu PD, Lander ES, Zhang F, Development and applications of CRISPR-Cas9 for genome engineering. Cell 157, 1262– 1278 (2014). doi: 10.1016/j.cell.2014.05.010 24906146 PMC4343198 source

In the paper

“even a single-nucleotide mutation in an essential gene can be incompatible with life if it disrupts that gene’s expression or function (72).”

What the sources say

• Partly supported

  The Introduction describes essential genes as serving indispensable cellular functions and discusses identifying them through gene inactivation. The ‘Essentiality drives gene strand bias’ section says nonfunctional peptides are especially counter-selected in essential functions. Neither section addresses whether a single-nucleotide mutation disrupting an essential gene’s expression or function can be incompatible with life. (LLM, conf=0.82)

  cites [72] Rocha EPC, Danchin A, Gene essentiality determines chromosome organisation in bacteria. Nucleic Acids Res. 31, 6570– 6577 (2003). doi: 10.1093/nar/gkg859 14602916 PMC275555 sourceExperimental — accuracy not yet measured

In the paper

“Diverse families of CRISPR-Cas systems are found throughout bacterial and archaeal life, such as Cas12- or Cas13-based systems that target DNA and RNA, respectively (61).”

What the sources say

• Supported

  The Abstract identifies CRISPR-Cas as bacterial and archaeal adaptive immunity systems, and the Diversity and classification section describes their substantial diversity. The Class 2 effectors section identifies Cas12 as a type V effector in complexes with target DNA and states that Cas13 effectors cleave target RNA. Our copy provides no page numbers for these passages. (LLM, conf=0.92)

  cites [61] Koonin EV, Makarova KS, Origins and evolution of CRISPR-Cas systems. Phil. Trans. R. Soc. B 374, 20180087 (2019). doi: 10.1098/rstb.2018.0087 PMC6452270 30905284 source

In the paper

“For example, directly sampling from a large natural language model typically produces sequences that are grammatically correct yet locally biased toward simpler sentence constructions and that are globally incoherent, especially at long lengths.”

What the sources say

• ?Could not verify

  The Introduction (p. 2) discusses large language models’ zero-shot task performance, not the quality of text produced by direct sampling. Searches for grammaticality, simpler sentence constructions, and global incoherence over long spans found no passage in our copy addressing those claims. (LLM, conf=0.75)

  cites [83] Wei J., Finetuned language models are zero-shot learners. arXiv: 2109. 01652 [cs.CL] (2022). sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Introduction (page unavailable) discusses factual errors, biased text, and failure to follow instructions, but not the claimed pattern in directly sampled text. Searches for grammaticality, simpler sentence constructions, and long-span incoherence found no passage in our copy that addresses the claim. (LLM, conf=0.73)

  cites [84] Ouyang L., Training language models to follow instructions with human feedback. arXiv: 2203. 02155 [cs.CL] (2022). sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Introduction discusses why language-model outputs may need steering, but does not describe the claimed grammaticality, preference for simpler sentences, or long-span incoherence. The Additional information section discusses meaningless responses from an unlikelihood-training baseline, not the typical results of directly sampling a large language model. Searches for the claim’s distinctive wording and for long-text incoherence found no supporting passage in our copy; page numbers were not available. (LLM, conf=0.76)

  cites [85] Rafailov R., “ Direct Preference Optimization: Your Language Model is Secretly a Reward Model” in Advances in Neural Information Processing Systems, vol. 36, Oh A. et al., Eds. (Curran Associates, Inc., 2023), pp. 53728– 53741. sourceExperimental — accuracy not yet measured

In the paper

“TGA and TAA stop codons appeared most frequently, whereas TAG was least common, consistent with previously observed patterns in prokaryotic genomes (Fig. 6H) (80).”

What the sources say

• Supported

  The Results (p. 3) report that TAA or TGA is the most frequent stop codon across the bacterial phyla examined, depending on phylum and genomic GC content, while TAG is generally the least frequent. This supports the cited comparison to prior prokaryotic patterns; the source does not need to describe the Evo-generated sequences themselves. (LLM, conf=0.93)

  cites [80] Korkmaz G, Holm M, Wiens T, Sanyal S, Comprehensive analysis of stop codon usage in bacteria and its correlation with release factor abundance. J. Biol. Chem 289, 30334– 30342 (2014). doi: 10.1074/jbc.M114.606632 25217634 PMC4215218 source

In the paper

“We also examined stop codon frequencies across reading frames, a conserved genomic feature in prokaryotes (79).”

What the sources say

• Partly supported

  The Discussion reports that related bacterial species often have similar stop-signal ratios in the second and third reading frames, and the Conclusion says signals across all three frames are interrelated. This supports a conserved pattern among related bacteria, but the paper does not establish that stop-codon frequencies are conserved across prokaryotes generally. (LLM, conf=0.83)

  cites [79] Xu L, Kuo J, Liu J-K, Wong T-Y, Bacterial phylogenetic tree construction based on genomic translation stop signals. Microb. Inform. Exp 2, 6 (2012). doi: 10.1186/2042-5783-2-6 22651236 PMC3466146 sourceExperimental — accuracy not yet measured

In the paper

“Promisingly, in these domains, algorithmic techniques have emerged to improve the quality of generations compared with sampling from the pretrained model alone (83–85).”

What the sources say

• Partly supported

  The Introduction (p. 2) describes instruction tuning a pretrained language model, and the Results (p. 4) report that it significantly improves performance over the untuned model on most evaluated NLP tasks, including some generative tasks. The paper does not establish improved quality of open-ended generations compared with sampling from the pretrained model alone, and it does not study image generation. (LLM, conf=0.82)

  cites [83] Wei J., Finetuned language models are zero-shot learners. arXiv: 2109. 01652 [cs.CL] (2022). sourceExperimental — accuracy not yet measured
• Partly supported

  Section 3.1 describes fine-tuning a pretrained language model using demonstrations, human comparisons, and reward optimization. Section 4.1 reports that labelers preferred the resulting InstructGPT outputs to GPT-3 outputs on the paper’s test prompts. This supports improved natural-language generation over the pretrained model, but the cited paper does not establish the image-generation part of the claim. (LLM, conf=0.87)

  cites [84] Ouyang L., Training language models to follow instructions with human feedback. arXiv: 2203. 02155 [cs.CL] (2022). sourceExperimental — accuracy not yet measured
• Partly supported

  The Related Work section says fine-tuning language models with human preferences improves performance on several text-generation tasks, and Section 6.2 reports better summarization results for DPO than for other methods. This supports the claim for natural-language generation, but the paper does not establish it for image generation. (LLM, conf=0.87)

  cites [85] Rafailov R., “ Direct Preference Optimization: Your Language Model is Secretly a Reward Model” in Advances in Neural Information Processing Systems, vol. 36, Oh A. et al., Eds. (Curran Associates, Inc., 2023), pp. 53728– 53741. sourceExperimental — accuracy not yet measured

In the paper

“Natural language models often struggle to maintain coherent and diverse generation over long sequences”

What the sources say

• ?Could not verify

  The Introduction (p. 2) says recent models can compose coherent multiparagraph text, while Appendix D.5 (p. 25) examines loss and longer-range correlations across a context. Neither passage addresses whether language models often struggle to maintain coherence and diversity when generating long sequences; searches for those terms found no supporting passage. (LLM, conf=0.76)

  cites [39] Kaplan J., Scaling Laws for Neural Language Models. arXiv: 2001. 08361 [cs.LG] (2020).Experimental — accuracy not yet measured
• ?Could not verify

  The Introduction (p. 2) discusses difficulty with zero-shot tasks, not maintaining coherence or diversity in long text. The qualitative examples (p. 34) include a story of at least 150 tokens but do not make the claimed general statement. Searches for coherence, diversity, and long-form generation found no passage addressing it. (LLM, conf=0.76)

  cites [83] Wei J., Finetuned language models are zero-shot learners. arXiv: 2109. 01652 [cs.CL] (2022). sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Introduction describes failures such as making up facts, producing biased or toxic text, and not following instructions, but does not discuss coherence or diversity over long sequences. Searches for long-form generation, repetition, and output diversity found no passage establishing the claimed difficulty; the References merely list a work on coherent long-form text generation. (LLM, conf=0.72)

  cites [84] Ouyang L., Training language models to follow instructions with human feedback. arXiv: 2203. 02155 [cs.CL] (2022). sourceExperimental — accuracy not yet measured
• ?Could not verify

  The Additional information section says that an unlikelihood-training baseline produced meaningless responses on summarization and dialogue tasks, but it does not discuss whether language models generally struggle with coherence and diversity over long sequences. Searches for coherent and diverse long-sequence generation found no passage addressing that claim; the copy provides no page number for the cited section. (LLM, conf=0.72)

  cites [85] Rafailov R., “ Direct Preference Optimization: Your Language Model is Secretly a Reward Model” in Advances in Neural Information Processing Systems, vol. 36, Oh A. et al., Eds. (Curran Associates, Inc., 2023), pp. 53728– 53741. sourceExperimental — accuracy not yet measured
• ?Could not verify

  The discussion of Few-shot-CoT (p. 2) says language models have struggled with multi-step reasoning, not with maintaining coherence or diversity in long text generation. Searches for long-sequence generation, coherence, diversity, and repetition found no passage in our copy addressing the attributed claim. (LLM, conf=0.76)

  cites [87] Kojima T, Gu SS, Reid M, Matsuo Y, Iwasawa Y, “ Large Language Models are Zero-Shot Reasoners” in Advances in Neural Information Processing Systems, vol. 35, Koyejo S., Eds. (Curran Associates, Inc., 2022), pp. 22199– 22213. sourceExperimental — accuracy not yet measured

In the paper

“Multistage sequence length pretraining has been shown to reduce the overall number of compute hours required to train long context models (116).”

What the sources say

• Partly supported

  Section 4.4 (Training Curriculum; page not available in our copy) compares two-stage sequence-length training with training at the long sequence length throughout and reports about 40% fewer FLOPs with almost no performance loss. That supports reduced computational work, but the paper does not report compute hours, so the claim's specific time-based measure is not established. (LLM, conf=0.82)

  cites [116] Xiong W., Effective long-context scaling of foundation models. arXiv: 2309. 16039 [cs.CL] (2023). sourceExperimental — accuracy not yet measured

In the paper

“We experimented with grouped-query attention (GQA) (117) and found minimal differences in final loss, suggesting the technique may be suited to DNA sequence modeling, to further reduce memory footprint during inference.”

What the sources say

• Supported

  The citation identifies grouped-query attention, which Section 2.2 defines as sharing key and value heads across groups of query heads and describes as reducing inference memory demands. The citing authors present the minimal change in final loss and the possible application to DNA modeling as findings and inferences from their own experiment, not as findings attributed to this source. (LLM, conf=0.93)

  cites [117] Ainslie J., GQA: Training generalized multi-query transformer models from multi-head checkpoints. arXiv: 2305. 13245 [cs.CL] (2023). source

In the paper

“Evo is a genomic foundation model with 7 billion parameters trained with a context length of up to 131,072 tokens, using single-nucleotide, byte-level tokenization.”

In the paper

“We estimate 250 billion to be the compute-optimal number of tokens for Evo 7B given the floating point operation (FLOP) budget, meaning the model was trained at a 17% offset from the compute-optimal model size during the initial 8192 sequence length pretraining phase of 300 billion tokens.”

In the paper

“Both Transformer++ and Mamba experienced numerical instability during training and suffered from a higher performance degradation of the scaling rate outside the compute-optimal frontier, in contrast to StripedHyena”

In the paper

“Evo uses a context length of 131,072 tokens and is based on the StripedHyena architecture (24), which hybridizes attention and data-controlled convolutional operators to efficiently process and recall patterns in long sequences.”

What the sources say

• Not checked yetcites [24] Poli M., StripedHyena: Moving Beyond Transformers with Hybrid Signal Processing Models, GitHub (2023); https://github.com/togethercomputer/stripedhyena.

In the paper

“One of the generations exhibited robust activity, which we named EvoCas9–1.”

In the paper

“We observed that more conserved sequences tended to be essential, with an association strength similar to that of Evo with gene-only context but weaker than that of Evo with genomic sequence context”

In the paper

“Although the predicted backbone structure of EvoCas9–1 resembles that of SpCas9, the predicted structure of EvoCas9–1 exhibits a more positive surface charge distribution”

In the paper

“when concatenating the promoter and RBS sequence together, Evo’s zero-shot likelihoods improved substantially (Spearman r = 0.61); this correlation is also higher than the zero-shot correlation of just the promoter sequence alone (Spearman r = 0.47)”

In the paper

“TnpA and TnpB proteins that were detected within these generated sequences varied widely in their distance from the nearest examples in the training set (Fig. 4C), with consistently high ESMFold pLDDT values for predicted structures that were >40 to 50% identity to the training set”

In the paper

“This activity was also dependent on the presence of a putative catalytic tyrosine and on having a ssDNA substrate instead of double-stranded DNA (dsDNA), consistent with the known mechanism for IS200/IS605 TnpA”

In the paper

“These correlations also exceed those of the sequence guanine-cytosine (GC) content (mean Spearman r = 0.35) and the zero-shot likelihoods of GenSLM (mean Spearman r = 0.09)”

In the paper

“Evo-designed sgRNA is 91.1% identical to the canonical SpCas9 sgRNA and exhibits secondary structure differences in the two terminal stem loops, notably extending the length of stem loops 2 and 3”

In the paper

“We observed that 11 out of 24 Evo-generated IS200-like elements and 3 out of 24 Evo-generated IS605-like elements demonstrated evidence for both excision and insertion in vitro”

In the paper

“The EvoCas9–1 amino acid sequence shares 79.9% identity with the closest Cas9 in the database of Cas proteins used for model fine-tuning and 73.1% identity with SpCas9.”

In the paper

“We also observed that providing the model with additional genomic context beyond the gene sequence results in a substantial improvement in performance, especially from gene-only context to 8k context. From 8k to 66k context, the average predictive performance is comparable, although performance on the lower range of examples does improve with longer context”

In the paper

“As a whole, the 14 active elements use a diverse set of hairpins (Fig. 4, E and H, and figs. S25 and S26) and encode functional TnpA proteins with sequence identity as low as 67% to the fine-tuning database.”

In the paper

“The CNN architecture substantially outperformed ridge regression across both embeddings, and the Evo embeddings substantially outperformed one-hot embeddings across both architectures”

In the paper

“The functional IS605-like elements, which contain putative TnpB coding sequences, also contain sequences with significant matches (cmsearch E value < 0.001) to a covariance model constructed from known ωRNAs”

In the paper

“Compared with HyenaDNA (33), a previous generation of DNA models leveraging a Hyena architecture (34), Evo is based on an improved hybrid design and scaled to 1000× larger model size and 100× more data.”

What the sources say

• ?Not checked — no full text of the sourcecites [33] Nguyen E., “ HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution” in Advances in Neural Information Processing Systems, vol. 36, Oh A. et al., Eds. (Curran Associates, Inc., 2023), pp. 43177– 43201. source
• ?Not checked — no full text of the sourcecites [34] Poli M., Hyena Hierarchy: Towards Larger Convolutional Language Models. arXiv: 2302. 10866 [cs.LG] (2023). source

In the paper

“Evo is a genomic foundation model trained on hundreds of billions of DNA tokens across the evolutionary diversity of prokaryotic life, capable of prediction and generation tasks at the scale of individual molecules, molecular complexes, systems, and even whole genomes.”

In the paper

“When visualized, both natural and generated sequences display similar patterns of coding organization (Fig. 6C), with sequences in close proximity typically found with the same strand orientation”

In the paper

“Based on a state-of-the art hybrid model architecture, Evo enables single-nucleotide-resolution language modeling at a context length of 131,072.”

In the paper

“We used Evo to sample 16 sequences each containing ~1 Mb, representing more than seven times the model’s context length of 131 kb.”

In the paper

“Evo accurately performed zero-shot prediction across diverse fitness or expression prediction tasks on proteins, ncRNAs, or regulatory DNA that matches or outperforms specialized models while also understanding how mutations to individual genes can affect broader organismal fitness.”

In the paper

“As a multimodal generative model, we use Evo to generate CRISPR-Cas proteins and their noncoding guide RNAs, multicomponent transposable systems, and megabase-long sequences that recapitulate the architecture of real genomes.”

In the paper

“We found strong correlations between species-specific generations and their corresponding natural reference sequences, with TUDs sufficiently accurate to reconstruct natural phylogenetic relationships among the generated sequences (Fig. 6, F and G).”

In the paper

“We conducted the first scaling laws analysis of DNA pretraining across several architectures, where we observed StripedHyena outperforming several baseline architectures, including Transformers.”

In the paper

“Notably, Evo generated sequences have nearly the same coding densities as natural genomes, and substantially higher than that of random sequences (Fig. 6B).”

In the paper

“Evo can demonstrate similar properties.”

In the paper

“We pretrained Evo on a dataset of 300 billion prokaryotic tokens, which represents a miniscule portion of petabytes of publicly available genomic data.”

In the paper

“At the genome-scale, Evo generates megabase-long sequences that demonstrate a high-level understanding of genome organization, but it struggles to include key marker genes, such as full sets of rRNAs.”

In the paper

“we found that, in some instances, adding an additional random nucleotide character would improve the quality of generations.”

In the paper

“We experimentally validated the functional activity of Evo-generated CRISPR-Cas molecular complexes as well as IS200 and IS605 transposable systems”

In the paper

“EvoCas9–1 was generated from just 11 code-signs, representing a robust success rate given the complexity of Cas9’s multistep mechanism”

In the paper

“Model hybridization, first proposed to address shortcomings of state-space models (30–32), has recently been shown to improve scaling performance on language modeling of both standalone Hyena and Transformer architectures (24).”

What the sources say

• Not checked yetcites [24] Poli M., StripedHyena: Moving Beyond Transformers with Hybrid Signal Processing Models, GitHub (2023); https://github.com/togethercomputer/stripedhyena.

In the paper

“Evo is a foundation model that is designed to capture two fundamental aspects of biology: the multimodality of the central dogma and the multiscale nature of evolution.”

In the paper

“Because Evo learns covariation patterns involving any genetic elements within its context window, the model should understand interactions between encoded protein and ncRNA molecules.”

In the paper

“Evo demonstrates a robust and general understanding of the constitutive protein coding sequences, ncRNA sequences, and regulatory elements.”

In the paper

“For comparison, the smallest “minimal” bacterial genomes are ~580 kb in length (76).”

What the sources say

• ?Not checked — no full text of the sourcecites [76] Blanchard A, Bébéar C, The evolution of Mycoplasma genitalium. Ann. N. Y. Acad. Sci 1230, E61– E64 (2011). doi: 10.1111/j.1749-6632.2011.06418.x 22417108 source

In the paper

“As a negative control, we generated 1000 sequences of length 131,072 in which the four DNA base pairs were sampled uniformly at random.”

In the paper

“To account for different start codons and to generate a more conservative percentage identity estimate, these alignments were trimmed to the middle 80% of each sequence, end gaps were trimmed, and the amino acid percent identity was recalculated, which we called a “trimmed percent identity.””

In the paper

“CheckM uses Prodigal (140) to identify coding sequences and computes the coding density as one metric of genome quality.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites CheckM (77)

In the paper

“To select candidates for experimental validation, Cas9 generations with an ORF sequence identity higher than 90% to a training sequence were first filtered out.”

In the paper

“As an additional control, we also used the gene’s linear position in the reference genome as the value with which to predict essentiality.”

In the paper

“We evaluated these generations with CheckM (77), a tool that computes basic genome quality metrics based on whether a given long DNA sequence has similar properties as known bacterial genomes.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites CheckM (77)

In the paper

“CheckM will also search for the presence of genes that are highly conserved across much of prokaryotic diversity.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites CheckM (77)

In the paper

“CheckM uses Prodigal (140) to identify coding sequences and computes the coding density as one metric of genome quality.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites CheckM (77)
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites Prodigal (140)

In the paper

“For final synthesis and experimental validation, a different 60 bp filler sequence was used for the ISHp608-like candidates compared to the ISStin10-like, ISSpn6-like, or ISDge10-like sequences to eliminate a primer-binding site containing a TTAC, which is the canonical ISHp608 target site.”

In the paper

“We assessed statistical significance of the AUROC with a permutation-based method in which a null distribution is constructed by permuting the binary labels and recomputing the subsequent AUROC.”

• Also stated in: § Gene essentiality prediction · plain

  “We performed 100,000 permutations to construct this null distribution.”

In the paper

“To ensure that the Spearman correlation is comparable across these settings, we computed the correlation over all 12,243 examples (which involves duplicating sequences in the promoter-alone or RBS-alone settings).”

In the paper

“We also used the conservation of a gene as another control.”

In the paper

“As a positive control, we randomly selected 100 bacterial genomes from GTDB and used CheckM to compute the coding densities for 131,072 bp crops from these genomes.”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites GTDB
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites CheckM (77)

In the paper

“We also modified the data loader such that each sample provided to the model during training would begin with the first token of the IS200/IS605 sequence and, if a sequence was shorter than the context length, we padded the sequence to the remaining context (where padding did not contribute to the loss computation), similar to the strategy described for CRISPR-Cas9 systems above.”

In the paper

“If a model were simply using positional information to make essentiality predictions, the performance would be similar to this control.”