Cited by 678Genomics and Phylogenetic StudiesChromosomal and Genetic VariationsRNA and protein synthesis mechanisms

The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index; 1 of the 15 sources its claims cite were checked for support.

BRAKER3: Fully automated genome annotation using RNA-seq and protein evidence with GeneMark-ETP, AUGUSTUS, and TSEBRA

Open access

Published version of a preprint

Abstract

Gene prediction has remained an active area of bioinformatics research for a long time. Still, gene prediction in large eukaryotic genomes presents a challenge that must be addressed by new algorithms. The amount and significance of the evidence available from transcriptomes and proteomes vary across genomes, between genes, and even along a single gene. User-friendly and accurate annotation pipelines that can cope with such data heterogeneity are needed. The previously developed annotation pipelines BRAKER1 and BRAKER2 use RNA-seq or protein data, respectively, but not both. A further significant performance improvement integrating all three data types was made by the recently released GeneMark-ETP. We here present the BRAKER3 pipeline that builds on GeneMark-ETP and AUGUSTUS, and further improves accuracy using the TSEBRA combiner. BRAKER3 annotates protein-coding genes in eukaryotic genomes using both short-read RNA-seq and a large protein database, along with statistical models learned iteratively and specifically for the target genome. We benchmarked the new pipeline on genomes of 11 species under an assumed level of relatedness of the target species proteome to available proteomes. BRAKER3 outperforms BRAKER1 and BRAKER2. The average transcript-level F1-score is increased by about 20 percentage points on average, whereas the difference is most pronounced for species with large and complex genomes. BRAKER3 also outperforms other existing tools, MAKER2, Funannotate, and FINDER. The code of BRAKER3 is available on GitHub and as a ready-to-run Docker container for execution with Docker or Singularity. Overall, BRAKER3 is an accurate, easy-to-use tool for eukaryotic genome annotation.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

BRAKER3 is a fully automated eukaryotic genome annotation pipeline that integrates RNA-seq and protein evidence using GeneMark-ETP, AUGUSTUS, and TSEBRA. It outperforms BRAKER1, BRAKER2, and other existing tools in benchmarks, with the largest gains on large and complex genomes.

Key findings

  • On 11 species with order-excluded protein databases, BRAKER3 had the highest average accuracy; the average transcript-level F1-score increased by about 20 percentage points relative to BRAKER1/BRAKER2, with the highest increase in *G. gallus* (gene/transcript F1 improved by 55/48 points compared with TSEBRA combining BRAKER1 and BRAKER2).
  • Compared with MAKER2 and Funannotate on eight species using close-relatives-included protein databases, BRAKER3’s F1-scores were higher than Funannotate by 10.2 points at exon level, 25.9 at gene level, and 21.6 at transcript level; Funannotate exceeded MAKER2 by 2.2, 3.8, and 4.4 points at exon, gene, and transcript levels, respectively. FINDER completed only 7 of 11 species and its best result (*D. melanogaster*, approximately 15 gene F1 and 11 transcript F1) was much below other methods.
  • Transcript-level sensitivity increased with expression: 23% of low-expression transcripts, 55% of medium-expression transcripts, and 76% of highly expressed transcripts were correctly identified. BRAKER3 predicted acceptor/donor splice sites with F1 >87% averaged over species, and start/stop codons with F1 of 70%/76%.
  • BUSCO completeness averaged 96% for well-annotated compact genomes, 83.6% for well-annotated large genomes, 91% for other genomes with reference annotation, and 93.7% for three recently sequenced genomes. Runtime ranged from 5 h 37 min for *A. thaliana* to 64 h 16 min for *M. musculus*.

Methods

  • BRAKER3 requires a genome sequence, short-read RNA-seq data (BAM, FASTQ, or SRA library IDs), and a protein database. It runs GeneMark-ETP for iterative training and prediction with RNA-seq/protein hints, trains AUGUSTUS on high-confidence genes, and uses TSEBRA to combine GeneMark-ETP and AUGUSTUS predictions while ensuring inclusion of high-confidence genes.
  • Benchmarks used 11 species plus three recently sequenced genomes without established reference annotations. Protein databases were prepared as species excluded, order excluded, and close relatives included; comparisons included BRAKER1, BRAKER2, TSEBRA, GeneMark-ETP, MAKER2, FINDER, and Funannotate.
  • Accuracy was assessed against reference annotations at exon, gene, and transcript levels using sensitivity, precision, and F1-score; BUSCO was used for genomes lacking reference annotations. Runtime was measured on an HPC node with 48 threads.

Limitations

  • The paper states that BRAKER3 cannot be used without RNA-seq evidence, performs worse for weakly expressed transcripts, and currently predicts only protein-coding genes. It also notes benchmark sampling bias toward well-annotated, early-sequenced species, limited RNA-seq input to at most six libraries, eliminated same-order homology evidence, and cautions that reference annotations may fall short and BUSCO does not fully evaluate gene-structure errors or false-positive genes.
  • The paper states that the close-relatives-included protein database was manually selected and would not scale well to very large numbers of annotated species.
  • Evident from the text: FINDER completed only 7 of 11 species; Funannotate or MAKER2 failed on some large genomes; MAKER2 was trained on the reference annotation, so its reported accuracy is an upper limit; and BRAKER3 had the longest runtime among the compared methods.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1101/gr.278090.123

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

1 of 15 cited sources checked · 0 supported · 0 flagged · accuracy not yet measured

  • Findingnone given

    The difference in performance is most pronounced for species with large and complex genomes.

    “The average transcript-level F1-score is increased by about 20 percentage points on average, whereas the difference is most pronounced for species with large and complex genomes.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The reference annotations used as ground truth are sufficiently accurate for benchmarking.
    • The authors state this as an assumption: The target species' proteome has an assumed level of relatedness to available proteomes.
    • Rests on an assumption: The five or six randomly selected short-read RNA-seq libraries per species are sufficient and representative for annotation.
    • Rests on an assumption: The OrthoDB-based protein databases from the broad clade contain useful homologous informants for the target species.
  • Findingown data

    The average transcript-level F1-score is increased by about 20 percentage points on average for BRAKER3 compared with BRAKER1 and BRAKER2.

    “The average transcript-level F1-score is increased by about 20 percentage points on average, whereas the difference is most pronounced for species with large and complex genomes.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The reference annotations used as ground truth are sufficiently accurate for benchmarking.
    • The authors state this as an assumption: The target species' proteome has an assumed level of relatedness to available proteomes.
    • Rests on an assumption: The five or six randomly selected short-read RNA-seq libraries per species are sufficient and representative for annotation.
    • Rests on an assumption: The OrthoDB-based protein databases from the broad clade contain useful homologous informants for the target species.
  • Findingown data

    BRAKER3 outperforms BRAKER1 and BRAKER2.

    “BRAKER3 outperforms BRAKER1 and BRAKER2.”
    • Also stated in: § BRAKER3 performance assessment · boosted
      “Notably, there was a significant improvement of BRAKER3 in comparison with BRAKER1 and BRAKER2 in species with GC-heterogeneous or large genomes (Fig. 3).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The reference annotations used as ground truth are sufficiently accurate for benchmarking.
    • The authors state this as an assumption: The target species' proteome has an assumed level of relatedness to available proteomes.
    • Rests on an assumption: The five or six randomly selected short-read RNA-seq libraries per species are sufficient and representative for annotation.
    • Rests on an assumption: The OrthoDB-based protein databases from the broad clade contain useful homologous informants for the target species.
  • Findingnone given

    BRAKER3 outperforms other existing tools, MAKER2, Funannotate, and FINDER.

    “BRAKER3 also outperforms other existing tools, MAKER2, Funannotate, and FINDER.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The reference annotations used as ground truth are sufficiently accurate for benchmarking.
    • The authors state this as an assumption: The target species' proteome has an assumed level of relatedness to available proteomes.
    • Rests on an assumption: The five or six randomly selected short-read RNA-seq libraries per species are sufficient and representative for annotation.
    • Rests on an assumption: The OrthoDB-based protein databases from the broad clade contain useful homologous informants for the target species.
  • Findingown data

    When given a large protein database as input (2,588,444 proteins), MAKER2 took 16 h and BRAKER3 took 2.5 h.

    “When given a large protein database as input (2,588,444 proteins), MAKER2 took 16 h and BRAKER3 took 2.5 h.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. 15
  • It may state two separate claims as one. 31
  • The quote only partly states this claim. 1
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    FINDER uses RNA-seq assemblies with predicted open reading frames, in conjunction with BRAKER1 and homologous protein predictions.

    “FINDER follows an approach similar to BRAKER3. It uses RNA-seq assemblies with predicted open reading frames, in conjunction with BRAKER1 and homologous protein predictions.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Banerjee et al. 2021 source

      The Implementation section says FINDER assembles RNA-Seq reads into transcripts, predicts their coding regions with GeneMarkS-T, and incorporates protein-derived gene models. It identifies the accompanying gene predictor as BRAKER2, not BRAKER1. (LLM, conf=0.92)

      Experimental — accuracy not yet measured
  • It may state two separate claims as one.

    Findingown data

    In each species, BRAKER3 was more precise in predicting exons compared with GeneMark-ETP, which in turn predicted exons more sensitively compared with BRAKER3.

    “In each species, BRAKER3 was more precise in predicting exons compared with GeneMark-ETP, which in turn predicted exons more sensitively compared with BRAKER3 (Supplemental Table S4).”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    BRAKER3 shows better performance than Funannotate and MAKER2 for all species and individual metrics, except at the exon level for C. elegans, for which BRAKER3 had a sensitivity 3.3 percentage points lower than Funannotate.

    “BRAKER3 shows better performance than Funannotate and MAKER2 for all species and individual metrics, except at the exon level for C. elegans, for which BRAKER3 had a sensitivity 3.3 percentage points lower than the one of Funannotate (Supplemental Table S8).”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Findingown data

    On average, the BRAKER3 transcript-level sensitivity was improved by approximately three percentage points, and the precision was improved slightly (less than one percentage point) when using the species excluded protein database.

    “On average, the BRAKER3 transcript-level sensitivity was improved by approximately three percentage points, and the precision was improved slightly (less than one percentage point).”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    The relative ranking of the methods and the comparison of BRAKER3 with other methods remain unchanged when using the species excluded protein database.

    “However, the relative ranking of the methods and the comparison of BRAKER3 with other methods remain unchanged.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    In the group of well-annotated large genomes, AUGUSTUS is 6.6 percentage points more BUSCO-complete than BRAKER3, but BRAKER3 is 30.5 percentage points more precise than AUGUSTUS.

    “For example, in the group of well-annotated large genomes, AUGUSTUS is 6.6 percentage points more BUSCO-complete than BRAKER3, but BRAKER3 is 30.5 percentage points more precise than AUGUSTUS.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Findingown data

    As a trend, a relatively large advantage of BRAKER3 in gene precision stands against a relatively small advantage of AUGUSTUS in BUSCO completeness.

    “As a trend, a relatively large advantage of BRAKER3 in gene precision stands against a relatively small advantage of AUGUSTUS in BUSCO completeness.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Findingown data

    The pattern that BRAKER3-unique transcripts uncover more reference transcripts than GeneMark-ETP-unique transcripts is consistent across all 11 species and applies to both spliced and unspliced transcripts.

    “This pattern is consistent across all 11 species and applies to both spliced and unspliced transcripts.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Findingown data

    TSEBRA adds single-exon genes predicted by AUGUSTUS, which increases the percentage of single-exon genes correctly identified by 5.5 percentage points on average.

    “However, it adds single-exon genes predicted by AUGUSTUS, which increases the percentage of single-exon genes correctly identified by 5.5 percentage points on average.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Findingown data

    For each of the groups of species, AUGUSTUS had a higher BUSCO completeness score compared with GeneMark-ETP, which in turn had a higher score compared with BRAKER3.

    “For each of the groups of species, AUGUSTUS had a higher BUSCO completeness score compared with GeneMark-ETP, which in turn had a higher score compared with BRAKER3.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    The average BUSCO completeness (single copy or duplicate) of BRAKER3 is 96% for the group of well-annotated and compact genomes, 83.6% for the group of well-annotated and large genomes, 91% for the group of other genomes with a reference annotation, and 93.7% for the group of three recent genomes.

    “The average BUSCO completeness (single copy or duplicate) of BRAKER3 is 96% for the group of well-annotated and compact genomes, 83.6% for the group of well-annotated and large genomes, 91% for the group of other genomes with a reference annotation (M. truncatula, P. tepidariorum, S. lycopersicum), and 93.7% for the group of three recent genomes (Supplemental Fig. S9).”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    On average, BRAKER3's F1-scores were higher than Funannotate's by 10.2 points at the exon level, 25.9 points at the gene level, and 21.6 points at the transcript level.

    “On average, BRAKER3's F1-scores were higher than the ones of Funannotate by 10.2 points at the exon level, 25.9 points at the gene level, and 21.6 points at the transcript level.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Findingown data

    BRAKER3 had the highest sensitivities and precisions for each species at the transcript and gene level but often had a somewhat lower exon-level F1-score compared with GeneMark-ETP.

    “BRAKER3 had the highest sensitivities and precisions for each species at the transcript and gene level but often had a somewhat lower exon-level F1-score compared with GeneMark-ETP (Supplemental Table S5).”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Findingown data

    There is a tendency that a tool that predicts more genes genome-wide also reports more of the universal single-copy genes that are counted by BUSCO.

    “However, this is also the order of tools sorted by decreasing numbers of predicted genes; that is, there is a (perhaps unsurprising) tendency that a tool that predicts more genes genome-wide also reports more of the universal single-copy genes that are counted by BUSCO.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    When averaging over all species, only 23% of low-expression transcripts are correctly identified, 55% of medium-expression transcripts, and 76% of highly expressed transcripts.

    “When averaging over all species, only 23% of low-expression transcripts are correctly identified, 55% of medium-expression transcripts, and 76% of highly expressed transcripts.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    The averaged F1-score for stop and start codons are 76% and 70%, respectively.

    “The averaged F1-score for stop and start codons are 76% and 70%, respectively (Supplemental Fig. S5).”
    Quote in the paperNot a substantial claim
  • The quote only partly states this claim.

    Findingown data

    In computational experiments with genomes of 11 species, BRAKER3 consistently outperformed the other gene finding tools.

    “We have shown that BRAKER3 consistently outperformed the other gene finding tools.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The reference annotations used as ground truth are sufficiently accurate for benchmarking.
    • The authors state this as an assumption: The target species' proteome has an assumed level of relatedness to available proteomes.
    • Rests on an assumption: The five or six randomly selected short-read RNA-seq libraries per species are sufficient and representative for annotation.
    • Rests on an assumption: The OrthoDB-based protein databases from the broad clade contain useful homologous informants for the target species.
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.

    Findingnone given

    BRAKER3's performance drops for transcripts that are weakly represented in the RNA-seq data.

    “BRAKER3's performance drops for transcripts that are weakly represented in the RNA-seq data.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Findingown data

    AUGUSTUS' average gene- and transcript-level F1-scores of 59.6 and 51.3, respectively, exceeded the F1-scores of the AUGUSTUS predictions in BRAKER1 and BRAKER2 and are slightly lower than the F1-scores of GeneMark-ETP.

    “AUGUSTUS’ average gene- and transcript-level F1-scores of 59.6 and 51.3, respectively, exceeded the F1-scores of the AUGUSTUS predictions in BRAKER1 and BRAKER2 and are slightly lower than the F1-scores of GeneMark-ETP (see Supplemental Table S10).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The reference annotations used as ground truth are sufficiently accurate for benchmarking.
    • The authors state this as an assumption: The target species' proteome has an assumed level of relatedness to available proteomes.
    • Rests on an assumption: The five or six randomly selected short-read RNA-seq libraries per species are sufficient and representative for annotation.
    • Rests on an assumption: The OrthoDB-based protein databases from the broad clade contain useful homologous informants for the target species.
  • It may state two separate claims as one.

    Premisecitation

    GeneMark-ETP integrates both sources of extrinsic evidence in a new workflow that outperforms all previously mentioned methods, particularly in species with large and complex genomes.

    “A more integrated approach is the GeneMark-ETP pipeline (Brůna et al. 2024), which integrates both sources of extrinsic evidence in a new workflow that outperforms all previously mentioned methods, particularly in species with large and complex genomes.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Brůna et al. 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • It may state two separate claims as one.

    Premisereasoning

    TSEBRA selects transcripts from BRAKER1 and BRAKER2 annotations, considering the joint extrinsic evidence, and generates a prediction based on both RNA-seq and protein evidence, thus improving F1-scores.

    “Therefore, we developed a combiner tool, TSEBRA (Gabriel et al. 2021). It selects transcripts from BRAKER1 and BRAKER2 annotations, considering thereby the joint extrinsic evidence, and therefore generates a prediction based on both RNA-seq and protein evidence, thus improving the F1-scores.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Gabriel et al. 2021 source

      Checked before — its earlier verdict is held back until a new check runs.

  • It may state two separate claims as one.

    Interpretationreasoning

    BRAKER3 annotates protein-coding genes in eukaryotic genomes using both short-read RNA-seq and a large protein database, along with statistical models learned iteratively and specifically for the target genome.

    “BRAKER3 annotates protein-coding genes in eukaryotic genomes using both short-read RNA-seq and a large protein database, along with statistical models learned iteratively and specifically for the target genome.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Interpretationown data

    BRAKER3 outperforms by a large margin its predecessors BRAKER1 and BRAKER2, as well as publicly available pipelines such as MAKER2, FINDER, and Funannotate.

    “BRAKER3 outperforms by a large margin its predecessors BRAKER1 and BRAKER2, as well as publicly available pipelines such as MAKER2, FINDER, and Funannotate.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Previous research indicates that gene finders generally show a decreased performance on unspliced transcripts.

    “These findings agree with previous research indicating that gene finders generally show a decreased performance on unspliced transcripts (Scalzitti et al. 2020).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Scalzitti et al. 2020 source

      Checked before — its earlier verdict is held back until a new check runs.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisenone given

    Funannotate was initially designed for fungal genomes and has since been further developed to support annotation of larger genomes.

    “The Funannotate pipeline, which was not described in an article, was initially designed as a pipeline for analyzing fungal genomes; however, it has since been further developed to support the annotation of larger genomes as well.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites https://github.com/nextgenusfs/funannotate

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    MAKER2 was designed to be used with a smaller protein database.

    “Funannotate failed to run on most of the large OrthoDB-based protein databases, and MAKER2 was designed to be used with a smaller protein database, too.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Holt and Yandell 2011 source

      Checked before — its earlier verdict is held back until a new check runs.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Premisecitation

    In Stanke et al. (2019), a manual selection of libraries from different experiments and tissues was compared with an automatic sampling from diverse libraries for complementary data; the latter yields, on average, slightly better performances in BRAKER1 when the number of reads was similar.

    “In their study, a manual selection of libraries from different experiments and tissues was compared with an automatic sampling from diverse libraries for complementary data. The latter yields, on average, slightly better performances in BRAKER1 when the number of reads was similar.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Stanke et al. (2019) source

      Checked before — its earlier verdict is held back until a new check runs.

  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Interpretationnone given

    Currently, BRAKER3 predicts only protein-coding genes.

    “Currently, BRAKER3 predicts only protein-coding genes.”
    Quote in the paperNot a substantial claim
  • It may state two separate claims as one.

    Premisereasoning

    TSEBRA heavily relies on intron position information, which can make it challenging to evaluate single-exon transcripts.

    “TSEBRA heavily relies on intron position information, which can make it challenging to evaluate single-exon transcripts.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Interpretationreasoning

    BRAKER3 greatly benefited from the high accuracy of GeneMark-ETP and managed to exceed the sensitivity and precision on gene and transcript levels even further.

    “Here, BRAKER3 greatly benefited from the high accuracy of GeneMark-ETP and managed to exceed the sensitivity and precision on gene and transcript levels even further.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The reference annotations used as ground truth are sufficiently accurate for benchmarking.
    • The authors state this as an assumption: The target species' proteome has an assumed level of relatedness to available proteomes.
    • Rests on an assumption: The five or six randomly selected short-read RNA-seq libraries per species are sufficient and representative for annotation.
    • Rests on an assumption: The OrthoDB-based protein databases from the broad clade contain useful homologous informants for the target species.
  • It may state two separate claims as one.

    Interpretationreasoning

    For these gene sets, a better BUSCO score comes at the expense of more false-positive genes, and BRAKER3 weights the trade-off between fewer missing genes and fewer false-positive genes in favor of the latter.

    “In conclusion, for these gene sets, a better BUSCO score comes at the expense of more false-positive genes, and BRAKER3 weights the trade-off between fewer missing genes and fewer false-positive genes in favor of the latter.”
    Quote in the paperSubstantial claim
  • It may state two separate claims as one.

    Premisecitation

    GeneMark-ETP benefited from GC-content-specific model training and from estimating species-specific repeat penalties.

    “The method also benefited from the GC-content-specific model training and from estimating species-specific repeat penalties.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Brůna et al. 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • It may state two separate claims as one.

    Interpretationnone given

    BUSCO's design does not encompass the detection of various gene structure errors, nor does it impose penalties for the presence of false-positive genes.

    “Specifically, BUSCO's design does not encompass the detection of various gene structure errors, nor does it impose penalties for the presence of false-positive genes.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Study design and conductnone given

    With the diverse species selection, it was not feasible to use RNA-seq sets that are comparable across different species.

    “With our diverse species selection, it was not feasible to use RNA-seq sets that are comparable across different species.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisereasoning

    The self-training of GeneMark.hmm models has to be performed outside of MAKER2.

    “Also, the self-training of GeneMark.hmm models (Lomsadze et al. 2005) has to be performed outside of MAKER2.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lomsadze et al. 2005

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim. It may state two separate claims as one.

    Study design and conductnone given

    The long-established approach of comparing predicted annotations with reference annotations allows measurement of grave as well as subtle errors and is informative in absolute terms if the reference annotations are of high quality.

    “This long-established approach to benchmark gene prediction methods allows measurement of grave as well as subtle errors and is informative in absolute terms if the reference annotations are of high quality.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The reference annotations used as ground truth are sufficiently accurate for benchmarking.
  • It may state two separate claims as one.

    Interpretationreasoning

    BRAKER3 combines sets of transcripts and can be seen as an ensemble learning approach that improves each of its inputs.

    “BRAKER3 combines these sets of transcripts and can be seen as an ensemble learning approach that improves each of its inputs.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Interpretationnone given

    The species for the close relatives included database were manually selected, and the procedure would not scale well when very large numbers of species are annotated.

    “It should be noted that the species for the close relatives included database were manually selected, and the procedure would not scale well when very large numbers of species are annotated.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.

    Interpretationreasoning

    Some reference annotations may fall short of the desired accuracy, potentially leading to an underestimation of accuracy or a bias in the analysis.

    “It should be noted that some of the reference annotations may fall short of the desired accuracy, potentially leading to an underestimation of accuracy or a bias in our analysis.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The reference annotations used as ground truth are sufficiently accurate for benchmarking.
  • Probably not a substantial claim. The paper may give no reason why it holds.

    Interpretationnone given

    BRAKER3 cannot be used without RNA-seq evidence.

    “Notably, BRAKER3 cannot be used without RNA-seq evidence.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim. The paper’s conclusions may not depend on it.

    Premisecitation

    Stanke et al. (2019) investigated the impact of an RNA-seq sampling method on the performance of BRAKER1.

    “However, Stanke et al. (2019) investigated the impact of an RNA-seq sampling method on the performance of BRAKER1.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Stanke et al. (2019) source

      Checked before — its earlier verdict is held back until a new check runs.

  • It may state two separate claims as one.

    Study design and conductreasoning

    The figures for MAKER2 do not include the considerable times for training gene finders and/or transcriptome assembly, whereas BRAKER3 performs these steps as part of the pipeline.

    “When comparing these runtimes, one has to consider that the figures for MAKER2 do not include the considerable times for training gene finders and or for transcriptome assembly. In contrast, BRAKER3 performs these steps as part of the pipeline.”
    Quote in the paperSubstantial claim
42 of 42
  • The authors state this as an assumption.

    The reference annotations used as ground truth are sufficiently accurate for benchmarking.

    “This long-established approach to benchmark gene prediction methods allows measurement of grave as well as subtle errors and is informative in absolute terms if the reference annotations are of high quality.”
  • The authors state this as an assumption.

    The target species' proteome has an assumed level of relatedness to available proteomes.

    “We benchmarked the new pipeline on genomes of 11 species under an assumed level of relatedness of the target species proteome to available proteomes.”
  • The authors state this as an assumption.

    The order excluded protein database is a conservative assumption.

    “we performed experiments with the particularly favorable case of the species excluded database and with the rather conservative assumption of the order excluded database.”
  • The paper relies on this without support.

    The five or six randomly selected short-read RNA-seq libraries per species are sufficient and representative for annotation.

    “For each species, we retrieved genome assemblies, five or six randomly selected short-read RNA-seq libraries from NCBI's Sequence Read Archive (https://www.ncbi.nlm.nih.gov/sra) (for detailed list, see Supplemental Table S15), a protein database, and a reference genome annotation (for detailed list, see Supplemental Table S1).”
  • The paper relies on this without support.

    The OrthoDB-based protein databases from the broad clade contain useful homologous informants for the target species.

    “The protein database is a FASTA file with proteins from the broad clade of the target genome in question, for example, a subset from the partitioning of OrthoDB that we provide (see Supplemental Methods).”
  • The paper relies on this without support.

    Modern genome annotation pipelines depend on an increasing number of tools, which can make their installation and maintenance difficult.

    “One problem with modern genome annotation pipelines is their dependence on an increasing number of tools, which can make their installation and maintenance difficult.”
  • The paper relies on this without support.

    BUSCO uses a set of genes that are well conserved and, therefore, arguably relatively easy to predict.

    “Moreover, BUSCO uses a set of genes that are well conserved and, therefore, arguably relatively easy to predict.”
  • The paper relies on this without support.

    A comprehensive gold-standard annotation for a set of test genomes representing the whole space of eukaryotic genomes does not exist.

    “However, a comprehensive gold-standard annotation for such a set of genomes does not exist.”
  • The paper relies on this without support.

    The error rates in eukaryotic gene annotation remain significant.

    “As long as the error rates in eukaryotic gene annotation remain significant, a trade-off between the numbers of false-positive and false-negative genes has to be made.”
9 of 9

Citations

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Metadata

Authors
Gabriel, L., Brůna, T., Hoff, K. J., Ebel, M., Lomsadze, A., Borodovsky, M., & Stanke, M.
Affiliations
  • Universität Greifswald (DE)
  • Lawrence Berkeley National Laboratory (US)
  • Georgia Institute of Technology (US)
Corresponding author
  • Hoff, Katharina J.
  • Lomsadze, Alexandre
Venue
Genome Research
Published
2024-05
Volume
34
Issue
5
Pages
769-777
ISSN
  • 1088-9051 (ISSN-L)
  • 1549-5469
OpenAlex
W4399560644
Cited by
676
References
26
Citations by year
2026: 360 · 2025: 245 · 2024: 63
Type
posted content
Language
en
Open access
Licence
cc-by-nc-sa
Locations
  • Genome Research (publishedVersion)
  • PubMed Central (publishedVersion, cc-by-nc-sa)
  • PubMed (publishedVersion)
  • OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion, other-oa)
  • HighWire Press Open Archive (submittedVersion)
Funders
U.S. National Institutes of Health (GM128145)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Genetics › Chromosomal and Genetic Variations
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › RNA and protein synthesis mechanisms
Keywords
BRAKER3, genome annotation, gene prediction, RNA-seq, Augustus, eukaryotic genomes, transcriptome, proteome, annotation pipeline, Funannotate, Finder, Docker containers
MeSH terms
RNA-Seq, Algorithms, Animals, Humans, Software, Genome, Computational Biology, Genomics, Molecular Sequence Annotation, Transcriptome

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review