CoChat
Sign Up
Cited by 139Genomics and Phylogenetic StudiesBioinformatics and Genomic NetworksChromosomal and Genetic Variations

Orthology prediction methods: A quality assessment using curated protein families

Open access
Trachana, Kalliopi, Larsson, Tomas A., Powell, Sean, Chen, Wei‐Hua, Doerks, Tobias, Muller, Jean, Bork, Peer·2011·BioEssays
DOI 10.1002/bies.201100062PMC PMC3193375
Open this paper in CoChat — free accountOpen PDF

Abstract

The increasing number of sequenced genomes has prompted the development of several automated orthology prediction methods. Tests to evaluate the accuracy of predictions and to explore biases caused by biological and technical factors are therefore required. We used 70 manually curated families to analyze the performance of five public methods in Metazoa. We analyzed the strengths and weaknesses of the methods and quantified the impact of biological and technical challenges. From the latter part of the analysis, genome annotation emerged as the largest single influencer, affecting up to 30% of the performance. Generally, most methods did well in assigning orthologous group but they failed to assign the exact number of genes for half of the groups. The publicly available benchmark set (http://eggnog.embl.de/orthobench/) should facilitate the improvement of current orthology assignment protocols, which is of utmost importance for many fields of biology and should be tackled by a broad scientific community.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/bies.201100062

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Gene annotation quality has a large influence on the accuracy of orthology inference.

    “First, gene annotation quality has a large influence on the accuracy of orthology inference (15).”

    From Integrating gene annotation with orthology inference at scale (2023) · cites this paper as [15]

  • The OrthoBench dataset consists of expert-curated reference orthogroups.

    “we compared this new version of OrthoFinder with a set of alternative methods on the OrthoBench dataset of expert-curated reference orthogroups20,21.”

    From OrthoFinder: improved phylogenetic orthology inference with enhanced accuracy and scalability (2026) · cites this paper as [20]

  • Orthogroup inference accuracy of OrthoFinder has already been tested using the independent Orthobench dataset, and it was the most accurate method tested in terms of overall F-score.

    “Orthogroup inference accuracy of OrthoFinder has already been tested using the independent Orthobench dataset [11]. This showed to be the most accurate method tested in terms of overall F-score (although other methods scored higher in terms of either precision or recall while scoring proportionally worse in the other) [10].”

    From OrthoFinder: phylogenetic orthology inference for comparative genomics (2019) · cites this paper as [11]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 53

  1. [1]Koonin EV Galperin MY Sequence - Evolution - Function. Computational Approaches in Comparative Genomics 2003 Boston Kluwer Academic 21089240
  2. [2]Fitch WM Distinguishing homologous from analogous proteins Syst Zool 1970 19 99 113 5449325
  3. [3]Sonnhammer EL Koonin EV Orthology, paralogy and proposed classification for paralog subtypes Trends Genet 2002 18 619 20 12446146
  4. [4]Tatusov RL Koonin EV Lipman DJ A genomic perspective on protein families Science 1997 278 631 7 9381173
  5. [6]Muller J Szklarczyk D Julien P Letunic I eggNOG v2.0. extending the evolutionary genealogy of genes with enhanced non-supervised orthologous groups, species and functional annotations Nucleic Acids Res 2010 38 D190 195 19900971 PMC2808932
  6. [7]Waterhouse RM Zdobnov EM Tegenfeldt F Li J OrthoDB, the hierarchical catalog of eukaryotic orthologs in 2011 Nucleic Acids Res 2011 39 D283 288 20972218 PMC3013786
  7. [8]Chen F Mackey AJ Stoeckert CJ Jr. Roos DS OrthoMCL-DB. Querying a comprehensive multi-species collection of ortholog groups Nucleic Acids Res 2006 34 D363 368 16381887 PMC1347485
  8. [9]Altenhoff AM Schneider A Gonnet GH Dessimoz C OMA 2011. Orthology inference among 1000 complete genomes Nucleic Acids Res 2011 39 D289 294 21113020 PMC3013747
  9. [10]Vilella AJ Severin J Ureta-Vidal A Heng L EnsemblCompara GeneTrees. Complete, duplication-aware phylogenetic trees in vertebrates Genome Res 2009 19 327 35 19029536 PMC2652215
  10. [11]Huerta-Cepas J Capella-Gutierrez S Pryszcz LP Denisov I PhylomeDB v3.0. An expanding repository of genome-wide collections of trees, alignments and phylogeny-based orthology and paralogy predictions Nucleic Acids Res 2011 39 D556 560 21075798 PMC3013701
  11. [12]Kristensen DM Wolf YI Mushegian AR Koonin EV Computational methods for Gene Orthology inference Brief Bioinform 2011 in press, DOI: 10.1093/bib/bbr030 PMC3178053 21690100 · DOI
  12. [13]Koonin EV Orthologs, paralogs, and evolutionary genomics Annu Rev Genet 2005 39 309 38 16285863
  13. [14]Lang T Alexandersson M Hansson GC Samuelsson T Gel-forming mucins appeared early in metazoan evolution Proc Natl Acad Sci USA 2007 104 16209 14 17911254 PMC2042186
  14. [15]Lang T Alexandersson M Hansson GC Samuelsson T Bioinformatic identification of polymerizing and transmembrane mucins in the puffer fish Fugu rubripes Glycobiology 2004 14 521 7 15044386
  15. [16]Gabaldon T Large-scale assignment of orthology. Back to phylogenetics? Genome Biol 2008 9 235 18983710 PMC2760865
  16. [17]Kuzniar A van Ham RC Pongor S Leunissen JA The quest for orthologs: finding the corresponding gene across genomes Trends Genet 2008 24 539 51 18819722
  17. [18]Pryszcz LP Huerta-Cepas J Gabaldon T MetaPhOrs. Orthology and paralogy predictions from multiple phylogenetic evidence using a consistency-based confidence score Nucleic Acids Res 2010 39 e32 21149260 PMC3061081
  18. [19]Hulsen T Huynen MA de Vlieg J Groenen PM Benchmarking ortholog identification methods using functional genomics data Genome Biol 2006 7 R31 16613613 PMC1557999
  19. [20]Chen F Mackey AJ Vermunt JK Roos DS Assessing performance of orthology detection strategies applied to eukaryotic genomes PLoS One 2007 2 e383 17440619 PMC1849888
  20. [21]Altenhoff AM Dessimoz C Phylogenetic and functional assessment of orthologs inference projects and methods PLoS Comput Biol 2009 5 e1000262 19148271 PMC2612752
  21. [23]Byrne KP Wolfe KH The yeast gene order browser: combining curated homology and syntenic context reveals gene fate in polyploid species Genome Res 2005 15 1456 61 16169922 PMC1240090
  22. [24]Wapinski I Pfeffer A Friedman N Regev A Automatic genome-wide reconstruction of phylogenetic gene trees Bioinformatics 2007 23 i549 58 17646342
  23. [25]Akerborg O Sennblad B Arvestad L Lagergren J Simultaneous Bayesian gene tree reconstruction and reconciliation analysis Proc Natl Acad Sci USA 2009 106 5714 9 19299507 PMC2667006
  24. [26]Salichos L Rokas A Evaluating ortholog prediction algorithms in a yeast model clade PLoS One 2011 6 e18755 21533202 PMC3076445
  25. [27]Jensen LJ Julien P Kuhn M von Mering C eggNOG. automated construction and annotation of orthologous groups of genes Nucleic Acids Res 2008 36 D250 254 17942413 PMC2238944
  26. [28]Kriventseva EV Rahman N Espinosa O Zdobnov EM OrthoDB: the hierarchical catalog of eukaryotic orthologs Nucleic Acids Res 2008 36 D271 5 17947323 PMC2238902
  27. [30]Rokas A The origins of multicellularity and the early history of the genetic toolkit for animal development Annu Rev Genet 2008 42 235 51 18983257
  28. [31]Schneider A Dessimoz C Gonnet GH OMA Browser-exploring orthologous relations across 352 complete genomes Bioinformatics 2007 23 2180 2 17545180
  29. [32]Funkhouser JD Aronson NN Jr. Chitinase family GH18. Evolutionary insights from the genomic history of a diverse protein family BMC Evol Biol 2007 7 96 17594485 PMC1945033
  30. [33]Muller J Oma Y Vallar L Friederich E Sequence and comparative genomic analysis of actin-related proteins Mol Biol Cell 2005 16 5736 48 16195354 PMC1289417
  31. [34]Thompson JD Plewniak F Ripp R Thierry JC Towards a reliable objective function for multiple sequence alignments J Mol Biol 2001 314 937 51 11734009
  32. [35]Doolittle RF The origins and evolution of eukaryotic proteins Philos Trans R Soc Lond B Biol Sci 1995 349 235 40 8577833
  33. [36]Tordai H Nagy A Farkas K Banyai L Modules, multidomain proteins and organismic complexity FEBS J 2005 272 5064 78 16176277
  34. [37]Copley RR Letunic I Bork P Genome and protein evolution in eukaryotes Curr Opin Chem Biol 2002 6 39 45 11827821
  35. [38]Ciccarelli FD von Mering C Suyama M Harrington ED Complex genomic rearrangements lead to novel primate gene function Genome Res 2005 15 343 51 15710750 PMC551560
  36. [39]Forslund K Henricson A Hollich V Sonnhammer EL Domain tree-based analysis of protein architecture evolution Mol Biol Evol 2008 25 254 64 18025066
  37. [40]Singh AH Doerks T Letunic I Raes J Discovering functional novelty in metagenomes: examples from light-mediated processes J Bacteriol 2009 191 32 41 18849420 PMC2612456
  38. [41]Letunic I Doerks T Bork P 2009 D229 232 SMART 6. Recent updates and new developments. Nucleic Acids Res 37 PMC2686533 18978020
  39. [42]Milinkovitch MC Helaers R Depiereux E Tzika AC 2x genomes–depth does matter Genome Biol 2010 11 R16 20144222 PMC2872876
  40. [43]Flicek P Amode MR Barrell D Beal K Ensembl 2011 Nucleic Acids Res 2011 39 D800 806 21045057 PMC3013672
  41. [44]Thompson JD Thierry JC Poch O RASCAL. Rapid scanning and correction of multiple sequence alignments Bioinformatics 2003 19 1155 61 12801878
  42. [45]Muller J Creevey CJ Thompson JD Arendt D AQUA. Automated quality improvement for multiple sequence alignments Bioinformatics 2009 26 263 5 19926669
  43. [46]Huynen MA Bork P Measuring genome evolution Proc Natl Acad Sci USA 1998 95 5849 56 9600883 PMC34486
  44. [47]Ostlund G Schmitt T Forslund K Kostler T InParanoid 7: new algorithms and tools for eukaryotic orthology analysis Nucleic Acids Res 2010 38 D196 203 19892828 PMC2808972
  45. [48]Deluca TF Wu IH Pu J Monaghan T Roundup: a multi-genome repository of orthologs and evolutionary distances Bioinformatics 2006 22 2044 6 16777906
  46. [49]van der Heijden RT Snel B van Noort V Huynen MA Orthology prediction at scalable resolution by phylogenetic tree analysis BMC Bioinformatics 2007 8 83 17346331 PMC1838432
  47. [51]Chapman JA Kirkness EF Simakov O Hampson SE The dynamic genome of Hydra Nature 2010 464 592 6 20228792 PMC4479502
  48. [52]Srivastava M Begovic E Chapman J Putnam NH The Trichoplax genome and the nature of placozoans Nature 2008 454 955 60 18719581
  49. [53]Altschul SF Gish W Miller W Myers EW Basic local alignment search tool J Mol Biol 1990 215 403 10 2231712
  50. [54]Edgar RC MUSCLE: multiple sequence alignment with high accuracy and high throughput Nucleic Acids Res 2004 32 1792 7 15034147 PMC390337
  51. [55]Larkin MA Blackshields G Brown NP Chenna R Clustal W and Clustal X version 2.0 Bioinformatics 2007 23 2947 8 17846036
  52. [56]Castresana J Selection of conserved blocks from multiple alignments for their use in phylogenetic analysis Mol Biol Evol 2000 17 540 52 10742046
  53. [57]Eddy SR A new generation of homology search tools based on probabilistic inference Genome Inform 2009 23 205 11 20180275

Cited by

Sorted by citations

Authors
Trachana, K., Larsson, T. A., Powell, S., Chen, W., Doerks, T., Muller, J., & Bork, P.
Affiliations
  • European Molecular Biology Laboratory (DE)
  • Centre National de la Recherche Scientifique (FR)
ORCID
Venue
BioEssays
Publisher
Wiley
Published
2011-08-19
Volume
33
Issue
10
Pages
769-780
ISSN
  • 0265-9247 (ISSN-L)
  • 1521-1878
DOI
10.1002/bies.201100062
PMC
PMC3193375
OpenAlex
W2030317329
Cited by
139
References
5
Citations by year
2026: 5 · 2025: 4 · 2024: 4 · 2023: 5 · 2022: 3 · 2021: 11 · 2020: 10 · 2019: 19 · 2018: 9 · 2017: 13 · 2016: 8 · 2015: 11 · 2014: 13 · 2013: 12 · 2012: 8
FWCI
1.08
Type
journal article
Language
en
Open access
Licence
onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations
  • BioEssays (publishedVersion)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • MDC Repository (Max-Delbrueck-Center for Molecular Medicine) (acceptedVersion)
  • HAL (submittedVersion, other-oa)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
  • Life Sciences › Agricultural and Biological Sciences › Plant Science › Chromosomal and Genetic Variations
Keywords
orthology prediction, protein families, orthogroups, benchmarking, animals, automatic annotation
MeSH terms
Algorithms, Animals, Genes, Mucins, Phylogeny, Proteins, Species Specificity, User-Computer Interface, Reproducibility of Results, Computational Biology, Internet, Databases, Genetic, Databases, Protein, Molecular Sequence Annotation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review