Biases in the Experimental Annotations of Protein Function and Their Effect on Our Understanding of Protein Function Space
Open accessThe DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Abstract
The ongoing functional annotation of proteins relies upon the work of curators to capture experimental findings from scientific literature and apply them to protein sequence and structure data. However, with the increasing use of high-throughput experimental assays, a small number of experimental studies dominate the functional protein annotations collected in databases. Here, we investigate just how prevalent is the "few articles - many proteins" phenomenon. We examine the experimentally validated annotation of proteins provided by several groups in the GO Consortium, and show that the distribution of proteins per published study is exponential, with 0.14% of articles providing the source of annotations for 25% of the proteins in the UniProt-GOA compilation. Since each of the dominant articles describes the use of an assay that can find only one function or a small group of functions, this leads to substantial biases in what we know about the function of many proteins. Mass-spectrometry, microscopy and RNAi experiments dominate high throughput experiments. Consequently, the functional information derived from these experiments is mostly of the subcellular location of proteins, and of the participation of proteins in embryonic developmental pathways. For some organisms, the information provided by different studies overlap by a large amount. We also show that the information provided by high throughput experiments is less specific than those provided by low throughput experiments. Given the experimental techniques available, certain biases in protein function annotation due to high-throughput experiments are unavoidable. Knowing that these biases exist and understanding their characteristics and extent is important for database curators, developers of function annotation programs, and anyone who uses protein function annotation data to plan experiments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Moreover, increasing numbers of functional annotations are based on phenotypes from high-throughput screens for genetic phenotypes or protein–protein interactions, both of which are prone to generating false positives [100].”
From Functional unknomics: Systematic screening of conserved genes of unknown function (2023) · cites this paper as [100]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gapped BLAST and PSI-BLAST: a new generation of protein database search programs
Altschul, S. (1997) · Nucleic Acids Research · doi:10.1093/nar/25.17.3389 · Cited by 73,302 · cited 5×
“Count of ECO terms in top-50 papers N ECO term ECO ID Articles 1 protein separation followed by fragment iden- tification evidence ECO:0000160 27 2 sequence similarity evidence ECO:0000044 27 3 imaging assay evidence ECO:0000324 24 4 cell fractionation evidence ECO:0000004 23 5 transmembrane domain prediction ev- iden…”
InfluentialGene Ontology: tool for the unification of biology
Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M., Davis, A. P., Dolinski, K., Dwight, S. S., Eppig, J. T., et al. (2000) · Nature Genetics · doi:10.1038/75556 · Cited by 43,206
Matplotlib: a 2D graphics environment
Hunter, J. D. (2007) · Computing in Science & Engineering · doi:10.1109/mcse.2007.55 · Cited by 39,249 · cited 1×
“The following tools were used in the analyses: Biopython [20], matplotlib [21].”
Biopython: freely available python tools for computational molecular biology and bioinformatics
Cock, P. J. A., Antao, T., Chang, J. T., Chapman, B. A., Cox, C. J., Dalke, A., Friedberg, I., Hamelryck, T., Kauff, F., Wilczynski, B., et al. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp163 · Cited by 6,080 · cited 3×
“The following tools were used in the analyses: Biopython [20], mat349 plotlib [21].”
InfluentialSelf-citationGlobal analysis of protein localization in budding yeast
Huh, W.-K., Falvo, J. V., Gerke, L. C., Carroll, A. S., Howson, R. W., Weissman, J. S., & O'Shea, E. K. (2003) · Nature · doi:10.1038/nature02026 · Cited by 4,248
Systematic functional analysis of the Caenorhabditis elegans genome using RNAi
Kamath, R. S., Fraser, A. G., Dong, Y., Poulin, G., Durbin, R., Gotta, M., Kanapin, A., Le Bot, N., Moreno, S., Sohrmann, M., et al. (2003) · Nature · doi:10.1038/nature01278 · Cited by 3,616
Towards a proteome-scale map of the human protein–protein interaction network
Rual, J.-F., Venkatesan, K., Hao, T., Hirozane-Kishikawa, T., Dricot, A., Li, N., Berriz, G. F., Gibbons, F. D., Dreze, M., Ayivi-Guedehoussou, N., et al. (2005) · Nature · doi:10.1038/nature04209 · Cited by 2,933
Proteome survey reveals modularity of the yeast cell machinery
Gavin, A.-C., Aloy, P., Grandi, P., Krause, R., Boesche, M., Marzioch, M., Rau, C., Jensen, L. J., Bastuck, S., Dümpelfeld, B., et al. (2006) · Nature · doi:10.1038/nature04532 · Cited by 2,575
Genes required for mycobacterial growth defined by high density mutagenesis
Sassetti, C. M., Boyd, D. H., & Rubin, E. J. (2003) · Molecular Microbiology · doi:10.1046/j.1365-2958.2003.03425.x · Cited by 2,544
A Mitochondrial Protein Compendium Elucidates Complex I Disease Biology
Pagliarini, D. J., Calvo, S. E., Chang, B., Sheth, S. A., Vafai, S. B., Ong, S.-E., Walford, G. A., Sugiana, C., Boneh, A., Chen, W. K., et al. (2008) · Cell · doi:10.1016/j.cell.2008.06.016 · Cited by 1,994
Cited by
Inferring the molecular and phenotypic impact of amino acid variants with MutPred2
Pejaver, V., Urresti, J., Lugo-Martinez, J., Pagel, K. A., Lin, G. N., Nam, H.-J., Mort, M., Cooper, D. N., Sebat, J., Iakoucheva, L. M., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-19669-x · Cited by 923
Zhou, N., Jiang, Y., Bergquist, T. R., Lee, A. J., Kacsoh, B. Z., Crocker, A. W., Lewis, K. A., Georghiou, G., Nguyen, H. N., Hamid, M. N., et al. (2019) · Genome Biology · doi:10.1186/s13059-019-1835-8 · Cited by 452
Self-citationCATH: comprehensive structural and functional annotations for genome sequences
Sillitoe, I., Lewis, T. E., Cuff, A., Das, S., Ashford, P., Dawson, N. L., Furnham, N., Laskowski, R. A., Lee, D., Lees, J. G., et al. (2015) · Nucleic Acids Research · doi:10.1093/nar/gku947 · Cited by 423
“Significant biases in GO annotations have recently been identified (6) suggesting that GO annotation data can only provide a partial picture of the function of a protein, which will affect functional classification.”
BackgroundMachine Learning in Enzyme Engineering
Mazurenko, S., Prokop, Z., & Damborsky, J. (2020) · ACS Catalysis · doi:10.1021/acscatal.9b04321 · Cited by 410
An expanded evaluation of protein function prediction methods shows an improvement in accuracy
Jiang, Y., Oron, T. R., Clark, W. T., Bankapur, A. R., D’Andrea, D., Lepore, R., Funk, C. S., Kahanda, I., Verspoor, K. M., Ben-Hur, A., et al. (2016) · Genome Biology · doi:10.1186/s13059-016-1037-6 · Cited by 396
“Various reasons contribute to this effect including: (1) the topological properties of the ontology such as the size, depth, and branching factor; (2) term predictability; for example, the BPO terms are considered to be more abstract in nature than theMFO and CCO terms; (3) the annotation status, such as the size of t…”
The language of proteins: NLP, machine learning & protein sequences
Ofer, D., Brandes, N., & Linial, M. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.03.022 · Cited by 339
“training on protein–protein interactions is more problematic due to the far smaller, sparser and more biased data [94,89].”
BackgroundGenomics, convergent neuroscience and progress in understanding autism spectrum disorder
Willsey, H. R., Willsey, A. J., Wang, B., & State, M. W. (2022) · Nature Reviews Neuroscience · doi:10.1038/s41583-022-00576-7 · Cited by 229
Tomczak, A., Mortensen, J. M., Winnenburg, R., Liu, C., Alessi, D. T., Swamy, V., Vallania, F., Lofgren, S., Haynes, W., Shah, N. H., et al. (2018) · Scientific Reports · doi:10.1038/s41598-018-23395-2 · Cited by 190
“Furthermore, previous GO annotation versions were shown to be affected by confounds and biases such as annotation bias, where most annotations are for only few well-studied genes 6,7 or literature bias, where few articles disproportionally contribute many experimental annotations 8 .”
BackgroundGene Ontology: Pitfalls, Biases, and Remedies
Gaudet, P., & Dessimoz, C. (2017) · Methods in Molecular Biology · doi:10.1007/978-1-4939-3743-1_14 · Cited by 168
Das, S., Lee, D., Sillitoe, I., Dawson, N. L., Lees, J. G., & Orengo, C. A. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btv398 · Cited by 155
(2017) · Methods in Molecular Biology · doi:10.1007/978-1-4939-3743-1 · Cited by 133
Koskinen, P., Törönen, P., Nokso-Koivisto, J., & Holm, L. (2015) · Bioinformatics · doi:10.1093/bioinformatics/btu851 · Cited by 145
Hong, J., Luo, Y., Zhang, Y., Ying, J., Xue, W., Xie, T., Tao, L., & Zhu, F. (2019) · Briefings in Bioinformatics · doi:10.1093/bib/bbz081 · Cited by 147
Akiva, E., Copp, J. N., Tokuriki, N., & Babbitt, P. C. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1706849114 · Cited by 137
“Historically, protein characterization efforts have been strongly skewed toward certain classes of proteins, protein families, and superfamilies, leaving the vast majority of superfamilies unexplored (68).”
BackgroundSelf-citationA Literature Review of Gene Function Prediction by Modeling Gene Ontology
Zhao, Y., Wang, J., Chen, J., Zhang, X., Guo, M., & Yu, G. (2020) · Frontiers in Genetics · doi:10.3389/fgene.2020.00400 · Cited by 126
“Two species with high homology have a large number of homologous genes, which should share similar (or even identical) GO annotations (Schnoes et al., 2013).”
BackgroundInfluentialUnderstanding how and why the Gene Ontology and its annotations evolve: the GO within UniProt
Huntley, R. P., Sawford, T., Martin, M. J., & O’Donovan, C. (2014) · GigaScience · doi:10.1186/2047-217x-3-4 · Cited by 94
Revealing Unexplored Sequence-Function Space Using Sequence Similarity Networks
Copp, J. N., Akiva, E., Babbitt, P. C., & Tokuriki, N. (2018) · Biochemistry · doi:10.1021/acs.biochem.8b00473 · Cited by 94
Self-citationTan, J., Doing, G., Lewis, K. A., Price, C. E., Chen, K. M., Cady, K. C., Perchuk, B., Laub, M. T., Hogan, D. A., & Greene, C. S. (2017) · Cell Systems · doi:10.1016/j.cels.2017.06.003 · Cited by 89
“These algorithms require curated pathway knowledge to work effectively, and in many species such resources are biased in various ways (Gillis and Pavlidis, 2013; Greene and Troyanskaya, 2012; Schnoes et al., 2013).”
BackgroundEvolution of Enzyme Superfamilies: Comprehensive Exploration of Sequence–Function Relationships
Baier, F., Copp, J. N., & Tokuriki, N. (2016) · Biochemistry · doi:10.1021/acs.biochem.6b00723 · Cited by 73
Functional unknomics: Systematic screening of conserved genes of unknown function
Rocha, J. J., Jayaram, S. A., Stevens, T. J., Muschalik, N., Shah, R. D., Emran, S., Robles, C., Freeman, M., & Munro, S. (2023) · PLOS Biology · doi:10.1371/journal.pbio.3002222 · Cited by 70
New Insights about Enzyme Evolution from Large Scale Studies of Sequence and Structure Relationships
Brown, S. D., & Babbitt, P. C. (2014) · Journal of Biological Chemistry · doi:10.1074/jbc.r114.569350 · Cited by 71
Self-citationA Survey of Computational Methods for Protein Function Prediction
Shehu, A., Barbará, D., & Molloy, K. (2016) · Springer eBooks · doi:10.1007/978-3-319-41279-5_7 · Cited by 70
“Human experts known as biocurators also often peruse published wet-laboratory studies to provide functional information on a protein [38, 308, 309].”
BackgroundBastian, F., Chibucos, M. C., Gaudet, P., Giglio, M. G., Holliday, G. L., Huang, H., Lewis, S., Niknejad, A., Orchard, S. E., Poux, S., et al. (2015) · Database · doi:10.1093/database/bav043 · Cited by 61
“gene ontology (GO) enrichment analyses on large datasets, or protein-protein interaction network analyses], this curation work provides us with standardized datasets that are essential for downstream analyses (1, 2).”
BackgroundBearne, S. L. (2017) · Biochimica et Biophysica Acta (BBA) - Proteins and Proteomics · doi:10.1016/j.bbapap.2017.02.006 · Cited by 54
Pejaver, V. R., Mooney, S. D., & Radivojac, P. (2017) · Human Mutation · doi:10.1002/humu.23258 · Cited by 57
Gene function prediction based on Gene Ontology Hierarchy Preserving Hashing
Zhao, Y., Fu, G., Wang, J., Guo, M., & Yu, G. (2018) · Genomics · doi:10.1016/j.ygeno.2018.02.008 · Cited by 53
Discovery and mining of enzymes from the human gut microbiome
Jia, B., Han, X., Kim, K. H., & Jeon, C. O. (2021) · Trends in biotechnology · doi:10.1016/j.tibtech.2021.06.008 · Cited by 53
Protein function in precision medicine: deep understanding with machine learning
Rost, B., Radivojac, P., & Bromberg, Y. (2016) · FEBS Letters · doi:10.1002/1873-3468.12307 · Cited by 51
Graph2GO: a multi-modal attributed network embedding method for inferring protein functions
Fan, K., Guan, Y., & Zhang, Y. (2020) · GigaScience · doi:10.1093/gigascience/giaa081 · Cited by 49
“This is due to the bias in the experimental annotation of proteins and has impeded the development of biomedical research to some extent [42, 43].”
Manoharan, L., Kushwaha, S. K., Hedlund, K., & Ahrén, D. (2015) · DNA Research · doi:10.1093/dnares/dsv026 · Cited by 47
“WMG could still be a way of obtaining such information in these particular cases but not to forget that annotating them could still be a subject of biases.(54) This study has for the first time implemented the sequence capture technique to study the functional diversity of enzymes degrading organic matter in natural s…”
BackgroundProtein function annotation using protein domain family resources
Das, S., & Orengo, C. A. (2015) · Methods · doi:10.1016/j.ymeth.2015.09.029 · Cited by 46
“Moreover, bias in protein function annotations [73] or mis-annotations affects our understanding of protein function space [11].”
Laukens, K., Naulaerts, S., & Berghe, W. V. (2014) · PROTEOMICS · doi:10.1002/pmic.201400296 · Cited by 49
“Functional annotations tend to be biased to certain areas [44].”
BackgroundPredicting Protein Function via Semantic Integration of Multiple Networks
Yu, G., Fu, G., Wang, J., & Zhu, H. (2015) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2015.2459713 · Cited by 42
Predicting enzymatic function of protein sequences with attention
Buton, N., Coste, F., & Cunff, Y. L. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad620 · Cited by 49
“One of the major questions of interest is to bridge the gap between these sequences and the functions of the corresponding proteins (Schnoes et al. 2013).”
BackgroundNewGOA: Predicting New GO Annotations of Proteins by Bi-Random Walks on a Hybrid Graph
Yu, G., Fu, G., Wang, J., & Zhao, Y. (2017) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2017.2715842 · Cited by 40
Automatic Gene Function Prediction in the 2020’s
Makrodimitris, S., van Ham, R. C. H. J., & Reinders, M. J. T. (2020) · Genes · doi:10.3390/genes11111264 · Cited by 44
“Moreover, experimental EC’s have also been shown to be highly biased [75], leading to the recent split between “experimental” and “high-throughput experimental” EC’s.”
BackgroundNegGOA: negative GO annotations selection using ontology structure
Fu, G., Wang, J., Yang, B. X., & Yu, G. (2016) · Bioinformatics · doi:10.1093/bioinformatics/btw366 · Cited by 39
Propagation, detection and correction of errors using the sequence database network
Goudey, B., Geard, N. L., Verspoor, K. M., & Zobel, J. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbac416 · Cited by 43
“As several papers have demonstrated [13, 17, 78], the rapid growth and limited ability to curate these databases is increasing the amount of erroneous information in these databases over time.”
BackgroundUrbanization leads to asynchronous homogenization of soil microbial communities across biomes
Zheng, B.-X., Hui, N., Jumpponen, A., Lu, C., Pouyat, R. V., Szlávecz, K., Wardle, D. A., Yesilonis, I. D., Setälä, H. M., & Kotze, D. J. (2025) · Environmental Science and Ecotechnology · doi:10.1016/j.ese.2025.100547 · Cited by 40
“However, determining a direct causal relationship between such traits and ecosystem processes requires further investigation [49,50].”
A multi-scale map of protein assemblies in the DNA damage response
Kratz, A., Kim, M., Kelly, M. R., Zheng, F., Koczor, C. A., Li, J., Ono, K., Qin, Y., Churas, C., Chen, J., et al. (2023) · Cell Systems · doi:10.1016/j.cels.2023.04.007 · Cited by 44
Network-driven plasma proteomics expose molecular changes in the Alzheimer’s brain
Jaeger, P. A., Lucin, K. M., Britschgi, M., Vardarajan, B., Huang, R.-P., Kirby, E. D., Abbey, R., Boeve, B. F., Boxer, A. L., Farrer, L. A., et al. (2016) · Molecular Neurodegeneration · doi:10.1186/s13024-016-0095-2 · Cited by 35
Predicting functions of maize proteins using graph convolutional network
Zhou, G., Wang, J., Zhang, X., Guo, M., & Yu, G. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-03745-6 · Cited by 33
DeepGOA: Predicting Gene Ontology Annotations of Proteins via Graph Convolutional Network
Zhou, G., Wang, J., Zhang, X., & Yu, G. (2019) · 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm47256.2019.8983075 · Cited by 32
“only a very tiny part of newly obtained sequences have experimentally validated GO annotations [3]–[5].”
BackgroundPust, M., & Tümmler, B. (2021) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2021.12.008 · Cited by 32
“The typical core microbes may have been studied more extensively in the past than the often-unknown and unculturable rare species, so there is a literature bias in gene databases in favour of the more abundant species [69,70].”
BackgroundCovalent Docking Predicts Substrates for Haloalkanoate Dehalogenase Superfamily Phosphatases
London, N., Farelli, J. D., Brown, S. D., Liu, C., Huang, H., Korczynska, M., Al-Obaidi, N. F., Babbitt, P. C., Almo, S. C., Allen, K. N., et al. (2014) · Biochemistry · doi:10.1021/bi501140k · Cited by 33
Self-citationAnnotation of gene product function from high-throughput studies using the Gene Ontology
Attrill, H. L., Gaudet, P., Huntley, R. P., Lovering, R. C., Engel, S. R., Poux, S., Auken, K. M. V., Georghiou, G., Chibucos, M. C., Berardini, T., et al. (2019) · Database · doi:10.1093/database/baz007 · Cited by 32
Yan, T., Yue, Z., Xu, H., Liu, Y., Hong, Y., Chen, G.-X., Tao, L., & Xie, T. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106446 · Cited by 32
Predicting protein function via downward random walks on a gene ontology
Yu, G., Zhu, H., Domeniconi, C., & Liu, J. (2015) · BMC Bioinformatics · doi:10.1186/s12859-015-0713-y · Cited by 25
“Our choice of Lin’s structure similarity is driven by the fact that functional annotations of proteins are incomplete and biased to the biologist research interest [4, 8, 12], the frequency of a term is often coarsely estimated.”
BackgroundInfluentialKhalid, M., Liu, X., Zheng, B.-X., Su, L., Kotze, D. J., Setälä, H. M., Ali, M., Rehman, A. U., Rahman, S.-U., & Hui, N. (2023) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2023.137275 · Cited by 24
Community-Wide Evaluation of Computational Function Prediction
Friedberg, I., & Predrag Radivojac, A. (2016) · Methods in molecular biology · doi:10.1007/978-1-4939-3743-1_10 · Cited by 24
“High-throughput experiments, on the other hand, provide a large number of annotations, but those tend to concentrate only on few functions, and generally provide annotations that have a lower information content [28].”
BackgroundInfluentialSelf-citationHornung, B., & Terrapon, N. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010881 · Cited by 26
Chauhan, S., & Ahmad, S. (2019) · Proteins Structure Function and Bioinformatics · doi:10.1002/prot.25763 · Cited by 27
“Unfortunately, a large number of proteins remain un-annotated and many more, although annotated—typically predicted by sequence similarity—do not have their functions well established or understood.(1,2) A widely used practice to annotate protein function is to do a sequence comparison with a dataset of previously ann…”
BackgroundZhang, C., Lane, L., Omenn, G. S., & Zhang, Y. (2019) · Journal of Proteome Research · doi:10.1021/acs.jproteome.9b00537 · Cited by 22
Protein functional site annotation using local structure embeddings
Derry, A., Tartici, A., & Altman, R. B. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2513219122 · Cited by 26
Aranda, K. V., López-Cortina, S., Hodgson, D. A., & Barona‐Gómez, F. (2014) · Microbial Biotechnology · doi:10.1111/1751-7915.12175 · Cited by 21
“This challenge has been addressed either by reductionist studies focusing in a single protein, or after high-throughput systems-level analyses involving many proteins (Laskowski et al., 2005; Redfern et al., 2009; Schnoes et al., 2013).”
BackgroundADAGE signature analysis: differential expression analysis with data-defined gene sets
Tan, J., Huyck, M., Hu, D., Zelaya, R. A., Hogan, D. A., & Greene, C. S. (2017) · BMC Bioinformatics · doi:10.1186/s12859-017-1905-4 · Cited by 23
“Manual annotation ensures the quality of the gene sets but is slow, can be tedious, and leads to gene sets with certain biases [6].”
BackgroundEffusion: prediction of protein function from sequence similarity networks
Yunes, J. M., & Babbitt, P. C. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty672 · Cited by 19
Self-citationNMFGO: Gene Function Prediction via Nonnegative Matrix Factorization with Gene Ontology
Yu, G., Wang, K., Fu, G., Guo, M., & Wang, J. (2018) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2018.2861379 · Cited by 20
“However, applying wet-lab experiments to detect the functions of gene products discovered from these data is not only lowthroughput and expensive, but also limited by the applied experimental protocols and contexts [4], [5].”
Detecting Gene Ontology misannotations using taxon-specific rate ratio comparisons
Wei, X., Zhang, C., Freddolino, L., & Zhang, Y. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btaa548 · Cited by 19
HashGO: hashing gene ontology for protein function prediction
Yu, G., Zhao, Y., Lu, C., & Wang, J. (2017) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2017.09.010 · Cited by 18
A close look at protein function prediction evaluation protocols
Kahanda, I., Funk, C. S., Ullah, F., Verspoor, K. M., & Ben‐Hur, A. (2015) · GigaScience · doi:10.1186/s13742-015-0082-5 · Cited by 20
“In the CAFA AFP problem this assumption probably does not hold because the training is performed on an older set of annotations whereas testing is performed on newer annotations; and it is known that the distribution of GO categories changes over time as a result of strong biases in the annotation process [9].”
BackgroundKEGG orthology prediction of bacterial proteins using natural language processing
Chen, J., Wu, H., & Wang, N. (2024) · BMC Bioinformatics · doi:10.1186/s12859-024-05766-x · Cited by 18
“Moreover, the growing reliance on high-throughput experiments has resulted in a skewed distribution of functional protein annotations in databases, leaving a considerable number of bacterial proteins unexplored in terms of their functions [20].”
BackgroundGut microbiome-derived hydrolases—an underrated target of natural product metabolism
He, J., Liu, X., Zhang, J., Wang, R., Cao, X., & Liu, G. (2024) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2024.1392249 · Cited by 21
Redefining the coenzyme A transferase superfamily with a large set of manually annotated proteins
Hackmann, T. J. (2022) · Protein Science · doi:10.1002/pro.4277 · Cited by 19
Dynamics of the discovery process of protein-protein interactions from low content studies
Wang, Z., Clark, N. R., & Ma’ayan, A. (2015) · BMC Systems Biology · doi:10.1186/s12918-015-0173-z · Cited by 16
Protein Function Prediction Using Deep Restricted Boltzmann Machines
Zou, X., Wang, G., & Yu, G. (2017) · BioMed Research International · doi:10.1155/2017/1729301 · Cited by 16
“Currently, most functional annotations of proteins are shallow and far from complete [3–5].”
BackgroundPredicting functions of uncharacterized gene products from microbial communities
Zhang, Y., Bhosle, A., Bae, S., Eckenrode, K. B., Huang, X., Tang, J., Lavrentovich, D. O., Awad, L., Ji, H., Wang, Y., et al. (2025) · Nature Biotechnology · doi:10.1038/s41587-025-02813-7 · Cited by 19
“Even in the well-characterized Escherichia coli pangenome, only 37.6% protein families were annotated with BP terms and 24.9% time-consuming and expensive process that cannot approach the scale of thousands of new protein families per year 16 .”
InfluentialProtein Function Prediction With Functional and Topological Knowledge of Gene Ontology
Zhao, Y., Yang, Z., Hong, Y., Yang, Y., Wang, L., Zhang, Y., Lin, H., & Wang, J. (2023) · IEEE Transactions on NanoBioscience · doi:10.1109/tnb.2023.3278033 · Cited by 15
Systems Analysis for Interpretation of Phosphoproteomics Data
Munk, S., Refsgaard, J. C., & Olsen, J. V. (2015) · Methods in molecular biology · doi:10.1007/978-1-4939-3049-4_23 · Cited by 15
Evaluating Computational Gene Ontology Annotations
Škunca, N., Roberts, R. J., & Steffen, M. A. (2016) · Methods in molecular biology · doi:10.1007/978-1-4939-3743-1_8 · Cited by 15
Grötzinger, S. W., Alam, I. S., Ba-Alawi, W., Bajić, V. B., Stingl, U., & Eppinger, J. (2014) · Frontiers in Microbiology · doi:10.3389/fmicb.2014.00134 · Cited by 17
“Further, functional annotation, which is only based on GO-terms can result in a considerable bias (Schnoes et al., 2013).”
BackgroundChallenges and opportunities for proteomics and the improvement of bread wheat quality
Bacala, R., Hatcher, D. W., Perreault, H., & Fu, B. X. (2022) · Journal of Plant Physiology · doi:10.1016/j.jplph.2022.153743 · Cited by 16
de Crécy‐Lagard, V., Dias, R., Sexson, N., Friedberg, I., Yuan, Y., & Swairjo, M. A. (2025) · G3 Genes Genomes Genetics · doi:10.1093/g3journal/jkaf169 · Cited by 19
Self-citationTowards the explainability of protein language models
Hunklinger, A., & Ferruz, N. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01232-w · Cited by 18
Bell, M. J., Collison, M., & Lord, P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0075541 · Cited by 12
“, who have shown that the annotation of many high-throughput experiments is based upon a very small amount of experimental data [29].”
BackgroundInterspecies gene function prediction using semantic similarity
Yu, G., Luo, W., Fu, G., & Wang, J. (2016) · BMC Systems Biology · doi:10.1186/s12918-016-0361-5 · Cited by 12
“Because of resource limitations, priority of GO curators, experimental ethics and protocols, the GO annotations of its neighborhood genes from the same species may be shallow, incomplete and biased [6, 8, 9].”
BackgroundNoGOA: predicting noisy GO annotations using evidences and sparse representation
Yu, G., Lu, C., & Wang, J. (2017) · BMC Bioinformatics · doi:10.1186/s12859-017-1764-z · Cited by 12
“[2], and are also biased by the research interests of biologists [3].”
BackgroundDiscovery of Two Novel Oxidases Using a High‐Throughput Activity Screen
Rembeza, E., Boverio, A., Fraaije, M. W., & Engqvist, M. K. M. (2021) · ChemBioChem · doi:10.1002/cbic.202100510 · Cited by 11
Cross-Species Protein Function Prediction with Asynchronous-Random Walk
Zhao, Y., Wang, J., Guo, M., Zhang, X., & Yu, G. (2019) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2019.2943342 · Cited by 12
“However, most data fusion based solutions focus on predicting functions of proteins from single species [9], [11]–[13], and the curated annotations of proteins in different species are biased, incomplete and imbalanced [4].”
BackgroundInfluentialBioprospecting in the Genomic Age
Hicks, M. A., & Prather, K. L. J. (2014) · Advances in applied microbiology · doi:10.1016/b978-0-12-800261-2.00003-7 · Cited by 15
GOstruct 2.0: Automated Protein Function Prediction for Annotated Proteins
Kahanda, I., & Ben‐Hur, A. (2017) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3107411.3107417 · Cited by 14
“In the CAFA AFP problem this assumption likely does not hold because the training is performed on an older set of annotations while testing is performed on newer annotations; and it is known that distribution of GO categories changes over time due to biases in the annotation process [29].”
BackgroundMueller, A. J., Peffers, M. J., Proctor, C. J., & Clegg, P. (2017) · Journal of Orthopaedic Research® · doi:10.1002/jor.23563 · Cited by 10
“These important issues have been realised [8] and methods to improve annotations are being developed [9].”
CommWalker: correctly evaluating modules in molecular networks in light of annotation bias
Luecken, M. D., Page, M., Crosby, A., Mason, S., Reinert, G., & Deane, C. M. (2017) · Bioinformatics · doi:10.1093/bioinformatics/btx706 · Cited by 11
van den Bent, I., Makrodimitris, S., & Reinders, M. J. T. (2021) · Evolutionary Bioinformatics · doi:10.1177/11769343211062608 · Cited by 10
Rappsilber, J. (2023) · Trends in Genetics · doi:10.1016/j.tig.2023.10.011 · Cited by 13
Utilizing “Omics” Tools to Study the Complex Gut Ecosystem
Fodor, A. A. (2014) · Advances in experimental medicine and biology · doi:10.1007/978-1-4939-0897-4_2 · Cited by 8
“Of course, this interpretation of these results is very dependent on the accuracy of functions that are in gene function databases and there has been some question as to how biased these databases may be [55].”
BackgroundBiological process annotation of proteins across the plant kingdom
Bargsten, J. W., Severing, E. I., Nap, J., Sanchez‐Perez, G. F., & van Dijk, A. D. J. (2014) · Current Plant Biology · doi:10.1016/j.cpb.2014.07.001 · Cited by 8
NoisyGOA: Noisy GO annotations prediction using taxonomic and semantic similarity
Lu, C., Wang, J., Zhang, Z., Yang, P., & Yu, G. (2016) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2016.09.005 · Cited by 8
Enumerating consistent sub-graphs of directed acyclic graphs: an insight into biomedical ontologies
Peng, Y., Jiang, Y., & Predrag Radivojac, A. (2018) · Bioinformatics · doi:10.1093/bioinformatics/bty268 · Cited by 8
Biocuration in the structure–function linkage database: the anatomy of a superfamily
Holliday, G. L., Brown, S. D., Akiva, E., Mischel, D., Hicks, M. A., Morris, J. H., Huang, C. C., Meng, E. C., Pegg, S. C. ‐., Ferrin, T. E., et al. (2017) · Database · doi:10.1093/database/bax006 · Cited by 8
Self-citationExploring Approaches for Detecting Protein Functional Similarity within an Orthology-based Framework
Weichenberger, C. X., Palermo, A., Pramstaller, P. P., & Domingues, F. S. (2017) · Scientific Reports · doi:10.1038/s41598-017-00465-5 · Cited by 8
Beyond sequence similarity: toward function-based screening of nucleic acid synthesis
Abel, G. R., Alexanian, T., Bartling, C., Beal, J., Curtis, S., Flyangolts, K., Foner, L. N., Forry, S. P., Godbold, G. D., Horvitz, E., et al. (2026) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2026.1832724 · Cited by 6
“…is overwhelmingly from naturally evolved sequences or close mutants, biased by common measurement techniques and functions that are easily measured in high throughput (Schnoes et al., 2013), and concentrated on a small number of model organisms (Kustatscher et al., 2022; Rocha et al., 2023).”
Zhao, Y., Yang, Z., Hong, Y., Wang, L., Zhang, Y., Lin, H., & Wang, J. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3221988 · Cited by 6
“cause incomplete function annotations [5], [6].”
Leveraging computational genomics to understand the molecular basis of metal homeostasis
Grosjean, N., & Blaby, C. (2020) · New Phytologist · doi:10.1111/nph.16820 · Cited by 6
“The sources of curated functional annotations are biased toward experimentation from a small number of organisms, and relatively few of those annotations are based on biochemical characterization of molecular function (Schnoes et al ., 2013).”
BackgroundPrabakaran, R., & Bromberg, Y. (2025) · Bioinformatics · doi:10.1093/bioinformatics/btaf035 · Cited by 6
“The challenges associated with protein functional annotation are multi-fold and have been discussed at length in earlier studies (Schnoes et al. 2013, Mangul et al. 2019, Makrodimitris et al. 2020, Ramola et al. 2022).”
Automatic consistency assurance for literature-based gene ontology annotation
Chen, J., Geard, N. L., Zobel, J., & Verspoor, K. M. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04479-9 · Cited by 5
Kennedy, L. S., Harper, M., & Čuperlović‐Culf, M. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf635 · Cited by 6
“Collectively,these biases in the form of over-and under-studied functions, and inconsistent or missing information within our knowledge bases present a challenging landscape for accurate function prediction [31–33].”
GOThresher: a program to remove annotation biases from protein function annotation datasets
Joshi, P., Banerjee, S., Hu, X., Khade, P. M., & Friedberg, I. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btad048 · Cited by 4
Self-citationProtein function prediction based on zero-one matrix factorization
Wang, J., Zhao, Y., Guo, M., Zhang, Z., & Yu, G. (2019) · Scientia Sinica Informationis · doi:10.1360/n112018-00331 · Cited by 4
Molecular evolution and gene function
Robinson‐Rechavi, M. (2019) · SERVAL (Université de Lausanne) · doi:10.48550/arxiv.1910.01940 · Cited by 6
Metadata
- Authors
- Schnoes, A. M., Ream, D. C., Thorman, A. W., Babbitt, P. C., & Friedberg, I.
- Affiliations
- University of California, San Francisco (US)
- Miami University (US)
- Corresponding author
- Friedberg, Iddo
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-05-30
- Volume
- 9
- Issue
- 5
- Pages
- e1003063
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC3667760
- OpenAlex
- W2169059428
- Cited by
- 129
- References
- 53
- Citations by year
- 2026: 9 · 2025: 10 · 2024: 3 · 2023: 12 · 2022: 9 · 2021: 7 · 2020: 7 · 2019: 12 · 2018: 9 · 2017: 16 · 2016: 12 · 2015: 11 · 2014: 11 · 2013: 1
- FWCI
- 5.23
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- eScholarship (California Digital Library) (submittedVersion)
- eScholarship (California Digital Library) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1146960, 0640476)
- National Institutes of Health (NIH-R01)
- National Institute of General Medical Sciences (R01GM60595, R01GM060595)
- Division of Biological Infrastructure
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- protein annotation, annotation bias, high-throughput experimentation, Gene Ontology, subcellular localization, embryonic development, mass spectrometry, RNA interference, literature curation
- MeSH terms
- Animals, Humans, Proteins, Computational Biology, Databases, Protein, High-Throughput Screening Assays, Molecular Sequence Annotation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.