FUNC: a package for detecting significant associations between gene sets and ontological annotations
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Genome-wide expression, sequence and association studies typically yield large sets of gene candidates, which must then be further analysed and interpreted. Information about these genes is increasingly being captured and organized in ontologies, such as the Gene Ontology. Relationships between the gene sets identified by experimental methods and biological knowledge can be made explicit and used in the interpretation of results. However, it is often difficult to assess the statistical significance of such analyses since many inter-dependent categories are tested simultaneously. RESULTS: We developed the program package FUNC that includes and expands on currently available methods to identify significant associations between gene sets and ontological annotations. Implemented are several tests in particular well suited for genome wide sequence comparisons, estimates of the family-wise error rate, the false discovery rate, a sensitive estimator of the global significance of the results and an algorithm to reduce the complexity of the results. CONCLUSION: FUNC is a versatile and useful tool for the analysis of genome-wide data. It is freely available under the GPL license and also accessible via a web service.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482 · cited 2×
“Several programs that analyse functional annotations have implemented FDR methods (e.g. [5,9,16-21]), most often using the procedure of Benjamini and Hochberg [12].”
BackgroundMethodSubramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506580102 · Cited by 53,625
Gene Ontology: tool for the unification of biology
Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M., Davis, A. P., Dolinski, K., Dwight, S. S., Eppig, J. T., et al. (2000) · Nature Genetics · doi:10.1038/75556 · Cited by 43,206
The control of the false discovery rate in multiple testing under dependency
Benjamini, Y., & Yekutieli, D. (2001) · The Annals of Statistics · doi:10.1214/aos/1013699998 · Cited by 10,490 · cited 2×
“Although it has been shown that the method also works well under positive dependency among the tests [33] and an easy (conservative) correction method can be used under different kinds of dependencies [33], it is not entirely clear whether it is well-suited for the kind of strong dependencies that…”
MethodMootha, V. K., Lindgren, C. M., Eriksson, K.-F., Subramanian, A., Sihag, S., Lehar, J., Puigserver, P., Carlsson, E., Ridderstråle, M., Laurila, E., et al. (2003) · Nature Genetics · doi:10.1038/ng1180 · Cited by 10,197 · cited 1×
“[5,9,16-21]), most often using the procedure of Benjamini and Hochberg [12].”
MethodSerial Analysis of Gene Expression
Velculescu, V. E., Zhang, L., Vogelstein, B., & Kinzler, K. W. (1995) · Science · doi:10.1126/science.270.5235.484 · Cited by 3,711
Adaptive protein evolution at the Adh locus in Drosophila
McDonald, J. H., & Kreitman, M. (1991) · Nature · doi:10.1038/351652a0 · Cited by 3,283 · cited 1×
“…human amino acid changes versus chimpanzee amino acid changes) and (iv) a 2 × 2 table test that is suitable for a McDonald Kreitman type test to infer selection on genes from divergence and polymorphism data at two types of sites, like synonymous and non-synonymous sites in a coding sequence [24].”
MethodInitial sequence of the chimpanzee genome and comparison with the human genome
The Chimpanzee Sequencing and Analysis Consortium (2005) · Nature · doi:10.1038/nature04072 · Cited by 2,459
Improved scoring of functional groups from gene expression data by decorrelating GO graph structure
Alexa, A., Rahnenführer, J., & Lengauer, T. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl140 · Cited by 2,188
Identifying differentially expressed genes using false discovery rate controlling procedures
Reiner, A., Yekutieli, D., & Benjamini, Y. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btf877 · Cited by 1,765 · cited 2×
“Several approaches to estimate the FDR have been proposed (see e.g. [ 14,15]) and some have been integrated in various functional profiling applications (e.g. [5 ,9,16-21]).”
MethodResampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment.
Martin, M. A., Westfall, P. H., & Young, S. S. (1994) · Biometrics · doi:10.2307/2533464 · Cited by 1,699 · cited 2×
“This permuted data, the random sets, can be used to estimate the family wise error rate, i.e. the probability that among the tests that are declared significant one or more are false positives [28].”
MethodGOstat: find statistically overrepresented Gene Ontologies within a group of genes
Beißbarth, T., & Speed, T. P. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth088 · Cited by 1,257
GoMiner: a resource for biological interpretation of genomic and proteomic data
Zeeberg, B. R., Feng, W., Wang, G., Wang, M. D., Fojo, A. T., Sunshine, M., Narasimhan, S., Kane, D. W., Reinhold, W. C., Lababidi, S., et al. (2003) · Genome Biology · doi:10.1186/gb-2003-4-4-r28 · Cited by 1,198 · cited 1×
“First, it does not provide any graphical representation of the results such as those provided e.g. by GOMiner [36].”
FatiGO: a web tool for finding significant associations of Gene Ontology terms with groups of genes
Al-Shahrour, F., Díaz-Uriarte, R., & Dopazo, J. (2004) · Bioinformatics · doi:10.1093/bioinformatics/btg455 · Cited by 1,122 · cited 2×
“Applications exist that use a simple Bonferroni correction or other more powerful FWER correction procedures [9-11].”
MethodResampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment
Chaubey, Y. P., Westfall, P. H., & Young, S. S. (1993) · Technometrics · doi:10.2307/1270279 · Cited by 920
Yekutieli, D., & Benjamini, Y. (1999) · Journal of Statistical Planning and Inference · doi:10.1016/s0378-3758(99)00041-5 · Cited by 753 · cited 2×
“These random sets are used to calculate for each category, i.e. each raw p-value cut-off, two corrected p-values: a resampling-based false discovery rate (FDR) [27] and the family-wise error rate (FWER) [28].”
MethodDiscovering statistically significant pathways in expression profiling studies
Tian, L., Greenberg, S. A., Kong, S. W., Altschuler, J., Kohane, I. S., & Park, P. J. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506577102 · Cited by 648
Drǎghici, S., Khatri, P., Martins, R. P., Ostermeier, G. C., & Krawetz, S. A. (2003) · Genomics · doi:10.1016/s0888-7543(02)00021-6 · Cited by 578
Resampling-based multiple testing for microarray data analysis
Ge, Y., Dudoit, S., & Speed, T. P. (2003) · Test · doi:10.1007/bf02595811 · Cited by 510
Characterizing gene sets with FuncAssociate
Berriz, G. F., King, O. D., Bryant, B., Sander, C., & Roth, F. P. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg363 · Cited by 473 · cited 1×
“This approach is more powerful than the conservative Bonferroni correction and has, in the context of GO analyses programs, been implemented for example in FuncAs-sociate [10] and is also implemented in FUNC.”
MethodZhang, B., Schmoyer, D., Kirov, S., & Snoddy, J. (2004) · BMC Bioinformatics · doi:10.1186/1471-2105-5-16 · Cited by 450 · cited 1×
“Various programs have implemented this approach (e.g. [ 2-7]).”
Barry, W. T., Nobel, A. B., & Wright, F. A. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti260 · Cited by 356 · cited 4×
“Several programs that analyse functional annotations have implemented FDR methods (e.g. [5,9,16-21]), most often using the procedure of Benjamini and Hochberg [12].”
BackgroundMethodInfluentialHuman specific loss of olfactory receptor genes
Gilad, Y., Man, O., Pääbo, S., & Lancet, D. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0535697100 · Cited by 307 · cited 1×
“…also identified here (Table 2), are thought to have evolved under less constraint in primates than in rodents since a higher fraction of pseudogenes is found in primates [35], indicating Illustration how the global p-value is calculated Figure 2 Illustration how the global p-value is calculated .”
BackgroundSelf-citationGeneMerge—post-genomic analysis, data mining, and hypothesis testing
Castillo-Davis, C. I., & Hartl, D. L. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg114 · Cited by 300 · cited 1×
“Applications exist that use a simple Bonferroni correction or other more powerful FWER correction procedures [9-11].”
MethodSequence Divergence, Functional Constraint, and Selection in Protein Evolution
Fay, J. C., & Wu, C.-I. (2003) · Annual Review of Genomics and Human Genetics · doi:10.1146/annurev.genom.4.020303.162528 · Cited by 294 · cited 1×
“An excess of fixed non-synonymous substitutions can indicate the action of positive selection, whereas an excess of non-synonymous polymorphisms can indicate the presence of slightly deleterious amino acid variants (for a review see [30]).”
BackgroundZeeberg, B. R., Qin, H., Narasimhan, S., Sunshine, M., Cao, H., Kane, D. W., Reimers, M., Stephens, R. M., Bryant, D., Burt, S. K., et al. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-168 · Cited by 279
eVOC: A Controlled Vocabulary for Unifying Gene Expression Data
Kelso, J., Visagie, J., Theiler, G., Christoffels, A., Bardien, S., Smedley, D., Otgaar, D., Greyling, G., Jongeneel, C. V., McCarthy, M. I., et al. (2003) · Genome Research · doi:10.1101/gr.985203 · Cited by 162 · cited 1×
“It is particular useful when analyzing ontological annotations such as provided by the Gene Ontology consortium [1] or eVOC [26], but can be easily adapted to any other annotation.”
BackgroundStatistical concerns about the GSEA procedure
Damian, D., & Gorfine, M. (2004) · Nature Genetics · doi:10.1038/ng0704-663a · Cited by 118 · cited 1×
“Hence, the category with the biggest size effect is not necessarily the most significant category and vice versa (see also [31] and reply).”
BackgroundComparison of Methods for Estimating the Number of True Null Hypotheses in Multiplicity Testing
Hsueh, H.-M., Chen, J. J., & Kodell, R. L. (2003) · Journal of Biopharmaceutical Statistics · doi:10.1081/bip-120024202 · Cited by 113 · cited 1×
“Different methods exist to estimate the FDR, differing in how they treat the case that no hypothesis is rejected and in how they estimate the number of falsely rejected hypotheses (see e.g. [14,15, 32 ] for a discussion).”
MethodMasseroli, M., Martucci, D., & Pinciroli, F. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh432 · Cited by 92
Comparing functional annotation analyses with Catmap
Breslin, T., Edén, P., & Krogh, M. (2004) · BMC Bioinformatics · doi:10.1186/1471-2105-5-193 · Cited by 87 · cited 2×
“Usually the gene lists are permuted, though some approaches also allow the permutation of sample labels [16,20,21,23], This allows not only a correction of dependence among the categories, but also for dependence among the genes [23].”
BackgroundMethodGenomics, Prior Probability, and Statistical Tests of Multiple Hypotheses
Manly, K. F., Nettleton, D., & Hwang, J. T. G. (2004) · Genome Research · doi:10.1101/gr.2156804 · Cited by 86 · cited 2×
“…it might instead be more useful to determine the false discovery rate (FDR) [12], i.e. the proportion of false positives among all significant features, since a known proportion of false positives can easily be tolerated for an increase of power in these contexts (see e.g. [13] for an overview).”
BackgroundBen‐Shaul, Y., Bergman, H., & Soreq, H. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bti149 · Cited by 60 · cited 2×
“…have been proposed that instead use a measure of choice to rank the genes, and then use rank-based tests such as the Wilcoxon-rank test or the Kolmogorov-Smirnov test to test if genes that belong to one category differ in their ranks from genes that do not belong to the category (e.g. [8 ]).”
BackgroundMethodOntologyTraverser: an R package for GO analysis
Young, A., Whitehouse, N., Cho, J., & Shaw, C. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bth495 · Cited by 43
GOAL: automated Gene Ontology analysis of expression profiles
Volinia, S., Evangelisti, R., Francioso, F., Arcelli, D., Carella, M., & Gasparini, P. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh443 · Cited by 42
Handling multiple testing while interpreting microarrays with the Gene Ontology Database
Osier, M. V., Zhao, H., & Cheung, K.-H. (2004) · BMC Bioinformatics · doi:10.1186/1471-2105-5-124 · Cited by 35 · cited 1×
“Many methods rely on permutations to estimate the FWER or FDR, since the dependency and structure of the annotations makes it difficult to find analytical methods to do the same [22].”
Method
Cited by
A high-coverage genome sequence from an archaic denisovan individual
Meyer, M., Kircher, M., Gansauge, M.-T., Li, H., Racimo, F., Mallick, S., Schraiber, J. G., Jay, F., Prüfer, K., de Filippo, C., et al. (2012) · Science · doi:10.1126/science.1224344 · Cited by 1,967
Self-citationThe genomic landscape of Neanderthal ancestry in present-day humans
Sankararaman, S., Mallick, S., Dannemann, M., Prüfer, K., Kelso, J., Pääbo, S., Patterson, N., & Reich, D. (2014) · Nature · doi:10.1038/nature12961 · Cited by 1,107
Self-citationDe novo assembly of soybean wild relatives for pan-genome analysis of diversity and agronomic traits
Li, Y.-H., Zhou, G., Ma, J., Jiang, W., Jin, L.-G., Zhang, Z., Guo, Y., Zhang, J., Sui, Y., Zheng, L., et al. (2014) · Nature Biotechnology · doi:10.1038/nbt.2979 · Cited by 704
A humanized version of Foxp2 affects cortico-basal ganglia circuits in mice
Enard, W., Gehre, S., Hammerschmidt, K., Hölter, S. M., Blass, T., Somel, M., Brückner, M. K., Schreiweis, C., Winter, C., Sohr, R., et al. (2009) · Cell · doi:10.1016/j.cell.2009.03.041 · Cited by 605
“Using Cohen’s D estimate of effect size, we correlated genotype-dependent effects with functional annotations (Prüfer et al., 2007), putative FOXP2 targets in a human neuroblastoma cell line (Vernes et al.”
MethodSelf-citationIorizzo, M., Ellison, S., Senalik, D., Zeng, P., Satapoomin, P., Huang, J., Bowman, M., Iovene, M., Sanseverino, W., Cavagnaro, P., et al. (2016) · Nature Genetics · doi:10.1038/ng.3565 · Cited by 542
Metadata
- Authors
- Prüfer, K., Muetzel, B., Do, H.-H., Weiss, G., Khaitovich, P., Rahm, E., Pääbo, S., Lachmann, M., & Enard, W.
- Venue
- BMC Bioinformatics
- Published
- 2007-02-06
- Volume
- 8
- Issue
- 1
- Pages
- 41-41
- ISSN
- 1471-2105 (ISSN-L)
- OpenAlex
- W1786194602
- Cited by
- 173
- References
- 36
- FWCI
- 3.95
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.