A general modular framework for gene set enrichment analysis
Open accessAbstract
BACKGROUND: Analysis of microarray and other high-throughput data on the basis of gene sets, rather than individual genes, is becoming more important in genomic studies. Correspondingly, a large number of statistical approaches for detecting gene set enrichment have been proposed, but both the interrelations and the relative performance of the various methods are still very much unclear. RESULTS: We conduct an extensive survey of statistical approaches for gene set analysis and identify a common modular structure underlying most published methods. Based on this finding we propose a general framework for detecting gene set enrichment. This framework provides a meta-theory of gene set analysis that not only helps to gain a better understanding of the relative merits of each embedded approach but also facilitates a principled comparison and offers insights into the relative interplay of the methods. CONCLUSION: We use this framework to conduct a computer simulation comparing 261 different variants of gene set enrichment procedures and to analyze two experimental data sets. Based on the results we offer recommendations for best practices regarding the choice of effective procedures for gene set enrichment analysis.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Genetic pathway analysis methods for GWAS data aim to identify which biological pathways show an excess of etiological association.
“Genetic pathway analysis methods for GWAS data have been developed5,6 and aim to identify which biological pathways show an excess of etiological association”
From Psychiatric genome-wide association study analyses implicate neuronal, immune and histone pathways (2015) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,706 · see citing works
Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506580102 · Cited by 53,625 · see citing works
Gene Ontology: tool for the unification of biology
Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M., Davis, A. P., Dolinski, K., Dwight, S. S., Eppig, J. T., et al. (2000) · Nature Genetics · doi:10.1038/75556 · Cited by 43,206 · see citing works
In silico prediction of protein-protein interactions in human macrophages
Souiai, O., Guerfali, F. Z., Miled, S. B., Brun, C., & Benkahla, A. (2014) · BMC Research Notes · doi:10.1186/1756-0500-7-157 · Cited by 10,785 · see citing works
Significance analysis of microarrays applied to the ionizing radiation response
Tusher, V. G., Tibshirani, R., & Chu, G. (2001) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.091062498 · Cited by 10,568 · see citing works
Mootha, V. K., Lindgren, C. M., Eriksson, K.-F., Subramanian, A., Sihag, S., Lehar, J., Puigserver, P., Carlsson, E., Ridderstråle, M., Laurila, E., et al. (2003) · Nature Genetics · doi:10.1038/ng1180 · Cited by 10,197 · see citing works
limma: Linear Models for Microarray Data
Smyth, G. K. (2005) · Statistics for Biology and Health · doi:10.1007/0-387-29362-0_23 · Cited by 5,796 · see citing works
Bioinformatics and Computational Biology Solutions Using R and Bioconductor
Gentleman, R. C., Carey, V. J., Huber, W. K., Irizarry, R. A., & Dudoit, S. (2005) · Statistics in the health sciences · doi:10.1007/0-387-29362-0 · Cited by 3,470 · see citing works
On the Interpretation of χ 2 from Contingency Tables, and the Calculation of P
Fisher, R. A. (1922) · Journal Of The Royal Statistical Society · doi:10.2307/2340521 · Cited by 2,975 · see citing works
Measures of Association for Cross Classifications
Goodman, L. A., & Kruskal, W. (1979) · Springer series in statistics · doi:10.1007/978-1-4612-9995-0 · Cited by 2,255 · see citing works
Improved scoring of functional groups from gene expression data by decorrelating GO graph structure
Alexa, A., Rahnenführer, J., & Lengauer, T. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl140 · Cited by 2,194 · see citing works
Identifying biological themes within lists of genes with EASE
Hosack, D. A., Dennis, G., Sherman, B. T., Lane, H. C., & Lempicki, R. A. (2003) · Genome biology · doi:10.1186/gb-2003-4-10-r70 · Cited by 1,950 · see citing works
Schäfer, J., & Strimmer, K. (2005) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1175 · Cited by 1,556 · see citing works
Self-citationA global test for groups of genes: testing association with a clinical outcome
Goeman, J. J., van de Geer, S., de Kort, F., & van Houwelingen, H. C. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg382 · Cited by 1,052 · see citing works
Large-Scale Simultaneous Hypothesis Testing: The Choice of a Null Hypothesis
Efron, B. (2004) · Journal of the American Statistical Association · doi:10.1198/016214504000000089 · Cited by 1,005 · see citing works
Gene-expression profiles in hereditary breast cancer.
Hedenfalk, I., Duggan, D., Chen, Y.-D., Radmacher, M., Bittner, M., Simon, R., Meltzer, P., Gusterson, B., Esteller, M., Raffeld, M., et al. (2001) · New England Journal of Medicine · doi:10.1056/nejm200102223440801 · Cited by 978 · see citing works
Ontological analysis of gene expression data: current tools, limitations, and open problems
Khatri, P., & Drăghici, S. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti565 · Cited by 850 · see citing works
Analyzing gene expression data in terms of gene sets: methodological issues
Goeman, J. J., & Bühlmann, P. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm051 · Cited by 795 · see citing works
On testing the significance of sets of genes
Efron, B., & Tibshirani, R. J. (2007) · The Annals of Applied Statistics · doi:10.1214/07-aoas101 · Cited by 785 · see citing works
PAGE: Parametric Analysis of Gene Set Enrichment
Kim, S., & Volsky, D. J. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-144 · Cited by 781 · see citing works
Discovering statistically significant pathways in expression profiling studies
Tian, L., Greenberg, S. A., Kong, S. W., Altschuler, J., Kohane, I. S., & Park, P. J. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506577102 · Cited by 649 · see citing works
Drăghici, S., Khatri, P., Martins, R. P., Ostermeier, G. C., & Krawetz, S. A. (2003) · Genomics · doi:10.1016/s0888-7543(02)00021-6 · Cited by 578 · see citing works
Zhang, B., Schmoyer, D. D., Kirov, S. A., & Snoddy, J. R. (2004) · BMC Bioinformatics · doi:10.1186/1471-2105-5-16 · Cited by 450 · see citing works
Transcriptional profiling of aging in human muscle reveals a common aging signature
Zahn, J. M., Sonu, R., Vogel, H., Crane, E., Mazan-Mamczarz, K., Rabkin, R., Davis, R. W., Becker, K. G., Owen, A. B., & Kim, S. K. (2006) · PLoS Genetics · doi:10.1371/journal.pgen.0020115 · Cited by 351 · see citing works
Barry, W. T., Nobel, A. B., & Wright, F. A. (2005) · Computer applications in the biosciences · doi:10.1093/bioinformatics/bti260 · Cited by 349 · see citing works
Gene-set approach for expression pattern analysis
Nam, D., & Kim, S.-Y. (2008) · Briefings in Bioinformatics · doi:10.1093/bib/bbn001 · Cited by 338 · see citing works
Improving gene set analysis of microarray data by SAM-GS
Dinu, I. A., Potter, J. D., Mueller, T., Liu, Q., Adewale, A. J., Jhangri, G. S., Einecke, G., Famulski, K. S., Halloran, P. F., & Yasui, Y. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-242 · Cited by 262 · see citing works
Random-set methods identify distinct aspects of the enrichment signal in gene-set analysis
Newton, M. A., Quintana, O. A., den Boon, J. A., Sengupta, S., & Ahlquist, P. (2016) · Cited by 230 · see citing works
Extensions to gene set enrichment
Jiang, Z., & Gentleman, R. C. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl599 · Cited by 216 · see citing works
Random-set methods identify distinct aspects of the enrichment signal in gene-set analysis
Newton, M. A., Quintana, F. A., den Boon, J. A., Sengupta, S., & Ahlquist, P. (2007) · The Annals of Applied Statistics · doi:10.1214/07-aoas104 · Cited by 191 · see citing works
Accurate Ranking of Differentially Expressed Genes by a Distribution-Free Shrinkage Approach
Opgen-Rhein, R., & Strimmer, K. (2007) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1252 · Cited by 191 · see citing works
Self-citationExact t and F Tests for Analyzing Studies with Multiple Endpoints
Läuter, J. (1996) · Biometrics · doi:10.2307/2533057 · Cited by 175 · see citing works
Transcriptional profiling of aging in human muscle reveals a common aging signature
Zahn, J. M., Sonu, R. J., Vogel, H. S., Crane, E., Mazan-Mamczarz, K., Rabkin, R., Davis, R. W., Becker, K. G., Owen, A. B., & Kim, S. K. (2005) · PLoS Genetics · doi:10.1371/journal.pgen.0020115.eor · Cited by 171 · see citing works
Gene expression network analysis and applications to immunology
Nacu, Ş., Critchley-Thorne, R. J., Lee, P. P., et al. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm019 · Cited by 151 · see citing works
GlobalANCOVA: exploration and assessment of gene group effects
Hummel, M., Meister, R., & Mansmann, U. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm531 · Cited by 148 · see citing works
A multivariate approach for integrating genome-wide expression data and biological knowledge
Kong, S. W., Pu, W. T., & Park, P. J. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl401 · Cited by 132 · see citing works
Calculating the Statistical Significance of Changes in Pathway Activity From Gene Expression Data
Rahnenführer, J., Domingues, F. S., Maydt, J., & Lengauer, T. (2004) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1055 · Cited by 125 · see citing works
EXPLORING GENE EXPRESSION DATA WITH CLASS SCORES
Pavlidis, P., Lewis, D. P., & Noble, W. S. (2001) · Pacific Symposium on Biocomputing (PSB) · doi:10.1142/9789812799623_0044 · Cited by 118 · see citing works
Mansmann, U., & Meister, R. (2005) · PubMed · Cited by 108 · see citing works
Simultaneous Inference: When Should Hypothesis Testing Problems Be Combined?
Efron, B. (2008) · Cited by 105 · see citing works
Comparative evaluation of gene-set analysis methods
Liu, Q., Dinu, I. A., Adewale, A. J., et al. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-431 · Cited by 104 · see citing works
Testing Differential Gene Expression in Functional Groups
Meister, R., & Mansmann, U. (2005) · Methods of Information in Medicine · doi:10.1055/s-0038-1633992 · Cited by 90 · see citing works
Simultaneous inference: When should hypothesis testing problems be combined?
Efron, B. (2008) · The Annals of Applied Statistics · doi:10.1214/07-aoas141 · Cited by 84 · see citing works
Microarray-based gene set analysis: a comparison of current methods
Song, S., & Black, M. A. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-502 · Cited by 73 · see citing works
Edelman, E. J., Porrello, A., Guinney, J., Balakumaran, B. S., Bild, A., Febbo, P. G., & Mukherjee, S. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl231 · Cited by 73 · see citing works
Grouping Gene Ontology terms to improve the assessment of gene set enrichment in microarray data
Lewin, A., & Grieve, I. C. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-426 · Cited by 55 · see citing works
Computation of significance scores of unweighted Gene Set Enrichment Analyses
Keller, A., Backes, C., & Lenhof, H. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-290 · Cited by 54 · see citing works
A MULTIVARIATE EXTENSION OF THE GENE SET ENRICHMENT ANALYSIS
Klebanov, L. B., Glazko, G. V., Salzman, P., YAKOVLEV, A., & Xiao, Y. (2007) · Journal of Bioinformatics and Computational Biology · doi:10.1142/s0219720007003041 · Cited by 37 · see citing works
Tutorial on Microarray Gene Expression Experiments
Mansmann, U., Brunner, E. J., Ziegler, A., et al. (2005) · Methods of Information in Medicine · doi:10.1055/s-0038-1633982 · Cited by 35 · see citing works
Significance analysis of groups of genes in expression profiling studies
Chen, J. J., Lee, T., Delongchamp, R. R., et al. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm310 · Cited by 34 · see citing works
Pathway Analysis of Microarray Data via Regression
Adewale, A. J., Dinu, I. A., Potter, J. D., et al. (2008) · Journal of Computational Biology · doi:10.1089/cmb.2008.0002 · Cited by 29 · see citing works
Läuter, J., Horn, F., Rosołowski, M., et al. (2009) · Biometrical Journal · doi:10.1002/bimj.200800207 · Cited by 24 · see citing works
ProbCD: enrichment analysis accounting for categorization uncertainty
Vêncio, R. Z. N., & Shmulevich, I. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-383 · Cited by 22 · see citing works
Formulating and testing hypotheses in functional genomics
Dopazo, J. (2008) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2008.08.003 · Cited by 22 · see citing works
Comments on: Augmenting the bootstrap to analyze high dimensional genomic data
Strimmer, K. (2008) · Test · doi:10.1007/s11749-008-0101-2 · Cited by 9 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Ackermann M A comparison of statistical methods for gene set enrichment analysis Diploma thesis 2008 Department of Statistics, Technische Universität Dortmund
Cited by
Sorted by citations
Psychiatric genome-wide association study analyses implicate neuronal, immune and histone pathways
The Network and Pathway Analysis Subgroup of the Psychiatric Genomics Consortium (2015) · Nature Neuroscience · doi:10.1038/nn.3922 · Cited by 755 · see citing works
Leeman, D. S., Hebestreit, K., Ruetz, T., Webb, A. E., McKay, A., Pollina, E. A., Dulken, B. W., Zhao, X., Yeo, R. W., Ho, T. T., et al. (2018) · Science · doi:10.1126/science.aag3048 · Cited by 528 · see citing works
ROAST: rotation gene set tests for complex microarray experiments
Wu, D., Lim, E., Vaillant, F., Asselin-Labat, M.-L., Visvader, J. E., & Smyth, G. K. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq401 · Cited by 560 · see citing works
Heterogeneity in old fibroblasts is linked to variability in reprogramming and wound healing
Mahmoudi, S., Mancini, E., Xu, L., Moore, A., Jahanbani, F., Hebestreit, K., Srinivasan, R., Li, X., Devarajan, K., Prélot, L., et al. (2019) · Nature · doi:10.1038/s41586-019-1658-5 · Cited by 285 · see citing works
Gene set analysis of genome-wide association studies: Methodological issues and perspectives
Wang, L., Jia, P., Wolfinger, R. D., Chen, X., & Zhao, Z. (2011) · Genomics · doi:10.1016/j.ygeno.2011.04.006 · Cited by 207 · see citing works
Rahmatallah, Y., Emmert-Streib, F., & Glazko, G. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btt687 · Cited by 110 · see citing works
Footprint-based functional analysis of multiomic data
Dugourd, A., & Saez-Rodriguez, J. (2019) · Current Opinion in Systems Biology · doi:10.1016/j.coisb.2019.04.002 · Cited by 53 · see citing works
Vivar, J. C., Pemu, P., McPherson, R., & Ghosh, S. (2013) · OMICS: A Journal of Integrative Biology · doi:10.1089/omi.2012.0083 · Cited by 44 · see citing works
A Network Approach to Wound Healing
Arodz, T., Bonchev, D., & Diegelmann, R. F. (2013) · Advances in Wound Care · doi:10.1089/wound.2012.0386 · Cited by 22 · see citing works
<scp>PSEA</scp>: Phenotype Set Enrichment Analysis—A New Method for Analysis of Multiple Phenotypes
Ried, J. S., Döring, A., Oexle, K., Meisinger, C., Winkelmann, J., Klopp, N., Meitinger, T., Peters, A., Suhre, K., Wichmann, H. ‐., et al. (2012) · Genetic Epidemiology · doi:10.1002/gepi.21617 · Cited by 21 · see citing works
- Authors
- Ackermann, M., & Strimmer, K.
- Affiliations
- Biotechnologisches Zentrum (DE)
- Leipzig University (DE)
- Corresponding author
- Strimmer, Korbinian
- ORCID
- Strimmer, Korbinian
- Venue
- BMC Bioinformatics
- Published
- 2009-02-03
- Volume
- 10
- Issue
- 1
- Pages
- 47-47
- ISSN
- 1471-2105 (ISSN-L)
- PMC
- PMC2661051
- OpenAlex
- W1983463159
- Cited by
- 358
- References
- 55
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 8 · 2023: 10 · 2022: 18 · 2021: 21 · 2020: 9 · 2019: 23 · 2018: 17 · 2017: 22 · 2016: 23 · 2015: 31 · 2014: 26 · 2013: 35 · 2012: 37
- FWCI
- 13.54
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Bioinformatics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Research Explorer (The University of Manchester) (submittedVersion)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
- Keywords
- gene set enrichment analysis, modular framework, metatheory, comparison of statistical methods, microarray data, high-throughput genomic data, method benchmarking, best practices
- MeSH terms
- Algorithms, Animals, Computer Simulation, Humans, Models, Statistical, Oligonucleotide Array Sequence Analysis, Databases, Genetic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
