The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Gene set enrichment analysis: performance evaluation and usage guidelines
Open accessAbstract
A central goal of biology is understanding and describing the molecular basis of plasticity: the sets of genes that are combinatorially selected by exogenous and endogenous environmental changes, and the relations among the genes. The most viable current approach to this problem consists of determining whether sets of genes are connected by some common theme, e.g. genes from the same pathway are overrepresented among those whose differential expression in response to a perturbation is most pronounced. There are many approaches to this problem, and the results they produce show a fair amount of dispersion, but they all fall within a common framework consisting of a few basic components. We critically review these components, suggest best practices for carrying out each step, and propose a voting method for meeting the challenge of assessing different methods on a large number of experimental data sets in the absence of a gold standard.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506580102 · Cited by 53,625
Huang, D. W., Sherman, B. T., & Lempicki, R. A. (2009) · Nucleic Acids Research · doi:10.1093/nar/gkn923 · Cited by 14,108
Mapping and quantifying mammalian transcriptomes by RNA-Seq
Mortazavi, A., Williams, B. A., McCue, K., Schaeffer, L., & Wold, B. (2008) · Nature Methods · doi:10.1038/nmeth.1226 · Cited by 13,696
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
Exploration, normalization, and summaries of high density oligonucleotide array probe level data
Irizarry, R. A. (2003) · Biostatistics · doi:10.1093/biostatistics/4.2.249 · Cited by 10,603
Statistical significance for genomewide studies
Storey, J. D., & Tibshirani, R. (2003) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1530509100 · Cited by 9,692
Summaries of Affymetrix GeneChip probe level data
Irizarry, R. A. (2003) · Nucleic Acids Research · doi:10.1093/nar/gng015 · Cited by 4,887
KAAS: an automatic genome annotation and pathway reconstruction server
Moriya, Y., Itoh, M., Okuda, S., Yoshizawa, A. C., & Kanehisa, M. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkm321 · Cited by 4,153
Controlling the false discovery rate in behavior genetics research
Benjamini, Y., Drai, D., Elmer, G., Kafkafi, N., & Golani, I. (2001) · Behavioural Brain Research · doi:10.1016/s0166-4328(01)00297-2 · Cited by 4,146
The positive false discovery rate: a Bayesian interpretation and the q-value
Storey, J. D. (2003) · The Annals of Statistics · doi:10.1214/aos/1074290335 · Cited by 2,204
Shaffer, J. P. (1995) · Annual Review of Psychology · doi:10.1146/annurev.ps.46.020195.003021 · Cited by 1,960
A systems biology approach for pathway level analysis
Draghici, S., Khatri, P., Tarca, A. L., Amin, K., Done, A., Voichita, C., Georgescu, C., & Romero, R. (2007) · Genome Research · doi:10.1101/gr.6202607 · Cited by 1,249
RNA-Seq gene expression estimation with read mapping uncertainty
Li, B., Ruotti, V., Stewart, R. M., Thomson, J. A., & Dewey, C. N. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btp692 · Cited by 1,198
A novel signaling pathway impact analysis
Tarca, A. L., Draghici, S., Khatri, P., Hassan, S. S., Mittal, P., Kim, J.-S., Kim, C. J., Kusanovic, J. P., & Romero, R. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btn577 · Cited by 1,031
Calibration of ρ Values for Testing Precise Null Hypotheses
Sellke, T., Bayarri, M. J., & Berger, J. O. (2001) · The American Statistician · doi:10.1198/000313001300339950 · Cited by 854
Ontological analysis of gene expression data: current tools, limitations, and open problems
Khatri, P., & Drăghici, S. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti565 · Cited by 854
On testing the significance of sets of genes
Efron, B., & Tibshirani, R. (2007) · The Annals of Applied Statistics · doi:10.1214/07-aoas101 · Cited by 798
PAGE: Parametric Analysis of Gene Set Enrichment
Kim, S.-Y., & Volsky, D. J. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-144 · Cited by 779
A Parsimony Approach to Biological Pathway Reconstruction/Inference for Genomes and Metagenomes
Ye, Y., & Doak, T. G. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000465 · Cited by 686
Discovering statistically significant pathways in expression profiling studies
Tian, L., Greenberg, S. A., Kong, S. W., Altschuler, J., Kohane, I. S., & Park, P. J. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506577102 · Cited by 648
Quantitative Proteomics Analysis of the Secretory Pathway
Gilchrist, A., Au, C. E., Hiding, J., Bell, A. W., Fernandez-Rodriguez, J., Lesimple, S., Nagaya, H., Roy, L., Gosline, S. J. C., Hallett, M., et al. (2006) · Cell · doi:10.1016/j.cell.2006.10.036 · Cited by 458
Transcriptional regulation and function during the human cell cycle
Cho, R. J., Huang, M., Campbell, M. J., Dong, H., Steinmetz, L., Sapinoso, L., Hampton, G., Elledge, S. J., Davis, R. W., & Lockhart, D. J. (2001) · Nature Genetics · doi:10.1038/83751 · Cited by 455
Proteomic survey of metabolic pathways in rice
Koller, A., Washburn, M. P., Lange, B. M., Andon, N. L., Deciu, C., Haynes, P. A., Hays, L., Schieltz, D., Ulaszek, R., Wei, J., et al. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.172183199 · Cited by 409
The Folded Normal Distribution
Leone, F. C., Nelson, L. S., & Nottingham, R. B. (1961) · Technometrics · doi:10.1080/00401706.1961.10489974 · Cited by 408
A general modular framework for gene set enrichment analysis
Ackermann, M., & Strimmer, K. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-47 · Cited by 358
Barry, W. T., Nobel, A. B., & Wright, F. A. (2005) · Bioinformatics · doi:10.1093/bioinformatics/bti260 · Cited by 356
LSimpute: accurate estimation of missing values in microarray data with least squares methods
Bo, T. H. (2004) · Nucleic Acids Research · doi:10.1093/nar/gnh026 · Cited by 348
Gene-set approach for expression pattern analysis
Nam, D., & Kim, S.-Y. (2008) · Briefings in Bioinformatics · doi:10.1093/bib/bbn001 · Cited by 338
Human endometriosis is associated with plasma cells and overexpression of B lymphocyte stimulator
Hever, A., Roth, R. B., Hevezi, P., Marin, M. E., Acosta, J. A., Acosta, H., Rojas, J., Herrera, R., Grigoriadis, D., White, E., et al. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0703451104 · Cited by 254
Hong, F., & Breitling, R. (2008) · Bioinformatics · doi:10.1093/bioinformatics/btm620 · Cited by 240
Extensions to gene set enrichment
Jiang, Z., & Gentleman, R. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btl599 · Cited by 217
Gene set enrichment analysis made simple
Irizarry, R. A., Wang, C., Zhou, Y., & Speed, T. P. (2009) · Statistical Methods in Medical Research · doi:10.1177/0962280209351908 · Cited by 194
Harbig, J. (2005) · Nucleic Acids Research · doi:10.1093/nar/gni027 · Cited by 141
A multivariate approach for integrating genome-wide expression data and biological knowledge
Kong, S. W., Pu, W. T., & Park, P. J. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl401 · Cited by 132
Calculating the Statistical Significance of Changes in Pathway Activity From Gene Expression Data
Rahnenführer, J., Domingues, F. S., Maydt, J., & Lengauer, T. (2004) · Statistical Applications in Genetics and Molecular Biology · doi:10.2202/1544-6115.1055 · Cited by 125
Brock, G. N., Shaffer, J. R., Blakesley, R. E., Lotz, M., & Tseng, G. C. (2008) · BMC Bioinformatics · doi:10.1186/1471-2105-9-12 · Cited by 120
Celton, M., Malpertuy, A., Lelandais, G., & de Brevern, A. G. (2010) · BMC Genomics · doi:10.1186/1471-2164-11-15 · Cited by 98
Gene-set analysis and reduction
Dinu, I. A., Potter, J. D., Mueller, T., Liu, Q., Adewale, A. J., Jhangri, G. S., Einecke, G., Famulski, K. S., Halloran, P. F., & Yasui, Y. (2008) · Briefings in Bioinformatics · doi:10.1093/bib/bbn042 · Cited by 91
Faulkner, G. J., Forrest, A. R. R., Chalk, A. M., Schroder, K., Hayashizaki, Y., Carninci, P., Hume, D. A., & Grimmond, S. M. (2008) · Genomics · doi:10.1016/j.ygeno.2007.11.003 · Cited by 82
Kitami, T., & Nadeau, J. H. (2002) · Nature Genetics · doi:10.1038/ng945 · Cited by 77
Hung, J., Whitfield, T. W., Yang, T.-H., Hu, Z., Weng, Z., & DeLisi, C. (2010) · Genome biology · doi:10.1186/gb-2010-11-2-r23 · Cited by 72
Self-citationNormalization and quantification of differential expression in gene expression microarrays
Steinhoff, C., & Vingron, M. (2006) · Briefings in Bioinformatics · doi:10.1093/bib/bbl002 · Cited by 68
Absolute enrichment: gene set enrichment analysis for homeostatic systems
Saxena, V., Orgill, D. P., & Kohane, I. S. (2006) · Nucleic Acids Research · doi:10.1093/nar/gkl766 · Cited by 58
Computation of significance scores of unweighted Gene Set Enrichment Analyses
Keller, A., Backes, C., & Lenhof, H. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-290 · Cited by 55
Nonparametric Bayesian Estimation of Positive False Discovery Rates
Tang, Y., Ghosal, S., & Roy, A. (2007) · Biometrics · doi:10.1111/j.1541-0420.2007.00819.x · Cited by 51
Meta-analytic methods, the Rorschach, and the MMPI.
Rosenthal, R., Hiller, J. B., Bornstein, R. F., Berry, D. T. R., & Brunell-Neuleib, S. (2001) · Psychological Assessment · doi:10.1037//1040-3590.13.4.449 · Cited by 48
Scheer, M. P., Klawonn, F., Münch, R., Grote, A., Hiller, K., Choi, C., Koch, I., Schobert, M., Härtig, E., Klages, U., et al. (2006) · Nucleic Acids Research · doi:10.1093/nar/gkl329 · Cited by 32
ADAPT: a database of affymetrix probesets and transcripts
Leong, H. S., Yates, T., Wilson, C. L., & Miller, C. J. (2005) · Computer applications in the biosciences · doi:10.1093/bioinformatics/bti359 · Cited by 31
Towards the uniform distribution of null P values on Affymetrix microarrays
Fodor, A. A., Tickle, T. L., & Richardson, C. A. (2007) · Genome biology · doi:10.1186/gb-2007-8-5-r69 · Cited by 28
Improving the scaling normalization for high-density oligonucleotide GeneChip expression microarrays
Lu, C. (2004) · BMC Bioinformatics · doi:10.1186/1471-2105-5-103 · Cited by 19
Normalization and Gene p-Value Estimation: Issues in Microarray Data Processing
Fundel, K., Küffner, R., Aigner, T. G., & Zimmer, R. (2008) · Bioinformatics and Biology Insights · doi:10.4137/bbi.s441 · Cited by 18
Cited by
GSVA: gene set variation analysis for microarray and RNA-Seq data
Hänzelmann, S., Castelo, R., & Guinney, J. (2013) · BMC Bioinformatics · doi:10.1186/1471-2105-14-7 · Cited by 15,440
Enrichr: a comprehensive gene set enrichment analysis web server 2016 update
Kuleshov, M. V., Jones, M. R., Rouillard, A. D., Fernandez, N. F., Duan, Q., Wang, Z., Koplev, S., Jenkins, S. L., Jagodnik, K. M., Lachmann, A., et al. (2016) · Nucleic Acids Research · doi:10.1093/nar/gkw377 · Cited by 10,799
Väremo, L., Nielsen, J., & Nookaew, I. (2013) · Nucleic Acids Research · doi:10.1093/nar/gkt111 · Cited by 697
A network-biology perspective of microRNA function and dysfunction in cancer
Bracken, C. P., Scott, H. S., & Goodall, G. J. (2016) · Nature Reviews Genetics · doi:10.1038/nrg.2016.134 · Cited by 668
Holland, C. H., Tanevski, J., Perales-Patón, J., Gleixner, J., Kumar, M. P., Mereu, E., Joughin, B. A., Stegle, O., Lauffenburger, D. A., Heyn, H., et al. (2020) · Genome Biology · doi:10.1186/s13059-020-1949-z · Cited by 323
Metadata
- Authors
- Hung, J.-H., Yang, T.-H., Hu, Z., Weng, Z., & DeLisi, C.
- Venue
- Briefings in Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2011-09-07
- Volume
- 13
- Issue
- 3
- Pages
- 281-291
- ISSN
- 1467-5463 (ISSN-L)
- 1477-4054
- OpenAlex
- W2027895157
- Cited by
- 266
- References
- 52
- FWCI
- 5.29
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.