The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
A Comparison of Gene Set Analysis Methods in Terms of Sensitivity, Prioritization and Specificity
Open accessAbstract
Identification of functional sets of genes associated with conditions of interest from omics data was first reported in 1999, and since, a plethora of enrichment methods were published for systematic analysis of gene sets collections including Gene Ontology and biological pathways. Despite their widespread usage in reducing the complexity of omics experiment results, their performance is poorly understood. Leveraging the existence of disease specific gene sets in KEGG and Metacore® databases, we compared the performance of sixteen methods under relaxed assumptions while using 42 real datasets (over 1,400 samples). Most of the methods ranked high the gene sets designed for specific diseases whenever samples from affected individuals were compared against controls via microarrays. The top methods for gene set prioritization were different from the top ones in terms of sensitivity, and four of the sixteen methods had large false positives rates assessed by permuting the phenotype of the samples. The best overall methods among those that generated reasonably low false positive rates, when permuting phenotypes, were PLAGE, GLOBALTEST, and PADOG. The best method in the category that generated higher than expected false positives was MRGSE.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“These tests have been found to give an effective ranking of biologically significant pathways (54)”
From limma powers differential expression analyses for RNA-sequencing and microarray studies (2015) · cites this paper as [54]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506580102 · Cited by 53,625
Gene Ontology: tool for the unification of biology
Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M., Davis, A. P., Dolinski, K., Dwight, S. S., Eppig, J. T., et al. (2000) · Nature Genetics · doi:10.1038/75556 · Cited by 43,206
KEGG: Kyoto Encyclopedia of Genes and Genomes
Ogata, H., Goto, S., Sato, K., Fujibuchi, W., Bono, H., & Kanehisa, M. (1999) · Nucleic Acids Research · doi:10.1093/nar/27.1.29 · Cited by 31,232
GSVA: gene set variation analysis for microarray and RNA-Seq data
Hänzelmann, S., Castelo, R., & Guinney, J. (2013) · BMC Bioinformatics · doi:10.1186/1471-2105-14-7 · Cited by 15,440
Mootha, V. K., Lindgren, C. M., Eriksson, K.-F., Subramanian, A., Sihag, S., Lehar, J., Puigserver, P., Carlsson, E., Ridderstråle, M., Laurila, E., et al. (2003) · Nature Genetics · doi:10.1038/ng1180 · Cited by 10,197
Quantitative Monitoring of Gene Expression Patterns with a Complementary DNA Microarray
Schena, M., Shalon, D., Davis, R. W., & Brown, P. O. (1995) · Science · doi:10.1126/science.270.5235.467 · Cited by 9,609
limma: Linear Models for Microarray Data
Smyth, G. K. (2005) · Statistics for Biology and Health · doi:10.1007/0-387-29362-0_23 · Cited by 5,796
Systematic RNA interference reveals that oncogenic KRAS-driven cancers require TBK1
Barbie, D. A., Tamayo, P., Boehm, J. S., Kim, S. Y., Moody, S. E., Dunn, I. F., Schinzel, A. C., Sandy, P., Meylan, E., Scholl, C., et al. (2009) · Nature · doi:10.1038/nature08460 · Cited by 3,922
Bioinformatics and Computational Biology Solutions Using R and Bioconductor
Gentleman, R. C., Carey, V. J., Huber, W. K., Irizarry, R. A., & Dudoit, S. (2005) · Statistics in the health sciences · doi:10.1007/0-387-29362-0 · Cited by 3,470
Systematic determination of genetic network architecture
Tavazoie, S., Hughes, J. D., Campbell, M. J., Cho, R. J., & Church, G. M. (1999) · Nature Genetics · doi:10.1038/10343 · Cited by 2,509
Huang, D. W., Sherman, B. T., Tan, Q., Kir, J., Liu, D., Bryant, D., Guo, Y., Stephens, R. M., Baseler, M., Lane, H. C., et al. (2007) · Nucleic Acids Research · doi:10.1093/nar/gkm415 · Cited by 2,216
Using GOstats to test gene lists for GO term association
Falcon, S., & Gentleman, R. C. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl567 · Cited by 2,019
Ten Years of Pathway Analysis: Current Approaches and Outstanding Challenges
Khatri, P., Sirota, M., & Butte, A. J. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002375 · Cited by 1,635
GAGE: generally applicable gene set enrichment for pathway analysis
Luo, W., Friedman, M. S., Shedden, K., Hankenson, K. D., & Woolf, P. J. (2009) · BMC Bioinformatics · doi:10.1186/1471-2105-10-161 · Cited by 1,333
Reactome: a knowledgebase of biological pathways
Joshi-Tope, G. (2004) · Nucleic Acids Research · doi:10.1093/nar/gki072 · Cited by 1,291
A systems biology approach for pathway level analysisPreprint
Drăghici, S., Khatri, P., Tarca, A. L., Amin, K., Done, A., Voichiţa, C., Georgescu, C., & Romero, R. J. (2007) · Genome Research · doi:10.1101/gr.6202607 · Cited by 1,263
Self-citationGOstat: find statistically overrepresented Gene Ontologies within a group of genes
Beißbarth, T., & Speed, T. P. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth088 · Cited by 1,256
A global test for groups of genes: testing association with a clinical outcome
Goeman, J. J., van de Geer, S., de Kort, F., & van Houwelingen, H. C. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg382 · Cited by 1,052
A novel signaling pathway impact analysis
Tarca, A. L., Draghici, S., Khatri, P., Hassan, S. S., Mittal, P., Kim, J.-S., Kim, C. J., Kusanovic, J. P., & Romero, R. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btn577 · Cited by 1,031
Self-citationCamera: a competitive gene set test accounting for inter-gene correlation
Wu, D., & Smyth, G. K. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks461 · Cited by 901
Analyzing gene expression data in terms of gene sets: methodological issues
Goeman, J. J., & Bühlmann, P. (2007) · Bioinformatics · doi:10.1093/bioinformatics/btm051 · Cited by 809
On testing the significance of sets of genes
Efron, B., & Tibshirani, R. J. (2007) · The Annals of Applied Statistics · doi:10.1214/07-aoas101 · Cited by 785
Machine learning and its applications to biology
Tarca, A. L., Carey, V. J., Chen, X.-W., Romero, R., & Drăghici, S. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030116 · Cited by 656
Self-citationDiscovering statistically significant pathways in expression profiling studies
Tian, L., Greenberg, S. A., Kong, S. W., Altschuler, J., Kohane, I. S., & Park, P. J. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506577102 · Cited by 649
Inferring Pathway Activity toward Precise Disease Classification
Lee, E., Chuang, H., Kim, J., Ideker, T., & Lee, D. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000217 · Cited by 603
Drăghici, S., Khatri, P., Martins, R. P., Ostermeier, G. C., & Krawetz, S. A. (2003) · Genomics · doi:10.1016/s0888-7543(02)00021-6 · Cited by 584
Profiling Gene Expression Using Onto-Express
Khatri, P., Drăghici, S., Ostermeier, G. C., & Krawetz, S. A. (2002) · Genomics · doi:10.1006/geno.2002.6698 · Cited by 459
Pathway level analysis of gene expression using singular value decomposition
Tomfohr, J. K., Lü, J., & Kepler, T. B. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-225 · Cited by 414
Barry, W. T., Nobel, A. B., & Wright, F. A. (2005) · Computer applications in the biosciences · doi:10.1093/bioinformatics/bti260 · Cited by 353
Gene set analysis methods: statistical models and methodological differences
Maciejewski, H. (2013) · Briefings in Bioinformatics · doi:10.1093/bib/bbt002 · Cited by 164
Down-weighting overlapping genes improves gene set analysis
Tarca, A. L., Drăghici, S., Bhatti, G., & Romero, R. J. (2012) · BMC Bioinformatics · doi:10.1186/1471-2105-13-136 · Cited by 154
Self-citationIntegrative analysis of RUNX1 downstream pathways and target genes
Michaud, J., Simpson, K., Escher, R., Buchet-Poyau, K., Beißbarth, T., Carmichael, C., Ritchie, M. E., Schütz, F., Cannon, P., Liu, M., et al. (2008) · BMC Genomics · doi:10.1186/1471-2164-9-363 · Cited by 138
Tarca, A. L., Lauria, M., Unger, M., Bilal, E., Boué, S., Dey, K. K., Hoeng, J., Koeppl, H., Martin, F. P., Meyer, P., et al. (2013) · Bioinformatics · doi:10.1093/bioinformatics/btt492 · Cited by 121
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- S Draghici, 2003, Global functional profiling of gene expression, Genomics, 81, 98
- VK Mootha, 2003, PGC-1alpha-responsive genes involved in oxidative phosphorylation are coordinately downregulated in human diabetes, Nat Genet, 34, 267, 10.1038/ng1180 · DOI · Paper page
Cited by
Sorted by citations
limma powers differential expression analyses for RNA-sequencing and microarray studies
Ritchie, M. E., Phipson, B., Wu, D., Hu, Y., Law, C. W., Shi, W., & Smyth, G. K. (2015) · Nucleic Acids Research · doi:10.1093/nar/gkv007 · Cited by 40,458
Szklarczyk, D., Gable, A. L., Lyon, D., Junge, A., Wyder, S., Huerta-Cepas, J., Simonovic, M., Doncheva, N. T., Morris, J. H., Bork, P., et al. (2019) · Nucleic Acids Research · doi:10.1093/nar/gky1131 · Cited by 17,749
Current best practices in single-cell RNA-seq analysis: a tutorial
Luecken, M. D., & Theis, F. J. (2019) · Molecular Systems Biology · doi:10.15252/msb.20188746 · Cited by 2,231
ReactomeGSA - Efficient Multi-Omics Comparative Pathway Analysis
Griss, J., Viteri, G., Sidiropoulos, K., Nguyen, V., Fabregat, A., & Hermjakob, H. (2020) · Molecular & Cellular Proteomics · doi:10.1074/mcp.tir120.002155 · Cited by 418
Topic modeling for untargeted substructure exploration in metabolomics
van der Hooft, J. J. J., Wandy, J., Barrett, M. P., Burgess, K. E., & Rogers, S. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1608041113 · Cited by 370
Metadata
- Authors
- Tarca, A. L., Bhatti, G., & Romero, R.
- Affiliations
- National Institutes of Health (US)
- Corresponding author
- Tarca, Adi L.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-11-15
- Volume
- 8
- Issue
- 11
- Pages
- e79217
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3829842
- OpenAlex
- W2084533385
- Cited by
- 203
- References
- 34
- Citations by year
- 2026: 9 · 2025: 13 · 2024: 21 · 2023: 13 · 2022: 16 · 2021: 11 · 2020: 28 · 2019: 19 · 2018: 12 · 2017: 17 · 2016: 20 · 2015: 19 · 2014: 5
- FWCI
- 6.57
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, public-domain)
- Europe PMC (PubMed Central) (publishedVersion, public-domain)
- PubMed Central (publishedVersion, public-domain)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- U.S. Department of Health and Human Services
- National Institutes of Health
- National Institute of Child Health and Human Development
- Eunice Kennedy Shriver National Institute of Child Health and Human Development
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
- Keywords
- gene set enrichment analysis, method comparison, sensitivity and specificity, false positive rate, KEGG, plage, microarray data, omics data interpretation
- MeSH terms
- Electronic Data Processing, Humans, Oligonucleotide Array Sequence Analysis, Databases, Genetic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
