CoChat
Sign Up
Cited by 224Bioinformatics and Genomic NetworksBiomedical Text Mining and OntologiesGene expression and cancer classification

The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.

A Comparison of Gene Set Analysis Methods in Terms of Sensitivity, Prioritization and Specificity

Open access
Tarca, Adi L., Bhatti, Gaurav, Romero, Roberto·2013·PLoS ONE
DOI 10.1371/journal.pone.0079217PMC PMC3829842
Open this paper in CoChat — free account

Abstract

Identification of functional sets of genes associated with conditions of interest from omics data was first reported in 1999, and since, a plethora of enrichment methods were published for systematic analysis of gene sets collections including Gene Ontology and biological pathways. Despite their widespread usage in reducing the complexity of omics experiment results, their performance is poorly understood. Leveraging the existence of disease specific gene sets in KEGG and Metacore® databases, we compared the performance of sixteen methods under relaxed assumptions while using 42 real datasets (over 1,400 samples). Most of the methods ranked high the gene sets designed for specific diseases whenever samples from affected individuals were compared against controls via microarrays. The top methods for gene set prioritization were different from the top ones in terms of sensitivity, and four of the sixteen methods had large false positives rates assessed by permuting the phenotype of the samples. The best overall methods among those that generated reasonably low false positive rates, when permuting phenotypes, were PLAGE, GLOBALTEST, and PADOG. The best method in the category that generated higher than expected false positives was MRGSE.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0079217

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Procedure used to compare 16 gene set analysis methods. 42 microarray datasets were used, each studying a phenotype that has a corresponding KEGG or Metacore disease pathway, that we call target pathway. Each method was applied on each datasets and the p-value and rank of the target pathway in each dataset was used to compare the methods.
Figure 1 Procedure used to compare 16 gene set analysis methods. 42 microarray datasets were used, each studying a phenotype that has a corresponding KEGG or Metacore disease pathway, that we call target pathway. Each method was applied on each datasets and the p-value and rank of the target pathway in each dataset was used to compare the methods.CC0 · PubMed Central
Figure 2 A comparison of sensitivity and prioritization ability of 16 gene set analysis methods. Each box contains 42 data points representing the p-value (left) and the rank (%) (right) that the target pathway received from a given method when using as input an independent dataset and a collection of gene sets (either KEGG or Metacore). Since the target pathways were designed by KEGG and Metacore for those diseases we expected that, in average, they will be found relevant by the different methods. Methods are ranked from best to worst according to the median p-value (left) and median rank (right).
Figure 2 A comparison of sensitivity and prioritization ability of 16 gene set analysis methods. Each box contains 42 data points representing the p-value (left) and the rank (%) (right) that the target pathway received from a given method when using as input an independent dataset and a collection of gene sets (either KEGG or Metacore). Since the target pathways were designed by KEGG and Metacore for those diseases we expected that, in average, they will be found relevant by the different methods. Methods are ranked from best to worst according to the median p-value (left) and median rank (right).CC0 · PubMed Central
Figure 3 False positive rates produced by 16 gene sets analysis methods. The null hypothesis is simulated by analyzing phenotype permuted versions of the original datasets. The percentage of all pathways found significant at different significance levels (α) is reported for each method with a vertical bar. The horizontal lines denote the expected levels of false positives at each α level. Note the logarithmic scale.
Figure 3 False positive rates produced by 16 gene sets analysis methods. The null hypothesis is simulated by analyzing phenotype permuted versions of the original datasets. The percentage of all pathways found significant at different significance levels (α) is reported for each method with a vertical bar. The horizontal lines denote the expected levels of false positives at each α level. Note the logarithmic scale.CC0 · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 34 references

Research works printed but not linked: 2

  1. S Draghici, 2003, Global functional profiling of gene expression, Genomics, 81, 98
  2. VK Mootha, 2003, PGC-1alpha-responsive genes involved in oxidative phosphorylation are coordinately downregulated in human diabetes, Nat Genet, 34, 267, 10.1038/ng1180 · DOI · Paper page

Cited by

Sorted by citations

Related

See all

Metadata

Authors
Tarca, A. L., Bhatti, G., & Romero, R.
Affiliations
National Institutes of Health (US)
Corresponding author
Tarca, Adi L.
ORCID
Venue
PLoS ONE
Publisher
Public Library of Science (PLoS)
Published
2013-11-15
Volume
8
Issue
11
Pages
e79217
ISSN
1932-6203 (ISSN-L)
DOI
10.1371/journal.pone.0079217
PMC
PMC3829842
OpenAlex
W2084533385
Cited by
203
References
34
Citations by year
2026: 9 · 2025: 13 · 2024: 21 · 2023: 13 · 2022: 16 · 2021: 11 · 2020: 28 · 2019: 19 · 2018: 12 · 2017: 17 · 2016: 20 · 2015: 19 · 2014: 5
FWCI
6.57
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS ONE (publishedVersion, public-domain)
  • Europe PMC (PubMed Central) (publishedVersion, public-domain)
  • PubMed Central (publishedVersion, public-domain)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
Funders
  • U.S. Department of Health and Human Services
  • National Institutes of Health
  • National Institute of Child Health and Human Development
  • Eunice Kennedy Shriver National Institute of Child Health and Human Development
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
  • Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
Keywords
gene set enrichment analysis, method comparison, sensitivity and specificity, false positive rate, KEGG, plage, microarray data, omics data interpretation
MeSH terms
Electronic Data Processing, Humans, Oligonucleotide Array Sequence Analysis, Databases, Genetic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review