CoChat

Predictability of human differential gene expression

Open access
Crow, Megan, Lim, Nathaniel, Ballouz, Sara, Pavlidis, Paul, Gillis, Jesse · 2019 · Proceedings of the National Academy of Sciences

The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index.

Abstract

Differential expression (DE) is commonly used to explore molecular mechanisms of biological conditions. While many studies report significant results between their groups of interest, the degree to which results are specific to the question at hand is not generally assessed, potentially leading to inaccurate interpretation. This could be particularly problematic for metaanalysis where replicability across datasets is taken as strong evidence for the existence of a specific, biologically relevant signal, but which instead may arise from recurrence of generic processes. To address this, we developed an approach to predict DE based on an analysis of over 600 studies. A predictor based on empirical prior probability of DE performs very well at this task (mean area under the receiver operating characteristic curve, ∼0.8), indicating that a large fraction of DE hit lists are nonspecific. In contrast, predictors based on attributes such as gene function, mutation rates, or network features perform poorly. Genes associated with sex, the extracellular matrix, the immune system, and stress responses are prominent within the “DE prior.” In a series of control studies, we show that these patterns reflect shared biology rather than technical artifacts or ascertainment biases. Finally, we demonstrate the application of the DE prior to data interpretation in three use cases: ( i ) breast cancer subtyping, ( ii ) single-cell genomics of pancreatic islet cells, and ( iii ) metaanalysis of lung adenocarcinoma and renal transplant rejection transcriptomics. In all cases, we find hallmarks of generic DE, highlighting the need for nuanced interpretation of gene phenotypic associations.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1073/pnas.1802973116

Cites

94 of 94 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1. Data workflow and description. (A) Database filtering workflow. (B) Plot indicates the number of datasets per experimental category included in this assessment. Experiments ranged widely in their goals and design. (C) Histogram of the number of differentially expressed genes per dataset. Genes are considered to be differentially expressed if they have an absolute log2 fold change of >2 at FDR of <0.05. The red line indicates the median. (D) Histogram of the number of samples per dataset. The red line indicates the median.
Fig. 1. Data workflow and description. (A) Database filtering workflow. (B) Plot indicates the number of datasets per experimental category included in this assessment. Experiments ranged widely in their goals and design. (C) Histogram of the number of differentially expressed genes per dataset. Genes are considered to be differentially expressed if they have an absolute log2 fold change of >2 at FDR of <0.05. The red line indicates the median. (D) Histogram of the number of samples per dataset. The red line indicates the median.CC BY-NC-ND · PubMed Central
Fig. 2. The global DE prior accurately predicts DE hit lists. (A) Recurrence of differentially expressed genes across datasets. The red line indicates the mean. On average, each gene is DE in 10 expression studies. However, the distribution has a long, right-sided tail, indicating a small number of genes that are frequently DE. (B) Distribution of AUROC scores using the global DE prior to predict hit lists across the 635 studies. The red line indicates the mean, and the dashed line indicates the null (0.5). On average, the prior has very high performance, distinguishing ∼80% of DE genes within each hit list, reflecting shared transcriptional features between studies.
Fig. 2. The global DE prior accurately predicts DE hit lists. (A) Recurrence of differentially expressed genes across datasets. The red line indicates the mean. On average, each gene is DE in 10 expression studies. However, the distribution has a long, right-sided tail, indicating a small number of genes that are frequently DE. (B) Distribution of AUROC scores using the global DE prior to predict hit lists across the 635 studies. The red line indicates the mean, and the dashed line indicates the null (0.5). On average, the prior has very high performance, distinguishing ∼80% of DE genes within each hit list, reflecting shared transcriptional features between studies.CC BY-NC-ND · PubMed Central
Fig. 3. Multiple independent gene processes underlie the high performance of the DE prior. (A) Heatmap of AUROCs for each gene property-related prior (rows) and each study in the compendium (columns). Row colors indicate the gene property type. No single property performs as well as the DE prior. However, a few studies are clearly predicted by a number of features, including sex-related studies and immunity-related studies (both highlighted with white boxes). These groups are well predicted by expression level in males and expression level in blood, respectively. (B) Network visualization of genes in the top 1% of the DE prior. Each circle represents a gene, and edges connect all genes that are nearest neighbors based on their coexpression in an aggregate network. Genes are colored by cluster and labeled by GO enrichment. We find that genes in the top 1% of the prior show strong coexpression patterns, representing distinct functional modules that are commonly repurposed across many different conditions. ExAC, Exome Aggregation Consortium; GO, Gene Ontology; GTEx, Genotype-Tissue Expression project; KEGG, Kyoto Encyclopedia of Genes and Genomes; LCLs, lymphoblastoid cell lines; pLI, probability of intolerance to a single loss-of-function variant; pNull, probability of tolerance to loss-of-function variation; pRec, probability of intolerance of two loss-of-function variants.
Fig. 3. Multiple independent gene processes underlie the high performance of the DE prior. (A) Heatmap of AUROCs for each gene property-related prior (rows) and each study in the compendium (columns). Row colors indicate the gene property type. No single property performs as well as the DE prior. However, a few studies are clearly predicted by a number of features, including sex-related studies and immunity-related studies (both highlighted with white boxes). These groups are well predicted by expression level in males and expression level in blood, respectively. (B) Network visualization of genes in the top 1% of the DE prior. Each circle represents a gene, and edges connect all genes that are nearest neighbors based on their coexpression in an aggregate network. Genes are colored by cluster and labeled by GO enrichment. We find that genes in the top 1% of the prior show strong coexpression patterns, representing distinct functional modules that are commonly repurposed across many different conditions. ExAC, Exome Aggregation Consortium; GO, Gene Ontology; GTEx, Genotype-Tissue Expression project; KEGG, Kyoto Encyclopedia of Genes and Genomes; LCLs, lymphoblastoid cell lines; pLI, probability of intolerance to a single loss-of-function variant; pNull, probability of tolerance to loss-of-function variation; pRec, probability of intolerance of two loss-of-function variants.CC BY-NC-ND · PubMed Central
Fig. 4. The global DE prior almost always outperforms targeted approaches to hit list prediction. (A) DE prior performance is plotted with respect to agency-specific prior performance. Each point represents an individual study, and studies are colored by their funding source, as in the boxplot in B. The identity line is shown in black. Hit lists are generally much better predicted by the global DE prior than by priors specified by funding source. (B) Difference in performance (global − agency). Boxplots show the quartiles, with whiskers extending to 1.5× the interquartile range. The majority of studies are better predicted by the DE prior, although there are a few exceptions among NIGMS funded studies (details in Dataset S1). (C) Distribution of AUROCs using funding agency-specific priors. The red line indicates the mean, and the dashed line shows the null. Mean performance is much lower than for the global prior. (D) Heatmap of enrichment for DE prior gene clusters (Fig. 3B). Rows indicate cluster labels, and columns are expression studies; colors indicate funding agency as in B. If the gene set is among the top 1% of enriched functions compared with all of the GO, the square is colored in black; otherwise, squares are gray. The majority of studies have enrichment for at least one of the clusters. Studies do not clearly group by funding agency. (E) Distribution of AUROCs using priors obtained after grouping hit lists into smaller subsets. Lines are as in C. (F) Distribution of AUROCs after selecting the maximally performing gene set from GO. Lines are as in C. NCI, National Cancer Institute; NCRR, National Center for Research Resources; NHLBI, National Heart, Lung and Blood Institute; NIAID, National Institute of Allergy and Infectious Diseases; NICHD, National Institute of Child Health and Human Development; NIDDK, National Institute of Diabetes and Digestive and Kidney Diseases; NIGMS, National Institute of General Medical Sciences; NINDS, National Institute of Neurological Disorders and Stroke.
Fig. 4. The global DE prior almost always outperforms targeted approaches to hit list prediction. (A) DE prior performance is plotted with respect to agency-specific prior performance. Each point represents an individual study, and studies are colored by their funding source, as in the boxplot in B. The identity line is shown in black. Hit lists are generally much better predicted by the global DE prior than by priors specified by funding source. (B) Difference in performance (global − agency). Boxplots show the quartiles, with whiskers extending to 1.5× the interquartile range. The majority of studies are better predicted by the DE prior, although there are a few exceptions among NIGMS funded studies (details in Dataset S1). (C) Distribution of AUROCs using funding agency-specific priors. The red line indicates the mean, and the dashed line shows the null. Mean performance is much lower than for the global prior. (D) Heatmap of enrichment for DE prior gene clusters (Fig. 3B). Rows indicate cluster labels, and columns are expression studies; colors indicate funding agency as in B. If the gene set is among the top 1% of enriched functions compared with all of the GO, the square is colored in black; otherwise, squares are gray. The majority of studies have enrichment for at least one of the clusters. Studies do not clearly group by funding agency. (E) Distribution of AUROCs using priors obtained after grouping hit lists into smaller subsets. Lines are as in C. (F) Distribution of AUROCs after selecting the maximally performing gene set from GO. Lines are as in C. NCI, National Cancer Institute; NCRR, National Center for Research Resources; NHLBI, National Heart, Lung and Blood Institute; NIAID, National Institute of Allergy and Infectious Diseases; NICHD, National Institute of Child Health and Human Development; NIDDK, National Institute of Diabetes and Digestive and Kidney Diseases; NIGMS, National Institute of General Medical Sciences; NINDS, National Institute of Neurological Disorders and Stroke.CC BY-NC-ND · PubMed Central
Fig. 5. Reinterpreting gene sets with the DE prior. (A) PAM50 genes and housekeepers are plotted with respect to their rank in the DE prior, and each point represents a single gene (0, never differentially expressed; 1, frequently differentially expressed). As expected, PAM50 genes have high DE prior ranks (mean, 0.9), whereas housekeepers are not frequently DE and have low DE prior ranks (mean, 0.35). (B) Alpha and beta cell markers derived from single-cell RNA-seq are plotted with respect to their DE prior rank, and their recurrence among five independent hit lists. Increasing recurrence is associated with higher DE ranks. Differences in prior ranks between highly recurrent genes suggests differential condition specificity. (C) DE genes from metaanalysis of 13 renal transplant rejection studies are plotted with respect to their DE prior rank and their recurrence. The only genes that recur among most studies are all very frequently DE (high DE prior ranks), and thus unlikely to be specifically associated with renal transplant rejection. (D) DE genes from metaanalysis of 13 lung adenocarcinoma studies are plotted with respect to their DE prior rank and their recurrence. RAGE, a known biomarker, has a relatively low DE prior rank.
Fig. 5. Reinterpreting gene sets with the DE prior. (A) PAM50 genes and housekeepers are plotted with respect to their rank in the DE prior, and each point represents a single gene (0, never differentially expressed; 1, frequently differentially expressed). As expected, PAM50 genes have high DE prior ranks (mean, 0.9), whereas housekeepers are not frequently DE and have low DE prior ranks (mean, 0.35). (B) Alpha and beta cell markers derived from single-cell RNA-seq are plotted with respect to their DE prior rank, and their recurrence among five independent hit lists. Increasing recurrence is associated with higher DE ranks. Differences in prior ranks between highly recurrent genes suggests differential condition specificity. (C) DE genes from metaanalysis of 13 renal transplant rejection studies are plotted with respect to their DE prior rank and their recurrence. The only genes that recur among most studies are all very frequently DE (high DE prior ranks), and thus unlikely to be specifically associated with renal transplant rejection. (D) DE genes from metaanalysis of 13 lung adenocarcinoma studies are plotted with respect to their DE prior rank and their recurrence. RAGE, a known biomarker, has a relatively low DE prior rank.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 94 references

Cited by

See all 130

Metadata

Authors
Crow, M., Lim, N., Ballouz, S., Pavlidis, P., & Gillis, J.
Affiliations
  • Cold Spring Harbor Laboratory (US)
  • University of British Columbia (CA)
Corresponding author
Gillis, Jesse
Venue
Proceedings of the National Academy of Sciences
Published
2019-03-07
Volume
116
Issue
13
Pages
6491-6500
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
OpenAlex
W2921170286
Cited by
117
References
92
Citations by year
2026: 2 · 2025: 9 · 2024: 13 · 2023: 15 · 2022: 22 · 2021: 29 · 2020: 24 · 2019: 3
FWCI
6.45
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • Cold Spring Harbor Laboratory Institutional Repository (Cold Spring Harbor Laboratory) (acceptedVersion)
  • UNSWorks (University of New South Wales, Sydney, Australia) (submittedVersion, cc-by-nc-nd)
Funders
  • HHS | NIH | National Institute of Mental Health (F32MH114501, R01MH113005, R01MH111099)
  • Brain and Behavior Research Foundation (26140)
  • HHS | NIH | U.S. National Library of Medicine (R01LM012736)
Topics
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
Keywords
differential gene expression, ascertainment bias, extracellular matrix, immune system genes, stress response genes, breast cancer subtyping, lung adenocarcinoma, renal allograft rejection
MeSH terms
Human Genetics, Adenocarcinoma, Electronic Data Processing, Breast Neoplasms, Female, Gene Expression Regulation, Graft Rejection, Humans, Lung Neoplasms, Probability, Recurrence, ROC Curve, Sensitivity and Specificity, Biomarkers, Tumor, Kidney Transplantation, Genes, Essential, Gene Expression Profiling, Genomics, Gene Regulatory Networks, Transcriptome

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Predictability of human differential gene expression. https://app.cochat.ai/papers/10.1073/pnas.1802973116
The paper
Crow, M., Lim, N., Ballouz, S., Pavlidis, P., & Gillis, J. (2019). Predictability of human differential gene expression. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.1802973116

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.