Cited by 260Bioinformatics and Genomic NetworksGene expression and cancer classificationGenomics and Phylogenetic Studies

The DOI matches its Crossref record; 45 of the 47 references it cites were found in a registry or an index; none of the 17 sources its claims cite is checked for support yet.

Guidance for RNA-seq co-expression network construction and analysis: safety in numbers

Open access

Ballouz, S., Verleyen, W., Gillis, J.·2015·Bioinformatics

DOI 10.1093/bioinformatics/btv118

Open this paper in CoChat — free account Open PDF

Abstract

Motivation: RNA-seq co-expression analysis is in its infancy and reasonable practices remain poorly defined. We assessed a variety of RNA-seq expression data to determine factors affecting functional connectivity and topology in co-expression networks. Results: We examine RNA-seq co-expression data generated from 1970 RNA-seq samples using a Guilt-By-Association framework, in which genes are assessed for the tendency of co-expression to reflect shared function. Minimal experimental criteria to obtain performance on par with microarrays were >20 samples with read depth >10 M per sample. While the aggregate network constructed shows good performance (area under the receiver operator characteristic curve ∼0.71), the dependency on number of experiments used is nearly identical to that present in microarrays, suggesting thousands of samples are required to obtain ‘gold-standard’ co-expression. We find a major topological difference between RNA-seq and microarray co-expression in the form of low overlaps between hub-like genes from each network due to changes in the correlation of expression noise within each technology. Contact: jgillis@cshl.edu or sballouz@cshl.edu Supplementary information: Networks are available at: http://gillislab.labsites.cshl.edu/supplements/rna-seq-networks/ and supplementary data are available at Bioinformatics online.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper benchmarks RNA-seq co-expression network construction against microarrays and finds that simple correlation-based networks, especially when aggregated across many datasets, recover known gene function comparably to microarrays. However, RNA-seq and microarray networks differ sharply in topology, largely because low-expression noise creates different hub genes.

Key findings

Individual RNA-seq co-expression networks performed modestly, with an average AUROC of 0.585; only 0.7% of GO terms averaged above 0.8 AUROC, and only 2% ever exceeded 0.9 AUROC in any assessed network.
Aggregating networks improved functional connectivity to about 0.71 AUROC for both RNA-seq and microarray data; roughly 20 experiments were enough for AUROC >0.6, while capturing unknown functional information may require more than 50 experiments.
RNA-seq and microarray aggregate networks had negatively correlated node degrees, and hub-like genes were significantly dissimilar between technologies (P < 1.4e-9), even though both captured individual GO functions similarly.
Low-expressing transcripts that failed SEQC reliability thresholds in at least 80% of experiments had high node degree in microarrays but low node degree in RNA-seq, explaining much of the topological mismatch.

Methods

The authors assembled 50 RNA-seq experiments totaling 1970 samples from SRA, Gemma, BrainSpan, and ReCount, plus 43 microarray experiments totaling 5134 arrays from GEO.
Expression was estimated with Cufflinks, RSEQtools, and RSEM; co-expression networks used rank-transformed correlation edge weights, with comparisons to mutual information, ARACNE, CLR, MRNET, and WGCNA thresholding.
Functional connectivity was assessed by guilt-by-association using neighbor voting and other machine-learning classifiers, with 3-fold cross-validation and AUROC against GO, KEGG, and Reactome annotations; aggregate networks were built by summing rank-standardized edges across datasets.

Limitations

Stated: the authors note that their use of GO as the definition of function is the most easily critiqued aspect, although KEGG and Reactome gave nearly identical results; they also state that they cannot readily assess RNA-seq in its most novel applications, such as novel or more diverse transcript assessment.
Stated: they do not claim to identify an ideal RNA-seq co-expression method, and they note that focused uses such as tissue-specific or developmental analyses may not be higher performing in their framework.
Evident: the study is limited to human datasets and existing annotations, ignores alternate isoforms in aggregates and performance measures, and uses median values when multiple measures exist, so isoform-level or cross-species generalization is not established.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/bioinformatics/btv118

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 17 cited sources checked · 0 supported · 0 flagged

A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On average, a given connection in the aggregate is only more tightly connected than in 35 out of the 50 underlying datasets”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > A B
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Performance in the collagen complex is very broadly distributed both between gene pairs and across different datasets.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > A B
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This diffuse improvement between gene-pairs depends on a large fraction of the data for each connection.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > A B
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, cross platform correlations are much lower (RNA-seq networks to microarray aggregate average q ¼ À0.02, microarray networks to RNA-seq aggregate average q ¼ 0.06) than the correlations between the aggregate and the individual networks of the same platform (RNA-seq networks to RNA-seq aggregate average q ¼ 0.38, microarray networks to microarray aggregate average q ¼ 0.26).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.5 Microarray and RNA-seq-derived networks have different hub genes
Load-bearingStrength:plain· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“top 3%, $500 genes) were significantly dissimilar in the two networks (P < 1.4eÀ9).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.5 Microarray and RNA-seq-derived networks have different hub genes
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This set of transcripts exhibits starkly different topological properties in the RNA-seq co-expression and microarray data, being very high node degree within microarray and very low in RNA-seq (Fig. 6B).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.6 Noisy expression variation dominates topology outside of functional connectivity
Load-bearingStrength:plain· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Since our results were virtually identical using KEGG and Reactome as references”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the correlation between the node degree between the RNA-seq aggregate and the microarray aggregate was negative (q < 0) across all methodologies”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.5 Microarray and RNA-seq-derived networks have different hub genes
Load-bearingStrength:plain· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“individual RNA-seq networks exhibit a sample size dependency very similar to microarray data”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:plain· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If we use the same criterion to define a set of transcripts (failing SEQC guidelines but in microarray), we see no such difference in node degree between microarray and RNA-seq co-expression (Fig. 6C).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.6 Noisy expression variation dominates topology outside of functional connectivity
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the functional connectivity of the co-occurrence network is very poor (average AUROC ¼ 0.56, Supplementary Fig. S2H).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.5 Microarray and RNA-seq-derived networks have different hub genes
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even so, both networks capture individual GO functions in a similar way (P ¼ 0.84, Fig. 4B and inset).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.5 Microarray and RNA-seq-derived networks have different hub genes
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While our analysis does not find evidence to support the view that such uses will be higher performing in any way”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Strength:hedged· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The correlation between node degrees of this network and the true co-expression network (q ¼ 0.45, Supplementary Fig. S5D)”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.5 Microarray and RNA-seq-derived networks have different hub genes
Load-bearingStrength:plain· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“implying that the results are not strongly method independent.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.4 Machine learning methods do not confound network performance results
Load-bearingStrength:boosted· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
In the paper
“In the absence of careful experimental design, distinguishing gene expression variability due to technical artifacts and biological signal is challenging (Auer and Doerge, 2010).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 1 Introduction
What the sources say
- Not checked yetcites Auer and Doerge, 2010 source
Load-bearingStrength:plain· not yet calibratedCausal:conditional· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
None of the sources cited for this claim have been checked yet.
In the paper
“Very few analyses to date have been performed on co-expression networks from RNA-seq data (although see, e.g.Iancu et al., 2012;Sekhon et al., 2013), with fewer conclusions drawn about their utility, nor has their presumed novelty been fully assessed.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 1 Introduction
What the sources say
- Not checked yetcites Iancu et al., 2012 source
- Not checked yetcites Sekhon et al., 2013 source
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes? Checkable: unclear✓ Load-bearing: passes✓ Based: passes
None of the sources cited for this claim have been checked yet.
In the paper
“Very few analyses to date have been performed on co-expression networks from RNA-seq data (although see, e.g.Iancu et al., 2012;Sekhon et al., 2013), with fewer conclusions drawn about their utility, nor has their presumed novelty been fully assessed.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 1 Introduction
What the sources say
- Not checked yetcites Iancu et al., 2012 source
- Not checked yetcites Sekhon et al., 2013 source
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
In the paper
“Even 'gold-standard' expression datasets perform relatively poorly under GBA when assessed in benchmarking exercises (e.g. the Mousefunc competition for genome-wide gene function prediction- Pena-Castillo et al., 2008).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 1 Introduction
What the sources say
- Not checked yetcites Pena-Castillo et al., 2008 source
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
In the paper
“GBA states that genes with similar functional properties will tend to interact or exhibit similar profiles in network data, such as co-expression.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 1 Introduction
What the sources say
- Not checked yetcites Oliver, 2000 source
Load-bearingStrength:plain· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“to comprehensively capture unknown functional information may require >50 separate experiments.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.3 Aggregating across many networks improves performance and removes noise
Load-bearingStrength:hedged· not yet calibratedCausal:conditional· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The aggregate network's performance on GO terms is a valid proxy for capturing unknown functional information.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This suggests, once again, that the very high performance in aggregate is distributed throughout many moderately significant pairwise relationships.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > A B
Load-bearingStrength:hedged· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
None of the sources cited for this claim have been checked yet.
In the paper
“Very few analyses to date have been performed on co-expression networks from RNA-seq data (although see, e.g.Iancu et al., 2012;Sekhon et al., 2013), with fewer conclusions drawn about their utility, nor has their presumed novelty been fully assessed.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 1 Introduction
What the sources say
- Not checked yetcites Iancu et al., 2012 source
- Not checked yetcites Sekhon et al., 2013 source
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
None of the sources cited for this claim have been checked yet.
In the paper
“However, meta-analysis across many datasets often improves performance dramatically (Gillis and Pavlidis, 2011a;Lee et al., 2004;Wren, 2009).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 1 Introduction
What the sources say
- Not checked yetcites Gillis and Pavlidis, 2011a
- Not checked yetcites Lee et al., 2004 source
- Not checked yetcites Wren, 2009 source
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Co-expression has frequently relied on the aggregation of diverse data and so many of the problems that affect RNA-seq as a pure measure of transcriptomic activity may not have downstream importance in co-expression use.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:hedged· not yet calibratedCausal:conditional· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our interpretation of the topological shuffling we see between RNA-seq and microarray is not that they are robustly different (even though the significance is high), but rather that the measurements are easily influenced by weak biases (e.g.gene size).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:plain· not yet calibratedCausal:conditional· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“implies that there is a strong influence of the presence of a gene pair contributing to a higher node degree correlation.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.5 Microarray and RNA-seq-derived networks have different hub genes
Load-bearingStrength:boosted· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“suggesting that technical issues are not at the heart of the 'noisiness' of co-expression data.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:hedged· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, RNA-seq is most robust in its topology where artifacts likely dominate, such as in within given pipelines or cooccurrence in meta-analysis.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.6 Noisy expression variation dominates topology outside of functional connectivity
Load-bearingStrength:hedged· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“implying that many highly connected genes in one network were now no longer well connected in the second.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.5 Microarray and RNA-seq-derived networks have different hub genes
Load-bearingStrength:hedged· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Altogether, this strongly suggests that technical variation drives the topological differences between microarray and RNA-seq.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.6 Noisy expression variation dominates topology outside of functional connectivity
Load-bearingStrength:boosted· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
None of the sources cited for this claim have been checked yet.
In the paper
“SEQC provides criteria under which transcript expression estimates are not expected to be reliable in RNA-seq pipelines (with moderately low expression-the bottom one-third of transcripts-being a major threshold).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.6 Noisy expression variation dominates topology outside of functional connectivity
What the sources say
- Not checked yetcites Li et al., 2014
- Not checked yetcites SEQC/MAQC-III Consortium, 2014 source
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Because these transcripts contribute many hub-like genes to the microarray data, this effect is fully sufficient to account for the lack of overlap we see.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.6 Noisy expression variation dominates topology outside of functional connectivity
Load-bearingStrength:boosted· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we suggest our results are likely robust to most choice of reference data.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:hedged· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Altogether, this strongly suggests that the topological reordering we see between microarray and RNA-seq reflects noisy expression”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.6 Noisy expression variation dominates topology outside of functional connectivity
Load-bearingStrength:boosted· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“RNA-seq is both more sensitive at detecting when this occurs (when expression is too low to be reliable)”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.6 Noisy expression variation dominates topology outside of functional connectivity
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.

It may state two separate claims as one. 33
Probably not a substantial claim. 6

A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The average AUROC varied slightly between methods (Supplementary Table S4), with the neighbor voting, logistic regres- sion and stochastic gradient descent giving the highest performance (average AUROC ¼ 0.69), and the random forest performing the worst of the six methods (average AUROC ¼ 0.63).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.4 Machine learning methods do not confound network performance results
Load-bearingStrength:plain· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, all the methods perform well on similar GO groups as neighbor voting (performances correlated at q ¼ 0.87-0.96, P < 2.2eÀ16)”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.4 Machine learning methods do not confound network performance results
Load-bearingStrength:plain· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the aggregate, almost half (47%) of the gene pairs exhibit high ranking (top 10%) co-expression, while a proportionately smaller fraction ($2%) exhibit very highly ranked (top 1%) co-expression.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > A B
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we verified by removing the connections ranked in the top 1% and seeing no significant drop in performance whereas removal of the connections in the top 10% had a large effect (AUROC $0.6).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > A B
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In co-expression analysis, increasing depth past 100 M aligned reads per sample and even thousands of samples continue to add value if our focus is on genome-wide functional inference for a given dataset.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“RNA-seq coexpression and microarray co-expression are very similar in their functional properties but very dissimilar in their overall topology (Fig. 6A).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.6 Noisy expression variation dominates topology outside of functional connectivity
Load-bearingStrength:plain· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“a simple rank-andsummate aggregation across multiple datasets generates a more robust network, capturing known biology with high efficacy.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our study suggests that 1 M reads per sample (with many samples) is sufficient to build a network that is slightly divergent from random and adding such networks has no negative impact on the performance of the aggregate network.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:hedged· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“simple, basic approaches were among the highest performing, from measuring network connectivity (Spearman correlation) to functional inference (neighbor voting).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Here, again, performances are similar in microarray and RNA-seq (mean GO AUROCs of 0.64 and 0.65, respectively), but topology is starkly different with a correlation of q ¼ À0.66 between the node degrees of the two networks constructed.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.6 Noisy expression variation dominates topology outside of functional connectivity
Load-bearingStrength:plain· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
In the paper
“These include dependencies on library preparation, sequencing depth, gene lengths, GC content and transcript lengths, as well as other laboratory-specific practices (Hitzemann et al., 2013).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 1 Introduction
What the sources say
- Not checked yetcites Hitzemann et al., 2013 source
Load-bearingStrength:plain· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Altogether, our examination of the collagen complex supports the view that aggregation will work best in functions where pairwise relationships are expected to be distributed (e.g. protein complexes rather than dynamic pathways) and weak aggregate effects can summate to provide a strong total signal.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > A B
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
None of the sources cited for this claim have been checked yet.
In the paper
“Node degree is central to network topology and a major factor in numerous other properties (Gillis and Pavlidis, 2011b;Newman, 2003).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 1 Introduction
What the sources say
- Not checked yetcites Gillis and Pavlidis, 2011b
- Not checked yetcites Newman, 2003 source
Load-bearingStrength:plain· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Users of co-expression data cannot naively apply those recommendations because the minimum number of samples is already relatively high, and robustly measuring correlations may require an even greater increase.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:plain· not yet calibratedCausal:conditional· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This indicates that the negative correlation between microarrays and RNA-seq data is a recurring feature of the individual networks, and not just the aggregates, and suggests a feature due to technological biases (in conjunction with network construction methods, Supplementary Fig. S5C).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.5 Microarray and RNA-seq-derived networks have different hub genes
Load-bearingStrength:hedged· not yet calibratedCausal:correlational· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This dissimilarity suggests again that while inference methods average away topological noise, the actual networks exhibit it to a substantial degree and this could be easily altered by varying methodology in other ways.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.5 Microarray and RNA-seq-derived networks have different hub genes
Load-bearingStrength:hedged· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“this may reflect our reliance on GO as a reference, even for narrowly defined functions.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Strength:hedged· not yet calibratedCausal:conditional· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passes
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unsupervised interpretation of RNA-seq data connectivity is unlikely to be robust to underlying methodological choices without careful filtering, while methods that extract signals using training data (supervised or semi-supervised methods) appear to safely recover known information.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:hedged· not yet calibratedCausal:conditional· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Further, the low impact of variation in node degree on semisupervised learning (e.g. through GBA) suggests that even without the appropriate filtering, many methods adequately identify the absence of learnable features in noisy expression.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Strength:hedged· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the machine learning algorithm selected has little influence on the results, which also implies that the features being selected to make predictions, determine connectivity and assess the networks are adequate for the task at hand.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“So profound is the performance difference between aggregate and individual networks that it is difficult to recommend any minimum depth or sample count which is adequate; certainly none in our case come close to producing the same apparent performance.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our analysis suggests that the reordering reflects the tendency for low-expressing genes to be correlated in microarray data, but that RNA-seq allows us to identify which genes are 'too' low to be trusted with much greater specificity.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:hedged· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and in not generating spurious correlation among these noisy values (low node degree).”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 3.6 Noisy expression variation dominates topology outside of functional connectivity
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
It may state two separate claims as one.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Co-expression analyses will still require many samples and a reliable quality reference network as a control and comparative measure.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:plain· not yet calibrated
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✓ Based: passes
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Because of our reliance on pre-existing knowledge (GO) or data (microarray) to provide reference knowledge, we cannot readily assess RNA-seq where its application is likely to prove most novel and exciting, e.g.novel or more diverse transcript assessment.”
§ Gene expression Guidance for RNA-seq co-expression network construction and analysis: safety in numbers > 4 Discussion
Load-bearingStrength:plain· not yet calibratedCausal:direct· not yet calibrated
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
Probably not a substantial claim.
- Rests on an assumption: The GO reference data adequately represents biological function for evaluating co-expression network performance.
- Rests on an assumption: The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.

36 of 36

The paper relies on this without support.
RNA-seq co-expression analysis is in its infancy and reasonable practices remain poorly defined.
“RNA-seq co-expression analysis is in its infancy and reasonable practices remain poorly defined.”
The paper relies on this without support.
The GO, KEGG, and Reactome annotations used as labels accurately reflect gene function and are sufficiently complete for evaluating functional connectivity.
“By using known annotations as labels for the genes (here we used GO, KEGG and Reactome labeled gene sets), one can determine how well the network recapitulates this information”
The paper relies on this without support.
The 10 RNA-seq experiments selected for method comparisons are representative of the broader corpus.
“We used a sample of 10 representative experiments, comparing the estimation expressions of Cufflinks, RSEQtools and RSEM and constructed co-expression networks using our default procedures”
The paper relies on this without support.
The aggregate network's performance on GO terms is a valid proxy for capturing unknown functional information.
“to comprehensively capture unknown functional information may require >50 separate experiments.”
The paper relies on this without support.
The RNA-seq and microarray datasets used are comparable enough that observed differences in network topology and sample-size dependency reflect technology rather than dataset composition.
“In a similar fashion, we generated an aggregate of the microarrays on a random sample of 30 datasets using 3320 individual microarrays.”
The paper relies on this without support.
GO slim's 109 terms are broadly representative of the ontology overall.
“For our machine learning analysis, we used GO slim, a version of GO that filters analyses to 109 GO terms, broadly representative of the ontology overall.”
The paper relies on this without support.
The GO reference data adequately represents biological function for evaluating co-expression network performance.
“our use of the GO as a way of defining 'function' is perhaps the most easily critiqued, albeit hard to avoid and common.”
The paper relies on this without support.
The 50 RNA-seq and 30 microarray datasets are representative of their respective technologies.
“We used the 30 networks from the microarray aggregate, and the 50 networks of the RNA-seq aggregates”

8 of 8

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

Conflicting minimum read-depth guidance
Major
The abstract gives more than 10 million reads per sample as a minimum for performance comparable to microarrays. The results section says that even modest performance requires close to 100 million reads per sample, a substantially higher threshold.
Machine-learning method dependence is described in opposite ways
Minor
The results say the findings are not strongly independent of method, whereas the discussion says the selected algorithm has little influence on them.
Figure 4 correlation is labeled as a P-value in the text
Minor
The text assigns P = 0.84 to the similarity shown in Figure 4B, but the figure caption identifies 0.84 as the correlation and gives its P-value as less than 1e−16.

3 of 3

Random Forests
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Fast gapped-read alignment with Bowtie 2
Langmead, B., & Salzberg, S. L. (2012) · Nature Methods · doi:10.1038/nmeth.1923 · Cited by 56,400
Gene Ontology: tool for the unification of biology
Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M., Davis, A. P., Dolinski, K., Dwight, S. S., Eppig, J. T., et al. (2000) · Nature Genetics · doi:10.1038/75556 · Cited by 43,206
KEGG: Kyoto Encyclopedia of Genes and Genomes
Kanehisa, M. (2000) · Nucleic Acids Research · doi:10.1093/nar/28.1.27 · Cited by 37,077
KEGG: Kyoto Encyclopedia of Genes and Genomes
Ogata, H., Goto, S., Sato, K., Fujibuchi, W., Bono, H., & Kanehisa, M. (1999) · Nucleic Acids Research · doi:10.1093/nar/27.1.29 · Cited by 31,232
RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome
Li, B., & Dewey, C. N. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-323 · Cited by 22,179
The Structure and Function of Complex Networks
Newman, M. E. J. (2003) · SIAM Review · doi:10.1137/s003614450342480 · Cited by 18,719
Cluster analysis and display of genome-wide expression patterns
Eisen, M. B., Spellman, P. T., Brown, P. O., & Botstein, D. (1998) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.95.25.14863 · Cited by 16,184
Transcript assembly and quantification by RNA-Seq reveals unannotated transcripts and isoform switching during cell differentiation
Trapnell, C., Williams, B. A., Pertea, G., Mortazavi, A., Kwan, G., van Baren, M. J., Salzberg, S. L., Wold, B. J., & Pachter, L. (2010) · Nature Biotechnology · doi:10.1038/nbt.1621 · Cited by 15,556
Mapping and quantifying mammalian transcriptomes by RNA-Seq
Mortazavi, A., Williams, B. A., McCue, K., Schaeffer, L., & Wold, B. (2008) · Nature Methods · doi:10.1038/nmeth.1226 · Cited by 13,696

Flagged first, then by citations

Show all 47 references

2 of the 47 references couldn’t be matched to a known paper.

Sorted by citations

Gene co-expression analysis for functional classification and gene–disease predictions
van Dam, S., Võsa, U., van der Graaf, A., Franke, L., & de Magalhães, J. P. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbw139 · Cited by 995
Impact of Genetic Polymorphisms on Human Immune Cell Gene Expression
Schmiedel, B. J., Singh, D., Madrigal, A., Valdovino-Gonzalez, A. G., White, B. M., Zapardiel-Gonzalo, J., Ha, B., Altay, G., Greenbaum, J. A., McVicker, G., et al. (2018) · Cell · doi:10.1016/j.cell.2018.10.022 · Cited by 867
Systems biology and gene networks in neurodevelopmental and neurodegenerative disorders
Parikshak, N. N., Gandal, M. J., & Geschwind, D. H. (2015) · Nature Reviews Genetics · doi:10.1038/nrg3934 · Cited by 430
Characterizing the replicability of cell types defined by single cell RNA-sequencing data using MetaNeighbor
Crow, M., Paul, A., Ballouz, S., Huang, Z. J., & Gillis, J. (2018) · Nature Communications · doi:10.1038/s41467-018-03282-0 · Cited by 365
Self-citation
CEMiTool: a Bioconductor package for performing comprehensive modular co-expression analyses
Russo, P. S. T., Ferreira, G. R., Cardozo, L. E., Bürger, M. C., Arias‐Carrasco, R., Maruyama, S. R., Hirata, T. D. C., de Lima, D. S., Passos, F. M., Fukutani, K. F., et al. (2018) · BMC Bioinformatics · doi:10.1186/s12859-018-2053-1 · Cited by 313
“around 20 samples (which is in accordance to previous findings [26]), indicating that the β x R(2) curve, and thus”
Result

See all 260

Authors: Ballouz, S., Verleyen, W., & Gillis, J.
Affiliations: Cold Spring Harbor Laboratory (US)
Corresponding author: Ballouz, S.
Gillis, J.
ORCID: Ballouz, S.
Gillis, J.
Venue: Bioinformatics
Publisher: Oxford University Press (OUP)
Published: 2015-02-28
Volume: 31
Issue: 13
Pages: 2123-2130
ISSN: 1367-4803 (ISSN-L)
1367-4811
DOI: 10.1093/bioinformatics/btv118
OpenAlex: W2104724027
Cited by: 215
References: 47
Citations by year: 2026: 7 · 2025: 15 · 2024: 7 · 2023: 21 · 2022: 15 · 2021: 31 · 2020: 18 · 2019: 27 · 2018: 24 · 2017: 23 · 2016: 20 · 2015: 6
FWCI: 10.55
Type: journal article
Language: en
Open access: bronze
doi.org
Locations: Bioinformatics (publishedVersion)
PubMed (publishedVersion)
Cold Spring Harbor Laboratory Institutional Repository (Cold Spring Harbor Laboratory) (acceptedVersion)
HighWire Press Open Archive (submittedVersion)
Topics: Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Gene expression and cancer classification
Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Genomics and Phylogenetic Studies
Keywords: guilt by association, gene co-expression, network topology, hub genes, sample size requirements, sequencing depth, stochastic gene expression, microarray comparison, functional connectivity, gene function prediction
MeSH terms: Algorithms, Brain, Humans, Models, Theoretical, Sequence Analysis, RNA, Computational Biology, Gene Expression Profiling, Gene Regulatory Networks, High-Throughput Nucleotide Sequencing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Guidance for RNA-seq co-expression network construction and analysis: safety in numbers

Abstract

Overview

AI snapshot

TL;DR

Key findings

Methods

Limitations

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks